How Mapillary Works with Cities
At the last Esri User Conference, I met an environmental consultant who worked on a project verifying fire hydrant locations for a small town. He checked out each coordinate on Google Street View -- “by hand!” he insisted, referring to restrictions on using the Google Maps API for deriving datasets and asset tracking. He hadn’t minded the laborious process so much as an unexpected problem: Street View was neither current nor complete. Street View cars had driven through the town over three years ago, and hadn’t captured smaller streets and hiking trails.
The problem isn’t just a small town one. Street View gets updated more frequently in major metropolitan areas, but even once a year isn’t enough for cities where the only constant is change.
“What we want is Street View that we can control,” a GIS manager in New York City told me.
Simple, Smart Street Photos
As better technology becomes more accessible to local government, interest has bubbled up in on-the-ground visuals to enhance applications for transportation, public works, parks, environmental compliance, city planning and disaster recovery. Unlike SAAS tools and cloud infrastructure, procuring street photos remains beyond most municipal budgets. Bigger cities that may have funds to hire professional rigs to cover their streets find that after months of image collection and post-processing, the images are already obsolete.
When we show cities how Mapillary turns photos from any device into 3D maps within minutes, reactions are laced with skepticism about image quality and geospatial accuracy. While it’s true that better devices make for better 3D reconstructions, footage from mobile phones and action cameras has surpassed “good enough” for most uses, and will only get better. The footage from such devices can be stitched together with photos captured with professional equipment (normally relegated to dust-gathering hard drives) to complement high quality imagery with recent coverage.
A park with playground facilities in Trollhattan, Sweden, taken with a mobile phone
An overpass in Buenos Aires, Argentina, taken with a mobile phone
A hiking trail in the Cleveland Metropark, OH, taken with a mobile phone
A residential street prior to a development project in Ashland, WI, taken with a Ricoh Theta S
The exterior of a railway station in St. Denis, France, taken with a Ricoh Theta M15
The streets of Helsingborg, Sweden, professional capture
Crowdsourcing and Open Data
The premise of creating 3D maps with consumer devices not only puts mapping in the hands of local government, but also drives civic engagement by empowering citizens. Citizens all over the world have contributed over 45 million photos to Mapillary to date. City-led crowdsourcing can improve street photo coverage in targeted areas while spurring conversations about issues like road safety, conservation, and sustainable development. By releasing street photos to the public as part of Open Data initiatives, local governments can push for more photo contributions and for participation in data-driven projects.
A public competition to create a basemap for urban planning in Lesotho
“OpenPompeii”: a project promoting open data related to the cultural heritage of the city of Pompeii
An open call to action by the Finnish Transportation Agency to map waterways in Finland
Street photos of Melbourne and Geelong, Australia from VicRoads, submitted as part of an open data effort
The Road Ahead
Turning photos into 3D maps isn’t just a neat party trick. What happens behind the scenes is key to improving the GPS positions of the images and determining precise locations of objects within the images.
Point Clouds from 2D photos -- no LiDAR required
The geospatial data can then be extracted with the Mapillary API for use in GIS applications. The Mapillary for ArcGIS app, developed in partnership with Esri, lets GIS teams add and edit features to their layers as they navigate virtual streets, syncing back to ArcGIS with one click.
More advanced tools have the potential to automate more GIS workflows. Today, Mapillary recognizes traffic signs based on North American and European standards, with lots of help from the Mapillary community to train the detection algorithm. Initial work on Optical Character Recognition (OCR) shows promise for reading highway signs, addresses, billboards, and more.
Automated traffic sign detection in San Francisco, CA
Optical Character Recognition (OCR) on a highway sign in the Bay Area, CA.
Soon, progress in deep learning will expand the types of objects that Mapillary can classify and detect. Point clouds today are just the beginning of what will become modelling and measuring capabilities in 3D. Combined with geospatial information, the world of data that computer vision opens up is the virtual equivalent of an inspector or surveyor walking through the streets taking notes -- all at one’s fingertips.
“There’s certainly some one who spends hours and hours on this stuff,” said an IT director for a large city in Asia. “I don’t want to put the guy out of a job, but I have a feeling he’ll appreciate spending his time on something else.”
Human-powered Mapping
To be sure, discounting the human element in Mapillary’s mission to make a 3D visual map of the world would miss the point. The advances in computer vision are only useful when the technology is within reach of communities everywhere, no matter how small or remote.
At another Esri event, I struck up a conversation with a one-man GIS team about how to capture street photos of his town. I suggested mounting mobile phones on garbage truck windshields, and his eyes lit up. “Oh yes”, he said, “One of our haulers broke down last month but I could check with our guy on the other one.”
/janine