How to Use Mapillary Data in OpenStreetMap

Mapillary has been processing street-level imagery with computer vision in order to extract traffic signs for several years, but in 2019 we’ve started to offer an expanded dataset. Point features such as fire hydrants, crosswalks, and manholes are also now available, with their location approximated on the map thanks to 3D reconstruction of scenes combined with segmentation of the images where these objects are recognized. To date, Mapillary has identified more than 191 million of these map features, and one of the results is GeoJSON data that can be used to enhance a map.

Crosswalk map features detected in Santa Monica on mapillary.com/app Crosswalk map features detected in Santa Monica on mapillary.com/app

While many feature classes are available (43, to be exact), only a handful of these are directly correlated to OpenStreetMap tags. We offer this data through two avenues: data served from the Mapillary API and data downloaded via an organization. Both of these require creating a Mapillary user account and organization, and are typically a commercial product. In the case of OpenStreetMap, we’re currently experimenting with the data quality in select communities, with plans in the works to make it more accessible to all OpenStreetMap users.

For this post, I have outlined the two ways it is possible to use Mapillary data for updating OpenStreetMap. First, I will walk through how to acquire, verify, and use the data downloaded via an organization. Second, I will briefly introduce Pic4Review, which offers a smooth workflow for verifying and creating new data on OpenStreetMap.

Option 1: Through an organization

Today we’ll walk through how to make use of a Mapillary dataset from Santa Monica, California. We’ll first take a glance at the process of acquiring the data, look at how the data is structured, then jump into how to get mapping.

For more information on how to set up an organization, see the Mapillary help pages. Through my organization, I created a shape for my area of interest, and have generated a download. My Mapillary organization count tells me to expect just under 100,000 unique map features in the download, which we’ll whittle down to a smaller number by choosing just one class: bike racks.

Creating a shape to extract Santa Monica data Creating a shape to extract Santa Monica data

While we’ll verify that each bike rack exists by checking Mapillary photos when adding the data to OpenStreetMap, you can also set up a verification project for your data. You can click here to verify bike racks in Santa Monica and see how verification works. Verifying the image detections means the downloaded data will improve in quality.

Verification Project for bike racks

Once I request a download of all data in the shape, I receive a zip file. For an area like Santa Monica with dense Mapillary coverage, the filesize will be a few gigabytes, but each map feature class comes in its own smaller GeoJSON. In my zip the is an object-point-feature directory that has a folder named after each map feature class available. I find object--bike-rack with a GeoJSON inside.

An example of a Mapillary data download An example of a Mapillary data download

Using QGIS for a quick preview, I can see that there are 51 bike racks spread throughout Santa Monica, as detected by Mapillary. Because I only just started my verification project, some of these are likely to be false positives, but I’ll inspect these manually as an alternative.

Bike racks in Santa Monica, on an OSM basemap Bike racks in Santa Monica, on an OSM basemap

Getting these map features into OpenStreetMap is a simple process. On the main OpenStreetMap website, I can search for or pan to Santa Monica then click Edit to open the OpenStreetMap iD editor. On the right side menu is an icon for Map Data, which can also be opened with the hotkey F.

Opening the Map Data menu in OpenStreetMap iD editor Opening the Map Data menu in OpenStreetMap iD editor

At the bottom of the Data Layers section, you can see a greyed-out option for Custom Map Data. Click the button for more options, and you’ll see a dialog pop up that allows you to choose a file. We’ll choose our GeoJSON.

Load the Mapillary GeoJSON as custom map data Load the Mapillary GeoJSON as custom map data

Click OK and you’ll see magenta circles on the map, representing the Mapillary bike racks. As we get started adding these to OpenStreetMap, be sure to note that the Mapillary data should match the OSM Wiki definition of bicycle_parking. One specific thing to keep in mind is that the bike racks must be intentionally created for bicycle parking, not another object such as a fence, railing, or utility pole that happens to be convenient for parking and securing a bike. Verification projects are extremely helpful when trying to prevent these kinds of errors, as it helps to invalidate similar-looking objects such as cafe chairs, railings, playground equipment, and even odd-shaped public art.

Mapillary-deteced bike racks in magenta, with OSM data and Digital Globe Premium basemap Mapillary-deteced bike racks in magenta, with OSM data and Digital Globe Premium basemap

To get started mapping, I like to hit the W hotkey, which toggles wireframe mode—this makes it easier to see where the magenta circles are located on the map, without so much noise from existing data. Next, I open the Map Data menu again (hotkey F) and turn on the Photo Overlay (Mapillary) layer. Finally, click on one of the green Mapillary image points to load the Mapillary viewer and get a look in the direction of your data.

No dice here: the data incorrectly identified bike racks No dice here: the data incorrectly identified bike racks

The first pair of points I checked includes what looks like a bike locked to a tree, and some cafe chairs. Both can be misleading when the computer vision algorithm analyzes the photos, but here we can invalidate the data. In the verification project I came across these specific two cases, so I know they would be dropped from the dataset automatically if the verification project was used beforehand.

The data is off, but close—the bike racks are between the second and third tree seen in the Digital Globe imagery as well as on Mapillary The data is off, but close—the bike racks are between the second and third tree seen in the Digital Globe imagery as well as on Mapillary

In this second case, we get closer—probably the benches and chairs between the first and second tree on the left were classified as bicycle parking, and I invalidated this in the verification project as well. However, there are posts between the second and third tree, so I can add those to the map.

Bingo! We’ve got bike racks here Bingo! We’ve got bike racks here

The third area I check looks great—a pair of bike racks beside the building, next to an alleyway and a tree at the curb. I’ve added this one to OSM successfully, and adjusted the position slightly to reflect that its near but not directly behind the tree.

Another great find—bike racks outside of businesses Another great find—bike racks outside of businesses

A few blocks away is another great case. I can see several bike racks outside of businesses, with Mapillary data placing them at about the right location, though it overlaps the building footprint. With a little adjustment, it’s now on the map!

A bike-share station already exists, but an unmapped bike rack is just down the street A bike-share station already exists, but an unmapped bike rack is just down the street

There are also cases like bike-share or rental platforms—in this case, Mapillary also detected the Breeze Bike Share rack, which is already on OSM. To the northwest however, there’s another piece of Mapillary data. We can see in the image, between the two palm trees at left, that it’s a bike rack but isn’t on OSM (until now).

I didn’t parse all 51 of the bike racks, but I completed a review of about 25% of them before submitting my changeset with 10 new bike racks. Much of my time was spent checking the data from multiple angles, to ensure it existed and was located in the right position. Much of this effort can be saved by first using the verification tool, as mentioned.

Option 2: Pic4Review

For a smoother workflow, it’s also easy to use Pic4Review, a tool created by Adrien Pavier from the OSM France community that allows both verifying OSM data with Mapillary, and creating new OSM data based on the Mapillary API for map features. For example of using a similar workflow to what we explored in this article but with Pic4Review missions, check out this great tutorial from Mapillary ambassador Nuno Caldeira.

Summary

We are committed to always making the images and data from the Mapillary platform freely available for any person to contribute to OpenStreetMap. There are currently over 528 million images on the Mapillary platform and this data is only beginning to show its potential for fixing and growing the OpenStreetMap database.

Now that our data offerings have expanded beyond traffic signs to dozens of other point classes, there is great potential to use Mapillary to add new data to the map. Fire hydrants, benches, wastebaskets, post boxes, and crosswalks are some other examples of overlapping data. We encourage users to create an organization and a shape for an area they are interested in, and to contact us about downloading the data for OpenStreetMap use.

Much of the Mapillary data represented in the icons here in Phoenix, Arizona are not on OpenStreetMap Much of the Mapillary data represented in the icons here in Phoenix, Arizona are not on OpenStreetMap

All Mapillary data is open for the OSM community to use. While adding data through Pic4Review is one option, anyone can also request Mapillary data for their non-commercial OSM community or project to use. Just follow these steps:

  1. Create a Mapillary organization for your local OSM community or group
  2. Draw a shape to represent the polygon where you want to extract data
  3. Request a map data subscription, but don’t mind the listed data price
  4. Send an email to support@mapillary.com with your organization name and some information about the project you’re working on
  5. If it meets our criteria for non-commercial, direct to OpenStreetMap use we’ll approve your request
  6. Set up a verification project for your shape, to refine the data quality
  7. Finally, download the data and get started mapping!

If you have any other questions about how to use the data, terms and conditions, or ideas for how to improve the workflow of adding Mapillary data to OpenStreetMap, feel free to reach out to us on social media or email. In the meantime, we look forward to working together to improve OpenStreetMap with street-level imagery, computer vision, and a strong community.

/Chris, Solutions Engineer

Continue the conversation
comments powered by Disqus