At Mapillary we build a lot of core technology for creating map data from images. Our approach centers around collaboration and openness, computer vision, and sharing of data. We think this is the way forward for the industry as a whole. It is a non-zero-sum game (you know, win-win!) best played in the open.

Collaboration

The only sensible, scalable way to map this planet is to do it together. One of the fundamental data sources for making maps is street-level imagery. Not only map companies but also cities, local governments, road authorities, and many more collect images. Pooling images from individuals, interest groups, companies, and cities means everyone gets more images, faster updates, and better coverage.

Next, these images need to be accessible and available for everyone to use. Keeping imagery offline limits innovation and new ideas for using this data. Startups, researchers, and companies not primarily in the business of collecting map imagery will have new opportunities.

Here’s a great example of side-effect data generated by sharing imagery, our raw 3D model of Amsterdam created from Mapillary images of the city.

3D point cloud that helps us understand relationships and position objects detected in images

Computer vision

As image collection grows exponentially over the next few years, it will become impossible to generate map data in reasonable time using human work (which is how it is done today!). Computer vision, on the other hand, scales infinitely. Computers need to do the heavy lifting to get map updates fast enough and then use human labour to verify and adjust details. This is the only scalable way.

My colleague Chris gave a great example of how a city can get traffic signs and objects automatically detected in the images and made available shortly after publishing their imagery. Here’s Amsterdam again:

Some examples of automatic detections using computer vision

Sharing data

Similar to how people treat image datasets, a lot of organizations generate training data for their computer vision algorithms that they are unwilling to share. For example, the type of densely annotated segmentation images you find in our Vistas Dataset are crucial to many automotive applications, and almost every company we talk to are tasking people to create similar datasets. With high level of detail and number of object classes, each dataset is extremely expensive and labor intensive to generate.

Unless you have a unique image database with types of imagery no one else has, it is naive to think that whatever you sent to your annotation team will be fundamentally different from what your competitors are sending to theirs. Instead, the best way forward is to collaborate and share data. This is especially true in mapping and automotive, where better map data and automotive AI will mean better, safer transportation—and lives saved.

At Mapillary we opened up our entire training set of 20,000 annotated images, available for free under a research license and fully available for commercial licensing. We make all of our images and all the extracted map data available to OpenStreetMap for map editing for free, and also available for commercial licensing.

Samples from our Vistas dataset, used to train object recognition systems

We’re always open to working with more people and companies out there and if you have an interest in collaborating on image collection, computer vision, or training data, send us a note.

Meet us at CVPR

Come meet us at CVPR 2017! Swing by our booth, come to our LSUN Challenge workshop, or see our paper presentations.

/Jan Erik