Point Clouds: Behind the Scenes of Our 3D Navigation
At Mapillary, we don’t want to simply show you photos on maps – we want you to feel immersed in whichever landscape you’re exploring. Behind-the-scenes, we have been 3D reconstructing every location our members photograph and now, we want to show you. We're giving our community the power to virtually explore their photos like never before.
Our members have taken over 40 million photos to date. Each of these are stitched together with computer vision – a type of artificial intelligence that extracts information from images. For every single photo uploaded, we can automatically match features to the ones in photos from the same geographic vicinity so that we can compute how the images relate to each other and how to navigate from one to the other. This is how you navigate between photos in the Mapillary app and web browser.
Taking it one step further, we can also recover a 3D model of every area photographed. What began as a means to improve positioning of each photo beyond the GPS accuracy given by smartphones turned out to be quite an magical viewing experience.
We couldn't keep this to ourselves so now you can explore our underlying 3D data by simply clicking the “show point clouds” option in the sidebar in your web browser. We’ve also added full support for panoramas, which enables you to move seamlessly between regular photos and panoramas in 3D, giving them a smooth and other-worldly viewing experience.
Navigate the World in 3D with Us
Check out the 3D data collected from a member walking across the courtyard at the Alcazar Palace in Seville, Spain - you’ll notice the camera positions as white rectangular frames. (You may recognize the place as the Water Gardens of Dorne in the Game of Thrones TV show)
Here’s the underlying 3D model of a cemetery building in Sweden.
3D reconstruction holds a wealth of value to all of Mapillary. Sure it looks cool, but it also continuously improves the positioning of each photo beyond the capabilities of GPS and enables our customers to make more precise measurements using the photos.
Computer vision is taking maps to the next level. Our algorithms have the ability to not only detect traffic signs and blur faces, but also to navigate photos and virtually explore landscapes in 3D. At Mapillary, we're constantly exploring what else computer vision can help us uncover and we’re excited to show you even more in the months to come.
Let us know what you think on Twitter or email.
/Jan Erik
Special thanks to Pau Gargallo, Yubin Kuang, Oscar Lorentzon, and all OpenSfM contributors who worked to make 3D navigation on Mapillary a reality.