Image Connections and Transitions with 3D

The new Mapillary viewer comes with improved navigation using our global 3D reconstruction. It is now easier for you to move across images of the same place and you will see much nicer 3D transitions when moving.

How to use it

There are three ways to move around in the new viewer: free movement, playing a sequence or dragging within a sequence.

Free movement is done with the navigation arrows in UI, the keyboard or with gestures. It is used for exploring everything in an area. There are possibilities to step in all directions as well as turning. Walk down a road, turn to an alley, change direction or take a closer look at the surroundings.

Try it here

Playing is used for automatically moving the camera to the next image in a sequence continuously. This is passive exploration of a route that a single user has mapped. Press play, sit back and enjoy the ride.

Dragging is similar to playing in the sense that it follows a specific sequence. Dragging can be initiated from any image. By dragging down the screen we move to the next image in a sequence and by dragging up the screen we move to the previous image in the sequence. It is possible to change moving direction during dragging at any time. Dragging is very effective on touch screens and when you want to look for a specific photo or view.

Try it here

The viewer uses WebGL, which is enabled by default on most desktop browsers and smartphones. For those of you that do not have WebGL, there is a fallback viewer that works and looks a lot like the previous one. You can check here if you have WebGL enabled.

Behind the scenes

We match images with common content to determine their spatial relation. This process is called Structure from Motion (SfM) and our implementation is based on our open source module OpenSfM. SfM gives us the 3D position of the images nearby and information about the 3D shape of the environment. The new viewer uses that information to move you from one image to another.

Here’s a video showing the camera position and points computed by SfM:

SfM works best for images with large overlaps. So walking sequences in cities will get the best transitions. Sequences on highways tend to get no connections. The viewer will just switch from one image to another when no connection is found. Trees, large open skies and fast turns often confuse SfM algorithms and transition can be shaky in such cases.

Our servers run SfM continuously in the background, constantly looking for new images to process. It is a computationally intensive process and it can take a while between the time you upload the images and the time the process is run. Please, be patient.

We hope you'll enjoy the smoother navigation! Let us know what you think in the comments, Twitter. or through GitHub issues!

/Yubin, Pau, Oscar & The Mapillary Team

Continue the conversation