Ever since we launched the new version of the Mapillary platform in June, we have been busy iterating and refining features and stability - often with the help from the Mapillary community who provide valuable feedback. In this post we summarize some of the most recent changes and updates.

Web App

• Improved display of sequences that are still processing: sequences that are still processing and have not been added to the map tiles yet, are now rendered dynamically on top of the map with a lighter blue and a white fill. This allows you to spot these sequences easily while you navigate and inspect them as processing finishes.

  

• Allow display of all map features in map data filters: you can now “select all” point features and traffic signs for display, which is convenient when you want to explore all extracted features in a given region.

  

• You can also now download the map data selected for an area. This is an easy alternative to using the API in order to download data.

  

• Filtering is now improved to make it easier to search and find for the sign you’re looking for.

  

• We redesigned the image details pop-up giving it a clean and fresh look, and also added a direct link to OSM editing with RapiD (in addition to iD and JOSM).

  

• We brought back the ability to change your email address in account settings.

Mobile Apps

Our mobile apps are the easiest way to capture and directly upload to Mapillary. We have followed up with various releases since June. For Android we released 5 updates since the initial 5.0 release including:

• Improving upload stability and fixing various bugs
• Resumeable upload if a file was only partially uploaded
• Expanded device and OS version support
• Improved map explore view and navigation

For iOS we released 9 updates since the initial 5.0 release:

• Better memory handling during upload
• Fixing issues on iOS 12
• Supporting older phones running iOS 10 and above (Usually iPhone 6 to the latest)
• Various UI/UX improvements

Desktop Uploader

The Desktop Uploader is the most convenient way to upload larger amounts of images captured with action cameras or other external cameras. Since June we followed up with 5 significant releases covering:

• Added support for Linux!
• Improved Windows support (prevent antivirus warnings, menu bar always visible, and more)
• Upload progress view now shows the total number of sequences
• Added button to view logs on upload error screen
• Fixed bug that prevented uploading to some organizations
• Added option to interpolate image directions

Mapillary CLI Tools

If you’d like to control upload from other tools or scripts, or want to build more complex processing pipelines, our command line interface (CLI) tools have been and remain the way to go. We’ve updated the tools to use the new platform and followed up with various updates since.

Mapillary API

The new Mapillary API and vector tiles have been launched, approaching Mapillary data from a different perspective. The API focuses on retrieving information about entities, such as an image, a traffic sign, or a map feature, while the vector tiles are served from a new endpoint that we encourage using for bulk data downloads. Since its initial launch in June we have added the following features to the API:

• We restored ability to search for images over a small area using a geographic bounding box and the ability to filter by date
• We restored bounding box search for map features, as well as filters for first seen and last seen dates and object value filters
• You may now add /detections to any image, traffic sign, or map feature API request to get a list of the image segmentations, such as graph.mapillary.com/ <IMAGE_KEY>/detections
• Detections can be searched in a bounding box, and filtered by the date of image capture and the detection value
• Map features and detections accept wildcards when filtering by values
• You now get attributes of an image or map feature based only on the what fields you specify, for example adding graph.mapillary.com/ <IMAGE_KEY>?fields=id,geometry,captured_at returns a slim response with other fields excluded

OSM (RapiD, iD and JOSM) integrations

Integrations of Mapillary imagery and data into OSM editing tools (JOSM, iD and RapiD) are essential to effectively and efficiently leverage the community’s Mapillary contributions to improve OpenStreetMap. We’ve updated all the integrations to leverage the new platform and APIs:

1. RapiD footway mapping coming in November, with a preview October 1 at FOSS4G.
2. OSM iD and RapiD are now using vector tiles directly from Mapillary, with increased performance.

Mapillary JS 4.0

MapillaryJS is an interactive, extendable street-level imagery and semantic mapping visualization platform and a reusable component on the web. Our recent 4.0 release brings major improvements across the board. A dedicated post goes into all the details.

A glimpse behind the scenes

When Mapillary joined Facebook, one of our goals was to leverage Facebook’s world-class infrastructure as a solid and scalable foundation for the future of Mapillary. It was our goal to ensure scalable and reliable upload and processing, as well as scalable and low-latency access to Mapillary data. This meant more than migrating from Amazon Web Services (AWS) to Facebook systems. Instead we replaced many of the key layers of our system with Facebook’s equivalents and re-architected the system from the ground up.

The biggest change that occurred during the transition was Mapillary’s overall processing model, where we moved from a pure streaming approach to batching + streaming. Historically we were processing everything using Apache Storm (+ Trident) and Apache Kafka as part of our architecture. When a user uploaded a new image, we were scheduling “affected” geographic areas for processing. This meant that sometimes certain active areas weren't scheduled for processing for long periods of time (hours), as more images were popping up and the system couldn't "close" the area for processing. In contrast, other areas were reprocessed multiple times. Add to this the time required to actually run the algorithms or generate vector tiles and you end up with very inconsistent update and processing times across the system, respective to geography. To work around this characteristic of our system, and to introduce more predictability into the system, we decided to introduce a daily batch model for parts of our system that needed it the most. We now have combined streaming and batching models, but separated surfaces which they affect (e.g. visualization of map data is batched, but processing imagery is still streaming). For the user of the platform it means: we can keep processing the data as it comes in as soon as we can and keep displaying the relevant data and then on a daily cadence we can regenerate the tiles for the whole world to ensure consistency. To do this data crunching and tile generation we use various of Facebook’s technologies, e.g. Hive and Hadoop and FBLearner flow.

For storage and delivery of binary data we moved from AWS s3 to Facebook’s storage systems and global CDN.

For our public v4 APIs, in the new model we're using Facebook-style Graph APIs and model everything that third-party developers who build applications on top of the Mapillary platform as a graph. This model is heavily inspired by GraphQL.

GraphQL is also the new way that our iOS, Android and Web apps now use to interact with the Facebook backend. It allows us to load only what we need to make loading faster and combine requests in order to minimize the number of API requests when we load complex data (user/org profiles and feeds, dashboard etc.). This directly contributed to performance improvements that we’ll show further down.

The benefit of this major refactor of the complete stack from storage to frontend is that:

• We rely on up-to date technologies
• Are able to leverage Facebook systems, processes, and teams for operations, scalability, and reliability
• We can continue building Mapillary on a much more stable foundation

The performance and stability improvements should be noticeable across the platform now, but one example of its impact is the frontend performance of our web app before and after the updates. For example, time to interactive improved by almost an order of magnitude:

Of course, such a re-architecture endeavour is also not without risk, and after launch we discovered a few areas that needed further optimization and refinement. Our engineering teams worked tirelessly to tackle these as fast as possible. Many of the improvements shared at the beginning of this post are a result of this iteration since initial launch.

The road ahead

With a new solid foundation in place, our goals for Mapillary are:

• Further improving the experience with an initial focus on capture and upload. We will continue relying on Mapillary user feedback to guide us in this process.
• Extending Mapillary with new features to continue and scale delivery of fresh street level imagery globally for use in open mapping and by other partners
• Integrating better in RapiD, but also in iD and JOSM editing tools
• Supporting new ways and devices of capture
• Continue building out 3D understanding and representation of the world

Onward!

/ Till