Mapillary’s approach to world-wide street-level image data collection is a collaborative one. A large community of engaged users has already gathered over 135 million images, covering more than 2.6 million kilometers. Combining this big data with our visual computing technology, we are disrupting digital mapping technologies that empower a broad range of emerging sectors like autonomous driving, intelligent transportation, and smart city planning.

We are very excited that the Austrian Research Promotion Agency, FFG, has decided to support our proposed research project that aims to further enhance our image processing technology for building next-generation maps. Conventional mapping solutions are typically restricted to dealing with static map content, thus limiting the power of applications in location-based services (e.g. ride sharing, automated delivery, and mobile gaming). In the context of our FFG-supported project, we will continue to develop AI-driven algorithms for fully automated generation and population of digital 3D maps based on large-scale image processing. The resulting maps will contain a broad range of street-level specific assets like traffic signs, traffic lights, manholes, catch basins, column poles, benches, etc., all of them geo-positioned and easily accessible for location-based services like autonomous driving.

In our funded project, we aim to develop smarter machine learning algorithms, ultimately merging findings from semantic street-level image segmentation, instance-specific object recognition, and image-based 3D modeling at scale for improved 3D positioning of map objects. Thanks to our recently announced Mapillary Vistas Dataset, we have already greatly improved our automated semantic segmentation results for street-level images as shown in some example test videos below (where color-codings associated with object classes are superimposed on the original video frames, e.g. blue corresponds to cars, green to vegetation, etc.).

The first example shows a 360° video (whereas the segmentation algorithms never learned from 360° training data) taken from a cyclist’s perspective and exhibits all kinds of real-world distortion (blur, rolling-shutter, camera motion). The second video has been taken from inside a car and shows semantic segmentations of street-level image data.

With the grant funding for the ~1.9 million EUR project we are ramping up our Research team and investing in more sophisticated computing infrastructure, so that we can eventually provide the desired next-generation maps for the majority of our global image repository—always up to date as our customers and users keep updating the places they care about.

We are delighted to announce that Samuel Rota Bulò has joined us as Senior Researcher in May 2017. Samuel is an expert in the fields of machine learning, computer vision, and optimization, graph and game theory, and has been awarded several best paper prizes (including the prestigious Marr Prize at ICCV 2015). Our second new hire is Lorenzo Porzi, who joins us as Postdoctoral Researcher with hands-on experience in machine learning, 3D modeling, mobile augmented reality applications, and gesture recognition.

With our new team members, we are very much looking forward to taking our image processing models to the next level and to sharing our findings with the computer vision community at research.mapillary.com.

/Peter and Mapillary Research