When we first opened our deep learning AI lab in Graz in 2016, there were 60 million images on the Mapillary platform—that number has now soared to over 440 million. Today we are celebrating the opening of a new research lab in Graz, and I sat down with Peter Kontschieder, Director of Research, and the newest addition to the roster Arno Knapitsch, to talk about how the team came together and what they will be working on this year.

Peter has worked on computer vision problems for the last ten years. Before Mapillary, he worked at Microsoft Research, where the IEEE Computer Society awarded his team (including Samuel Rota Bulò, another researcher at Mapillary) the Marr Prize, one of the most prestigious awards in the field. When approached about starting a research lab at Mapillary, his response was a resounding “Let’s do this!”. For Peter, the freedom of a research lab in the startup environment allows him to focus on what is needed to drive product development while still dealing with the most challenging questions out there.

“We were able to create a research lab around the way we thought it should be. Product development at Mapillary is tightly coupled with research, and it is important to have a team that keeps us on top of the state of the art.” - Peter Kontschieder, Director of Research

The Mapillary research and computer vision teams, September 2018

When asked about how the team has grown since 2016, Peter’s response was honest: “We’ve had more work to do and more problems to solve than people to solve them.”

Finding the right mix of talents has been key to continuously improving our methods and results. It is the combination of 3D modeling and semantic segmentation that allows us to produce a 3D model of the world through street-level images, and to understand the positioning of real-world objects on the map. Starting in 2017, Mapillary received a grant from the Austrian Research Promotion Agency (FFG) that allowed us to bring onboard new talent and ramp up development—including releasing the Mapillary Vistas Dataset, the world’s most diverse street-level training dataset for teaching machines to see.

Semantic segmentation allows computers to understand images and recognize the objects within them

While a portion of the team works internationally, the Graz office is the heart of the science behind the Mapillary platform—a research lab that combines the information-sharing spirit of the academic community, with the passion and excitement of a startup. One of their goals is to strengthen relationships with the entire research community by hosting workshops and interacting with many different academic and corporate partners. In 2017 they co-hosted a workshop with Berkeley University at CVPR, and again in 2018 they co-hosted another recognition challenge workshop with COCO, co-located with ECCV. At the housewarming of the new Graz office, Adrien Gaidon, Machine Learning Lead at Toyota Research Institute, gave a talk on some of the latest technology behind automated driving. Mapillarians from all over the world tuned in for the live feed.

As the most recent addition to the team, joining the Graz office was like coming home for Arno Knapitsch—he studied computer vision and machine learning at the Graz University of Technology. Arno also worked at the world-renowned CERN in Switzerland at the Institute of Atomic & Subatomic Physics where he was involved in experiments to improve the performance of cancer-detecting medical scanners. Having kept in touch with his friends in the computer vision field and following exciting developments in the area, he decided to make a return to machine learning.

“I always followed the progress of the field, and the exciting new developments of friends and former colleagues. Looking at Mapillary, I thought damn, this is really cool.” - Arno Knapitsch

Arno’s strong background in 3D modeling makes him a perfect fit to tackle the challenge of teaching autonomous vehicles to see and understand the world around them. For him, the biggest challenge that self driving cars face isn’t whether the technology will work, but how they will gain public acceptance and trust.

“I am hoping that the computer vision community can show the public that we are on track for autonomous vehicles and that it’s going to be helpful to people. It will increase safety, reduce traffic, and be beneficial to users.” - Arno

The computer vision technology behind automatically generated map data from user-contributed images on the Mapillary platform

Our research team is coming off of a successful and productive 2018. They won the CVPR Semantic Segmentation Challenge and the ECCV Semantic Segmentation Challenge for Autonomous Navigation in Unstructured Environments. When it comes to going up against some of the bigger research institutes at universities and corporations, Mapillary has had to make do with fewer computing resources.

“That’s forced us to to invent smarter ways to be efficient with our data—to try to make the methods smarter and better exploit the data we have.” - Peter

This need to do more with less has led to the development of a computing approach that saves up to 50% of GPU memory when training deep neural networks. For us, this approach allows our semantic segmentation models to handle more and much larger data samples, but this is work that pushes the entire research community forward and makes artificial intelligence more accessible than ever before.

Keeping that momentum going, this year the research team is looking forward to improving our recognition and positioning techniques, to produce the highest quality map data possible.

“The hype around deep learning triggered resources being put into the field, but that is starting to level off. People were very enthusiastic, but we are beginning to learn that deep learning isn’t going to solve every problem.” - Peter

The trend of machine learning and deep neural networks will definitely continue into 2019, and both Peter and Arno are looking forward to seeing new ideas grow the field. For us, this means continuing to invent smart solutions that make the most out of our resources and continue to have an immediate impact on Mapillary’s community and customers.

/Lindsey, Communications Assistant and Geographer