Our new traffic sign recognition algorithm supports 1,500 traffic sign classes—an increase of 600 classes—across over 100 countries, and has a detection rate of 98%.

Traffic signs are one of the most common and essential map objects on the road. They show us the way and advise us in everything from speed limits to when to stop and where. And if we want to understand the world, we need to understand traffic signs at scale. That’s why Mapillary uses computer vision to detect and recognize traffic signs in street-level imagery.

Developing scalable—and, of course, accurate—traffic sign recognition on a global level is challenging even for the most advanced computer vision technology. First, traffic signs that are in the same class and therefore have the same meaning, such as stop signs, don’t all look the same. This becomes even more problematic when considering that there are shifts in what traffic signs look like depending on the weather, or if the sign is partially hidden by, say, a tree or a car.

Secondly, traffic signs can be visually similar to other objects in the scene, e.g. tail lights on vehicles or even random patterns formed by tree branches. Lastly, a traffic sign taxonomy is needed to organize the different traffic signs into semantic classes and countries.

With the advancement of our algorithms and the help from our community (who’s playing the Verifier Game), we have made great progress.

Traffic sign recognition in city and highway scenes

Towards a global traffic sign taxonomy: introducing 1,500 traffic sign classes

We see that each type of traffic sign has its importance when it comes to navigation and mapping. As part of our ongoing efforts, we have expanded our support to 1,500 traffic sign classes globally (from 900 classes previously).

Here are some of the classes we will support in the new release. A few notable examples are the complementary signs, end of speed limit signs, as well as signs related to road constructions, which have crucial indications for navigation and map updates.

Examples of new signs

Given the further expansion of signs, we see the importance of a traffic sign taxonomy to unify traffic sign similarities and variations globally. As part of this effort, we have also mapped MUTCD codes in the US to the Mapillary traffic sign taxonomy, which will be available in our API soon. The next step is to create such a mapping between Mapillary traffic sign taxonomy for all other countries.

The Mapillary recognition system: detection and classification

Our traffic sign recognition system consists of two basic components: detection and classification. Detection is the step of finding where traffic signs are in images. Classification is the step of classifying each detected traffic sign into the corresponding traffic sign class.

To tackle the problem of traffic sign detection under varying capture conditions, we have integrated our traffic sign recognition with semantic segmentation globally as part of the detection step.

Given that the semantic segmentation network is trained with Mapillary Vistas Dataset in diverse lighting conditions and different countries, we achieve a detection rate of 98% for traffic signs, with false positives on vehicles, buildings and vegetations properly skipped. The detection is completed with a refinement process on the traffic sign segmentation to identify individual traffic sign instances as image patches.

Semantic segmentation and classification of traffic signs

The classification module is a light-weighted neural network trained for classification of image patches. Its key role is to tell the difference between different sign classes and assign a traffic sign class to each image patch.

The training of this classification network benefits immensely from the data verified by the Mapillary community members who have contributed 1,000,000 verified traffic signs. The verified data from the community identifies hard cases of false detections, confusing classification, and occluded signs, which are essential perspectives for the neural network to learn from human-verified data.

Left: statistics on verification of the top 20 traffic signs by the Mapillary community. Right: significant improvements in recognition accuracy when using more verified signs

We are in the process of deploying this release on all newly added Mapillary images. At the same time, we will reprocess all images uploaded before this update as fast as we can. There will be some waiting time for everything to be updated but the result will be notably better map data when it comes to traffic signs.

We would really like thank all our community members for their contributions to the verification game, making traffic sign recognition better every day!

/Tobias