One of the key components for building a good object recognition system is labeled data. Given the diversity of the images uploaded by Mapillary users, labeled data is even more important. We started with a very limited set of labeled images of EU/US traffic signs. The accuracy of the traffic sign recognition algorithm has been affected by that. Users’ feedback is particularly valuable in identifying the misclassified signs as well as confirming the correct signs.

By retraining our recognition algorithm with the user feedback, we are able to improve the recognition accuracy. The precision-recall curve below shows the improvement for EU Speed limit 50 signs after retraining. Specifically, we now obtain both higher recall (i.e. missing fewer EU Speed limit 50 signs) and higher precision (i.e. less likely to recognize other signs as EU Speed limit 50 signs). At the 75% recall level, after retraining, we improve the precision by 45% (from 50% to 95%). For all the EU traffic sign classes we retrained, on average, we gain approximately 20% increase in precision at the same recall level.

Below are two examples of US traffic signs. Overall, we observe approximately 10% increase in average precision for the US traffic signs with retraining.

So keep your traffic sign feedback coming and play the traffic sign game. The traffic sign recognition will get better and better with your help! We love all your other feedback too. Feel free to comment or contact us on email and twitter.

/yubin