Map Your Hometown in a Day: An Evaluation of Mapillary
This is a guest post about the experiences of Dutch students from a photo mapping day in Almere. They see great potential as well as crucial points of improvement in using Mapillary.
Map your hometown in a day! At least that is our plan. Our first tests have been very positive. On a sunny afternoon in September, Geo, Media & Design students photo mapped parts of the City of Almere, a new town in the Netherlands, using Mapillary. The training programme at the Aeres College Almere (a university of applied science) blends new media and geography. An excellent match with Mapillary.
In the second year, during the module Big Data in a Smart City, students learn about the possibilities of big and open data. They learn how they are able to subtract relevant information and deduce insights from a large amount of data. Topics like data quality and data value are also on the programme. Before the students dive into big data, it was especially useful to think about “small data”: data that students collect themselves with Mapillary. So we had a group of eleven students mapping Almere around the Weerwater—the urban lake of this Dutch city. You can take a look at our story map of the results (text in Dutch but plenty of visuals).
The waterways on Lake Weerwater in Almere
Observations by the students
Our students are positive about the possibilities and opportunities with the products of Mapillary. Compared with for instance Google Street View, Mapillary is less accurate and it brings lower image quality when using your mobile phone. But it is plenty more topical. Where Google visits many places only once every 5 years (or on tourist hot-spots once a year), with Mapillary you can obtain much more up-to-date information. And that is a big step forward. A shop closes its doors? Tomorrow is it on the map. Mapillary is therefore much more real-time-topical and thus even more useful, for both users and administrators of buildings and public spaces.
In addition, you can use Mapillary in more places—also on spots you cannot reach by a car. "The tool and the underlying platform deliver a highly reliable map," said one of the students, "because I created the data." Also, you can easily see the source: who has uploaded the data. This is a matter of trust: wisdom of the crowd arises in this field instead of data by major companies.
Students evaluating the reliability of the data they've collected through photos
Another large difference from Google Street View is the open character of the data. Students find it very important that the Mapillary data remains open: accessible to everyone without paying for (also in the future). It made students wonder about Mapillary’s business model. Maybe a topic for an interview by Skype?
Points for attention
It's great that the software has automatic image analysis. Road signs were distilled out of the photo material in two days: location and type of sign. Impressive! But there were some minor flaws. For instance, a road sign mirrored in the water was recognized as a real sign. On the map this resulted in showing a road sign in the water. This is still a task for the Mapillary team to fine tune.
But even more important is that the team works further on image recognition: that almost all of the information on the pictures can be placed in a category. From waste bins to lines on the road, from stalled bikes to waste on streets. Is the team behind Mapillary working on this?
Privacy is another important issue. Faces and license plates of cars were already blurred, but not always correctly. On some pictures the face was recognisable, while later on in the timeline it was blurred. The same with number signs. Here the software has to improve.
Also the accuracy of the lines of the track is not always correct. This is probably due to the inaccuracy of GPS. But possibly an algorithm, coupled with infrastructure on the map, is a solution for this issue? Of course without conscious abnormalities that a user meant to do.
All in all, our students and teachers enjoyed working with Mapillary. That this kind of mapping is possible on your own smart phone with such quality nowadays is a great progress. We are just on the eve of opportunities and possibilities this is going to offer. Probably faster than we will expect. Will it become standard in every car, tram and/or truck in the next years? For the Netherlands we also see plenty of opportunities as the default on bikes.
Photo mapping from all thinkable means of transportation
In spring 2017 we will organise a Mapday with a much larger group of students. Let’s see how much data we can capture then, and revisit some of the data already captured in the fall. We are looking forward to the comments from this new group of students. And hopefully we encourage others to photo map their home town in no time!
About the authors
Jan Willem van Eck works at Esri Nederland and lives in Almere.
Kees Jansen is lecturer at Aeres University of Applied Science Almere.
End notes by Mapillary
 Mapillary as a business deals with commercial use of images and data. For individual, NGO and educational purposes, the use of Mapillary will always be free.
 The Mapillary team is constantly working on detecting new categories of objects from images as well as improving the accuracy of the detections. Here is an update of our progress with semantic segmentation.
 We are also working to improve the blurring algorithm. Meanwhile, anyone can edit blurs on images manually to increase privacy as well as help train the algorithm, since the manual fixes can be used as feedback to the model.
 The accuracy of the tracks on the map indeed depends on the accuracy of GPS. At the same time we are using and developing computer vision technology to help correct the positions on the map. Read more about the accuracy of Mapillary and the ways it can be improved.