UPDATE: We'd like to thank all who participated in the 2nd Global Verification Challenge. Your contributions were impressive and consequential. Gerhard outlines how these verifications have improved precision, recall, and specificity in his blog post.

As a thank you for their amazing work so far (over 4,400 verifications between them), the current top three competitors will be receiving gifts from us shortly. Remember, each verification equals one wildcard raffle ticket. We have had 500,000 clicks totaling 200,000 verifications so far, which means there is plenty of room to add up those chances to win. Why not try out the utility poles and parking meter tasks?

Earlier this year, Mapillary launched a competition that encouraged the community to help us determine whether objects we had detected automatically had been identified correctly. Knowing instances where we have incorrectly identified an object lets us remove those false detections from our database and further train the algorithms by telling them which detections they got wrong. During the challenge, over 250,000 verifications were completed, something which has been enormously beneficial for our computer vision capabilities, improving the quality of the map data we provide.

Due to the success of this last challenge, we are now launching a much larger verification challenge with an ambitious goal of 1,000,000 verifications across 40 object classes. You can participate by following this link and choosing a task.

Start verifying here

What is a verification?

One verification is where two users provide the same positive or negative indication of whether we have identified an object correctly in an image. In other words, two people must agree that an object has been correctly (or incorrectly) identified for it to count as one verification.

The scoring system is outline in more detailed in our help pages.

What are the object classes?

Object classes represent different types of objects that we are seeking to identify in each image. For each class, we are interested in the function or nature of the object rather than the way it appears.

For example, a fire hydrant in the Faroe Islands looks very different from a fire hydrant in Germany. They are serving the same function though, so for the purpose of verification projects we want to train the algorithms to pick up both types of fire hydrants. The same is true of other object classes such as bike racks, manholes, and trash cans.

Fire hydrant in Tórshavn, Faroe Islands Fire hydrant in Heidelberg, Germany

Head over to the project page to see the full list of the classes you can verify.
To have an easier time deciding over each detection, take a look at this forum post with the descriptions of each class so you'll know what to look for.

Why participate?

Participating in the Global Verification challenge helps the community and Mapillary in two ways.

1. It helps to remove false detections from the database. This means that detections that we have incorrectly identified will not be used for generating map data, which improves the quality of the dataset when you’re working with it later. For example, if you are adding street lights to the map, you can use a verification project to make sure your dataset is human verified.
2. It helps us train algorithms with a global perspective. A bike rack in Uganda and a bike rack in Mexico might look slightly different, but by participating in the global verification project you can help the algorithm recognize the object in both instances. The changes come into effect when we retrain the algorithms on the new verification data. This takes place periodically throughout the year.

Instructions on participating

When you select a task, you will see images in which Mapillary's algorithms have identified a particular type of object. Your task is to approve or reject this detection after assessing whether what's contained in the blue bounding box is the object that we're looking for.

For detections of certain classes, you will need to see the context of the objects. In this case, you can simply click or tap the image crops to zoom into the full image. Please take the time to open the full image when you are unsure about the detection. For instance, to determine whether the detection on the left is a bird or not is not straightforward from the crop. With the context provided by the full image, one can be confident that the object in question is a bird. If after seeing the full image you are still not sure, please click the "Unsure" button instead of taking a guess.

Full instructions on how to verify detected objects can be found here.

The dates

This campaign will run until October 26th 2019 at 23:59 UTC.

The prizes

Five prizes are up for grabs in this competition. Three of the prizes are to be awarded to the three participants with the highest verification numbers at the end of the challenge. Two of the prizes will be awarded as a wildcard.

The prizes will be distributed when we reach our overall goal of 1,000,000 verifications across the 34 categories. The first and second place winners are welcome to swap out their prize for one of the others if they prefer.

1st prize: Blackvue DR900S - 1CH with 128GB SD card

2nd prize: GoPro Hero 7 Black

3rd prize: this prize is a ticket to an upcoming State of the Map of your choice. The prize only covers the cost of the ticket itself, so you will need to work out your own travel arrangements. A $100 USD donation will be made to the OpenStreetMap Foundation in the event you can't or don’t want to attend a State of the Map.

Wildcard prizes: every verification you make counts as one raffle ticket towards:

• a Mapillary swag pack (t-shirt, bag, selfie stick, pen, stickers, bike mount, and car mount)
• a Blackvue DR900S - 1CH

The wildcard prizes will be generated at the end of the challenge.

Fair play

In the last challenge there were some isolated instances of people collaborating with others to boost their score in ways that could damage the integrity of the data. This time around we'll be looking out for suspicious verification behavior. You are also welcome to let us know about any suspicious activity you observe. Incorrectly validating images to boost your score reduces our ability to identify images correctly, so focus on quality over quantity. Anyone found cheating will become ineligible for prizes.

Thanks for taking an interest in Global Verifications V2. Let’s work together to reach 1,000,000 verifications and better map data globally.

/Ed, Strategic Partnerships Manager

Start verifying →

* The five additional wildcard prizes will be Mapillary branded hoodies. Any customs fees must be paid by the winners.