Data on demand with the Mapillary Python SDK
Access to data
With the release of Mapillary API v4, working with Mapillary data at scale comes with a greater focus on tiles and a speedy graph API. A great medium for interacting with the new API is Python, where additional libraries can facilitate downloading data across large regions or reusing small code snippets in various loops.
Some example code has become popular for many users, including how to use a bounding box to download images, sequences, JPGs, or map features. Rather than reusing and adapting these scripts to various developer needs, we decided to think long term and build the Mapillary Python SDK.
This open source library was made possible by the planning and engineering efforts of Omar Muhammed Ali and Saif Ul Islam, who joined up with the Mapillary team at Meta through the Major League Hacking Fellowship. The MLH Fellowship is a remote internship program, which connected these two talented and emerging engineers with us to collaborate for 12 weeks building skills with a real-world project.
The chosen project started out with a simple goal: build a Python library that allows users to query, download, and analyze Mapillary data. Working as a team, we developed a timeline, and scoped out use cases for this library. Many of these use cases centered around the specific user needs like downloading data with particular filter requirements, or looking up image detections given a known image ID.
Features in brief
To briefly restate what we are looking at, the Mapillary Python SDK is a Python package that enables the user to query and download spatial data from Mapillary with simple commands.
Using Mapillary's Python SDK you can retrieve data available in the API:
- Images
- Sequences
- Map Features
- Detections
Mapillary's Python SDK provides interfaces that remove and abstract away many underlying technical data processing techniques to output an easy to understand and use GeoJSON format. Ideally used in a Python notebook, this SDK includes such functions as:
- `get_image_close_to` - retrieve all images near a specified location
- `get_image_looking_at` - retrieve nearby images viewing the specified location
- `get_detections_with_image_id` - get all the detection segmentations within an image `get_detections_with_map_feature_id` - get all detections composing a map feature
- `image_thumbnail` - get the thumbnail URL of an image
- `images_in_bbox` - retrieve all images in a bounding box
- `map_features_in_geojson` - get all map features within a GeoJSON's boundaries
- `save_locally` - save data to a local file
There are many other functions that enable easy access to Mapillary data, visible in the full documentation.
Building the library
The project started off from scratch where the main aim was to develop a software development kit or SDK that was compatible with Mapillary’s new API v4. It was critical to think about how an open source project should be built, maintained, and applied.
Important steps included developing source code, pushing to PyPi as a package, documentation, testing, automation, contributor friendliness, well written code changes and edits.
There was also an emphasis on design and practical implementation of the project. This includes a focus on code readability, maintainability, modularity, and the right sense of coupling and cohesion. The end result is a legible open source library, easily explored by developers looking to contribute to the source code or borrow functionality.
Using the SDK
There are a broad number of capabilities built into the Mapillary Python SDK. Getting started is as easy as running “pip install mapillary”, then running the following lines of code:
import mapillary.interface as mlyFrom here, you need to set your API access token. If you don’t have one, you can generate it in your developer dashboard. Save the token as a variable then set it:
my_token = ‘MLY|XXXX|0000000000’ mly.set_access_token(my_token) With the access token set, you can now make requests to the API. A popular request, for example, is to get all images in a bounding box:
bbox = { ‘west’: 8.944175, ‘south’: 46.001107, ‘east’: 8.954383, ‘north’: 46.004565 } data = mly.images_in_bbox(bbox) print(data) A great time saver is also to reference your own GeoJSON, and request images inside of it:
import json with open('fiji.geojson') as f: fiji = json.load(f) data = mly.images_in_geojson(fiji) print(data) Getting the URL for an image, if you know the image ID already, is also one simple command:
img_url = mly.image_thumbnail( image_id='2947453935512309', resolution=1024 ) Using the same bounding box as above, you can also request all point features and traffic signs within it:
trafficsign_data = mly.traffic_signs_in_bbox(bbox) point_data = mly.map_feature_points_in_bbox(bbox) There are dozens of other examples you can find in the `mapillary.interface` section of the documentation. A complete list of interfaces abstracted from the API can be accessed through the module `mapillary.interfaces`.
The SDK provides access to internal logic directly, and includes `config`, `controllers`, `utils`, `models`. Each of these are designed to modularize the complex aspects of the code base, in order to provide custom usage for developers looking to go beyond just data requests.
For more details, check out the codebase directly on Github.
There is also a full demo available in a shared Python notebook.
Meeting the engineers
As we wrapped up the project, I asked Saif and Omar to recap some of their experience working on the project, to share with the Mapillary community. I posed a few broad questions to get them started sharing their perspectives:
What challenges did you face working on the project?
Saif: For me, the main starting challenges were:
- How to structure and organize the project
- Asking as many questions as I can because of a new domain
- Understanding how to work professionally in a remote team because this is my first professional experience
So my obstacles were both in communication skills and technical skills. Overcoming these gave me a lot of exposure over how teams, especially remote teams, operate.
Omar: This was my first experience working on a major open-source project, let alone implementing an SDK from scratch which facilitates an API that deals with geospatial information (a whole new domain of knowledge for me). From the technical side, I wasn't familiar with some of the technologies that we initially used. Overall there was a lot of new material to learn. Being a novice in most of what we were doing drove my interest in the project, enabling me to learn and apply something new every day.
What did you learn working on this project?
Saif: Because it was a completely new domain, I learned about geospatial data and geoinformatics, as well as new Python libraries and processes like Structure From Motion (OpenSfM). I improved on my software engineering ideas and my thinking about how to design an SDK. I also became accustomed to working remotely professionally. Finally, I met some fantastic people on Mapillary's team and learned a lot from them about their work.
Omar: Most of the things I used and learned while working on the SDK were new to me, starting from learning how to actually build an SDK, getting acquainted with the different patterns and processes we used, learning about geospatial data and its different representations such as GeoJSON, and many, many more new concepts and technologies related to the project and its use cases.
What reflections do you have in general?
Saif: Coming into the project, I was kind of scared because Facebook is a gigantic company which made me feel I should deliver a lot of results. Over time, these fears stopped because I was given room to find my own pace and ask questions. I was able to share what blocked me, discuss the direction of the project, and share new ideas. I felt closely involved in the project and invested in how it would work out.
I am eager to see how the project works out down the road, and how I can still help maintain it and move it forward. After spending a lot of time on a project like this, you really want to see it succeed. Just this week, someone not from Mapillary, not from FB, wanted to contribute to the project. It made me extremely happy to see this, as well as seeing new traffic coming into the project repository. It definitely feels like I've made something very valuable for others!
Omar: Getting introduced to the Mapillary product gave me a high-level overview of how huge, global mapping systems are built, which was always an intriguing concept to me. I'm really thankful that I got to be part of its development even for this brief period, and hopefully can contribute to future development as well. Geospatial data analysis is a vast, new, and very interesting domain of knowledge to me. I'm looking forward to learning more about it and its applications. Of course, working on a Facebook project is an extremely interesting experience for me both professionally and personally. It comes without question that there was always something new to learn from more experienced and knowledgeable engineers.
What advice do you have for future contributors to this project?
Saif: Always, always feel free to ask as many questions as possible. I had some moments writing the code where I said to myself: “Is this good enough? Can it be better?” The answer always is, yes, everything can be improved.
Take it slow and have fun when you're making something! That's the spirit of open source - making something you believe has value, all the while having fun when you're working on the project in your own way. It doesn't have to be writing code all day long. It can be documentation, giving feature request/feedback/roadmap ideas, writing tutorials, or sharing stories about how the project is useful for you and others.
Overall, working with Saif and Omar was a great experience on our side too--they learned fast but also brought a wealth of technical knowledge, were strong communicators, and creative problem solvers. Their hard work on the foundations of this project will help set it up for a successful future, and serves as a proud example of the future work they will contribute to the open source community as well. A huge thanks to these two for their work, and bravo to the Major League Hacking Fellowship for allowing this collaboration to happen!
Contributing
Following the advice of Omar and Saif above, there are many ways to get involved contributing to and maintaining the Github repository for the Mapillary Python SDK. As an open source project, this is meant to serve the needs of the developer community and be molded in the future to keep up with growing and changing use cases. Make sure to read the succinct contribution guidelines as you look to get involved.
If you have a feature request, find a bug, or have another idea for contribution and improvement, do not hesitate to open an issue. In the long term, we hope to see new projects and analyses emerge based on this project, and invite you to share how you use the Mapillary Python SDK to solve challenges and find new insights by tagging Mapillary on social media or in the Mapillary forum.
As always, thanks for building with us and helping create a better map of the world!
/Chris