Mapillary’s computer vision technology automatically detects objects depicted in images, such as buildings, cars, pedestrians, bicycle racks, and much more. This is based on a method called semantic segmentation—an AI algorithm is trained to detect and assign a category label to every pixel in the image. Since each image has geocoordinates, it is possible to also visualize those object labels on the map. This will allow anyone to easily get a visual reference to different kinds of objects.
The visualizations at this stage reflect positions of the images where certain objects are visible. This is not the same as map features, which is one step further—triangulating the location of an object that has been detected in several images, so we can determine the location of that object in the real world and place it on the map as a map feature.
In this article, you can read about how to browse object labels, and view the list of all 65 object classes that are available for licencing.
Browsing object labels
To turn on object labels, click on the "Map data" icon at the top of the Mapillary web app and flip the switch for "Object labels". You will see the map change to give you a better visualization of the labels. You can use the same switch to return to the normal map view.
You will see some example object classes to help you start exploring labels. Click on e.g. cars, crosswalks, or trash cans to filter for images where these objects can be seen. Alternatively, you can use the search box to search for whatever objects interest you. As you type, you will see a list appear that helps you match your search to the available classes. To exit the list, press Esc on your keyboard.
The currently applied filters appear above the search box and you can remove any of them by clicking on the cross on the respective label. Note that you will be filtering for images that contain all the object classes you have selected, not just any one of them (in terms of search operators, this works as “AND” not “OR”). To clear all the applied search filters at the same time, flip off the switch for object labels—when you turn it on again, your previous searches will be cleared.
You can close the label browsing window at any time by clicking the cross at its top right corner or the “Map data” button. Your applied searches will remain active and you can access the search box any time by just clicking on the “Map data” menu again.
As you apply object class filters, you might not see anything on the map at first if you are at a low zoom level (less than level 10). To start exploring, zoom into an area and you will see yellow dots appear on the map, representing images where your filtered classes can be seen.
Click on a dot and you will see the image open, overlaid with a mask that represents the semantic segmentation, i.e. all the different object classes that have been detected in the image. The classes that you have filtered for are displayed in a highlighted color (you can check for the difference by turning your filter off and on again when looking at the image). As you hover your mouse over the segments, you'll see the label of the object class that has been detected.
Detections filtered for cars, bicycles, and cyclists
Detections turned on but not filtered for any particular object classes
Object labels are available for licencing as map data. Below is the list of the 65 classes of objects that are provided.
|Lane marking (dashed)
Lane marking (solid)
Lane marking (other)
Lane marking (stop line)
Lane marking (text)