At Mapbox, we map as well as inspect the data that is coming into OpenStreetMap for quality. This is the definitive guide that we follow - from detecting potentially harmful changes to inspecting them and eventually fixing the map.
What is a problematic edit?
Every change to OpenStreetMap is contained in a changeset. They consist of either additions (new feature), modifications (change existing tags, feature location, or location of a referred feature) or deletions by a mapper.
When working with data in OpenStreetMap, you sometimes come across places where another mapper might have added bad features or tags, changed or moved or deleted a large set of data or an important set of tags. Changes that spoil the quality of OpenStreetMap data are considered problematic edits.
There are several kinds of potentially problematic changes. For example:
- Use of proprietary data (like Google Maps) to add data in OpenStreetMap.
- Modification to significant tags like
- Undocumented imports.
- Modifications with outdated imagery.
- Unintentional dragging of nodes in way that results in geometry changes.
Creation and deletion of features, modification to tags and modification to a node’s location, result in a
version increment. Every change by every user is retained as a separate version of the object in the OpenStreetMap database. Changes to nodes within a way, or to members of a relation, result in displayed changes to the way/relation but do not increment
Identifying problematic edits.
- OSMCha: Changeset level notification, giving an overview of how many objects were affected in a changeset. A set of suspects words automatically flag some suspicious changes. Custom filters can be used with changeset comment, source, and other combinations to look for suspicious edits. Read more about using OSMCha to identify and inspect OpenStreetMap data.
- Map Feedback from Mapbox users: Sometimes we receive notice through Map feedback where the Mapbox users report the changes.
- OSM communication channels.
After identifying potentially problematic data on the map, you will want to determine few things to start:
- the user responsible
- the time they made the edit
- what made the changes - the changeset id
- the features edited
This information will help us investigate the change and identify the cause and type of change. Let’s follow an example of misaligned roads in New York City.
Inspecting the change
- Open the area in
- Select the problematic feature, and check the modified object’s history using
- The History window displays how that selected object was changed over time by different mappers. The display will vary based on the type of object (node/way/relation) selected. The left panel shows the object’s versions, modification dates, and responsible users.
Tagstab of History window has the history of tags changed between versions.
Nodestab of History window has the history of addition/deletion/modification of nodes associated with a way’s geometry between versions.
Coordinatestab of history window has the history of the object if its moved for a significant distance.
JOSM History window
In this example, someone moved a node a significant distance. From the JOSM history window, you can see the change was made by user
osm_user_name, in changeset
Understand the mapper
The mapper’s OSM user profile and HDYC profile gives an overview of the user’s activity and reputation. New mappers are more likely to make unintentional changes; though sometimes a new user account is used for intentional edits that break the map. Experienced mappers might have more detailed, trustworthy knowledge of an area, or sometimes a history of bad edits.
Click on the OSM username in the JOSM History window, and load the OSM user page where you can view the user’s
Edits history, any
Active Blocks, any other details on the user, and
You cal also gauge a mapper’s experience by looking at the number of project days, type of mapper and other points in HDYC.
Investigate the changeset
Click on the changeset ID in the JOSM History window to open the OpenStreetMap changeset page, and load details on how many objects were added/modified/deleted. You’ll also see the changeset comment, which may contain more self-reported details from the user, as well as information on the editor and sources used.
The OpenStreetMap changeset pages list the objects touched by the user in that particular changeset, but it won’t highlight the specific details of changed objects.
This is where Changeset Map comes in handy. Changeset Map visualizes all specific features changes made to OpenStreetMap in a changeset.
Submit the changeset id to changeset-map to see the changes affected in that changeset.
Changeset Map shows the features added/modified/deleted or touched during the changeset. Identify all the problematic changed objects by exploring the edits.
At this point, you know who, when, and which changeset are associated with the problematic change.
Most bad edits are accidental, and the general strategy to respond to such changes is education. Work to share knowledge with the mapper about the mistake and how they can avoid the mistake in the future. Communicate feedback through public changeset comments, visible to all, or sometimes direct messages if more direct contact is helpful. This helps build our community and improve the quality of future contributions to the map.
In cases where it is determined that the intent of the mapper was to damage the map, an urgent response may be required by reverting the change and reporting the issue to an OSM administrator or Data Working Group.
- Leave a comment on the changeset with a description of what was wrong and how it can be fixed. Link to an OpenStreetMap wiki or link to help material that is relevant.
- If the user does not respond or fix non-urgent issues within a week, alert a nearby active mapper of the issue.
- Also look for help on IRC or the local forums or mailing list of the area.
- For specific malicious changes, consider fixing the issue (and reporting it immediately).
- For a changeset that is wholly inaccurate, revert it. Send a message to the mapper responsible and ask for a description of what happened.
- For simple issues, communicate with the mapper to discuss on how they can fix it.
Revert using the JOSM Plugin
The JOSM Reverter Plugin is a convenient way to revert changes in OpenStreetMap. After every revert, make sure you upload the changes by sharing a changeset comment on why you reverted the changeset.
Reverting changesets can be tricky, and you can unintentionally break things yourself. Attempt to revert a changeset if you are 100% sure that the changeset is bad and if something goes wrong – you should be able to fix it.
Using the revert plugin
- For urgent issues that require an experienced response, contact the Data Working Group.
How to assess urgency to respond
The level of response depends on how severe the change is to OpenStreetMap. Urgency is somewhat subjective, but here is some guidance.
- Changes to significant features like the following should trigger an immediate repair.
- Major boundaries
- Highway connectivity
- Highway hierarchy
- Highways misalignment (like our NYC example)
- Broken multipolygons (water/border) which effect admin boundaries
- Changes to important but less critical features should start with communication, and repair later only if the user does not respond.
- Land use features like forests
- Water bodies
- If the mapper is using the proprietary data sources like Google, Bing Maps, Here Maps or other sources which are not open licensed then comment on the user changeset and mention Copyright guidance for OpenStreetMap. Encourage them to revert the changes and follow local knowledge, or stick to Bing and Mapbox Satellite Imagery.
Here are a few common scenarios of bad edits in OpenStreetMap.
Mistakes by inexperienced mappers
New users are always welcome to edit OpenStreetMap! During their learning process sometimes they may mistakingly attempt to modify/delete major land features like boundaries, important place tags, highways, and relations which can cause faulty data. In cases like this it’s good to educate the users by directing them to proper guides, and instructions they need to follow while mapping and encourage them to correct their mistakes. Revert immediately if it’s a serious error.
Humanitarian OpenStreetMap Team projects
Many new mappers join OpenStreetMap during HOT Activation. HOT tasks have custom imagery; the changeset review should be done using them. If data added does not match either imagery or the custom imagery, or added data is not of good quality, comment on user changeset to help improve their edits and mapping skills.
Intentional bad edits
There are few users who intentionally add bad data to OpenStreetMap which should be reverted immediately. One such example is shown below. Here the user intentionally added family names to towns, buildings, and footpaths.
You can import open datasets into OpenStreetMap. Imports should follow the Import Guidelines and add proper documentation to the Imports catalogue, and use a dedicated import accounts. If you see any undocumented imports, comment on the changeset to request documentation details.
Automated edits are changes made to OpenStreetMap with less direct human involvement which involves bots, algorithmic imports, etc.
Finding suspicious map edits using OSMCha
OSM Changeset Analyzer or OSMCha is a tool to filter and analyze changesets on OpenStreetMap using the changeset metadata. The tool has a live listing of every changeset on the map, flags potentially suspicious edits and has various filters to find changesets that match certain criteria.
Latest changesets listed in OSMCha
The changeset is shown with id, date and time of creation, OSM user, creations in green, modification in yellow, deletions in red, a keyword describing if the changeset fits any set behaviors along with a cross mark to signify that the changeset is unreviewed.
The filters available in OSMCha allow for a various combination of searches to help find problematic edits.
OSMCha filter dialog
OSMCha automatically tags changeset with certain parameters like
Mass creations or
Filter based on automatic flagging
Examining the quality of the changeset
OSMCha uses Changeset map to visually represent the features that were added, modified and deleted along with tag changes. This helps check the quality of the edits for obvious mistakes.
A changeset showing an object that incorrectly made into a circle
Workflow to find suspicious changes
Suspicious editing behavior
Inexperienced users, deletions, and imports:
- Edits from Maps.me editor
- Mass deletions
- Mass deletions + iD editor
- Mass modifications + iD editor
- Mass creations + iD editor/ Possible import
Strategy for filters:
- iD editor is targeted towards simple editing from new users, and any advanced editing happens in JOSM. So, if a mass deletion takes place using iD editor, there is a probability that the user could be inexperienced and the deletions unreasonable. Example of a harmful changeset.
- Maps.me users tend to be new to OSM. It is suggested to look for changesets that add an unreasonable amount of similar POIs in one area. This could be due to a lack experience of the user in mapping coupled with a lack of amenities available through the app preset. Example of a harmful changeset.
- Edits from Maps.me editor are automatically uploaded through the app. The app generates an automatic changeset comment based on the kind of amenities that were added. Filtering Maps.me edits with most creations first as sort order helped us in finding most of the bad edits from Maps.me users so far.
- This is similar to the first assumption. Mass modifications using iD editor could be a suspicious changeset. Mass modification could be two scenarios:
- When the user modifies tags of a long way (coastline, administrative boundaries, long motorway, etc.) and polygons, these edits are registered to the way/polygon and all the nodes in the feature object.
- When the user changes the geometry of an object on OSM, even though only one node is added the whole way/polygon is updated to be modified. Example of a harmful changeset.
- Mass creations can also be coupled with the iD editor in the filters. The idea is to review changesets which have most creations and see if all the editions fall under the OSM editing standards and conventions. Example of a harmful changeset.
- Sometimes, we were able to detect suspicious changesets by going through reason filters such as
edited an object with a significant place tag,
edited an object with a significant tag,
edited a wikipedia landmark. Example of a harmful changeset.
- Clever acts of vandalism are hard to detect as the changesets may not be big. We can verify these modifications to
nametags, deletion of tags from small changesets that are iD editor contributions or through random search in OSMCha. Examples harmful changesets: 1, 2.
Suspicious changeset keywords
There are certain keywords we have stored to quickly look through changesets that have these words in the changeset comments. For example “Google” and any profane language.
Finding vandalism detected by the community and reverted by Data Working Group
The community is very active in finding and commenting on suspicious changesets. If the edits in the changesets are destructive to the map then the community will inform Data Working Group and Data Working Group will revert these changesets with a changeset comment.
- Using Keywords
- Using keywords such as
fixin changeset comment filter, we can find most of the changesets community has taken action on. This list in OSMCha includes fixes, reverts Data Working Group as well.
- Using keywords such as
- Using OSM - comments
- OSM-comments API allows querying changesets with specific parameters
- Link to see changeset comments from all users in OSM - URL for web view
Currently, there is no direct way of using OSM-Comments-API to see all the changesets on which Data Working Group members have commented. Issue #44
Finding reverts by Data Working Group
Finding Data Working Group reverts using OSMCha
- OSMCha allows us to query changesets from specific users.
- We can query all changesets from revert accounts of Data Working Group members such