Biking-Improver-Import

Biking-Improver-Import is an import of the dataset retrieved by an organized mapathon using Biking-Improver as a tool. This dataset contains new tags regarding ways and nodes of a city.

The project wish is to increment the data regarding cyclability inside of OSM.

About The Database

The database is created looking at different nodes and ways inside of OSM. If a way or node misses certain tags regarding cyclability then a question is created. Questions can be answered inside the web app by users and once this happens the answer is translated in a OSM tag. Once an answer receives 2 validations then the updated tag is ready to be sent to OSM.

In the table below you can see a portion of how the database table regarding the questions is structured:

TYPE

QUESTION

ID

SCORE

VALIDATING

ANSWERS

ICON

TAGANSWER

ANSWER

USERANSWERED

NUMBEROFVALIDATIONS

USERSWHOVALIDATED

SENT

TIME

WAY

Can I use the bicycle in this way?

22986084

1

no

yes,no

pedal_bike

bicycle

0

no

0

• TYPE: represent the type of the geometry (it can be node or way)
• QUESTION: the question that the user have to answer
• ID: the id of the geometry
• SCORE: the score gained by the user after answering
• VALIDATING: whether or not it is a validation questions (validation questions are only used in the web-app and the data is not sent to OSM)
• ANSWERS: the possible answers that the user can give
• ICON: used by the webapp, shows an icon near the question
• TAGANSWER: The tag that is linked to the answer. So in the example the data sent to OSM would be "bicycle: answer" where answer can either be yes or no
• ANSWER: the answer given by the user
• USERANSWERED: the user that answered the question. If the user logged in via OSM then the data in OSM is updated using his/her credentials, otherwise a custom account will send the data to OSM
• NUMBEROFVALIDATIONS: the number of times users validated the answer. When this value reaches 2 the data can be sent to OSM. If this data doesn't reach at least 2 then the data is not sent to OSM
• USERSWHOVALIDATED: the users that validated the answer
• SENT: whether or not the answer was sent to OSM
• TIME: a value used to check wheether or not we should send the data using the user credentials or a custom account.

There's also another table that saves the changeset and the imports that are created and sent to OSM. This table looks like this:

CHANGESETID	NEWDATAIMPORT	ID	TYPE

• CHANGESETID: the id of the changeset that was created before the import
• NEWDATAIMPORT: The new tags that have been imported to OSM. (for example: bicycle:yes)
• ID: The id of the geometry
• TYPE: The type of the geometry

Goals

The goal of the import is to increment the data regarding cyclability inside of OSM. The goal is to update only the tags without affecting the geometries.

Imports will be made during the mapathon once every hour. The imports will be made using a custom account or using the users accounts.

Schedule

The imports will be done by the end of september or beginning of november, the exact date still has to be decided

Import Data

Background

Provide links to your sources.

Data source site: https://bikingimprover.gisdev.io, the data is inside the database contained in the web-app.
Data license: https://github.com/DigitalCommonsLab/bikingimprover/blob/main/LICENSE
Type of license (if applicable): ODbl License.
Link to permission (if required): We are using OSM data, So ODbl, https://www.openstreetmap.org/copyright
ODbL Compliance verified: yes

OSM Data Files

OSM files are generated starting from the database and creating a .osm file ready to be used for a put call to the OSM Api. Only tags are updated. Since the database in stored inside the web-app the database isn't publicly available, anyway it is possible to check how the data is created from the repository on https://github.com/DigitalCommonsLab/bikingimprover/

Import Type

The imports will be done during organized events and will be done evey 40 minutes since the start of the event. The imports will be done automatically when an answer given by a user reaches 2 validations or more. If the OSM API returns any error then no changes are applied. The method used for the imports is the use of the OSM API. In particular we get the elements that we are going to update, we add the new tags, create a changeset, perform the import and close the changeset.

Data Preparation

Data Reduction & Simplification

Since the imports are only about editing tags and keys, we are updating only the tags. First we check that the tag isn’t already present in the element, if it is then we discard our new tag and then proceed to check the others.

A mapathon is organized where users have to answer simple questions about cyclability. The questions are all linked to an OSM tag attribute so that we can easily send the answer to the OSM database. The tags sent to OSM will be compatible with OSM tagging system.

Tagging Plans

A mapathon is organized where users have to answer simple questions about cyclability. The questions are all linked to an OSM tag attribute so that we can easily send the answer to the OSM database. The tags sent to OSM will be compatible with OSM tagging system.

Changeset Tags

The import will be made using an account set up for biking-improver. A tag containing the source will be added as well as a tag with the comment “add tag … “. We’re also using a tag “bot":"yes”.

Data Transformation

To convert the database data to osm format we’re using a js script. In particular we’re creating the osm file to be sent to the osm api by extracting the tags and its values. Then the tag comment and source are added.

Data Transformation Results

The reults looks like this: Changeset:

 <osm>
   <changeset>
   <tag k="source" v="Biking-Improver" />
   <tag k="bot" v="yes" />
   <tag k="comment" v="Adding tags to some OSM elements. Tags are about cyclability and were validated by users using Biking-Improver." />
   </changeset>
 </osm>

Import:

 <osm version="0.6">
   <changeset>
   <tag k="source" v="BikingImprover"/>
   <tag k="comment" v="Adding tags tagsList with values valuesList using Biking-Improver"/>
   </changeset>
   <type id="id" changeset="changesetID" version="version" ${type === 'node' ? ` lat="lat" lon="lon"` : }>
       <tag k="segregated" v="yes"/>
       <tag k="oneway" v="yes"/>
   </type>
 </osm>

Data Merge Workflow

Team Approach

This will be done by a script automatically during the organized event.

References

The import is done automatically, if there are errors then no import is done. If the OSM API returns any errors at all then the import is canceled and no edits are made.

Workflow

- Step by step instructions: Obtain the city xml from here: https://osmit-estratti.wmcloud.org/. The database is created following the instruction in the github page: https://github.com/DigitalCommonsLab/bikingimprover. The users answer questions using the web-app:https://bikingimprover.gisdev.io/. Every 40 minutes the app will check whether or not answers received 2 vaidations. The answers that received 2 validations are grouped based on the player that gave the answer. Then the web-app will send the data to OSM using the user’s token if he logged in via OSM, otherwise the data will be sent via our custom account. Before the data is sent a GET call to the OSM API regarding the way or node modified is performed (to check whether or not the node or way is still in OSM). If this get call receives any errors then the updates regarding that certain element are discarded. If there are no errors then we compare the data received by the GET call to the new data that we are going to import. We add the new tags dits done to an element are saved in the database under the table “changeset”, so that if an element has to be reverted we can easily check which data was modified and revert by removing the new tags.

See also

the discussion on the import mode took place at this the italian community forum

https://community.openstreetmap.org/t/importing-data-bikingimprover/104275