Fortaleza/PMF Buildings Import/Data Conversion

From OpenStreetMap Wiki
Jump to navigation Jump to search
Proposal Work Plan Data Conversion Import Guide Corrections Progress

This page describes how the original data was acquired and converted, putting them in the OSM tag form. For the import itself, it is not necessary to follow this page, as the converted data is already available for download, as described in the import guide.

Data download

The data can be accessed on the Fortaleza in Maps platform (in Portuguese), through the following procedure:

  • You must initially select any data in the panel located on the left side;
  • After the data is loaded on the screen, click on Downloads - Shape and;
  • Remove Shape do Mapa.shp (Map Shape) selection and select Edificações de Fortaleza.shp (Fortaleza Buildings), proceeding with the download.

If desired, in order to facilitate orientation, the roads from Fortaleza can also be downloaded, through Eixos viários de Fortaleza.shp file.

Data conversion

  • Open the downloaded file, as previous step, in QGIS.
  • If desired, also import the Roads (Eixos Viários) layer, to facilitate orientation. To add a new layer just go to the Menu - Layer - Add Layer - Add Vector Layer, or use the shortcut Ctrl + Shift + V.
  • When you finish importing data into QGIS, you must change the name ALTURA to height, such as adding a new OSM tag for buildings, as follows:
    • Right-click on the Buildings (Edificações) layer (left side menu, where all the layers are) and select Properties - Fields - Switch to editing mode (click on the top pencil) - Double click on ALTURA - change to height.
  • In the same menu, add a new field and place the tag building=yes in all features:
    • Click on Field Calculator on the top menu;
    • Select Create a new field and fill in:
      • Output field name: building
      • Output field type: Text (string)
      • Output field length: 3
      • Expression: 'yes' (with single quotes included)
    • Click on Ok and then Apply and Ok.
  • Then, after modifying all the polygons, select the desired features for export (select features by area or by a simple click, one by one).
  • After selecting the desired features, export the Shapefile with them: Menu - Layer - Save As:
    • Format: Shapefile;
    • File name: choose a name and location to save the file;
    • CRS: check if it is still in EPSG: 31984;
    • Field selection: select only the height, building and source fields;
    • Select: Save only selected features and save SHP file.

The addition of the Roads (Eixos Viários) layer has a purpose to add the streets in QGIS, for better user guidance. In addition, it is NOT recommended to export all features at once, given the high amount of data that must be processed manually in JOSM.

Data split

After converting the building data to OSM pattern, the buildings were split into block chunks, to facilitate the import and to help using a Tasking Manager. Creating the blocks out of the streets was very simple using QGIS. The data was then split programmatically, with the wonderful help from System-users-3.svgGlassman (on osm, edits, contrib, heatmap, chngset com.), in which we are extremely grateful (without his help we would not be able to create tasks like this).

Firstly, the Roads (Eixos Viários) were used to create blocks to split the data into small tasks. Blocks data are available from Fortaleza in Maps platform, but some errors in the data were identified, and the following procedure returned better results.

  • Open file Roads (Eixos Viários) on QGIS;
  • Processing toolbox (Ctrl+Alt+T);
  • Search for 'Polygonize' (vector geometry) and use the roads layer.

After creating the Polygonized Blocks, it is necessary to create an ID and giving to each feature a web address:

  • Right-click on the created layer (left side menu, where all the layers are) and select Properties - Fields - Switch to editing mode (click on the top pencil)
  • Click on Field Calculator on the top menu;
  • Select Create a new field and fill in:
  • Output field name: FID
    • Output field type: Number (integer)
    • Output field length: 6
    • Expression: $id
  • Click on Ok and then Apply and Ok.
  • Now perform the same procedure, adding the URL:
  • Right-click on the created layer (left side menu, where all the layers are) and select Properties - Fields - Switch to editing mode (click on the top pencil)
  • Click on Field Calculator on the top menu;
  • Select Create a new field and fill in:
  • Output field name: import_url
    • Output field type: Text (string)
    • Output field length: 100
    • Expression: 'https://www.mywebsite.com/osm/' || "FID" || '.osm.gz' (with quotes included, and you should change mywebsite.com/osm to desired URL)
  • Click on Ok and then Apply and Ok.

Each polygon will then have an unique ID, and their related URL address, to be used to be downloaded from Tasking Manager.

After creating them out of the roads, the polygonized blocks were used to split buildings data. This procedure were done by System-users-3.svgGlassman (on osm, edits, contrib, heatmap, chngset com.), using PostGIS/shell script, in which we are not familiar to, so we cannot describe the complete procedure here. As a result, that procedure created over 17,000 polygons, in SHP and OSM files.

Tasking Manager

Following those previous procedures, the buildings data are finally split into blocks, and are ready to be used in the Tasking Manager (TM). However, there is a large amount of tasks, not permitted to be used in the TM. Therefore, it is necessary to split the tasks into smaller groups (neighborhoods). The following procedure is performed:

  • Import poligonyzed blocks (previous step) in QGIS;
  • Import neighborhood shapefile (also available in Fortaleza in Maps platform - Limites Territoriais (Estado, Região Metropolitana, Município, Regionais e Bairros));
  • Select desired neighborhood;
  • Menu - Vector - Geoprocessing Tools - Clip;
  • Input layer: Poligonyzed Blocks;
  • Clip layer: Neighboorhood - Use only selected features;
  • Clean any visible errors;
  • Export as a SHP file (GeoJSON and KML was not accepted in TM for some reason);
  • Create a ZIP file with created files.

Finally, in Tasking Manager:

  • Create a new project;
  • Select Option 2 - Import a GeoJSON, KML, or zipped SHP file - and import the created ZIP;
  • In Step 2, choose Arbitrary Tasks;
  • In following steps, fill in desired information.
  • After creating the project, go to Edit Project - Instructions - Per Task Instructions and fill in (with all star keys * included):

**Download the neighboorhood buildings [here](http://localhost:8111/import?new_layer=true&url={import_url}) to load it into JOSM (it must be already opened on your computer).**

Then, all tasks are created, and each task has its own file assigned. Happy editing!

Acknowledgments

We are very grateful to Clifford Snow (Glassman), for the patience and kindness in helping us creating the tasks. This would not be possible at all without his help.

Also, we are grateful to the Portuguese community for letting us use their Tasking Manager. Portugal OSM is a very strong, organized, and committed community, with continous efforts in improving OSM.