bulk_upload.py

bulk_upload.py is a Python script for performing bulk imports. It uses ElementTree for XML parsing which allows for a subset of the abilities of Bulk_upload.pl but is compatible with API v0.6.

Imports and automated edits should only be carried out by those with experience and understanding of the way the OpenStreetMap community creates maps, and only with careful planning and consultation with the local community.
See Import/Guidelines and Automated Edits code of conduct for more information. Imports/automated edits which do not follow these guidelines might be reverted!

It is currently available in SVN (click to view online). There is also an untested bulk_upload.php script included.

To check the scripts out of subversion, use

svn co https://svn.openstreetmap.org/applications/utils/import/bulk_upload_06/

Usage

$ python bulk_upload.py --help
Usage: bulk_upload.py -i input.osm -u user -p password

Options:

 -h, --help            show this help message and exit
 -i INFILE, --input=INFILE
                       read data from input.osm
 -u USER, --user=USER  username
 -p PASSWORD, --password=PASSWORD
                       password
 -c COMMENT, --comment=COMMENT
                       ChangeSet Comment

Installation on Ubuntu 9.04

aptitude install python2.5 python-httplib2 python-celementtree python-graph

Execution on Windows

Current version may fail running in Windows because os.rename() does not behave as in UNIX-like environments. The symptom is that 2000 nodes get loaded and the program aborts:

Created changeset: ###### Uploading to changeset ###### Uploading to changeset ###### Traceback (most recent call last):

 File "bulk_upload.py", line 333, in <module>
   importProcessor.parse(options.infile)
 File "bulk_upload.py", line 103, in parse
   self.addToChangeset(elem)
 File "bulk_upload.py", line 120, in addToChangeset
   self.currentChangeset.addChange(action, elem)
 File "bulk_upload.py", line 209, in addChange
   self.currentDiffSet.addChange(action,item)
 File "bulk_upload.py", line 249, in addChange
   self.upload()
 File "bulk_upload.py", line 276, in upload
   self.idMap.save()
 File "bulk_upload.py", line 150, in save
   os.rename(self.filename+".tmp", self.filename)

WindowsError: [Error 183] Cannot create a file when that file already exists

My quick hack is to add two lines around this step:

        os.rename(self.filename, self.filename+".old") # ensure prior version is renamed to something else
        os.rename(self.filename+".tmp", self.filename)
        os.remove(self.filename+".old") # once actual rename performed delete old file

Obviously a test on the value returned by os.name would make the behavior more graceful.

Alternative version(s)

See also

• Bulk upload sax.py - for huge imports where the XML file is too big for the default script (e.g. the Corine Land Cover France with 1.4 GB single file), a special version has been created in July 2009 to replace the DOM parsing loading the whole file in memory by the SAX parsing.