Adding field with filename when merging shapefiles with ogr2ogr

Question

I'm merging some shapefiles and I had some problems doing so inside QGIS, so i'm using ogr2ogr directly. I'm doing this (in a batch):

ogr2ogr -overwrite %destination% %n1%
ogr2ogr -update -append %destination% %n2% -nln all_new
ogr2ogr -update -append %destination% %n3% -nln all_new
ogr2ogr -update -append %destination% %n4% -nln all_new

It works fine, but now I need to have in the resulting shapefile, a field with the names of the original shapefiles I merged. Doesn't sound very difficult, but I'm not managing to do it.

Can anyone help?

Answer 1

With small scripting it would be doable. With something like following you should be able to add column to a shapefile in all shapefiles in a folder, and merge them to merged.shp file

for %f in (*.shp) do (
 ogrinfo %f -sql "ALTER TABLE %f ADD COLUMN filename character(15)"
 ogrinfo %f -sql "UPDATE TABLE %f filename = '%f'"
 ogr2ogr -update -append merged.shp %f -f "esri shapefile" -nln merge 
)

The same as a Bash script, with some changes to make it work:

for f in *.shp
do 
 base=${f%.shp}
 ogrinfo $f -sql "ALTER TABLE $base ADD COLUMN filename character(15)"
 ogrinfo $f -dialect SQLite -sql "UPDATE $base SET filename = '$base'"
 ogr2ogr -update -append merged.shp $f
done

Answer 2

I would use the -sql option, and import the shapefile in the following way:

ogr2ogr -update -append %destination% %n2% -sql 'SELECT "%n2%" as SHAPE_ORIG, field1, field2, ... FROM %n2%'

Answer 3

there are some ways for merging shapefiles.

• if you want to merge layers as a one layer, you can use MMqgis tools for merging...

---

mmqgis

• if you want to merge all shapefiles under a folder, you can use DARREN COPE simple code here.

---

mkdir merged
for %f in (*.shp) do (
if not exist merged\merged.shp (
ogr2ogr -f "esri shapefile" merged\merged.shp %f) else (
ogr2ogr -f "esri shapefile" -update -append merged\merged.shp %f -nln Merged )
)

• beside this can use GeoMerge free tool for merging lots of file but dont forget to consider your file size for workin with it.

and adding attribute to shapefile @dango directon is good. you can use layer.CreateField(field_name) for creating a new column which is populated from

import os
shapeFileName = os.path.splitext("your_shape_file_path")[0]

Answer 4

Here is how I accomplished this problem by using a Python script to daisy-chain several ogr2ogr instructions together. You could easily convert it to a batch script, basically I just concatenate together ogr2ogr instructions (cmd), then execute them by calling os.system(cmd), passing-in the ogr2ogr command I concatenated together.

The secret weapon is (as @capooti demonstrated) applying OGR_SQL to impose the filename as a constant value of the source dataset you are appending into your merge result.

In my example, the -sql flag handles this, in the code it's like this:

-sql "SELECT \'' + filename + '\' AS filename, * FROM ' + filenameNoExt + '"'

But that's confusing to read because I need to apply single quotes and double quotes in the resulting concatenation. To do that I have to escape the single quotes (i.e. ') to use them "for real". So for readability, it helps to see it without variables and escape sequences. If you pretend the filename was "roads1" for a particular iteration, the resulting concatenation would look like this in the ogr2ogr sentence:

-sql "SELECT 'roads1.shp' AS filename, * FROM roads1"

This .py script is an amalgamation of three tricks I stole from matt wilkie (an empty, clone of a shapefile), j03lar50n (adding a column to a shapefile using ogrinfo and ogr_sql), and capooti (using ogr_sql to impose a fixed column value on all records in a shapefile). So here's the full script:

---

# merge_shps.py
import os 
path = "D:/GIS/01_tutorials/ND_Roads/extracted" # path to your folder of .shp files
merge = "merge_filename" # this will be the name of your merged result
directory = os.listdir(path)
count = 0
for filename in directory:
 if ".SHP" in filename.upper() and not ".XML" in filename.upper():
 
 # On the first pass, create a clone and add the filename column.
 if count == 0:
 # Make a clone (matt wilkie)..
 cmd = 'ogr2ogr ' + path + '/' + merge + '.shp ' + path + '/' + filename + ' -where "FID < 0"'
 os.system(cmd)
 
 # Add the field (j03lar50n)..
 cmd = 'ogrinfo ' + path + '/' + merge + '.shp -sql "ALTER TABLE ' + merge + ' ADD COLUMN filename character(50)"'
 os.system(cmd)
 
 # Now populate the data (capooti)..
 print "Merging: " + str(filename)
 
 # You'll need the filename without the .shp extension for the OGR_SQL..
 filenameNoExt = filename.replace(".shp","")
 
 cmd = 'ogr2ogr -f "esri shapefile" -update -append ' + \
 path + '/' + merge + '.shp ' + \
 path + '/' + filename + \
 ' -sql "SELECT \'' + filename + '\' AS filename, * FROM ' + filenameNoExt + '"'
 
 # Uncomment this line to spit the ogr2ogr sentence to the terminal..
 #print "\n" + cmd + "\n"
 
 os.system(cmd)
 
 count += 1

Answer 5

Add column with source filename from folder of shapefiles. Requires GDAL 1.10dev, my attempt to drop .shp extension isn't working - but overall, works. - I imagine it could be added to the lines that do merging with OGR.

for f in *.shp;
do
name=${f%.shp}
/Users/you/gdal_src/bin/ogrinfo $f -sql "ALTER TABLE $name ADD COLUMN filename character(21)"
/Users/you/gdal_src/bin/ogrinfo $f -dialect SQLite -sql "UPDATE $name SET filename = '$f'"
done;

Answer 6

There is also ogrmerge:

ogrmerge.py -single -o merged.shp *.shp -src_layer_field_content {DS_BASENAME}

Answer 7

Inside QGIS you can add the Merge Shapefile plugin. There is an option to "Add column with file name"enter image description here

Answer 8

A slightly modified version of JaaKL's answer. Note that the -append foo.shp and -nln foo need to match. Also, note the use of the SQLite dialect (GDAL apparently doesn't accept the keyword 'Update', so the SQLite dialect must be used instaed), and the absence of the keyword 'TABLE' after the word 'UPDATE' (not needed or accepted by SQLite).

for %%f in (*.shp) do (
 if not "%%f" == "merge.shp" (
 ogrinfo %%f -sql "ALTER TABLE %%~nf ADD COLUMN fname character(15)"
 ogrinfo %%f -dialect SQLite -sql "UPDATE %%~nf SET fname = '%%~nf'"
 ogr2ogr -update -append merge.shp %%f -f "ESRI SHAPEFILE" -nln merge 
 )
)