Passing GDAL commands in Python using subprocess: errors

Question

Linux Ubuntu 22.04, latest GDAL installation (3.4.1).

Note: skip to 'UPDATE' section further down in this question

gdalinfo don’t recognise Ubuntu file paths or shapefiles:

gdalinfo "/home/username/docs/test.shp"

Leads to this error when run from the default terminal:

ERROR 4: /home/username/docs/test.shp: No such file or directory
gdalinfo failed - unable to open '/home/username/docs/test.shp'.

Strangely a different error appears when Terminal is launched from the directory of the shapefile:

ERROR 4: `/home/username/docs/test.shp' not recognized as a supported file format.
gdalinfo failed - unable to open '/home/username/docs/test.shp'.

Python has no issues recognising the folder paths. For example this returns all files in the folder:

import os
dir = r’/home/username/docs/‘
for file in os.listdir(dir):
 print(file)

returns:

test.shp
test.prj
test.dbf
test.shx

So it appears to be an issue with GDAL. Any ideas?

UPDATE: as pointed out below, I need to use ogrinfo when working with vector data, not gdalinfo which is for raster data. However, I still have an issue referencing an absolute path to a shapefile, only when using gdal in Python (using subprocess call function). For example, using ogr2ogr in Terminal, this works:

ogr2ogr -f "ESRI Shapefile" /home/data/new.shp /home/data/old.shp

but in python this doesn't work:

from subprocess import call
command = r'ogr2ogr -f "ESRI Shapefile" /home/data/new.shp /home/data/old.shp'
call(command)

returns this error:

FileNotFoundError: [Errno 2] No such file or directory: 'oogr2ogr -f "ESRI Shapefile" /home/data/new.shp /home/data/old.shp'

(I'm using dummy paths for this example) From the error it looks like the entire gdal command is being used as a directory??

• user30184: When you get the name/path problem resolved you may notice that gdalinfo is made for raster files. Use orginfo gdal.org/programs/ogrinfo.html for vector data.

Answer 1

If its a relative path, you need a period in front of the file path, such as

gdalinfo './Path/to/my/file.shp'

or for the whole path you would specify with tilde such as

gdalinfo '~/home//Path/to/my/file.shp'

so for you if its a relative path it should be

gdalinfo './home/username/docs/test.shp'

Your bigger problem is that gdalinfo is for raster data

So you should actually be using ogrinfo instead, such as:

ogrinfo -so ./home/username/docs/test.shp

• user2856: /home/etc... is an absolute path so the 1st part of your answer is not really relevant to the question, but you are correct about gdalinfo and ogrinfo
• Theo F: I prefer to work only with absolute paths (in case I move my script files around). Thanks for reminding me about ogrinfo (it's been a while away from GDAL for me). Running terminal from the file directory, this works: ogrinfo test.shp, but I'm struggling to work out the correct syntax for absolute path...
• user2856: @TheoF ogrinfo /home/username/docs/test.shp
• Theo F: @user2856 so I'm running a python script which calls subprocess to run the ogr2ogr command. The python script sits in /home/username/github/repo/script.py . The script references the shapefile in /home/username/docs/test.shp'. I can't figure out how to reference this location as an absolute path in the python script. Even using your suggestion throws FileNotFoundError...`

Answer 2

It turns out the call method of subprocess doesn't work as well on Linux machines vs Windows. This script has worked for me before (on a Windows setup, with python 3.8):

from subprocess import call
command = r'ogr2ogr -f "ESRI Shapefile" C:/data/new.shp C:/data/old.shp'
call(command)

I've now found that even when swapping the file paths for Linux friendly versions, call would treat the entire command as one long directory when I run this on an Ubuntu machine. My solution (from mightypile's answer here) is to use subprocess.run on a list instead:

import subprocess
subprocess.run(["ogr2ogr", "-f", "ESRI Shapefile", f"/home/data/new.shp", f"/home/data/old.shp"])

Notice how I didn't have to wrap "ESRI Shapefile" in additional single quotes. Also note the absolute paths.

In summary, I'm not sure if something has changed in the subprocess library since I last played with using python to pass gdal commands, or perhaps I had to try different approach on a Linux machine vs my previous Windows machines.

• user2856: Look at gdal.VectorTranslate
• Theo F: @user2856 is that one of their python methods? Thanks
• user2856: Yes - gdal.org/python/…