Python subprocess communication

Question

I'm trying to run a python script that will open a command prompt(OSGeo4W.bat is a command prompt line). I can get it to open but now I would like to send the command prompt commands.

import subprocess
myProcess = subprocess.Popen(['C:\OSGeo4W64\OSGeo4W.bat'],shell = False) #opens command prompt
myProcess.communicate('gdal2tiles -p raster -z 0-1 new.jpg abc') 
myProcess.wait()
print("my process has terminated")

I've also tried

subprocess.check_call('gdal2tiles -p raster -z 0-1 new.jpg abc', shell=False)

I keep getting errors that say "WindowsError: [Error 2] The system cannot find the file specified"

although, if I were to keep the command prompt that it opens and type in " 'gdal2tiles -p raster -z 0-1 new.jpg abc' " then it will work just as I wanted. Help would be great, thanks!

• Charles Duffy: Windows makes it... interesting. In general, ignoring the platform, you'd want to be sure to use stdin=subprocess.PIPE if you want to be able to feed stdin to your process.
• Charles Duffy: That said -- if you don't actually need OSGeo4W.bat, have you tried working with subprocess.Popen(['gdal2tiles', '-p', 'raster', '-z', '0-1', 'new.jpg', 'abc'], shell=False)? If all the batch file is doing is setting up your PATH, you might include that in the script as an argument, or fully-qualify the executable name (r'C:\OSGeo4W64\bin\gdal2tiles', or such).
• jfs: @CharlesDuffy: command string is a native interface on Windows. There is no point to convert it to a list here.

Answer 1

Try:

check_call('gdal2tiles -p raster -z 0-1 new.jpg abc', shell=True)

shell=True changes how the executable is searched on Windows.

Or if gdal2tiles works only in the environment created by OSGeo4W.bat:

shell = Popen(r'C:\OSGeo4W64\OSGeo4W.bat', stdin=subprocess.PIPE)
shell.communicate('gdal2tiles -p raster -z 0-1 new.jpg abc')
# you don't need shell.wait() here

Notice: r"" literal. It is necessary to avoid escaping the backslashes in the path.

• BFlint: I would like to use to use the env made by OSgeo4W.bat, I tried your second segment of code, it opens command line quick and closes and doesn't create an output folder as it should. I then tried your first segment of code and it didnt work for this call subprocess.check_call('gdal2tiles -p raster -z 0-1 new.jpg abc', shell=True) but instead of running gdal I ran a simple python script and that had worked. When I tried the gdal2tiles I got this error Command 'gdal2tiles -p raster -z 0-1 new.jpg abc' returned non-zero exit status 1
• jfs: @BFlint: Run python scripts in console (command-line). Edit your question and provide the full traceback for the code that fails. What happens if you write both commands OSGeo4W.bat and gdal2tiles into a batch file and run it instead?
• BFlint: Actually, I was thinking...I'm starting a subprocess with OGGeo4W.bat, what that batch file does it start a command line with a preloaded path. So when I'm telling it to communicate, is it sending my commands to the command line, or the batch file that opens the command line(which I would assume isn't doing anything)

Answer 2

For those of you that are still trying to figure this one out, this is what I found. The "stdin=subprocess.PIPE" method above works with some command line tool in OSGeo4W64 but not all. It works with gdal_translate but not pdal translate for example. Not sure why;(

My Solution:

OSGeo4Wenv = r'CALL "C:/OSGeo4W64/bin/o4w_env.bat" '

pdal_translate_String = r'c:/OSGeo4W64/bin/pdal translate c:\inputFile c:\outputFile radiusoutlier --filters.radiusoutlier.min_neighbors=2 --filters.radiusoutlier.radius=8.0 --filters.radiusoutlier.extract=true'

Cmd = str(OSGeo4Wenv)+' & '+str(pdal_translateCmd)

shell = subprocess.call(Cmd, stdout=None, shell=True)

What is going on? 1) Open shell and set up the OSGeo4W environment by calling "OSGeo4Wenv". This is normally called by the OSGeo4W.bat file. Without this, the command line programs don't know where to look for the libraries.

2) The pdal_translate command is then sent to the dos shell because in Windows, multiple commands can be separated by the "&" symbol. I use the .call method of python 2.7. It has the advantage that it waits for the end of the process. That is nice if you use the multiprocessing map.pool method to launch multiple processes at the same time.

Hope this help others! Nicolas