Python subprocess call precompiled java

Question

This works from the windows command line:

c:\mallet\bin\mallet run

I've tried

subprocess.call(['c:\mallet\bin\mallet', 'run'])

and get an error

WindowsError: [Error 2] The system cannot find the file specified

and I've tried

subprocess.call(['c:/mallet/bin/mallet', 'run'])

and get the error

WindowsError: [Error 193] %1 is not a valid Win32 application

What do I have to pass to subprocess.call()?

For completeness sake the complete command I would like to pass is:

bin\mallet run cc.mallet.topics.tui.DMRLoader texts.txt features.txt instance.mallet

My vague idea is that this is a precompiled java class that I'm calling somehow, but I don't really understand what I'm doing here.

Here are the two mallet-files in the folder bin:

mallet.bat

@echo off
rem This batch file serves as a wrapper for several
rem MALLET command line tools.
if not "%MALLET_HOME%" == "" goto gotMalletHome
echo MALLET requires an environment variable MALLET_HOME.
goto :eof
:gotMalletHome
set MALLET_CLASSPATH=%MALLET_HOME%\class;%MALLET_HOME%\lib\mallet-deps.jar
set MALLET_MEMORY=1G
set MALLET_ENCODING=UTF-8
set CMD=%1
shift
set CLASS=
if "%CMD%"=="import-dir" set CLASS=cc.mallet.classify.tui.Text2Vectors
if "%CMD%"=="import-file" set CLASS=cc.mallet.classify.tui.Csv2Vectors
if "%CMD%"=="import-smvlight" set CLASS=cc.mallet.classify.tui.SvmLight2Vectors
if "%CMD%"=="train-classifier" set CLASS=cc.mallet.classify.tui.Vectors2Classify
if "%CMD%"=="train-topics" set CLASS=cc.mallet.topics.tui.Vectors2Topics
if "%CMD%"=="infer-topics" set CLASS=cc.mallet.topics.tui.InferTopics
if "%CMD%"=="estimate-topics" set CLASS=cc.mallet.topics.tui.EstimateTopics
if "%CMD%"=="hlda" set CLASS=cc.mallet.topics.tui.HierarchicalLDATUI
if "%CMD%"=="prune" set CLASS=cc.mallet.classify.tui.Vectors2Vectors
if "%CMD%"=="split" set CLASS=cc.mallet.classify.tui.Vectors2Vectors
if "%CMD%"=="bulk-load" set CLASS=cc.mallet.util.BulkLoader
if "%CMD%"=="run" set CLASS=%1 & shift
if not "%CLASS%" == "" goto gotClass
echo Mallet 2.0 commands: 
echo import-dir load the contents of a directory into mallet instances (one per file)
echo import-file load a single file into mallet instances (one per line)
echo import-svmlight load a single SVMLight format data file into mallet instances (one per line)
echo train-classifier train a classifier from Mallet data files
echo train-topics train a topic model from Mallet data files
echo infer-topics use a trained topic model to infer topics for new documents
echo estimate-topics estimate the probability of new documents given a trained model
echo hlda train a topic model using Hierarchical LDA
echo prune remove features based on frequency or information gain
echo split divide data into testing, training, and validation portions
echo Include --help with any option for more information
goto :eof
:gotClass
set MALLET_ARGS=
:getArg
if "%1"=="" goto run
set MALLET_ARGS=%MALLET_ARGS% %1
shift
goto getArg
:run
java -Xmx%MALLET_MEMORY% -ea -Dfile.encoding=%MALLET_ENCODING% -classpath %MALLET_CLASSPATH% %CLASS% %MALLET_ARGS%
:eof

and mallet

#!/bin/bash
malletdir=`dirname 0ドル`
malletdir=`dirname $malletdir`
cp=$malletdir/class:$malletdir/lib/mallet-deps.jar:$CLASSPATH
#echo $cp
MEMORY=1g
JAVA_COMMAND="java -Xmx$MEMORY -ea -Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -classpath $cp"
CMD=1ドル
shift
help()
{
cat <<EOF
Mallet 2.0 commands: 
 import-dir load the contents of a directory into mallet instances (one per file)
 import-file load a single file into mallet instances (one per line)
 import-svmlight load SVMLight format data files into Mallet instances
 train-classifier train a classifier from Mallet data files
 classify-dir classify data from a single file with a saved classifier
 classify-file classify the contents of a directory with a saved classifier
 classify-svmlight classify data from a single file in SVMLight format
 train-topics train a topic model from Mallet data files
 infer-topics use a trained topic model to infer topics for new documents
 evaluate-topics estimate the probability of new documents under a trained model
 hlda train a topic model using Hierarchical LDA
 prune remove features based on frequency or information gain
 split divide data into testing, training, and validation portions
Include --help with any option for more information
EOF
}
CLASS=
case $CMD in
 import-dir) CLASS=cc.mallet.classify.tui.Text2Vectors;;
 import-file) CLASS=cc.mallet.classify.tui.Csv2Vectors;;
 import-svmlight) CLASS=cc.mallet.classify.tui.SvmLight2Vectors;;
 train-classifier) CLASS=cc.mallet.classify.tui.Vectors2Classify;;
 classify-dir) CLASS=cc.mallet.classify.tui.Text2Classify;;
 classify-file) CLASS=cc.mallet.classify.tui.Csv2Classify;;
 classify-svmlight) CLASS=cc.mallet.classify.tui.SvmLight2Classify;;
 train-topics) CLASS=cc.mallet.topics.tui.Vectors2Topics;;
 infer-topics) CLASS=cc.mallet.topics.tui.InferTopics;;
 evaluate-topics) CLASS=cc.mallet.topics.tui.EvaluateTopics;;
 hlda) CLASS=cc.mallet.topics.tui.HierarchicalLDATUI;;
 prune) CLASS=cc.mallet.classify.tui.Vectors2Vectors;;
 split) CLASS=cc.mallet.classify.tui.Vectors2Vectors;;
 bulk-load) CLASS=cc.mallet.util.BulkLoader;;
 run) CLASS=1ドル; shift;;
 *) echo "Unrecognized command: $CMD"; help; exit 1;;
esac
$JAVA_COMMAND $CLASS $*

• Artturi Björk: I typed assoc c:\mallet\bin\mallet to windows command line and the return was File association not found for extension c:\mallet\bin\mallet I tried it also with / with same results.
• Zizouz212: Have you tried any of the suggestions in the answers?
• Padraic Cunningham: The problem is that c:\mallet\bin\mallet is not exceutable, what exactly is c:\mallet\bin\mallet? If it is a java file why not run it with java?
• Artturi Björk: Unfortunately I don't know. There are two files in the folder named mallet: mallet.bat (begins @echo off) and mallet (begins #!/bin/bash).
• Padraic Cunningham: And this is on windows?

Answer 1

When you call a program without extension, the Windows shell will try several standard extensions (.BAT, .EXE, ...) in order to guess the file you are trying to call.

If you want to execute your program without a shell to perform that look-up phase, you need to pass the full name of the batch you are trying to execute -- incl. the .BAT extension:

subprocess.call(['c:/mallet/bin/mallet.bat', 'run'])

• jfs: related: on the difference between CreateProcess (default) and cmd.exe (shell=True) command lookup.

Answer 2

Make sure that you pass the shell = True argument to subprocess.call(). However, it poses security issues, so make sure you look at the documentation and understand how it works.

subprocess.call(['c:/mallet/bin/mallet', 'run'], shell = True)

Also, when using strings to identify paths that contain backslashes, make it a raw string (r"This is a raw string!"), so that it doesn't implement anything else (such as newline tokens).

If my above suggestions don't work, there are only two things that I can thing of:

1. The file you are trying to execute may not be an application file (.exe file). I don't work with Windows so I'm not sure about this one, but it could likely be a possibility.
2. Something in the file(s) is broken or something like that.

subprocess.call() docs

subprocess.call() Security Issues

• Artturi Björk: Thanks for the suggestion. Do you have any idea what kind of security issues using shell=True poses? I think it quite unlikely that mallet would be a security risk. I tried the r in front of the string, but got the same WindowsError: [Error 193] %1 is not a valid Win32 application error.
• Zizouz212: The only security issues that shell = True imposes is if commands are based from external input. I'll set a link to the docs. I doubt you'll have to worry about it in this case anyway though ;)
• Artturi Björk: Your answer works and I'm going to accept it in a day or two, but would just like to hear if anyone has a suggestion how to get the subprocess.call() to work without shell = True.
• Zizouz212: Sure no problem. I should probably point out that you will always need shell = True. In other functions, such as os.system, it's like processing subprocess.call with the shell = True argument.

Answer 3

Well, maybe the problem is with the backslashes.

From their docs:

The backslash (\) character is used to escape characters that otherwise have a special meaning, such as newline, backslash itself, or the quote character.

So you probably should do:

subprocess.call(['c:\\mallet\\bin\\mallet', 'run'])

• jedwards: OP notes that they tried forward slashes, which are interpreted the same way as backslashes and don't need to be escaped.
• Artturi Björk: Thanks for the suggestion. I tried and got the WindowsError: [Error 193] %1 is not a valid Win32 application error.