passing argument to python from shell script

Question

I have run through some similar questions but none seem to answer my doubt, I am trying to pass the argument to python script from my bash script. I don't see errors nor I get the required output. What am I doing wrong here?

python:

import os
import glob
import unicodecsv as csv
import pandas as pd
import codecs
import sys
OUTPUT_PATH=sys.argv[2] 
def createFolder(directory):
 # print('createFolder')
 try:
 if not os.path.exists(directory):
 # print('createFolder')
 os.makedirs(directory)
 # print(directory)
 except OSError:
 print ('Error: Creating directory. ' + directory)
createFolder(OUTPUT_PATH + 'csv20_out/')

bash script:

INPUT_PATH='home/pg/Public/test_data/test0002_2files_to_1flow/csv64_in/'
OUTPUT_PATH='home/pg/Public/test_data/test0002_2files_to_1flow/'
cd /
cd home/pg/Documents/LMS/kanjiforbeginner/convertToYAML/
cd converts_csv642csv20
python csvFile.py $INPUT_PATH $OUTPUT_PATH

Should create a folder csv20_out

when I print OUTPUT_PATH, I get 'home/pg/Public/test_data/test0002_2files_to_1flow/'

• isstiaung: What's the bash script you're using? Please provide that as well
• Wazaki: Can you try with OUTPUT_PATH=sys.argv[2] instead ? If it doesn't work, try to print the content of OUTPUT_PATH to see what it is
• user6462977: I made a mistake, actually, it points to sys.argv [2]

Answer 1

The contents of sys.argv look like this:

['csvFile.py', 'inputPath', 'outputPath']

To get the outputPath you need to access sys.argv[2] in your case.

Answer 2

So, as the other commenters have pointed out, you need to use argv[2] instead of argv[1] This is because in bash argv[0] is the python script being executed and argv[1] in your case becomes '$INPUT_PATH'

But, that won't solve your problem. In your script, you need to remove the ' surrounding $INPUT_PATH and $OUTPUT_PATH so that the last line becomes,
python csvFile.py $INPUT_PATH $OUTPUT_PATH

'$INPUT_PATH' sends the string $INPUT_PATH to your python script, whereas,

$INPUT_PATH sends 'home/pg/Public/test_data/test0002_2files_to_1flow/csv64_in/'

or whatever value you choose to assign to $INPUT_PATH

And lastly, you need to change, the values of $INPUT_PATH and $OUTPUT_PATH
INPUT_PATH='home/pg/Public/test_data/test0002_2files_to_1flow/csv64_in/' OUTPUT_PATH='home/pg/Public/test_data/test0002_2files_to_1flow/'

needs to become

INPUT_PATH='/home/pg/Public/test_data/test0002_2files_to_1flow/csv64_in/' OUTPUT_PATH='/home/pg/Public/test_data/test0002_2files_to_1flow/'

The / at the beginning gives the full path to the directory or file, if you didn't use the / then it becomes a relative path.

In your bash script, you're navigating folders using cd as a result, the folder

'home/pg/Public/test_data/test0002_2files_to_1flow/csv_out'

will get created under the directory,

home/pg/Documents/LMS/kanjiforbeginner/convertToYAML/

Adding the / at the beginning to $INPUT_PATH and $OUTPUT_PATH as shown before would ensure that your code:

• searches for input in the right directory
• creates the folder in the right directory

edit:

As Kim has suggested below, the last line in the script should be:

python csvFile.py "$INPUT_PATH" "$OUTPUT_PATH"

This is, in case $INPUT_PATH or $OUTPUT_PATH has spaces or special characters.

• Kim: Your bash script should use "$INPUT_PATH" instead of $INPUT_PATH. If $INPUT_PATH includes spaces or other special character then $INPUT_PATH will expand into multiple arguments - which won't work as expected. "$INPUT_PATH" will expand into a single argument (which may include spaces or other special characters).

Answer 3

By convention, the first argument is the name of the program itself—that is, the first thing specified on the command line (whatever_name.py).

So argv[0] is the program's name, argv[1] is your input path, and argv[2] is your output path.