Run bash command line script in python

Question

Is it possible to run a command line bash script in python? I know of subprocess.call/check_output to run a single command with parameters. However, how do I run

for tag in `git branch -r | grep "tags/" | sed 's/ tags\///'`; do
 git tag -a -m"Converting SVN tags" \
 `echo $tag | grep "tags/" |sed 's/.*tags\///'` \
 refs/remotes/$tag
done 

in python? I don't want to put the "line" in a shell script and have python call it.

Thanks!

• A user: Possible duplicate of How to run bash command inside python script?
• ghoti: It is indeed possible to spawn shell command lines from within python, but this is an XY Problem. You don't want to do that. With this code, a better question would be something like "I'm currently running the following in shell. It works, but could be done better. In particular, I need to implement something in Python that generates the same results. Here is my attempt, followed by the error message I get when I run it."
• chepner: A good first step, though, would be to simplify the shell code you have now. For instance, git branch -r | grep "tags/" can be replaced with git branch --list -r 'tags/*'. The command substitution used as an argument for git tag can (probably) be replaced with simply "${tag#*tags/}".
• Victor: @Pri So are you saying the answer is to use a method of subprocess, which I have mentioned? If that is what you are saying how do I place the for loop in a list for subprocess.call?
• tripleee: `echo $tag | grep "tags/" |sed 's/.*tags\///'` is wrong on so many levels. You want simply "${tag##*tags/}"

Answer 1

When converting from bash script to python you can go two ways:

1. you take over the program calls and just replace the looping / string processing, e.g. like this:

   from subprocess import check_output
   for line in check_output('git branch -r', shell=True).split("\n"):
    line = line.strip()
    # do your replaces here, etc. leaving that for an exercise for you
    # run more `check_call` or `check_output` here
   

2. you do it "the right way" and use a e.g. a python git module. That's initially more work, but long term beneficial as you get more out of the box features, and have less bugs as in your "quickly hacked together shell calls"

Edit: As a commenter rightly suggests, you should try to avoid shell=True. In this case, check_output(['git', 'branch', '-r']) is certainly better, in other cases, when you know that the python script will be run on a Linux systems and under a specific shell (e.g. bash), then having shell=True allows you to access environment variables specified under .bashrc, do globbing, etc. For a detailed discussion see here.

• tripleee: You're better off with check_output(['git', 'branch', '-r']) with no shell=True as the shell isn't contributing anything useful here, apart from splitting the command on spaces, which you can easily do yourself. (Granted, it's slightly less legible, but the benefits hugely outweigh this minor inconvenience.) See also stackoverflow.com/questions/3172470/…
• hansaplast: @tripleee: thanks for the pointer, generally I think shell=True smoothes out the way from shell to python, but of course in the end you'd want a script which works everywhere, which is certainly not the case with shell=True. I have added a section to the answer
• tripleee: Very good update, +1. I tend to caution against shell=True because people tend to copy/paste code from Stack Overflow without understanding the implications, and so constructs with substantial side effects are problematic, at least if you don't call them out.