Getting git fetch output to file through python

Question

I am trying to save git fetch output to file through python, using:

subprocess.check_output(["git", "fetch", "origin", ">>", "C:/bitbucket_backup/backup.log", "2>&1"], cwd='C:/bitbucket_backup/loopx')

but I believe there is something missing in subprocess.check_output args because when adding >> C:/bitbucket_backup/backup.log 2>&1 I receive this error:

Traceback (most recent call last):
 File "<pyshell#28>", line 1, in <module>
 subprocess.check_output(["git", "fetch", "origin", ">>", "C://bitbucket_backup//backup.log", "2>&1"], cwd='C://bitbucket_backup//loopx')
 File "C:\Users\fabio\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 336, in check_output
 **kwargs).stdout
 File "C:\Users\fabio\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 418, in run
 output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['git', 'fetch', 'origin', '>>', 'C://bitbucket_backup//backup.log', '2>&1']' returned non-zero exit status 128.

• zwer: Don't forward STDOUT/STDERR as the part of the command, that's not how it's meant to be run from Python. subprocess.check_output will automatically pipe your STDOUT, and if you're expecting an error use subprocess.communicate() instead to capture both streams. You can use subprocess.Popen() to fully control the piping, including automatic piping to a file handle if you want to save the output as a file.
• Jean-François Fabre: to complete what zwer just said, your command line would need shell=True (but that's dirty :))
• torek: It's also not a great idea to depend on git fetch's output: it's not designed for machine parsing. Moreover it behaves differently if stdout is a pipe vs a tty. (There are some flags to try to control this but I ran into a bug trying to use them to show progress in parallel fetching.)
• marafado88: I know, I had to do some work arounds to be able to depend on git fetch.

Answer 1

Quickfix: enable shell features to handle redirection arguments:

subprocess.check_output(["git", "fetch", "origin", ">>", "C:/bitbucket_backup/backup.log", "2>&1"], cwd='C:/bitbucket_backup/loopx', shell=True)

But that's really dirty as python is able to do that really nicely:

output = subprocess.check_output(["git", "fetch", "origin"], stderr=subprocess.STDOUT, cwd='C:/bitbucket_backup/loopx')
with open("C:/bitbucket_backup/backup.log","ab") as f: # append to file
 f.write(output)

That said, if you're to rewrite all git commands in python, maybe you should use a git python API like GitPython for instance.

• marafado88: Thanks for the solution and also the "better" alternative eheh =P
• marafado88: also with the "better" alternative I had to do f.write(str(output)) to be able to write in backup.log, but through that it lost line breaks from the default git fetch formatation output.
• Jean-François Fabre: check my edit. Just open the file in binary mode. that should do the trick instead of converting to string.