piping with subprocess

Sat Feb 1 09:00:59 EST 2014

On Saturday, February 1, 2014 7:54:34 AM UTC-6, Rick Dooling wrote:
> On Saturday, February 1, 2014 6:54:09 AM UTC-6, Peter Otten wrote:
>> > Rick Dooling wrote:
>> > 
>> > 
>> > 
>> > > I spent half a day trying to convert this bash script (on Mac)
>> > 
>> > > 
>> > 
>> > > textutil -convert html 1ドル -stdout | pandoc -f html -t markdown -o 2ドル
>> > 
>> > > 
>> > 
>> > > into Python using subprocess pipes.
>> > 
>> > > 
>> > 
>> > > It works if I save the above into a shell script called convert.sh and
>> > 
>> > > then do
>> > 
>> > > 
>> > 
>> > > subprocess.check_call(["convert.sh", file, markdown_file])
>> > 
>> > > 
>> > 
>> > > where file and markdown_file are variables.
>> > 
>> > > 
>> > 
>> > > But otherwise my piping attempts fail.
>> > 
>> > 
>> > 
>> > It is always a good idea to post your "best effort" failed attempt, if only 
>> > 
>> > to give us an idea of your level of expertise.
>> > 
>> > 
>> > 
>> > > Could someone show me how to pipe in subprocess. Yes, I've read the doc,
>> > 
>> > > especially
>> > 
>> > > 
>> > 
>> > > http://docs.python.org/2/library/subprocess.html#replacing-shell-pipeline
>> > 
>> > > 
>> > 
>> > > But I'm a feeble hobbyist, not a computer scientist.
>> > 
>> > 
>> > 
>> > Try to convert the example from the above page
>> > 
>> > 
>> > 
>> > """
>> > 
>> > output=`dmesg | grep hda`
>> > 
>> > # becomes
>> > 
>> > p1 = Popen(["dmesg"], stdout=PIPE)
>> > 
>> > p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
>> > 
>> > p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits.
>> > 
>> > output = p2.communicate()[0]
>> > 
>> > """
>> > 
>> > 
>> > 
>> > to your usecase. Namely, replace
>> > 
>> > 
>> > 
>> > ["dmesg"] --> ["textutil", "-convert", "html", infile, "-stdout"]
>> > 
>> > ["grep", "hda"] --> ["pandoc", "-f", "html", "-t", "marktown", "-o",
>> > 
>> > outfile]
>> > 
>> > 
>> > 
>> > Don't forget to set
>> > 
>> > 
>> > 
>> > infile = ... 
>> > 
>> > outfile = ... 
>> > 
>> > 
>> > 
>> > to filenames (with absolute paths, to avoid one source of error).
>> > 
>> > If that doesn't work post the code you wrote along with the error messages.
>>>> p1 = subprocess.Popen(["textutil", "-convert", "html", file], stdout=subprocess.PIPE)
>> p2 = subprocess.check_call(["pandoc", "-f", "html", "-t", "markdown", "-o", markdown_file], stdin=p1.stdout, stdout=subprocess.PIPE)
>> p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits.
>> output = p2.communicate()[0]
>>>> Errors
>>>> Traceback (most recent call last):
>> File "/Users/me/Python/any2pandoc.py", line 70, in <module>
>> convert_word_file(file, markdown_file)
>> File "/Users/me/Python/any2pandoc.py", line 59, in convert_word_file
>> output = p2.communicate()[0]
>> AttributeError: 'int' object has no attribute 'communicate'
>>>> I get a markdown_file created but it's empty.
>>>> Thanks,
>>>> RD
>>>> ps - Daniel's works fine but I still don't learn to pipe :)

Okay, sorry. I fixed that obvious goof
 p1 = subprocess.Popen(["textutil", "-convert", "html", file], stdout=subprocess.PIPE)
 p2 = subprocess.Popen(["pandoc", "-f", "html", "-t", "markdown", "-o", markdown_file], stdin=p1.stdout, stdout=subprocess.PIPE)
 p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits.
 output = p2.communicate()[0]
Now I get no errors, but I still get a blank markdown file.