Interacting with bash from python [duplicate]

Question

This question already has answers here:

Interactive input/output using Python (4 answers)

Closed 7 months ago.

I've been playing around with Python's subprocess module and I wanted to do an "interactive session" with bash from python. I want to be able to read bash output/write commands from Python just like I do on a terminal emulator. I guess a code example explains it better:

>>> proc = subprocess.Popen(['/bin/bash'])
>>> proc.communicate()
('user@machine:~/','')
>>> proc.communicate('ls\n')
('file1 file2 file3','')

(obviously, it doesn't work that way.) Is something like this possible, and how?

Thanks a lot

Answer 1

Try with this example:

import subprocess
proc = subprocess.Popen(['/bin/bash'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
stdout = proc.communicate('ls -lash')
print(stdout)

You have to read more about stdin, stdout and stderr. This looks like good lecture: http://www.doughellmann.com/PyMOTW/subprocess/

EDIT:

Another example:

import subprocess
process = subprocess.Popen(['/bin/bash'], shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
process.stdin.write(b'echo it works!\n')
process.stdin.flush()
print(process.stdout.readline()) # 'it works!\n'
process.stdin.write(b'date\n')
process.stdin.flush()
print(process.stdout.readline()) # b'Fri Aug 30 18:34:33 UTC 2024\n'

Since the stream is buffered by default you will need to either call flush after sending a command or disable buffering by setting bufsize=0 on the Popen call, both will work.
You can check the subprocess.Popen documentation for more information on buffering.

• justinas: The first .communicate() call works well, but if I try to communicate again, this happens: ValueError: I/O operation on closed file. Is there any way to keep it running?
• jfs: 1- the first code example can be written as stdout = subprocess.check_output(['ls', '-lash']). To run a bash command, you could check_output("some && command $(< file)", shell=True, executable='/bin/bash') 2- the second code example is very fragile -- if the input/output are not in sync.; deadlock may happen. 3- if stdin/stdout are not a tty; the program may change their output. See Q: Why not just use a pipe (popen())?
• waitingkuo: Your "another example" doesn't work in Python3.5. Any way to solve it?
• UpmostScarab: @waitingkuo that is because in Python 3 buffsize defaults to -1 and in 2 it is 0, so set it to 0 in Popen and it should work. For me it did.
• e-info128: You example is not a interactive shell, is a static instruction execution.

Answer 2

This should be what you want

import subprocess
import threading
p = subprocess.Popen(["bash"], stderr=subprocess.PIPE,shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
exit = False
def read_stdout():
 while not exit:
 msg = p.stdout.readline()
 print("stdout: ", msg.decode())
def read_stderro():
 while not exit:
 msg = p.stderr.readline()
 print("stderr: ", msg.decode())
threading.Thread(target=read_stdout).start()
threading.Thread(target=read_stderro).start()
while not exit:
 res = input(">")
 p.stdin.write((res + '\n').encode())
 p.stdin.flush()

Test result:

>ls
>stdout: 1.py
stdout: 2.py
>ssss
stderr: bash: line 2: ssss: command not found

Answer 3

Use this example in my other answer: https://stackoverflow.com/a/43012138/3555925

You can get more details in that answer.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import sys
import select
import termios
import tty
import pty
from subprocess import Popen
command = 'bash'
# command = 'docker run -it --rm centos /bin/bash'.split()
# save original tty setting then set it to raw mode
old_tty = termios.tcgetattr(sys.stdin)
tty.setraw(sys.stdin.fileno())
# open pseudo-terminal to interact with subprocess
master_fd, slave_fd = pty.openpty()
# use os.setsid() make it run in a new process group, or bash job control will not be enabled
p = Popen(command,
 preexec_fn=os.setsid,
 stdin=slave_fd,
 stdout=slave_fd,
 stderr=slave_fd,
 universal_newlines=True)
while p.poll() is None:
 r, w, e = select.select([sys.stdin, master_fd], [], [])
 if sys.stdin in r:
 d = os.read(sys.stdin.fileno(), 10240)
 os.write(master_fd, d)
 elif master_fd in r:
 o = os.read(master_fd, 10240)
 if o:
 os.write(sys.stdout.fileno(), o)
# restore tty settings back
termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_tty)

Answer 4

An interactive bash process expects to be interacting with a tty. To create a pseudo-terminal, use os.openpty(). This will return a slave_fd file descriptor that you can use to open files for stdin, stdout, and stderr. You can then write to and read from master_fd to interact with your process. Note that if you're doing even mildly complex interaction, you'll also want to use the select module to make sure that you don't deadlock.

Answer 5

I wrote a module to facilitate the interaction between *nix shell and python.

def execute(cmd):
if not _DEBUG_MODE:
 ## Use bash; the default is sh
 print 'Output of command ' + cmd + ' :'
 subprocess.call(cmd, shell=True, executable='/bin/bash')
 print ''
else:
 print 'The command is ' + cmd
 print ''

Check out the whole stuff at github: https://github.com/jerryzhujian9/ez.py/blob/master/ez/easyshell.py

• Jerry T: you now can install via pip: pip install ez