homepage

Popen.communicate() raises an exception if passed no input when stdin=PIPE and universal_newlines=True. With universal_newlines=False, no exception is raised. For example, the following yields--
args = [sys.executable, '-c', 'pass']
popen = Popen(args, universal_newlines=False, stdin=PIPE, stdout=PIPE, stderr=PIPE)
print(popen.communicate())
popen = Popen(args, universal_newlines=True, stdin=PIPE, stdout=PIPE, stderr=PIPE)
print(popen.communicate())
(b'', b'[41449 refs]\n')
Traceback (most recent call last):
 ...
 File ".../Lib/subprocess.py", line 1581, in _communicate_with_poll
 self._input = self._input.encode(self.stdin.encoding)
AttributeError: 'NoneType' object has no attribute 'encode'
It seems like communicate() should not be trying to encode input if there is none.
I can provide a patch with tests if it is agreed this is an issue (along with a fix assuming it is reasonable).

See issue 12623 for another issue related to communicate() and universal newline support.

_communicate_with_select has the same problem as _communicate_with_poll.
I don't understand why input has encoded if universal_newlines and passed unchanged otherwise.
From my perspective input should be encoded (converted to bytes) if it is str regardless of universal_newlines value.

> From my perspective input should be encoded (converted to bytes) if it is str regardless of universal_newlines value.
I don't know the reason, but the limitation is documented:
"The optional input argument should be data to be sent to the child process, or None, if no data should be sent to the child. The type of input must be bytes or, if universal_newlines was True, a string."
http://docs.python.org/dev/library/subprocess.html#subprocess.Popen.communicate
Maybe "explicit better than implicit"?

This issue may be the bug referenced here:
# BUG: can't give a non-empty stdin because it breaks both the
# select- and poll-based communicate() implementations.
(stdout, stderr) = p.communicate()
http://hg.python.org/cpython/file/843e0da7e91f/Lib/test/test_subprocess.py#l618
The reason is that the above test would fail if stdin were not None since the input to communicate is None.
I am preparing a test and patch for this issue.

I like to accept both str and bytes if universal_newlines is False and raise explicit exception for bytes if universal_newlines is True.
Looks like universal_newlines for bytes doesn't make sense, also we prefer to use str everywhere. Bytes should be used if it is really byte-stream or if unicode cannot be passed by some limitations of underlying OS etc.
There are couple dark corners:
1. What to do if stdin doesn't have `encoding` attribute? Convert to bytes using filesystemencoding? Raise explicit exception? Adding `encode` parameter for .communicate doesn't looks elegant.
2. How .communicate works on Windows? I see completely different implementation of this method for win32.
I need some time to investigate before publish proposal to python-dev list.

Attaching a failing test case.
Also, to confirm, this issue does not seem to affect 3.2.

Attaching patch.

New changeset 839bd8f98539 by Andrew Svetlov in branch '3.2':
Add test to explicit check the absence regression in subprocess (issue #15592).
http://hg.python.org/cpython/rev/839bd8f98539
New changeset 9d86480cc177 by Andrew Svetlov in branch 'default':
Issue #15592. Fix regression: subprocess.communicate() breaks on no input with universal newlines true.
http://hg.python.org/cpython/rev/9d86480cc177 

Close the issue. Thanks, Chris.
It was the real regression, 3.2 works fine.
See also #15649 for accepting str if not universal_newlines.

Thanks, Andrew!