homepage

It would be nice to support the PEP 383 (surrogateescape) for environment variables in os.execvpe(). Attached patch uses PyUnicode_AsEncodedString(val, Py_FileSystemDefaultEncoding, "surrogateescape") to encode an environment variable value.
I'm not sure that PyUnicode_AsEncodedString(val, Py_FileSystemDefaultEncoding, "surrogateescape") does always return a PyBytes object.
I not patched environment keys, but it might be useful.

See also issue #4036.

See also #8393.

My patch doesn't work for types bytes and bytearray.
I noticed that py3k uses surrogateescape to encode environment variable values ;-)

Other notes: Environment variable *names* use also surrogateescape "encoding". os.spawnve() and os.spawnvpe() should also be patched (the code should also be factorized).

New version of the patch:
 - factorize code between execve(), spawnve() and spawnvpe()
 - support also surrogates in environment variable names
 - support bytes and bytearray (bytearray cannot be used as a dictionary key, but my patch supports it)
 - remove unrelated fix (my first patch contains a fix for os.system(), also about surrogates)
Because of the factorization, the error messages doesn't contain the function name anymore. spawnve() and spawnvpe() omit BEGINLIBPATH and ENDLIBPATH, as execve(): "that Would Confuse Programs if Passed On". I suppose that if execve() ignore them, spawn*e() should also ignore them.
I don't have an OS/2, so I'm unable to test my patch on this OS :-/
Note: The patch fixes also subprocess to support bytes and bytearray in the environment dictionary.

Current code of execve() has a bug: it uses the length of the environment variable value in *characters* and not in *bytes* to allocate the "p" buffer. I remember that someone wrote a comment somewhere about that... The result is that the environment variable value is truncated by 1 byte.
Example (copy of http://dpaste.com/184803/):
-----------
$ cat test.py
#!/usr/bin/python
# -*- coding: utf-8 -*-
import os
env = {"VAR": "ćd"}
os.execve("test.sh", [], env)
$ cat test.sh
#!/bin/bash
declare -p VAR
$ python2.6 test.py
declare -x VAR="ćd"
$ python3.1 test.py
declare -x VAR="ć"
-----------

Commited: r80421 (py3k), blocked in 3.1 (80422). The commit fixes also os.getenv() to support bytes environment name.

I blocked the fix in Python 3.1 because it's non trivial and I prefer to avoid complex changes in Python 3.1. But then I realized that Python 3.1 has two bugs about environment variables.
It uses sys.getfilesystemencoding()+surrogateecape to decode variables and sys.getdefaultencoding()+strict to encode variables: the encoding is different!
It counts the number of *characters* to allocate the *byte* string buffer and so non-ASCII values are truncated.
So I decided to backport the fix: r80494.