homepage

While investigating #11638, I encountered another encoding issue related to tarballs. Consider this command:
python -c "import gzip; gzip.GzipFile(u'\xe5rchive', 'w', fileobj=open(u'\xe5rchive', 'wb'))"
When run, it triggers the following traceback:
Traceback (most recent call last):
 File "<string>", line 1, in <module>
 File "c:\python\lib\gzip.py", line 127, in __init__
 self._write_gzip_header()
 File "c:\python\lib\gzip.py", line 172, in _write_gzip_header
 self.fileobj.write(fname + '000円')
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe5' in position 0: ordinal not in range(128)
Based on the resolution of #13639, I believe the recommended fix is to handle unicode here much like Python 3 does--specifically, detect unicode, encode to 'latin-1' if possible or leave the filename blank if not.

The actual fix in the previous issue, as in Python 3, was to always write the filename, but with errors replaced with '?/.

> The actual fix in the previous issue, as in Python 3, was to always write the filename, but with errors replaced with '?/.
Filename is optional in gzip file. If it can't be encoded to Latin1, it should be just omitted.
Here is a patch which backports the solution from Python 3 (accumulated f37016d42729, fb069eafaf89, 8cff949323c9, and e044fa016c85).

I there are no objections I'll commit the patch soon.

fine with me

New changeset 272c78c9c47e by Serhiy Storchaka in branch '2.7':
Issue #13664: GzipFile now supports non-ascii Unicode filenames.
https://hg.python.org/cpython/rev/272c78c9c47e 

Thank you Terry for the review.

This broke a FreeBSD buildbot (koobs-freebsd10), complete log attached.

koobs@10-STABLE-amd64:~ % locale
LANG=
LC_CTYPE="C"
LC_COLLATE="C"
LC_TIME="C"
LC_NUMERIC="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_ALL=

I rechecked and test_gzip passes on 2.7 Win7, I checked revision history and this is the only gzip or test_gzip patch for several months.

Ah, ASCII locale...

New changeset 7657cc08d29b by Serhiy Storchaka in branch '2.7':
Fixed the test of issue #13664 on platforms without unicode filenames support.
https://hg.python.org/cpython/rev/7657cc08d29b 

test_gzip passed after this patch.