homepage

Currently, ZipFile only accepts ascii or utf8 as file
name encodings. On Windows (Japanese), usually CP932
is used for it. So currently, when we melt ZipFile
via py3k, non-ascii file name becomes strange. Can we handle
this issue? (ie: adding encoding option for ZipFile#__init__)

The ZIP format specification mentions only cp437 and utf8: http://www.pkware.com/documents/casestudies/APPNOTE.TXT see Apeendix D.
Do zip files created on Japanese Windows contain some information about the encoding they use?
Or do some programs write cp932 where they are supposed to use one of the encodings above?

No, there is no indication in the zipfile that it deviates from the spec. That doesn't stop people from creating such zipfiles, anyway; many zip tools ignore the spec and use instead CP_ACP (which, of course, will then get misinterpreted if extracted on a different system).
I think we must support this case somehow, but must be careful to avoid creating such files unless explicitly requested. One approach might be to have two encodings given: one to interpret the existing filenames, and one to be used for new filenames (with a recommendation to never use that parameter since zip now supports UTF-8 in a well-defined manner).

@Hirokazu: Can you attach a small test archive?
Yes, we can add a "default_encoding" attribute to ZipFile and add an optional default_encoding argument to its constructor.

I'm not sure why, but I got BadZipFile error now. Anyway,
here is cp932 zip file to be created with python2.7.

In #10972, I propose to add an option for the filename encoding to UTF-8. But I would like to force UTF-8 to create a ZIP file, it doesn't concern the decompression of a ZIP file.
Proposal of a specification to fix both issues at the same time.
"default_encoding" name is confusing because it doesn't specify if it is the encoding of (text?) file content or the encoding the filename. Why not simply "filename_encoding"?
The option can be added in multiple places:
 - argument to ZipFile constructor: this is needed to decompress
 - argument to ZipFile.write() and ZipInfo, because they are 3 different manners to add files
ZipFile.filename_encoding (and ZipInfo.filename_encoding) will be None by default: in this case, use the current algorithm (try cp437 or use UTF-8). Otherwise, use the encoding. If the encoding is UTF-8: set unicode flag.
Examples:
---
zipfile.ZipFile("non-ascii-cp932.zip", filename_encoding="cp932")
f = zipfile.ZipFile("test.zip", "w")
f.write(filename, filename_encoding="UTF-8")
info = ZipInfo(filename, filename_encoding="UTF-8")
f.writestr(info, b'data')
---
Don't add filename_encoding argument to ZipFile.writestr(), because it may conflict if a ZipInfo is passed and ZipInfo.filename_encoding and filename_encoding are different.

I closed issue #12048 as a duplicate of this issue: yaoyu wants to uncompress a ZIP file having filenames encoded to GBK.

I fixed this problem.
I make new methos _decode_filename().

umedoblock: your patch is incorrect, as it produces moji-bake. if there is a file name b'f\x94n', it will decode as sjis under your patch (to u'f\u99ac'), even though it was meant as cp437 (i.e. u'f\xf6n').

Hi, Martin.
I tried your test case with attached file.
And I got below result.
p3 ./encodings.py
encoding: sjis, filename: f馬
encoding: cp437, filename: fön
sjis_filename = f馬
cp437_filename = fön
There are two success cases.
So I think that the patch needs to change default_encoding
before or in _decode_filename().
But I have no idea about how to change a default_encoding.

I'd like to submit patch to support zip archives created on systems that use non-US codepage (e.g. russian CP866).
Codepage would be specified in additional parameter of ZipFile constructor, named "codepage".
If it is not specified, old behavior is preserved (use CP437).
--- zipfile.py-orig 2013年09月18日 16:45:56.000000000 +0400
+++ zipfile.py 2013年10月15日 00:24:06.105157572 +0400
@@ -885,7 +885,7 @@
 fp = None # Set here since __del__ checks it
 _windows_illegal_name_trans_table = None
- def __init__(self, file, mode="r", compression=ZIP_STORED, allowZip64=False):
+ def __init__(self, file, mode="r", compression=ZIP_STORED, allowZip64=False, codepage='cp437'):
 """Open the ZIP file with mode read "r", write "w" or append "a"."""
 if mode not in ("r", "w", "a"):
 raise RuntimeError('ZipFile() requires mode "r", "w", or "a"')
@@ -901,6 +901,7 @@
 self.mode = key = mode.replace('b', '')[0]
 self.pwd = None
 self._comment = b''
+ self.codepage = codepage
 # Check if we were passed a file-like object
 if isinstance(file, str):
@@ -1002,7 +1003,7 @@
 filename = filename.decode('utf-8')
 else:
 # Historical ZIP filename encoding
- filename = filename.decode('cp437')
+ filename = filename.decode(self.codepage)
 # Create ZipInfo instance to store file information
 x = ZipInfo(filename)
 x.extra = fp.read(centdir[_CD_EXTRA_FIELD_LENGTH])
@@ -1157,7 +1158,7 @@
 # UTF-8 filename
 fname_str = fname.decode("utf-8")
 else:
- fname_str = fname.decode("cp437")
+ fname_str = fname.decode(self.codepage)
 if fname_str != zinfo.orig_filename:
 raise BadZipFile(

Please rename codepage to encoding. By the way, 437 is a codepage, cp437 is
a (python) encoding.
I don't think that ZIP is limited to windows. I uncompressed zip files many
times on various OSes, github also produces zip (and github is probably not
using windows). And codepage term is only used on windows. Mac OS 9 users
might produce mac roman filenames.

OK, here you are:
--- zipfile.py-orig 2013年09月18日 16:45:56.000000000 +0400
+++ zipfile.py 2013年10月19日 01:59:07.444346674 +0400
@@ -885,7 +885,7 @@
 fp = None # Set here since __del__ checks it
 _windows_illegal_name_trans_table = None
- def __init__(self, file, mode="r", compression=ZIP_STORED,
allowZip64=False):
+ def __init__(self, file, mode="r", compression=ZIP_STORED,
allowZip64=False, encoding='cp437'):
 """Open the ZIP file with mode read "r", write "w" or append
"a"."""
 if mode not in ("r", "w", "a"):
 raise RuntimeError('ZipFile() requires mode "r", "w", or "a"')
@@ -901,6 +901,7 @@
 self.mode = key = mode.replace('b', '')[0]
 self.pwd = None
 self._comment = b''
+ self.encoding = encoding
 # Check if we were passed a file-like object
 if isinstance(file, str):
@@ -1001,8 +1002,8 @@
 # UTF-8 file names extension
 filename = filename.decode('utf-8')
 else:
- # Historical ZIP filename encoding
- filename = filename.decode('cp437')
+ # Historical ZIP filename encoding, default is CP437
+ filename = filename.decode(self.encoding)
 # Create ZipInfo instance to store file information
 x = ZipInfo(filename)
 x.extra = fp.read(centdir[_CD_EXTRA_FIELD_LENGTH])
@@ -1157,7 +1158,7 @@
 # UTF-8 filename
 fname_str = fname.decode("utf-8")
 else:
- fname_str = fname.decode("cp437")
+ fname_str = fname.decode(self.encoding)
 if fname_str != zinfo.orig_filename:
 raise BadZipFile(
On Fri, Oct 18, 2013 at 11:47 AM, STINNER Victor <report@bugs.python.org>wrote:
>
> STINNER Victor added the comment:
>
> Please rename codepage to encoding. By the way, 437 is a codepage, cp437 is
> a (python) encoding.
>
> I don't think that ZIP is limited to windows. I uncompressed zip files many
> times on various OSes, github also produces zip (and github is probably not
> using windows). And codepage term is only used on windows. Mac OS 9 users
> might produce mac roman filenames.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue10614>
> _______________________________________
>

See also issue28080.

Thanks. Patch posted in issue28080 looks better than mine.