homepage

Trying to unpack a ZIP file where some packet files contain danish letters results in:
 zipfile.BadZipFile: File name in directory 'filename_with_æoå.txt'
 and header b'filename_with_\x91o\x86.txt' differ.
Using Py 3.2b2 on Win7.
Unpack the attached ZIP file and run the Py script, which will show the problem using the enclosed two ZIP files.

The attached patch fixes it for me. No time to write tests right now.

FWIW, having just looked at related code in zipfile recently, this patch looks correct to me.

I'll try to produce a test in the next hour or two

I'm attaching a patch with a test for Martin's fix. I had trouble programmatically generating a "bad" zip for this bug, since it has different encodings for the header and filename (probably created by WinZip?). So I created a directory in test/ and placed the problematic zipfile M.Z. submitted in there, and wrote an appropriate test in test_zipfile.py
I verified the test fails on py3k trunk before Martin's fix, and succeeds after it, both by running the test file directly and through regrtest.
Note: Tested only on Ubuntu

Committed patch and test in r87604.

OK, looks like there is a problem on some buildbots: 
http://www.python.org/dev/buildbot/all/builders/AMD64%20Gentoo%20Wide%203.x/builds/863/steps/test/logs/stdio 

OK, I think r87606 fixed it: it doesn't extract the files, instead calls only open().

Georg, did you figure out the root cause of the problem on that buildbot? Seeing it fails in open(targetpath, "wb"), extracting the file may have failed if the bot had no write permissions to the current directory, but the ascii encoding error is not what I'd expect in such a case.

Well, it looks like the filesystem encoding is set to ASCII on these machines.

Issue #12048 is a duplicate of this bug, but with Python 3.1. Should we backport the fix to Python 3.1?

New changeset 1f0f0e317873 by Victor Stinner in branch '3.1':
Backport commit 33543b4e0e5d from Python 3.2: #10801: In zipfile, support
http://hg.python.org/cpython/rev/1f0f0e317873 

New changeset 243c78fbbb49 by Victor Stinner in branch '3.1':
Ooops, add the missing file of the backport of commit 33543b4e0e5d from Python
http://hg.python.org/cpython/rev/243c78fbbb49 

These changes cause test failure on 3.1 branch when verbose mode is disabled:
# python3.1 -m test.regrtest test_zipfile
test_zipfile
test test_zipfile produced unexpected output:
**********************************************************************
*** line 2 of actual output doesn't appear in expected output after line 1:
+ /usr/lib64/python3.1/test/zip_cp437_header.zip
**********************************************************************
1 test failed:
 test_zipfile

New changeset 9ef8fc5454cb by Victor Stinner in branch '3.1':
Issue #10801: Remove a debug print() from test_zipfile
http://hg.python.org/cpython/rev/9ef8fc5454cb 

> These changes cause test failure on 3.1 branch when verbose mode is disabled
What a shame! I commited a debug "print()":
 1.39 + print(fname)
It should be fixed by my last commit.

Victor: you should have a look at <http://bitbucket.org/birkenfeld/hgcodesmell>.

New changeset 33b7428e65b4 by Victor Stinner in branch '3.1':
Issue #10801: Fix test_unicode_filenames() of test_zipfile
http://hg.python.org/cpython/rev/33b7428e65b4