Message102996
| Author |
vstinner |
| Recipients |
lemburg, loewis, vstinner |
| Date |
2010年04月13日.00:39:53 |
| SpamBayes Score |
1.8970048e-11 |
| Marked as misclassified |
No |
| Message-id |
<1271119197.6.0.824303125234.issue8383@psf.upfronthosting.co.za> |
| In-reply-to |
| Content |
Python3 uses unicode surrogates to store undecodable filenames. Eg. the filename b"abc\xff.py" is encoded as "abc\xdcff.py" if the file system encoding is ASCII. Pickle is unable to store them:
./python -c 'import pickle; pickle.dumps("abc\udcff")'
(...)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff' in position 20: surrogates not allowed
This is a limitation of pickle (in the binary mode): Python accepts to store any unicode character, but pickle doesn't.
Using "surrogatepass" error handler should be enough to fix this issue.
Related issue: #3672 (Reject surrogates in utf-8 codec) -> r72208 creates "surrogatepass" error handler. |
|
History
|
|---|
| Date |
User |
Action |
Args |
| 2010年04月13日 00:39:57 | vstinner | set | recipients:
+ vstinner, lemburg, loewis |
| 2010年04月13日 00:39:57 | vstinner | set | messageid: <1271119197.6.0.824303125234.issue8383@psf.upfronthosting.co.za> |
| 2010年04月13日 00:39:55 | vstinner | link | issue8383 messages |
| 2010年04月13日 00:39:54 | vstinner | create |
|