This issue tracker has been migrated to GitHub ,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2014年08月07日 15:06 by jleedev, last changed 2022年04月11日 14:58 by admin. This issue is now closed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| issue22165.patch | serhiy.storchaka, 2014年08月07日 18:24 | review | ||
| issue22165_2.patch | serhiy.storchaka, 2014年08月16日 19:24 | review | ||
| test_undecodable_filename.diff | serhiy.storchaka, 2014年08月17日 10:17 | review | ||
| Messages (16) | |||
|---|---|---|---|
| msg225016 - (view) | Author: Josh Lee (jleedev) | Date: 2014年08月07日 15:06 | |
While SimpleHTTPServer from Python2 would happily spit out whatever bytes were in the directory listing, Python3's http.server logs an error and closes the connection without responding to the HTTP request.
$ mkdir $'\xff'
$ ls
377円/
$ python3 -m http.server
Serving HTTP on 0.0.0.0 port 8000 ...
----------------------------------------
Exception happened during processing of request from ('74.125.59.145', 19648)
Traceback (most recent call last):
File "/home/josh/local/lib/python3.5/socketserver.py", line 321, in _handle_request_noblock
self.process_request(request, client_address)
File "/home/josh/local/lib/python3.5/socketserver.py", line 347, in process_request
self.finish_request(request, client_address)
File "/home/josh/local/lib/python3.5/socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/home/josh/local/lib/python3.5/socketserver.py", line 681, in __init__
self.handle()
File "/home/josh/local/lib/python3.5/http/server.py", line 398, in handle
self.handle_one_request()
File "/home/josh/local/lib/python3.5/http/server.py", line 386, in handle_one_request
method()
File "/home/josh/local/lib/python3.5/http/server.py", line 677, in do_GET
f = self.send_head()
File "/home/josh/local/lib/python3.5/http/server.py", line 716, in send_head
return self.list_directory(path)
File "/home/josh/local/lib/python3.5/http/server.py", line 772, in list_directory
% (urllib.parse.quote(linkname), html.escape(displayname)))
File "/home/josh/local/lib/python3.5/urllib/parse.py", line 688, in quote
string = string.encode(encoding, errors)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff' in position 0: surrogates not allowed
----------------------------------------
|
|||
| msg225030 - (view) | Author: R. David Murray (r.david.murray) * (Python committer) | Date: 2014年08月07日 16:53 | |
It should return a server error, I think. |
|||
| msg225031 - (view) | Author: Augie Fackler (durin42) * | Date: 2014年08月07日 16:54 | |
Why not treat the filename as opaque bytes, and let the client fetch it anyway? |
|||
| msg225035 - (view) | Author: R. David Murray (r.david.murray) * (Python committer) | Date: 2014年08月07日 17:50 | |
Because http traffic is supposed to be either latin-1 or whatever charset is specified (at least, to my understanding that is the case), so sending incorrectly encoded data seems wrong. On the other hand, we support unix files systems not having well defined charsets, so extending this to directory listings in http isn't crazy. That does raise the question, though, of passing the bytes through python3's string model without breaking anything, so some careful thought may be required. I haven't looked at the details, though, so it might well be pretty simple. |
|||
| msg225036 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2014年08月07日 18:24 | |
Here is a patch which fixes handling of undecodable paths in SimpleHTTPRequestHandler. |
|||
| msg225407 - (view) | Author: Senthil Kumaran (orsenthil) * (Python committer) | Date: 2014年08月16日 17:49 | |
Attached patch looks good to me. If an unittest can be provided for this situation in test_httpservers.py it will be comprehensive and will be good to go. |
|||
| msg225409 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2014年08月16日 19:24 | |
Here is a patch with a test. |
|||
| msg225426 - (view) | Author: Roundup Robot (python-dev) (Python triager) | Date: 2014年08月17日 05:25 | |
New changeset f180a9156cc8 by Serhiy Storchaka in branch '3.4': Issue #22165: SimpleHTTPRequestHandler now supports undecodable file names. http://hg.python.org/cpython/rev/f180a9156cc8 New changeset 3153a400b739 by Serhiy Storchaka in branch 'default': Issue #22165: SimpleHTTPRequestHandler now supports undecodable file names. http://hg.python.org/cpython/rev/3153a400b739 |
|||
| msg225427 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2014年08月17日 05:31 | |
Thank you for the review Senthil. |
|||
| msg225428 - (view) | Author: Senthil Kumaran (orsenthil) * (Python committer) | Date: 2014年08月17日 07:59 | |
Looks like we hit with an encoding issue, which is due to way os.fsdecode() and os.listdir() decode the filenames. >>> support.TESTFN_UNDECODABLE b'@test_99678_tmp\xe7w\xf0' >>> dir_list = os.listdir(self.tempdir) >>> dir_list ['@test_99678_tmp%E7w%F0.txt', 'test'] >>> filename = os.fsdecode(support.TESTFN_UNDECODABLE) + '.txt' >>> filename '@test_99678_tmp\udce7w\udcf0.txt' ====================================================================== FAIL: test_undecodable_filename (test.test_httpservers.SimpleHTTPServerTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/buildbot/buildarea/3.4.murray-snowleopard/build/Lib/test/test_httpservers.py", line 282, in test_undecodable_filename .encode('utf-8', 'surrogateescape'), body) AssertionError: b'href="%40test_62069_tmp%ED%B3%A7w%ED%B3%B0.txt"' not found in b'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">\n<html>\n<head>\n<meta http-equiv="Content-Type" content="text/html; charset=utf-8">\n<title>Directory listing for tmp0asrs9ei/</title>\n</head>\n<body>\n<h1>Directory listing for tmp0asrs9ei/</h1>\n<hr>\n<ul>\n<li><a href="%40test_62069_tmp%25E7w%25F0.txt">@test_62069_tmp%E7w%F0.txt</a></li>\n<li><a href="test">test</a></li>\n</ul>\n<hr>\n</body>\n</html>\n' |
|||
| msg225430 - (view) | Author: Roundup Robot (python-dev) (Python triager) | Date: 2014年08月17日 09:22 | |
New changeset a894b629bbea by Serhiy Storchaka in branch '3.4': Issue #22165: Fixed test_undecodable_filename on non-UTF-8 locales. http://hg.python.org/cpython/rev/a894b629bbea New changeset 7cdc941d5180 by Serhiy Storchaka in branch 'default': Issue #22165: Fixed test_undecodable_filename on non-UTF-8 locales. http://hg.python.org/cpython/rev/7cdc941d5180 |
|||
| msg225431 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2014年08月17日 10:17 | |
Oh, I missed that os.listdir() on Mac returns really strange result. Thank you Senthil. Here is a patch which try to workaround this. I'm not sure that it is enough. May be we should fix os.listdir(). Or conclude that this issue can't be fixed on Mac OS. |
|||
| msg225433 - (view) | Author: Ronald Oussoren (ronaldoussoren) * (Python committer) | Date: 2014年08月17日 10:51 | |
OSX returns a strange value in os.listdir because the HFS+ filesystem itself has unicode filenames and transforms byte strings that are assumed to contain UTF-8 into something the filesystem can handle (and seems to replace bytes that aren't valid UTF-8 into a percent-encoded value). |
|||
| msg225435 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2014年08月17日 11:58 | |
Well, then the workaround should work. |
|||
| msg225444 - (view) | Author: Roundup Robot (python-dev) (Python triager) | Date: 2014年08月17日 13:59 | |
New changeset b05d4f3ee190 by Serhiy Storchaka in branch '3.4': Issue #22165: Fixed test_undecodable_filename on Mac OS. http://hg.python.org/cpython/rev/b05d4f3ee190 New changeset 58e0d2c3ead8 by Serhiy Storchaka in branch 'default': Issue #22165: Fixed test_undecodable_filename on Mac OS. http://hg.python.org/cpython/rev/58e0d2c3ead8 |
|||
| msg233449 - (view) | Author: Roundup Robot (python-dev) (Python triager) | Date: 2015年01月05日 09:06 | |
New changeset 1bc41bbbe02d by Ned Deily in branch '3.4': Issue #22165: Skip test_undecodable_filename on OS X prior to 10.5. https://hg.python.org/cpython/rev/1bc41bbbe02d New changeset 85258e08b69b by Ned Deily in branch 'default': Issue #22165: merge from 3.4 https://hg.python.org/cpython/rev/85258e08b69b |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022年04月11日 14:58:06 | admin | set | github: 66361 |
| 2015年01月05日 09:06:09 | python-dev | set | messages: + msg233449 |
| 2014年08月19日 03:19:15 | orsenthil | set | resolution: fixed stage: resolved |
| 2014年08月19日 03:19:01 | orsenthil | set | status: open -> closed |
| 2014年08月17日 13:59:13 | python-dev | set | messages: + msg225444 |
| 2014年08月17日 11:58:25 | serhiy.storchaka | set | messages: + msg225435 |
| 2014年08月17日 10:51:55 | ronaldoussoren | set | messages: + msg225433 |
| 2014年08月17日 10:17:45 | serhiy.storchaka | set | files:
+ test_undecodable_filename.diff components: + macOS nosy: + ronaldoussoren, vstinner, ned.deily, hynek messages: + msg225431 resolution: fixed -> (no value) stage: resolved -> (no value) |
| 2014年08月17日 09:22:49 | python-dev | set | messages: + msg225430 |
| 2014年08月17日 08:00:18 | orsenthil | set | status: closed -> open |
| 2014年08月17日 07:59:13 | orsenthil | set | messages: + msg225428 |
| 2014年08月17日 05:31:53 | serhiy.storchaka | set | status: open -> closed resolution: fixed messages: + msg225427 stage: patch review -> resolved |
| 2014年08月17日 05:25:54 | python-dev | set | nosy:
+ python-dev messages: + msg225426 |
| 2014年08月16日 19:24:52 | serhiy.storchaka | set | files: + issue22165_2.patch |
| 2014年08月16日 19:24:20 | serhiy.storchaka | set | assignee: serhiy.storchaka messages: + msg225409 |
| 2014年08月16日 17:49:01 | orsenthil | set | nosy:
+ orsenthil messages: + msg225407 |
| 2014年08月09日 01:35:31 | martin.panter | set | nosy:
+ martin.panter |
| 2014年08月07日 19:34:43 | demian.brecht | set | nosy:
+ demian.brecht |
| 2014年08月07日 18:24:50 | serhiy.storchaka | set | files:
+ issue22165.patch nosy: + serhiy.storchaka messages: + msg225036 keywords: + patch stage: needs patch -> patch review |
| 2014年08月07日 17:50:24 | r.david.murray | set | messages: + msg225035 |
| 2014年08月07日 16:54:29 | durin42 | set | messages: + msg225031 |
| 2014年08月07日 16:53:17 | r.david.murray | set | versions:
+ Python 3.4 nosy: + r.david.murray messages: + msg225030 type: behavior stage: needs patch |
| 2014年08月07日 15:06:33 | jleedev | create | |