homepage

I was able to get to a website by using both IE and 
FireFox but my Python code kept giving HTTP 400 Bad 
request error. To debug, I set set_http_debuglevel(1) as 
in the following code:
hh = urllib2.HTTPHandler() 
hh.set_http_debuglevel(1)
opener = urllib2.build_opener
(hh,urllib2.HTTPCookieProcessor(self.cj))
The printed debug messages show that this crash 
happens when there is a space in the redirected 
location. Here's a cut-and-paste of the relevant debug 
messages (note the line starting with send that 
http_error_302 is sending):
reply: 'HTTP/1.1 302 Moved Temporarily\r\n'
header: Connection: close
header: Date: 2005年2月27日 19:52:51 GMT
header: Server: Microsoft-IIS/6.0
<---other header data-->
send: 'GET /myEmail/User?asOf=02/26/2005 11:38:12 
PM&
ddn=87cb51501730
<---remaining header data-->
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Content-Type: text/html
header: Date: 2005年2月27日 19:56:45 GMT
header: Connection: close
header: Content-Length: 20
To fix this, I first tried to encode the redirected location 
in the function http_error_302() in urllib2 using the 
methods urllib.quote and urllib.urlencode but to no avail 
(they encode other data as well). 
A temporary solution that works is to replace any space 
in the redirected URL by'%20'. Below is a snippet of the 
function http_error_302 in urllib2 with this suggested fix:
def http_error_302(self, req, fp, code, msg, headers):
 # Some servers (incorrectly) return multiple 
Location headers
 # (so probably same goes for URI). Use first 
header.
 if 'location' in headers:
 newurl = headers.getheaders('location')[0]
 elif 'uri' in headers:
 newurl = headers.getheaders('uri')[0]
 else:
 return
 newurl=newurl.replace(' ','%20') # <<< TEMP FIX - 
inserting this line temporarily fixes this problem
 newurl = urlparse.urljoin(req.get_full_url(), newurl)
 <--- remainder of this function -->
Thanks!

Logged In: YES 
user_id=2772
When the server sends the 302 response with 'Location:
http://example.com/url%20with%20whitespace', urllib2 seems
to work just fine.
I believe based on reading rfc2396 that a URL that contains
spaces must contain quoted spaces (%20) not literal spaces,
because space is not an "unreserved character" [2.3] and
"[d]ata must be escaped if it does not have a representation
using an unreserved character" [2.4].

Logged In: YES 
user_id=261020
Sure, but if Firefox and IE do it, probably we should do the
same.
I think cookielib.escape_path(), or something similar
(perhaps without the case normalisation) is probably the
right thing to do. That's not part of any documented API; I
suppose that function or a similar one should be added to
module urlparse, and used by urllib2 and urllib when
redirecting.

As always with urllib, the fix is trivial but adding a test is hard.

Appears to be the same as issue 918368 

This bug refers to urllib2. Issue 918368 refers to urllib. It's the
same problem in each case, though.

fixed in revision 43132 ( smaller 'r' for the roundup to auto-hyperlink). :)

Sorry, I meant fixed in revision 72351.

> Sorry, I meant fixed in revision 72351.
Commit in Git:
commit 690ce9b353bc0a86d0886470adbaa50e813de3b8
Author: Senthil Kumaran <orsenthil@gmail.com>
Date: Tue May 5 18:41:13 2009 +0000
 Fix for issue1153027, making Py3k changes similar to fix in issue918368.
 This will address:
 a) urllib/ in py3k,
 b) urllib in py2x is addressed by issue918368.
 c) urllib2 in py2x was already addressed in Revision 43132.