homepage

If I try to download a inexistent file I get a TypeError which is thrown during exception handling.
>>> import urllib.request
>>> urllib.request.urlretrieve('missing')
Traceback (most recent call last):
 File "/usr/lib/python3.1/urllib/request.py", line 1705, in open_local_file
 stats = os.stat(localname)
OSError: [Errno 2] No such file or directory: 'missing'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/usr/lib/python3.1/urllib/request.py", line 133, in urlretrieve
 return _urlopener.retrieve(url, filename, reporthook, data)
 File "/usr/lib/python3.1/urllib/request.py", line 1507, in retrieve
 fp = self.open_local_file(url1)
 File "/usr/lib/python3.1/urllib/request.py", line 1707, in open_local_file
 raise URLError(e.errno, e.strerror, e.filename)
TypeError: __init__() takes at most 3 positional arguments (4 given)

Looks like a regression - I can't remember such messages in Python 2.

Clearly regressing. In Python 2 it was IOError exception:
>>> import urllib
>>> urllib.urlretrieve('missing')
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "C:\pp\lib\urllib.py", line 93, in urlretrieve
 return _urlopener.retrieve(url, filename, reporthook, data)
 File "C:\pp\lib\urllib.py", line 239, in retrieve
 fp = self.open(url, data)
 File "C:\pp\lib\urllib.py", line 207, in open
 return getattr(self, name)(url)
 File "C:\pp\lib\urllib.py", line 462, in open_file
 return self.open_local_file(url)
 File "C:\pp\lib\urllib.py", line 476, in open_local_file
 raise IOError(e.errno, e.strerror, e.filename)
IOError: [Errno 2] The system cannot find the file specified: 'missing'
>>>

The problem is in open_local_file (Lib/urllib/request.py:1885). It passes 3 arguments to URLError, but the constructor only accepts 2: reason and filename (Lib/urllib/error.py:29).
This used to be an IOError when urllib was a single module rather than a package.

The presence of "During handling of the above exception, another exception occurred:" is part of an intentional change between 2.x and 3.x. Exception handling has been tweaked a bit more in the past year. 3.2.3 only gives the second half of the traceback, which I suppose is worse.
In 3.3.0, the exception raising an exception behavior is fixed for this particular input on my Win 7 system.
>>> import urllib.request
>>> urllib.request.urlretrieve('missing')
Traceback (most recent call last):
 File "<pyshell#1>", line 1, in <module>
 urllib.request.urlretrieve('missing')
 File "C:\Programs\Python33\lib\urllib\request.py", line 185, in urlretrieve
 with contextlib.closing(urlopen(url, data)) as fp:
 File "C:\Programs\Python33\lib\urllib\request.py", line 160, in urlopen
 return opener.open(url, data, timeout)
 File "C:\Programs\Python33\lib\urllib\request.py", line 458, in open
 req = Request(fullurl, data)
 File "C:\Programs\Python33\lib\urllib\request.py", line 279, in __init__
 self._parse()
 File "C:\Programs\Python33\lib\urllib\request.py", line 284, in _parse
 raise ValueError("unknown url type: %s" % self.full_url)
ValueError: unknown url type: missing
urlretrieve no longer assumes the file: scheme. (It was never documented to do so, and there is no good reason to do so.) If given explicitly with an invalid name, it gives the current equivalent of the 2.x error and message.
>>> urllib.request.urlretrieve('file:missing')
...
urllib.error.URLError: <urlopen error [WinError 2] The system cannot find the file specified: 'missing'>
Nonetheless, this 3 arg call should be fixed
1887 except OSError as e:
1888 raise URLError(e.errno, e.strerror, e.filename)

While fixing this issue is easy, there is a wider problem with the use of URLError.
The constructor accepts two args, reason and filename. About half of the errors in Lib/urllib/request.py use only one argument, and in the other half not a single one uses the second arg for the filename:
raise URLError('file error', 'proxy support for file protocol currently not implemented')
raise URLError('local file error', 'not on local host')
raise URLError('ftp error', 'proxy support for ftp protocol currently not implemented')
if not host: raise URLError('ftp error', 'no host given')
raise URLError('ftp error', msg).with_traceback(sys.exc_info()[2])
raise URLError('data error', 'proxy support for data protocol currently not implemented')
raise URLError('ftp error', reason).with_traceback(
raise URLError('ftp error', reason) from reason
Note that the only the first arg ends up in the error message.
This should be reported in another issue though.
In the attached patch I passed only the first argoment and did
- raise URLError(e.errno, e.strerror, e.filename)
+ raise URLError('local file error')
The result is
Traceback (most recent call last):
 File "/home/wolf/dev/py/3.2/Lib/urllib/request.py", line 1769, in open_local_file
 stats = os.stat(localname)
OSError: [Errno 2] No such file or directory: 'foo'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/wolf/dev/py/3.2/Lib/urllib/request.py", line 1771, in open_local_file
 raise URLError('local file error')
urllib.error.URLError: <urlopen error local file error>
If that's OK I'll add a couple of tests and commit.

> This should be reported in another issue though.
Has this been done?

Nick reported #16247 today which is very similar.

For tracking purposes, I created issue 16250 for what you observed above, which as you said is related to but not the same as Nick's issue 16247.

@Ezio, regarding msg172988. That's embarrassing. It should be fixed soon.
Review on the patch, how about having the strerror and filename from the exception?
- raise URLError(e.errno, e.strerror, e.filename)
+ raise URLError(e.strerror, e.filename)
I believe, that could have been the original intention which got overlooked.
And yes to fixing the other URLError msg related issues too, with just changing to single msg first, instead of wrongly sending the filename.

Here is the patch which captures both HTTPError and URLError at the open_file and thus preventing multiple exceptions to be raised ( URLError and next IOError). This can go in 3.4 and since this is bug, where correct exception is not being caught and wrong args are sent, I think the catching of correct exception can be backported to old versions

New changeset daef5526d2ac by Senthil Kumaran in branch 'default':
Issue #10836: Fix exception raised when file not found in urlretrieve
http://hg.python.org/cpython/rev/daef5526d2ac 

This one is fixed in 3.4. I can see the previous versions raised IOError exception and it is not a good idea to change by backporting. But the wrong arguments on URLError should be fixed tough.

Should the bug be closed as it seems to be fixed now?

Nope, I have backport it to other versions. I shall close it then.

This has been backported.
3.2 5e71f2712076
3.3 30547e2cd04d