homepage

I now have a working PEP 3151 implementation, working on various platforms (tested on Linux, Windows, FreeBSD and OpenIndiana buildbots).
The implementation has all the semantic changes and additions in PEP 3151. It still lacks the deprecation mechanism mentioned in "step 1" of the PEP.
The implementation is at http://hg.python.org/features/pep-3151/ in the branch named "pep-3151".

(I'm sorry for the filenames. I'm using the "create patch" feature in the hope that a code review is possible through the Rietveld integration :-))

The patch looks good. I can’t comment on details of the C code, but I’ve seen the changed Python code and the tests and I like this.

Updated patch with latest changes from the PEP. The implementation is now complete.

Hi,
I have tested on OpenBSD -current, and it seems to work fine:
$ ./python -E -bb Lib/test/test_mmap.py 
test_access_parameter (__main__.MmapTests) ... ok
test_anonymous (__main__.MmapTests) ... ok
test_bad_file_desc (__main__.MmapTests) ... ok
test_basic (__main__.MmapTests) ... ok
test_context_manager (__main__.MmapTests) ... ok
test_context_manager_exception (__main__.MmapTests) ... ok
test_double_close (__main__.MmapTests) ... ok
test_entire_file (__main__.MmapTests) ... ok
test_error (__main__.MmapTests) ... ok
test_extended_getslice (__main__.MmapTests) ... ok
test_extended_set_del_slice (__main__.MmapTests) ... ok
test_find_end (__main__.MmapTests) ... ok
test_io_methods (__main__.MmapTests) ... ok
test_length_0_large_offset (__main__.MmapTests) ... ok
test_length_0_offset (__main__.MmapTests) ... ok
test_move (__main__.MmapTests) ... ok
test_non_ascii_byte (__main__.MmapTests) ... ok
test_offset (__main__.MmapTests) ... ok
test_prot_readonly (__main__.MmapTests) ... ok
test_read_all (__main__.MmapTests) ... ok
test_read_invalid_arg (__main__.MmapTests) ... ok
test_rfind (__main__.MmapTests) ... ok
test_subclass (__main__.MmapTests) ... ok
test_tougher_find (__main__.MmapTests) ... ok
test_around_2GB (__main__.LargeMmapTests) ... ok
test_around_4GB (__main__.LargeMmapTests) ... ok
test_large_filesize (__main__.LargeMmapTests) ... ok
test_large_offset (__main__.LargeMmapTests) ... ok
----------------------------------------------------------------------
Ran 28 tests in 0.141s
OK
$ ./python -E -bb Lib/test/test_exceptions.py
...........................
----------------------------------------------------------------------
Ran 27 tests in 0.026s
OK
$ ./python -E -bb Lib/test/test_pep3151.py
test_blockingioerror (__main__.AttributesTest) ... ok
test_errno_translation (__main__.AttributesTest) ... skipped 'Windows-specific test'
test_posix_error (__main__.AttributesTest) ... ok
test_windows_error (__main__.AttributesTest) ... ok
test_builtin_errors (__main__.HierarchyTest) ... ok
test_errno_mapping (__main__.HierarchyTest) ... ok
test_ioerror_subclass_mapping (__main__.HierarchyTest) ... ok
test_select_error (__main__.HierarchyTest) ... ok
test_socket_errors (__main__.HierarchyTest) ... ok
----------------------------------------------------------------------
Ran 9 tests in 0.001s
OK (skipped=1)
Don't hesitate to indicate me if you need more tests, or if I did something wrong.
Cheers,
Remi.

New patch incorporating Ezio's comments and synchronized with latest default.

Here is a new patch implementing the latest PEP changes.

I still need to understand the full impact of the PEP, but it seems
very sound. I've left two small comments on Rietveld, and there's
an additional question:
Should the input of OSError be checked?
>>> OSError("not an errno", "Bad file descriptor")
OSError('not an errno', 'Bad file descriptor')
>>> o = OSError("not an errno", "Bad file descriptor")
>>> raise o
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
OSError: [Errno not an errno] Bad file descriptor
>>>

> Should the input of OSError be checked?
It could, but pre-PEP it is not, so I assumed it's better to minimize compatibility-breaking changes.

Patch update against latest default. There shouldn't be anything interesting to see.

New changeset 41a1de81ef2b by Antoine Pitrou in branch 'default':
PEP 3151 / issue #12555: reworking the OS and IO exception hierarchy.
http://hg.python.org/cpython/rev/41a1de81ef2b 

Thanks everyone for the reviews!

Is the following change in behavior caused by the fix for this issue?
$ python3.2 -c $'class A(IOError):\n def __init__(self, arg): pass\nA(arg=1)'
$ python3.3 -c $'class A(IOError):\n def __init__(self, arg): pass\nA(arg=1)'
Traceback (most recent call last):
 File "<string>", line 3, in <module>
TypeError: A does not take keyword arguments

Indeed, this seems to be the most likely culprit for the current buildbot failures in the new urllib2 tests:
http://www.python.org/dev/buildbot/all/builders/AMD64%20Gentoo%20Wide%203.x/builds/2868/steps/test/logs/stdio 

New changeset a3ddee916808 by Jason R. Coombs in branch 'default':
Pass positional arguments - HTTPError is not accepting keyword arguments. Reference #13211 and #12555.
http://hg.python.org/cpython/rev/a3ddee916808 

> Is the following change in behavior caused by the fix for this issue?
> 
> $ python3.2 -c $'class A(IOError):\n def __init__(self, arg): pass\nA(arg=1)'
> $ python3.3 -c $'class A(IOError):\n def __init__(self, arg): pass\nA(arg=1)'
> Traceback (most recent call last):
> File "<string>", line 3, in <module>
> TypeError: A does not take keyword arguments
It must be because IOError now has a significant __new__ method.
I could change it to accept arbitrary arguments but I'm not sure that's
the right solution.
Another approach would be:
- if IOError is instantiated, initialize stuff in IOError.__new__
- otherwise, initialize stuff in IOError.__init__
What do you think?

> > Is the following change in behavior caused by the fix for this issue?
> > 
> > $ python3.2 -c $'class A(IOError):\n def __init__(self, arg): pass\nA(arg=1)'
> > $ python3.3 -c $'class A(IOError):\n def __init__(self, arg): pass\nA(arg=1)'
> > Traceback (most recent call last):
> > File "<string>", line 3, in <module>
> > TypeError: A does not take keyword arguments
> 
> It must be because IOError now has a significant __new__ method.
> I could change it to accept arbitrary arguments but I'm not sure that's
> the right solution.
> Another approach would be:
> - if IOError is instantiated, initialize stuff in IOError.__new__
> - otherwise, initialize stuff in IOError.__init__
To make things clearer, IOError.__new__ would detect if a subclass is
asked for, and then defer initialization until __init__ is called so
that argument checking is done in __init__.

There's a fairly sophisticated tapdance in object.__new__ that deals with this problem at that level.
See: http://hg.python.org/cpython/file/default/Objects/typeobject.c#l2869
The new IOError may require something similarly sophisticated to cope with subclasses that only override __init__.

Here is a patch. Can someone take a look?

Updated patch. I've renamed oserror_post_init to oserror_init. Also addressed Nick's other comments.

New changeset d22c99e77768 by Antoine Pitrou in branch 'default':
Fix OSError.__init__ and OSError.__new__ so that each of them can be
http://hg.python.org/cpython/rev/d22c99e77768 

Fixed now.