homepage

Yet one inconsistency in charmap codec.
>>> import codecs
>>> codecs.charmap_decode(b'\x00', 'strict', '\U0002000B')
('𠀋', 1)
>>> codecs.charmap_decode(b'\x00', 'strict', {0: '\U0002000B'})
('𠀋', 1)
>>> codecs.charmap_decode(b'\x00', 'strict', {0: 0x2000B})
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
TypeError: character mapping must be in range(65536)
The suggested patch removes this unnecessary limitation in charmap decoder.

Could you add a test to your patch?
Is the issue 3.3-specific?

Fixing for 3.2 and lesser is possible, but expensive, because of narrow build limitation. If necessary, I will give the patch, but it is easier to mark it as "wont fix" for pre-3.3 versions.
Here is a tests for charmap decoding. Tests added not only for this issue, but for all non-covered cases with int2str and int2str mappings.

In 3.2, narrow build is also broken when the "charmap" is a string:
>>> codecs.charmap_decode(b'0円', 'strict', '\U0002000B')
returns ('𠀋', 1) with a wide unicode build, but ('\ud840', 1) with a narrow build.
3.2 could be fixed to allow characters up to sys.maxunicode, though.

Well, here is a patch for 3.2.

About the patch for 3.2:
 "needed = 6 - extrachars"
Where does this 6 come from? There is another part which uses this "extrachars". Why is the allocation strategy different here?

It's the same strategy.
"needed = (targetsize - extrachars) + (targetsize << 2)". targetsize == 2.

Ah, I was worried by the possible quadratic behavior. So the other (existing) case is quadratic as well (I was mislead by the <<, which made me think there is something clever there).
That's good enough for 3.2, I guess.

Ping.

Patches updated. Added a few new tests, used MAX_UNICODE, a little changed extrachars grow step.

New changeset 620d23f7ad41 by Antoine Pitrou in branch '3.2':
Issue #15379: Fix passing of non-BMP characters as integers for the charmap decoder (already working as unicode strings).
http://hg.python.org/cpython/rev/620d23f7ad41
New changeset c64dec45d46f by Antoine Pitrou in branch 'default':
Issue #15379: Fix passing of non-BMP characters as integers for the charmap decoder (already working as unicode strings).
http://hg.python.org/cpython/rev/c64dec45d46f 

Thank you, I've committed the patches. There was a test failure in test_codeccallbacks in 3.2, which I fixed simply by replacing sys.maxunicode with a hardcoded 0x110000.

We forgot about 2.7 (because I had not thought to apply it even for a 3.2). Here is backported patch.

The 2.7 patch is just a backport of 3.2 patch (including the last Antoine's fix). Please look and commit.

New changeset c7ce91756472 by Antoine Pitrou in branch '2.7':
Issue #15379: Fix passing of non-BMP characters as integers for the charmap decoder (already working as unicode strings).
http://hg.python.org/cpython/rev/c7ce91756472 

Thanks for the backport, committed!