homepage

While working on the regex code in sre_compile.py I came across the
following code in the handling of charset ranges in _optimize_charset:
 for i in range(fixup(av[0]), fixup(av[1])+1):
 charmap[i] = 1
The function fixup converts the ends of the range to lower case if the
ignore-case flag is present. The problem with this approach is
illustrated below:
>>> import re
>>> print re.match(r'[9-A]', 'A')
<_sre.SRE_Match object at 0x00A78058>
>>> print re.match(r'[9-A]', 'a')
None
>>> print re.match(r'[9-A]', '_')
None
>>> print re.match(r'[9-A]', 'A', re.IGNORECASE)
<_sre.SRE_Match object at 0x00D0BFA8>
>>> print re.match(r'[9-A]', 'a', re.IGNORECASE)
<_sre.SRE_Match object at 0x00A78058>
>>> print re.match(r'[9-A]', '_', re.IGNORECASE)
<_sre.SRE_Match object at 0x00D0BFA8>
>>> 
'_' doesn't lie between '9' and 'A', but it does lie between '9' and 'a'.
Surely the ignore-case flag should not affect whether non-letters are
matched or not?

I think this is even more complicated when you consider that
localization my be an issue. Consider "Á": is this grammatically before
 "A" or after "a"? From a character set point of view, it is typically
after "a" but when Locale is taken into account, all that is done is
there is a change to relative ordering, so Á appears somewhere before A
and B. But when this is done, does that mean that [9-Á] is going to
cover ALL uppercase and ALL lowercase and ALL characters with ord from
91 to 96 and 123 to 127 and all kinds of other UNICODE symbols? And how
will this effect case-insensitivity.
In a sense, I think it may only be safe to say that character class
ranges are ONLY appropriate over Alphabetic character ranges or numeric
character ranges, since the order of the ASCII symbols between 0 and 47,
56 and 64, 91 adn 96 and 123 and 127, though well-defined, are none the
less implementation dependent. When we bring UNICODE into this, things
get even more befuddled with some Latin characters in Latin-1, some in
Latin-2, Cyrillic, Hebrew, Arabic, Chinese, Japanese and Korean
character sets just to name a few of the most common! And how does a
total ordering of characters apply to them?
In the end, I think it's just dangerous to define character group ranges
that span the gap BETWEEN numbers and alphabetics. Instead, I think a
better solution is simply to implement Emacs / Perl style named
character classes as in issue 2636 sub-item 8.
I do agree this is a problem, but as I see it, the solution may not be
that simple, especially in a UNICODE world.

I'd close this as "won't fix", because (IMHO) ranges like [9-A]
shouldn't be used at all, so I won't expect it to work properly.
FWIW Perl doesn't seem to match the '_', even with the 'i' flag. Tested
with: perl -e '$s = ("_" =~ /[9-A]/); print $s' and perl -e '$s = ("_"
=~ /[9-A]/i); print $s'. It matches ":" with [9-A] and "_" with [9-a]
though (both without the 'i' flag).

"[9-A]" is equivalent to "[9:;<=>?@A]", or should be.
It'll be fixed in issue #2636.

If there's already a patch, then it's fine (and useful for ranges of
Unicode chars), but I wouldn't like to find a range like [9-A] around ;)

EM and MB seemed to agree on closing this.

Fixed in issue17381 (which has more realistic example than [9-A]).