homepage

Proposed patch removes following deprecated re features:
* Three unused not documented functions: isident(), isdigit() and isname(). They were deprecated since Python 3.3 (issue14462).
* '\' + ASCII character now is error if this combination is not defined. This allows to add new control combinations without breaking compatibility. This was deprecated since Python 3.5 (issue23622).
* Support for re.LOCALE with string patterns or with re.ASCII. This was deprecated since Python 3.5 (issue22407).

New changeset 09d1af3fe332 by Serhiy Storchaka in branch 'default':
Issue #27030: Unknown escapes consisting of ``'\'`` and ASCII letter in
https://hg.python.org/cpython/rev/09d1af3fe332 

Thanks Jim for the review.

New changeset 8ed3880e94e5 by Serhiy Storchaka in branch 'default':
Issue #27030: The re.LOCALE flag now can be used only with bytes patterns.
https://hg.python.org/cpython/rev/8ed3880e94e5 

New changeset a2482e805dff by Martin Panter in branch '3.5':
Fix buggy RE "\parrot_example.py", uncovered by Issue #27030
https://hg.python.org/cpython/rev/a2482e805dff
New changeset be193f8dbe4c by Martin Panter in branch 'default':
Issue #27030: Merge RE fix from 3.5
https://hg.python.org/cpython/rev/be193f8dbe4c
New changeset c5411cfa0bd3 by Martin Panter in branch '2.7':
Fix buggy RE "\parrot_example.py", uncovered by Issue #27030
https://hg.python.org/cpython/rev/c5411cfa0bd3 

FWIW, this breaks Mailman 3.1 (and probably 2.1)

Specifically, point #2; undefined combinations of \ + ASCII becoming an error.

Could Mailman be fixed? Undefined combinations of \ + ASCII emitted warnings in 3.5. And now they emit warnings even just in string literals (issue27364). If Mailman use undefined escape combinations, it could suffer from issue27364 too.

On Nov 22, 2016, at 04:13 PM, Serhiy Storchaka wrote:
>Could Mailman be fixed? Undefined combinations of \ + ASCII emitted warnings
>in 3.5. And now they emit warnings even just in string literals
>(issue27364). If Mailman use undefined escape combinations, it could suffer
>from issue27364 too.
No, I think this is a valid bug/regression.
The Mailman code is basically trying to do this:
 p = re.compile('%\d*d')
 p.sub(r'\s*\d+\s*', some_string)
And so we get the error:
sre_constants.error: bad escape \s at position 0
But this directly contradicts the documentation for re.sub():
"... if it is a string, any backslash escapes in it are processed. That is, \n
is converted to a single newline character, \r is converted to a carriage
return, and so forth. Unknown escapes such as \& are left alone."
Clearly \s is not being left alone, so this is a real regression.

This part of the documentation was just overlooked. Issue28450 is opened for this.

If you insist I could revert converting warnings to errors (only in replacement string or all?) in 3.6. But I think they should left errors in 3.7. The earlier we make undefined escapes the errors, the earlier we can define new special escape sequences without confusing users. It is bad if the escape sequence is valid in two Python versions but has different meaning.

There is still the argument that we shouldn't break 2.7 compatibility unnecessarily until 2.7 is out of maintenance. That is: warnings are good, removals are bad. (I haven't read through this issue, so I may be off base.)

Here is a patch that partially reverts changes of issue27030. It allows using unknown escapes in re.sub() replacement template, but they are deprecated and will be errors in 3.7.
If this is good to you Barry, and Ned allows, it can be committed in 3.6 only.
But this could delay adding support of new escapes (like \xXX, \uXXXX, etc) in replacement templates.

It is unfortunate that the deprecation note did not explicitly mention replacement templates as well as regexp patterns. While there are good arguments to be made for either case, I think it makes more sense to treat the replacement template as following the rules for strings rather than for regexps which would argue for making the proposed change to cause only a deprecation warning in templates for 3.6 - although I'm not happy about making such a change at the last minute. Serhiy, please push to 3.6 for rc1. There should also be a note in the What's New for 3.6 document.

New changeset 1b162d6e3d01 by Serhiy Storchaka in branch '3.6':
Issue #27030: Unknown escapes in re.sub() replacement template are allowed
https://hg.python.org/cpython/rev/1b162d6e3d01
New changeset 5904d2ced3d8 by Serhiy Storchaka in branch 'default':
Merge documentation for issue #27030 from 3.6.
https://hg.python.org/cpython/rev/5904d2ced3d8 

Thanks, Serhiy, for reverting the error handling for 3.6.0 (in 3.6.0rc1). I'm going to mark this as closed. Issue28450 is still open at the moment regarding the documentation and possible 3.7 changes.