homepage

locale.format() doesn't insert correct decimal separator to string
representation when 'format' argument has '\r' or '\n' symbols in it.
This bug has been reproduced on Python 2.5.2 and svn-trunk.
Python 2.4.5 (#2, Mar 12 2008, 14:42:24)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "ru_RU.UTF-8")
'ru_RU.UTF-8'
>>> a = 1.234
>>> print locale.format("%f", a)
1,234000
>>> print locale.format("%f\n", a)
1,234000
>>> print locale.format("%f\r", a)
1,234000
Python 2.6a1+ (trunk:62083, Mar 31 2008, 19:24:56)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "ru_RU.UTF-8")
'ru_RU.UTF-8'
>>> a = 1.234
>>> print locale.format("%f", a)
1,234000
>>> print locale.format("%f\n", a)
1.234000
>>> print locale.format("%f\r", a)
1.234000
Python 2.5.2 (r252:60911, Mar 12 2008, 13:36:25)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "ru_RU.UTF-8")
'ru_RU.UTF-8'
>>> a = 1.234
>>> print locale.format("%f", a)
1,234000
>>> print locale.format("%f\n", a)
1.234000
>>> print locale.format("%f\r", a)
1.234000

I've uploaded a patch that fixes this concrete issue, though
locale.format() continues to silently ignore other types of malformed
strings (e.g. locale.format('%fSPAMf')).
I don't think this is correct behavior. Maybe there should be reg-exp
that locale.format() will use to avoid such issues.

This bug is more subtle than it first appears. As far as I've been able
to figure out, there is in fact no way to reliably detect that there is
non-format text after the format specifier short of completely parsing
the format specifier. I went through several possibilities and found a
counter example for each that shows it would introduce new bugs:
See if last character of formatted string is different from last char of
formatter: format('%s', 'things') would then incorrectly be an error.
Make sure last character of format string is a valid format character: 
format('%fx', a) would be valid, but it has a 'x' on the end which is
not part of the format string. (The suggested patch has a false
negative in this case as well.)
Check for a decimal in the formatted string and if it didn't get
transformed, complain: format('%s', '1.234') would fail incorrectly.
One could argue that at least \n and \r should be checked for since the
output from those cases is least obviously "wrong", but I don't think
that is a strong argument. The extra control character is almost as
"visible" in the output as the trailing 'x' would be in the above
example, and the effects of the trailing x are equally mysterious.
To fix this correctly would require reimplementing format parsing in the
locale module, which would be a maintenance headache.
I'm inclined to close this "won't fix", unless someone can come up with
a heuristic that won't give false positives or false negatives.

AFAIK, locale.format() is supposed to be used with a single format
specifier, not a complete format string. It's up to you to concatenate
the various parts afterwards.

That is true, however the code contains the comment "this is only for
one-percent-specifier strings and this should be checked", implying
that the intent is to make sure only a single format specifier
has been passed. I don't think it is reasonable to perfect that
check, however.

It occured to me last night that it could be checked using a regular
expression, and indeed the locale module already has a regular
expression that matches percent codes. I've uploaded a patch that uses
this regex to fix this issue. I've removed 2.6 and 3.0 as this change
could break existing code that is misusing format.
I added georg.brandl to the nosy list since svn blame shows him as the
author of the code being modified.

Nice to see this moving forward. Your patch looks nicer than my naive
approach and I hope it's going to be applied. Thanks for investigation. :)

Fixed in r70936/r70938.

Hmm. See bug 10379 for fallout from this change. I'm not saying it should be reverted but see that issue for further discussion.

I mean issue 10379