homepage

if i configure logging into a file with encoding = 'cp1251' and do
logger.debug(u'...') then i get crash with UnicodeError
i suggest reimplementing method FileHandler.emit():
...
if isinstance(msg, unicode):
 stream.write(f % msg) # it works!
...

The attached test script and output file appear to show logging working
correctly. The script writes a log message including the Cyrillic text
доброе утро (Good morning) to a CP1251-encoded file, test.log. Opening
this file in a Unicode-aware editor (I used BabelPad,
http://www.babelstone.co.uk/Software/BabelPad.html) appears to show the
Cyrillic characters correctly: see the screenshot at
http://img5.imageshack.us/img5/799/cp1251zw2.png
Marking as pending. Can you give more specifics about your problem?

test_log.py:
-----------
#! -*- coding: windows-1251 -*-
import logging
logger = logging.getLogger('test_log')
logger.addHandler(logging.FileHandler('test.log', encoding='cp1251'))
logger.setLevel(logging.DEBUG)
logger.debug(u'Привет') # russian Hello
exception:
---------
Traceback (most recent call last):
 File "e:\bin\python0円\lib\logging\__init__.py", line 765, in emit
 self.stream.write(fs % msg.encode("UTF-8"))
 File "e:\bin\python0円\lib\codecs.py", line 686, in write
 return self.writer.write(data)
 File "e:\bin\python0円\lib\codecs.py", line 351, in write
 data, consumed = self.encode(object, self.errors)
 File "e:\bin\python0円\lib\encodings\cp1251.py", line 12, in encode
 return codecs.charmap_encode(input,errors,encoding_table)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0:
ordinal not in range(128)

Sorry, misread the original issue and tested with Python 2.5 rather than
2.6. This is a regression; fix and additional test case checked into
trunk and release26-maint.

(python 2.6.2, WinXP)
logging to console stopped working. Here is a workaround:
logging/__init__.py:
class StreamHandler(Handler):
 ...
 def emit(self, record):
 ...
 if (isinstance(msg, unicode) or
 getattr(stream, 'encoding', None) is None):
- stream.write(fs % msg)
+ if stream == sys.stdout:
+ print(msg)
+ else:
+ stream.write(fs % msg)

Can you retry with setting the "encoding" attribute of the file to
"cp1251"? That should work and that should be the appropriate method to
avoid the problem.
test_logging.py in the Python distribution has a test which exercises
Unicode functionality using cp1251, does that test work in your environment?

>
> Can you retry with setting the "encoding" attribute of the file to
> "cp1251"? That should work and that should be the appropriate method to
> avoid the problem.
>
> test_logging.py in the Python distribution has a test which exercises
> Unicode functionality using cp1251, does that test work in your
> environment?
logging to file is OK (since 2.6.2), logging to console __was__ OK (2.6.1),
not now (2.6.2)
shamil

Trunk and release26-maint were recently changed (r71657, r71658) to use
the following logic, which differs from the code snippet you posted.
 if (isinstance(msg, unicode) and
 getattr(stream, 'encoding', None)):
 stream.write(fs.decode(stream.encoding) % msg)
 else:
 stream.write(fs % msg)
If the stream is stderr and you are passing a unicode msg, the else
branch will not be taken; as long as the stream has an encoding
attribute, it should output correctly.
The change was made when another, similar issue was posted by another
user (issue #5768).
Can you confirm what happens with the current code as it is in
release26-maint? спасибо!

>
> Trunk and release26-maint were recently changed (r71657, r71658) to use
> the following logic, which differs from the code snippet you posted.
>
> if (isinstance(msg, unicode) and
> getattr(stream, 'encoding', None)):
> stream.write(fs.decode(stream.encoding) % msg)
> else:
> stream.write(fs % msg)
>
> If the stream is stderr and you are passing a unicode msg, the else
> branch will not be taken; as long as the stream has an encoding
> attribute, it should output correctly.
>
> The change was made when another, similar issue was posted by another
> user (issue #5768).
>
> Can you confirm what happens with the current code as it is in
> release26-maint? спасибо!
it still doesn't work for console (but OK for files).
the following works in both cases:
if (isinstance(msg, unicode) and
 getattr(stream, 'encoding', None) and
 (stream == sys.stdout or stream == sys.stderr)):
 stream.write(fs % msg.encode(stream.encoding))
else:
 stream.write(fs % msg)
i think it's all about the difference betwin print(msg) and
sys.stdout.write('%s\n' % msg)
shamil

> i think it's all about the difference betwin print(msg) and
> sys.stdout.write('%s\n' % msg)
Yes. Actually, it's about what streams will accept. For example, a
stream opened with codecs.open(encoding='cp1251') will accept a Unicode
string and encode it before output. Likewise, a StringIO wrapped inside
a writer obtained from the codecs module.
However, stdout and stderr are of type file and seem to behave
differently - they behave as expected if I encode the value written to
them, but fail if I write a Unicode string to them.
I don't think special-casing for sys.stdout and sys.stderr in the
logging code is the correct approach. Better to put a conditional
encoding step when it's needed - now I just need to figure out all the
cases when it's needed ;-)

Fix checked into trunk and release26-maint.