homepage

Hi, in switching to Python 3.0 I've run into an issue with displaying
Unicode characters via curses. In Python 2.x a simple hello-world looks
like:
#!/usr/bin/python
# coding=UTF-8
import curses
import locale
locale.setlocale(locale.LC_ALL,"")
def doStuff(stdscr):
 message = u"hello わたし!"
 stdscr.addstr(0, 0, message.encode("utf-8"), curses.A_BLINK)
 stdscr.getch() # pauses until a key's hit
curses.wrapper(doStuff)
This works. However, when I try to come up with an equivalent for Python
3.0:
#!/usr/bin/python
import curses
import locale
locale.setlocale(locale.LC_ALL,"")
def doStuff(stdscr):
 message = "hello わたし!"
 stdscr.addstr(0, 0, message, curses.A_BLINK)
 stdscr.getch() # pauses until a key's hit
curses.wrapper(doStuff)
It fails (printing gibberish to the console). It seems that the problem
is that the curses module isn't respecting the system's preferred
encoding (utf-8) which was set via the setlocale function (as per
instructions at
http://docs.python.org/dev/3.0/library/curses.html#module-curses).
My apologies in advance if this is my mistake. Cheers! -Damian

I tested your example on Linux (Ubuntu Gutsy) and it works correctly. 
What is your:
- OS (name, version)
- locale (charset?)

My OS is Ubuntu 8.04 (Hardy) and the locale is utf-8:
>>> locale.setlocale(locale.LC_ALL,"")
'en_US.UTF-8'
You do mean the Python 3.0 example didn't work, right? The Python3.0
header is:
Python 3.0 (r30:67503, Dec 21 2008, 02:16:52) 
[GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)] on linux2
in case that makes a difference.

Ack - sorry, typo. I meant "You do mean the Python 3.0 example did work,
right?"

> You do mean the Python 3.0 example didn't work, right?
I only tested yje Python3 example and it works correctly on my 
computer. I'm using Python3 trunk but I don't think that the curses 
module changed after the 3.0 release.
$ ./python
Python 3.1a0 (py3k:67973M, Dec 28 2008, 17:53:52)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
>>> import locale; locale.setlocale(locale.LC_ALL, '')
'fr_FR.UTF-8'

Doing a checkout of the trunk - I'll let you know if it works. Thanks!

I think that I catched the problem:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=379447
_curses.so should be linked to libncursesw.so.5 and not 
libncurses.so.5. I tested on Hardy and I doesn't work because 
_curses.so was linked to libncurses.so.5.
Few commands...
 sudo apt-get install libncursesw5-dev
 make clean
 make
and it works ;-) Possible workaround: reject non-unicode ncurses 
library in setup.py. The ugly patch:
Index: setup.py
===================================================================
--- setup.py (révision 67973)
+++ setup.py (copie de travail)
@@ -848,23 +848,6 @@
 panel_library = 'panelw'
 exts.append( Extension('_curses', ['_cursesmodule.c'],
 libraries = curses_libs) )
- elif (self.compiler.find_library_file(lib_dirs, 'ncurses')):
- curses_libs = ['ncurses']
- exts.append( Extension('_curses', ['_cursesmodule.c'],
- libraries = curses_libs) )
- elif (self.compiler.find_library_file(lib_dirs, 'curses')
- and platform != 'darwin'):
- # OSX has an old Berkeley curses, not good enough for
- # the _curses module.
- if 
(self.compiler.find_library_file(lib_dirs, 'terminfo')):
- curses_libs = ['curses', 'terminfo']
- elif 
(self.compiler.find_library_file(lib_dirs, 'termcap')):
- curses_libs = ['curses', 'termcap']
- else:
- curses_libs = ['curses']
-
- exts.append( Extension('_curses', ['_cursesmodule.c'],
- libraries = curses_libs) )
 else:
 missing.append('_curses')
Ugly because it may break the curses module compilation on many OS 
(BSD and Solaris?).

See also the old (and closed) issue #1428494: Prefer linking against 
ncursesw over ncurses library.

Just finished recompiling and works perfectly. My hat's off to you -
many thanks! -Damian

ncursesw looks to be available for:
 - Linux: eg. packaged in Ubunbut
 - NetBSD: 
ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/devel/ncursesw/README.html
 - FreeBSD: ncurses package with USE="unicode"
 - Mac OS X: http://ncursesw.darwinports.com/
 - Windows: by Cygwin (libncurses5 package)
I think that libncursesw is working well on Linux and BSD, but Windows 
port may be buggy.

For Solaris, see this bug report of nano editor:
https://savannah.gnu.org/bugs/?24028
libcurses has been added to OpenSolaris in Septembre 2008. But it 
don't see the unicode version (libncusesw) :-/

Looks like this was my mistake, not a bug. According to:
http://mail.python.org/pipermail/python-list/2007-July/450133.html
Python 2.5 also requires the addition of libcursesw but it was working
for the Ubuntu release because they specifically added it. There's no
missing functionality - just my mistake. Cheers! -Damian

> Python 2.5 also requires the addition of libcursesw 
> but it was working for the Ubuntu release because 
> they specifically added it.
What do you mean by "the addition of libcursesw"? _curses.so of Python 
2.5 is linked to libncursesw.so on Ubuntu Gutsy and Hardy.
If _curses.so is linked to libncurses.so, you can not used multibyte 
charset like UTF-8.

On FreeBSD there's no need for the ncurses package in most case. Since
somewhere along 6.x and 7.x line Rong-En Fan switched FreeBSD to do what
I did for DragonFly BSD a long time: to have both normal and wide curses
available.
As far as Python is concerned, it supports linking to ncursesw, so I
think this issue can be safely closed since it is not a problem within
Python (as also the original author said).
If this is not ok, feel free to reopen.