homepage

When attempting to raise an error with a message containing non-BMP characters (Unicode ordinals above U+0xFFFF), the python shell displays a long traceback including several UnicodeEncodeErrors and then restarts. Example:
>>> raise Exception('\U0001f603')
Traceback (most recent call last):
 File "<pyshell#84>", line 1, in <module>
 raise Exception('\U0001f603')
Traceback (most recent call last):
 File "<pyshell#84>", line 1, in <module>
 raise Exception('\U0001f603')
Traceback (most recent call last):
 File "D:\Python37\lib\idlelib\run.py", line 474, in runcode
 exec(code, self.locals)
 File "<pyshell#84>", line 1, in <module>
Traceback (most recent call last):
 File "D:\Python37\lib\idlelib\run.py", line 474, in runcode
 exec(code, self.locals)
 File "<pyshell#84>", line 1, in <module>
Exception: 
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "D:\Python37\lib\idlelib\run.py", line 144, in main
 ret = method(*args, **kwargs)
 File "D:\Python37\lib\idlelib\run.py", line 486, in runcode
 print_exception()
 File "D:\Python37\lib\idlelib\run.py", line 234, in print_exception
 print_exc(typ, val, tb)
 File "D:\Python37\lib\idlelib\run.py", line 232, in print_exc
 print(line, end='', file=efile)
 File "D:\Python37\lib\idlelib\run.py", line 362, in write
 return self.shell.write(s, self.tags)
 File "D:\Python37\lib\idlelib\rpc.py", line 608, in __call__
 value = self.sockio.remotecall(self.oid, self.name, args, kwargs)
 File "D:\Python37\lib\idlelib\rpc.py", line 220, in remotecall
 return self.asyncreturn(seq)
 File "D:\Python37\lib\idlelib\rpc.py", line 251, in asyncreturn
 return self.decoderesponse(response)
 File "D:\Python37\lib\idlelib\rpc.py", line 271, in decoderesponse
 raise what
UnicodeEncodeError: 'UCS-2' codec can't encode characters in position 11-11: Non-BMP character not supported in Tk
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "D:\Python37\lib\idlelib\run.py", line 158, in main
 print_exception()
 File "D:\Python37\lib\idlelib\run.py", line 234, in print_exception
 print_exc(typ, val, tb)
 File "D:\Python37\lib\idlelib\run.py", line 220, in print_exc
 print_exc(type(context), context, context.__traceback__)
 File "D:\Python37\lib\idlelib\run.py", line 232, in print_exc
 print(line, end='', file=efile)
 File "D:\Python37\lib\idlelib\run.py", line 362, in write
 return self.shell.write(s, self.tags)
 File "D:\Python37\lib\idlelib\rpc.py", line 608, in __call__
 value = self.sockio.remotecall(self.oid, self.name, args, kwargs)
 File "D:\Python37\lib\idlelib\rpc.py", line 220, in remotecall
 return self.asyncreturn(seq)
 File "D:\Python37\lib\idlelib\rpc.py", line 251, in asyncreturn
 return self.decoderesponse(response)
 File "D:\Python37\lib\idlelib\rpc.py", line 271, in decoderesponse
 raise what
UnicodeEncodeError: 'UCS-2' codec can't encode characters in position 11-11: Non-BMP character not supported in Tk
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "<string>", line 1, in <module>
 File "D:\Python37\lib\idlelib\run.py", line 162, in main
 traceback.print_exception(type, value, tb, file=sys.__stderr__)
 File "D:\Python37\lib\traceback.py", line 105, in print_exception
 print(line, file=file, end="")
 File "D:\Python37\lib\idlelib\run.py", line 362, in write
 return self.shell.write(s, self.tags)
 File "D:\Python37\lib\idlelib\rpc.py", line 608, in __call__
 value = self.sockio.remotecall(self.oid, self.name, args, kwargs)
 File "D:\Python37\lib\idlelib\rpc.py", line 220, in remotecall
 return self.asyncreturn(seq)
 File "D:\Python37\lib\idlelib\rpc.py", line 251, in asyncreturn
 return self.decoderesponse(response)
 File "D:\Python37\lib\idlelib\rpc.py", line 271, in decoderesponse
 raise what
UnicodeEncodeError: 'UCS-2' codec can't encode characters in position 11-11: Non-BMP character not supported in Tk
=============================== RESTART: Shell ===============================
>>>
I presume the error is caused by Tk being unable to display the characters in the error message, but being forced to anyway by the traceback, causing a series of UnicodeEncodeErrors. Perhaps the error handler should use repr() or similar methods to convert the message into a displayable form?

You are correct about reason. This is essentially a duplicate of #22742, but I am leaving it open for now because the behavior is worse and there is additional information in the tracebacks. The lines for IDLE internals are normally removed, but the exception during the exception prevented that.

Mostly fixed by PR 16545 (see issue13153). The original example is now passed.
>>> raise Exception('\U0001f603')
Traceback (most recent call last):
 File "<pyshell#1>", line 1, in <module>
 raise Exception('\U0001f603')
Exception: 😃
>>> 
But you can still cause a restart of the shell by using surrogate characters:
>>> raise Exception('\ud800')
Traceback (most recent call last):
 File "<pyshell#0>", line 1, in <module>
 raise Exception('\ud800')
Traceback (most recent call last):
 File "<pyshell#0>", line 1, in <module>
 raise Exception('\ud800')
================================ RESTART: Shell ================================
>>>

As with #22742, the problem, without PR 16583, only occurs with *nix. On Windows, the traceback ends with "Exception: �" and a prompt is printed.

New changeset b690a2759e62d9ee0b6ea1b20e8f7e4b2cdbf8bb by Serhiy Storchaka in branch 'master':
bpo-36698: IDLE no longer fails when write non-encodable characters to stderr. (GH-16583)
https://github.com/python/cpython/commit/b690a2759e62d9ee0b6ea1b20e8f7e4b2cdbf8bb

New changeset a1f45008f190b0d340ec0eac623f05ef98e6b4c9 by Miss Islington (bot) in branch '3.8':
bpo-36698: IDLE no longer fails when write non-encodable characters to stderr. (GH-16583)
https://github.com/python/cpython/commit/a1f45008f190b0d340ec0eac623f05ef98e6b4c9

New changeset 4f962ecfa1aadc45facc250841208f6dd2ce690f by Miss Islington (bot) in branch '3.7':
bpo-36698: IDLE no longer fails when write non-encodable characters to stderr. (GH-16583)
https://github.com/python/cpython/commit/4f962ecfa1aadc45facc250841208f6dd2ce690f