homepage

test_gdb started failing on me within the past few days. (I would have reported it sooner but I figured the omniscient CPython core dev community already knew about it.)
The machine it's failing on is Ubuntu 13.10, 64-bit.

Please see the attached output from the test run. But here's a sample:
======================================================================
FAIL: test_lists (test.test_gdb.PrettyPrintTests)
Verify the pretty-printing of lists
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/larry/src/python/nodocstrings/Lib/test/test_gdb.py", line 258, in test_lists
 self.assertGdbRepr(list(range(5)))
 File "/home/larry/src/python/nodocstrings/Lib/test/test_gdb.py", line 228, in assertGdbRepr
 cmds_after_breakpoint)
 File "/home/larry/src/python/nodocstrings/Lib/test/test_gdb.py", line 196, in get_gdb_repr
 import_site=import_site)
 File "/home/larry/src/python/nodocstrings/Lib/test/test_gdb.py", line 179, in get_stack_trace
 self.assertEqual(unexpected_errlines, [])
AssertionError: Lists differ: ["Python Exception <class 'UnicodeDecodeEr[197 chars]e: "] != []
First list contains 2 additional elements.
First extra element 0:
Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0xfc in position 1: invalid start byte: 
+ []
- ["Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode "
- 'byte 0xfc in position 1: invalid start byte: ',
- "Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode "
- 'byte 0xfc in position 1: invalid start byte: ']

> test_gdb started failing on me within the past few days
Can you please bisect to find the offending changeset?

The first bad revision is:
changeset: 87426:9f2a0043396b
user: Antoine Pitrou <solipsis@pitrou.net>
date: Sat Nov 23 17:40:36 2013 +0100
files: Lib/test/test_gdb.py Tools/gdb/libpython.py
description:
Issue #19308: fix the gdb plugin on gdbs linked with Python 3

If you go back one revision, test_gdb isn't run at all, right?
(it should be skipped)

That's correct, it is skipped.

Ok, it seems the failures are due to a non-pydebug build. I'll investigate tomorrow.

The test passes if I turn on "--with-pydebug". And Antoine told me in IRC that he sees the failure if he turns off "--with-pydebug". So we're in business.

New changeset 7a14cde3c4df by Antoine Pitrou in branch 'default':
Issue #19743: fix test_gdb on some optimized Python builds
http://hg.python.org/cpython/rev/7a14cde3c4df 

New changeset e42b449f73fd by Antoine Pitrou in branch '3.3':
Issue #19743: fix test_gdb on some optimized Python builds
http://hg.python.org/cpython/rev/e42b449f73fd 

Here's what I've figured out. If I build trunk then run:
% gdb --batch -iex "add-auto-load-safe-path /home/larry/src/python/buildtrunk/python-gdb.py" --eval-command="set breakpoint pending yes" --eval-command="break builtin_id" --eval-command=run --eval-command=backtrace --args /home/larry/src/python/buildtrunk/python -S -c "id([0, 1, 2, 3, 4])"
I see UnicodeDecodeErrors spat out in the middle of the traceback:
#0 builtin_id (self=<module at remote 0x7ffff7fbfc28>, v=[0, 1, 2, 3, 4]) at Python/bltinmodule
.c:991
#1 0x00000000004d8695 in call_function (oparg=<optimized out>, pp_stack=0x7fffffffdc00) at Pyth
on/ceval.c:4212
#2 PyEval_EvalFrameEx (f=f@entry=Frame 0x7ffff7f3e788, for file <string>, line 1, in <module> (
), throwflag=throwflag@entry=0) at Python/ceval.c:2826
Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0x80 in position 0
: invalid start byte: 
Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0x80 in position $: invalid start byte: 
#3 0x00000000004d9809 in PyEval_EvalCodeEx (closure=0x0, [...]
If I run the same sequence of commands manually (remove the --batch and the batched commands from the gdb command-line, then type them in myself) I still see the decode error.
If I then set a breakpoint on PyUnicodeDecodeError_Create() and run backtrace again, it still prints the decode errors, but it doesn't stop on the breakpoint. (I guess this would be like getting to the assault on the ski chalet in Inception?)
Most worrisome of all: if I change the Makefile from -O3 to -O0, the decode errors go away.

Sorry, both exceptions in the output of the traceback are:
Python Exception <class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte:

Antoine: your checked-in workaround makes the test pass. But I'm leaving this open because I'm still hoping David will take a look into it.

Agreed.

FWIW, I feel that it's worth just expecting failures with an *optimized* build: with an optimizing compiler, there's likely to always be some program counter location where the debugger is going to get confused for some variables. Given umpteen different compiler variants, debugger variants etc, this is never going to be perfect.
So IMHO we should run the test on a --pydebug build, but accept that there will be some failures when optimization is on.

If skipping them for optimized builds is the right call, then aren't we done here? Can we close this bug as fixed?

Seems as if this can be closed.