homepage

The json decoder doesn't pass tests when the C optimizations in the _json module aren't available. The actual test failures appear fairly superficial, but aren't the limit of the problems.
Empty objects are converted to empty lists instead of empty dictionaries when _json isn't available; test attached.
Issue 5723 notes that the pure-Python implementation isn't tested; fixing that might have avoided these problems.
Test failures from Python 2.6.5:
======================================================================
ERROR: test_c_encode_basestring_ascii (json.tests.test_encode_basestring_ascii.TestEncodeBaseStringAscii)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_encode_basestring_ascii.py", line 29, in test_c_encode_basestring_ascii
 self._test_encode_basestring_ascii(json.encoder.c_encode_basestring_ascii)
 File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_encode_basestring_ascii.py", line 32, in _test_encode_basestring_ascii
 fname = encode_basestring_ascii.__name__
AttributeError: 'NoneType' object has no attribute '__name__'
======================================================================
ERROR: test_c_scanstring (json.tests.test_scanstring.TestScanString)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_scanstring.py", line 13, in test_c_scanstring
 self._test_scanstring(json.decoder.c_scanstring)
 File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_scanstring.py", line 17, in _test_scanstring
 scanstring('"z\\ud834\\udd20x"', 1, None, True),
TypeError: 'NoneType' object is not callable
======================================================================
FAIL: test_encode_basestring_ascii (json.tests.test_speedups.TestSpeedups)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_speedups.py", line 13, in test_encode_basestring_ascii
 self.assertEquals(encoder.encode_basestring_ascii.__module__, "_json")
AssertionError: 'json.encoder' != '_json'
======================================================================
FAIL: test_scanstring (json.tests.test_speedups.TestSpeedups)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_speedups.py", line 9, in test_scanstring
 self.assertEquals(decoder.scanstring.__module__, "_json")
AssertionError: 'json.decoder' != '_json'
Test failures from 2.7:
======================================================================
ERROR: test_c_scanstring (json.tests.test_scanstring.TestScanString)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/fdrake/src/Python-2.7/Lib/json/tests/test_scanstring.py", line 13, in test_c_scanstring
 self._test_scanstring(json.decoder.c_scanstring)
 File "/home/fdrake/src/Python-2.7/Lib/json/tests/test_scanstring.py", line 17, in _test_scanstring
 scanstring('"z\\ud834\\udd20x"', 1, None, True),
TypeError: 'NoneType' object is not callable
======================================================================
FAIL: test_encode_basestring_ascii (json.tests.test_speedups.TestSpeedups)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/fdrake/src/Python-2.7/Lib/json/tests/test_speedups.py", line 12, in test_encode_basestring_ascii
 self.assertEquals(encoder.encode_basestring_ascii.__module__, "_json")
AssertionError: 'json.encoder' != '_json'
======================================================================
FAIL: test_scanstring (json.tests.test_speedups.TestSpeedups)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/fdrake/src/Python-2.7/Lib/json/tests/test_speedups.py", line 8, in test_scanstring
 self.assertEquals(decoder.scanstring.__module__, "_json")
AssertionError: 'json.decoder' != '_json'

The attached test only fails for Python 2.7; removing Python 2.6 from this issue.

For what it's worth, I don't think any of the failures mentioned here are in the current version of simplejson. There aren't any tests that check the speedup module name, the speedup tests are skipped when speedups are not detected, and the empty json object bug was fixed in June 2009.
http://code.google.com/p/simplejson/issues/detail?id=57 

Just a note from downstream with PyPy: we've cherry-picked the commit Bob linked into our copy of the 2.7 stdlib since we don't have an _json (yet).

I ran the tests again on 2.7 and got 3 failures:
======================================================================
ERROR: test_c_scanstring (json.tests.test_scanstring.TestScanString)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/wolf/dev/py/issue9233/Lib/json/tests/test_scanstring.py", line 13, in test_c_scanstring
 self._test_scanstring(json.decoder.c_scanstring)
 File "/home/wolf/dev/py/issue9233/Lib/json/tests/test_scanstring.py", line 17, in _test_scanstring
 scanstring('"z\\ud834\\udd20x"', 1, None, True),
TypeError: 'NoneType' object is not callable
======================================================================
FAIL: test_encode_basestring_ascii (json.tests.test_speedups.TestSpeedups)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/wolf/dev/py/issue9233/Lib/json/tests/test_speedups.py", line 14, in test_encode_basestring_ascii
 self.assertEqual(encoder.encode_basestring_ascii.__module__, "_json")
AssertionError: 'json.encoder' != '_json'
======================================================================
FAIL: test_scanstring (json.tests.test_speedups.TestSpeedups)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/wolf/dev/py/issue9233/Lib/json/tests/test_speedups.py", line 11, in test_scanstring
 self.assertTrue(decoder.scanstring is decoder.c_scanstring)
AssertionError: False is not true
The attached patch checks if _json is importable and skip the tests if it's not.

New changeset 500063f6ae5a by Ezio Melotti in branch '2.7':
#9233: skip _json-specific tests when _json is not available.
http://hg.python.org/cpython/rev/500063f6ae5a 

Do those new check finish this issue and should it be closed?
If not, what is left?

New changeset d58c63ff5bb2 by Ezio Melotti in branch '2.7':
#9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
http://hg.python.org/cpython/rev/d58c63ff5bb2 

New changeset 36e676a8541d by Ezio Melotti in branch '3.1':
#9233: skip _json-specific tests when _json is not available.
http://hg.python.org/cpython/rev/36e676a8541d
New changeset 7019fc1a9663 by Ezio Melotti in branch '3.1':
#9233: Fix json to work properly even when _json is not available.
http://hg.python.org/cpython/rev/7019fc1a9663
New changeset a220458179ed by Ezio Melotti in branch '3.1':
#9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
http://hg.python.org/cpython/rev/a220458179ed 

Fixed in 2.7 and 3.1 and ported to 3.2 (b279611146d7) and 3.3 (e8e3f2b72a32).
In 3.1 json was completely broken without _json, so I fixed that too.
I also did some minor cleanup in ec6d881f5b02.