non-pickle persistance for dicts?

Wed May 16 19:07:59 EDT 2012

On Wed, May 16, 2012 at 4:53 PM, Charles Hixson
<charleshixsn at earthlink.net> wrote:
> On 05/16/2012 03:11 PM, Ian Kelly wrote:
>>>> On Wed, May 16, 2012 at 3:52 PM, Charles Hixson
>> <charleshixsn at earthlink.net>  wrote:
>>>>>>>> I want to persist simple dicts, but due to the security problems with
>>> (un)pickle, I'd prefer to not use shelve, and the only way I could see to
>>> persist them onto sqlite also invoked pickle.
>>>>>> As (un)pickle allows arbitrary system commands to be issued, I'd really
>>> rather just use a simple convert to and from either bytes or strings.
>>>  repr
>>> works well for the conversion into string (I said they were simple), but
>>> I'd
>>> really rather be able to turn "{'a': 'A', 1: 23, 2: ['b', 2]}" back into
>>> a
>>> dict without allowing the execution of arbitrary commands.
>>>>>> Any suggestions?
>>>>>>> Either json, or repr with ast.literal_eval will be safe.
>>>>>>>>>>>>>> import json
>>>>> d = {'a': 'A', 1: 23, 2: ['b', 2]}
>>>>> json.dumps(d)
>>>>>>>>> '{"a": "A", "1": 23, "2": ["b", 2]}'
>>>>>>>>>>>> json.loads(json.dumps(d))
>>>>>>>>> {'a': 'A', '1': 23, '2': ['b', 2]}
>>>>>>>>>>>> import ast
>>>>> ast.literal_eval(repr(d))
>>>>>>>>> {'a': 'A', 1: 23, 2: ['b', 2]}
>>>> Cheers,
>> Ian
>>>>>> Thanks.  It looks like either would do what I need.  Any suggestion as to
> how to choose between them?  E.g., is AST better supported?  faster?  (I'm
> tending towards AST purely because it seems more tied to Python, but of
> course that *could* be a disadvantage, if there were more external tools for
> working with json.)

You pretty much just summed it up. JSON is more portable because it's
a well-known standard with implementations in a lot of different
languages. On the downside, since it essentially involves translating
your data into *JavaScript* literals and back, it's not going to be
quite as faithful. Notice in the example that the numeric dictionary
keys got turned into strings. AST will do a cleaner job since you're
just converting Python objects into Python literals and back, but the
serialized data will only be easily readable using Python.