Convert a String representation of a Dictionary to a dictionary

Question

How can I convert the str representation of a dictionary, such as the following string, into a dict?

s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"

I prefer not to use eval. What else can I use?

• Ned Batchelder: If you can't use Python 2.6, you can use a simple safeeval implmenentation like code.activestate.com/recipes/364469 It piggybacks on the Python compiler so you don't have to do all the gross work yourself.
• Martijn Pieters: Note: For those that come here with deceptively similar looking JSON data, you want to go read Parse JSON in Python instead. JSON is not the same thing as Python. If you have " double quotes around your strings you probably have JSON data. You can also look for null, true or false, Python syntax uses None, True and False.

Answer 1

You can use the built-in ast.literal_eval:

>>> import ast
>>> ast.literal_eval("{'muffin' : 'lolz', 'foo' : 'kitty'}")
{'muffin': 'lolz', 'foo': 'kitty'}

This is safer than using eval. As its own docs say:

>>> help(ast.literal_eval)
Help on function literal_eval in module ast:
literal_eval(node_or_string)
 Safely evaluate an expression node or a string containing a Python
 expression. The string or node provided may only consist of the following
 Python literal structures: strings, numbers, tuples, lists, dicts, booleans,
 and None.

For example:

>>> eval("shutil.rmtree('mongo')")
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "<string>", line 1, in <module>
 File "/opt/Python-2.6.1/lib/python2.6/shutil.py", line 208, in rmtree
 onerror(os.listdir, path, sys.exc_info())
 File "/opt/Python-2.6.1/lib/python2.6/shutil.py", line 206, in rmtree
 names = os.listdir(path)
OSError: [Errno 2] No such file or directory: 'mongo'
>>> ast.literal_eval("shutil.rmtree('mongo')")
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/opt/Python-2.6.1/lib/python2.6/ast.py", line 68, in literal_eval
 return _convert(node_or_string)
 File "/opt/Python-2.6.1/lib/python2.6/ast.py", line 67, in _convert
 raise ValueError('malformed string')
ValueError: malformed string

• Paulo Matos: I should add that you need to sanitize the string for use with ast.literal_eval. (ensure quotes/double quotes in string are escaped)
• user1176501: i get this error I am on python 2.6 (x86) on windows 7 x64 File "D:\Python26\lib\ast.py", line 48, in literal_eval node_or_string = parse(node_or_string, mode='eval') File "D:\Python26\lib\ast.py", line 36, in parse return compile(expr, filename, mode, PyCF_ONLY_AST) File "<unknown>", line 1 ^ SyntaxError: invalid syntax
• n611x007: what about "dict(a=1)" style strings?
• JuanB: why don't use json.dumps and json.loads insead, I found this solution more elevant thant using eval
• Michael Wheeler: @JuanB json.loads doesn't accept single-quoted strings. It also doesn't accept None, only null.

Answer 2

https://docs.python.org/library/json.html

JSON can solve this problem, though its decoder wants double quotes around keys and values. If you don't mind a replace hack...

import json
s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"
json_acceptable_string = s.replace("'", "\"")
d = json.loads(json_acceptable_string)
# d = {u'muffin': u'lolz', u'foo': u'kitty'}

NOTE that if you have single quotes as a part of your keys or values this will fail due to improper character replacement. This solution is only recommended if you have a strong aversion to the eval solution.

More about json single quote: jQuery.parseJSON throws "Invalid JSON" error due to escaped single quote in JSON

• Finn Årup Nielsen: Another problem is for "{0: 'Hello'}".
• gdvalderrama: This also fails if you have trailing commas (not JSON compliant), eg: "{'muffin' : 'lolz', 'foo' : 'kitty',}"
• AuthorOfTheSurf: Single-quoted strings, tuple literals, and trailing commas are not valid JSON. json.loads will only work on a valid JSON string. See the spec here: json.org Using json.loads is the safest solution, so use if possible. I would recommend transforming your input into valid JSON if necessary.
• Matias Gonzalez: Also this solution does not work if you have unicode strings
• 林果皞: Not working if value None (without single quotes, so can' t replace), e.g. "{'d': None}"

Answer 3

using json.loads:

>>> import json
>>> h = '{"foo":"bar", "foo2":"bar2"}'
>>> d = json.loads(h)
>>> d
{u'foo': u'bar', u'foo2': u'bar2'}
>>> type(d)
<type 'dict'>

• technazi: I dont think it answers the OP's answer. How do we use json.laads to convert a string s = "{'muffin' : 'lolz', 'foo' : 'kitty'}" to dict?
• ntg: @technazi: json.loads(h.replace("'",'"'))
• ntg: However, there are limits, e.g.: h= '{"muffin" : "lolz", "foo" : "kitty",}', also h= '{"muffin's" : "lolz", "foo" : "kitty"}', (just noticed part of the same comments in a similar answer... still leaving here for completeness...)
• nostradamus: In my opinion, that's the shortest and easiest way... Definitely the one I personally prefer.
• MrR: @nostradamus Too many exceptions, floating point values, tuples, etc. etc.

Answer 4

To OP's example:

s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"

We can use Yaml to deal with this kind of non-standard json in string:

>>> import yaml
>>> s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"
>>> s
"{'muffin' : 'lolz', 'foo' : 'kitty'}"
>>> yaml.load(s)
{'muffin': 'lolz', 'foo': 'kitty'}

• Eric Marcos: This will cause 'yes' and 'no' strings to be converted to True / False
• shailu: i got my value that works fine....but i get a error with it "AMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read msg.pyyaml.org/load for full details." what is it??
• Yasin Zähringer: Only use this yaml parser for trusted input. Preferably use safe_load to avoid security implications.
• it_works_on_my_computer: config = yaml.load(ymlfile, Loader=yaml.Loader)

Answer 5

To summarize:

import ast, yaml, json, timeit
descs=['short string','long string']
strings=['{"809001":2,"848545":2,"565828":1}','{"2979":1,"30581":1,"7296":1,"127256":1,"18803":2,"41619":1,"41312":1,"16837":1,"7253":1,"70075":1,"3453":1,"4126":1,"23599":1,"11465":3,"19172":1,"4019":1,"4775":1,"64225":1,"3235":2,"15593":1,"7528":1,"176840":1,"40022":1,"152854":1,"9878":1,"16156":1,"6512":1,"4138":1,"11090":1,"12259":1,"4934":1,"65581":1,"9747":2,"18290":1,"107981":1,"459762":1,"23177":1,"23246":1,"3591":1,"3671":1,"5767":1,"3930":1,"89507":2,"19293":1,"92797":1,"32444":2,"70089":1,"46549":1,"30988":1,"4613":1,"14042":1,"26298":1,"222972":1,"2982":1,"3932":1,"11134":1,"3084":1,"6516":1,"486617":1,"14475":2,"2127":1,"51359":1,"2662":1,"4121":1,"53848":2,"552967":1,"204081":1,"5675":2,"32433":1,"92448":1}']
funcs=[json.loads,eval,ast.literal_eval,yaml.load]
for desc,string in zip(descs,strings):
 print('***',desc,'***')
 print('')
 for func in funcs:
 print(func.__module__+' '+func.__name__+':')
 %timeit func(string) 
 print('')

Results:

*** short string ***
json loads:
4.47 μs ± 33.4 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
builtins eval:
24.1 μs ± 163 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
ast literal_eval:
30.4 μs ± 299 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
yaml load:
504 μs ± 1.29 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
*** long string ***
json loads:
29.6 μs ± 230 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
builtins eval:
219 μs ± 3.92 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
ast literal_eval:
331 μs ± 1.89 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
yaml load:
9.02 ms ± 92.2 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Conclusion: prefer json.loads

• Michael Campbell: Except this won't work with his single-quoted string, which was part of his initial problem. Performance was never mentioned.
• MestreLion: +1 for the benchmarks (it helps making an informed decision), -1 for the conclusion: as mentioned many times, json fails in many cases. Should be up to the user to choose between features vs performance.

Answer 6

If the string can always be trusted, you could use eval (or use literal_eval as suggested; it's safe no matter what the string is.) Otherwise you need a parser. A JSON parser (such as simplejson) would work if he only ever stores content that fits with the JSON scheme.

• Eli Courtwright: Starting in 2.6, simplejson is included in the Python standard library as the json module.
• Ben Hoyt: Yeah, that's a good answer, but note that officially JSON doesn't support single-quoted strings, as given in the original poster's example.

Answer 7

Use json. the ast library consumes a lot of memory and and slower. I have a process that needs to read a text file of 156Mb. Ast with 5 minutes delay for the conversion dictionary json and 1 minutes using 60% less memory!

• ntg: but has its limits: try converting the string "{'foo':'bar',}"

Answer 8

string = "{'server1':'value','server2':'value'}"
#Now removing { and }
s = string.replace("{" ,"")
finalstring = s.replace("}" , "")
#Splitting the string based on , we get key value pairs
list = finalstring.split(",")
dictionary ={}
for i in list:
 #Get Key Value pairs separately to store in dictionary
 keyvalue = i.split(":")
 #Replacing the single quotes in the leading.
 m= keyvalue[0].strip('\'')
 m = m.replace("\"", "")
 dictionary[m] = keyvalue[1].strip('"\'')
print dictionary

• Om Sao: Many mistakes in this approach. What if value of a key contains { or }. What if it is nested dict. What if value contains , ??

Answer 9

Optimized code of Siva Kameswara Rao Munipalle

s = s.replace("{", "").replace("}", "").split(",")
 
dictionary = {}
for i in s:
 dictionary[i.split(":")[0].strip('\'').replace("\"", "")] = i.split(":")[1].strip('"\'')
 
print(dictionary)

Answer 10

no any libs are used (python2):

dict_format_string = "{'1':'one', '2' : 'two'}"
d = {}
elems = filter(str.isalnum,dict_format_string.split("'"))
values = elems[1::2]
keys = elems[0::2]
d.update(zip(keys,values))

NOTE: As it has hardcoded split("'") will work only for strings where data is "single quoted".

NOTE2: In python3 you need to wrap filter() to list() to get list.

• Aravind Krishnakumar: elems = filter(str.isalnum,dict_format_string.split("'")) should be list(elems = filter(str.isalnum,dict_format_string.split("'"))) without converting to list it would still be 'filter' object

Answer 11

I couldn't use any of the above answers cause I had a string with the dtypes specified, so I used the json.load as follow:

string_dict = """{'contexts': array(['Programmed cell death (PCD) is the regulated death of cells within an organism.', ..., '...'], dtype=object)},..."""
# Replace array( with np.array(
string_dict = string_dict.replace("array(", "np.array(")
# Evaluate the string as python code
python_dict = ast.literal_eval(string_dict)

Answer 12

If you have a string representation of a Python object that cannot be parsed by ast.literal_eval or json, then try asteval module. You can install it via pip first to use it: pip install asteval.

Its main advantage over the built-in ast or eval is that it can evaluate string representations of complex Python objects much easier than ast and much more safely than eval. Use it especially if your string contains math expressions such as nan, Decimal, numpy objects etc.

To use it, we must instantiate an Interpreter object and call it with the string to evaluate. In the example below, the string representation of the dictionary which is not JSON and contains NaN which cannot be converted by ast.literal_eval; however, asteval.Interpreter evaluates it correctly.

import ast
import json
from asteval import Interpreter
s = "{1: nan, 2: 3}"
ast.literal_eval(s) # ValueError: malformed node or string
json.loads(s) # JSONDecodeError
aeval = Interpreter()
aeval(s) # {1: nan, 2: 3}

Some other examples where literal_eval or json.loads fails but asteval works.

1. If you have the string representation of numpy objects and if numpy is installed on your system, then it's much easier to convert to the proper object with asteval as well.

   aeval = Interpreter()
   aeval("{'a': array([ 1., 2., nan])}") # {'a': array([ 1., 2., nan])}
   

2. By default, asteval evaluates numpy functions; however, if you want to make it parse some symbols in a certain way (e.g. numpy dtypes), you can define a custom symbol table and pass it to Interpreter at creation.

   Below is an example where a symbol table that defines how to evaluate numpy dtypes, decimal.Decimal type and null value is created and is passed to the Interpreter.

   from asteval import Interpreter, make_symbol_table
   from decimal import Decimal
   s = "{'a': array(['text'], dtype=object), 'b': Decimal('3.33'), 'c': null, 'd': array([1, 2], dtype=int8)}"
   symtable = make_symbol_table(object=object, Decimal=Decimal, null=None, int8='int8')
   aeval = Interpreter(symtable=symtable)
   d = aeval(s)
   print(d) # {'a': array(['text'], dtype=object), 'b': Decimal('3.33'), 'c': None, 'd': array([1, 2], dtype=int8)}
   

---

Finally, keep in mind that since it evaluates any numpy/math functions (really any function that is fed into it as long as it is defined in the symtable argument), please be mindful of the input you pass into it.

Answer 13

My string didn't have quotes inside:
s = 'Date: 2022年11月29日T10:57:01.024Z, Size: 910.11 KB'

My solution was to use str.split:
{k:v for k, v in map(lambda d: d.split(': '), s.split(', '))}

• A. West: @EricAya, why did you edit my question out? How can I do this without a map?
• Eric Aya: Because the answers section is just for answers, not for questions, this is not a discussion forum. If you have a new question you need to post an actual new question.