Return to Revisions

ast.literal_eval(...)

If we can remove your calls to ast.literal_eval(...) we should see a nice reduction in the run time of your loops. But, why can we remove this you ask? Consider:

 m = '[0, 1, 2, ... , 9,999]' # a str representation of list w/ 10k elements, 0-9999
 n = '[0, 1, 2]'
 x = ast.literal.eval(m)
 y = ast.literal.eval(n)
 x == range(10000) # true

As you can see from the snippet above, ast.literal_eval(...) will parse and evaluate whatever string you pass it, and return a literal representation of that string (assuming of course that the string represents a valid literal). Clearly, it is more efficient to compare m and n than it is to compare x and y. Also, it doesn't appear that you are concerned with whether or not val or val_2 is a valid python literal because under the scenario that ast.literal_eval(...)throws an exception, you default to just comparing the strings returned by getter(val) and getter(val_2). Long story short you can remove the try: / except: and just use the statements you have under the except clause.

for key in val_2.keys()

The above loop occurs as the inner-most loop of both loops 1 and 2. With each iteration you check that key isn't equivalent to 7 other possible key values. 6 of these key values occur in the data you've presented and the 7th (condition) doesn't. It should be more efficient to replace:

for key in val_2.keys():
 if(key != "block" and key != "username" and key != "setid" 
 and key != "setid_hash" and key != "predecessor" 
 and key != "time_string" and key != "condition"):
 val[key]=val_2[key]

with:

# put this at the top of the test function 
x_keys = set(['block', 'username', 'setid', 'setid_hash', 'predecessor', 'time_string', 'condition'])
# ...
for key in set(val_2.keys()) - x_keys:
 val[key] = val_2[key]