Convert dictionary of lists of tuples to list of lists/tuples in python

Question

I've got dictionary of lists of tuples. Each tuple has timestamp as first element and values in others. Timestamps could be different for other keys:

dict = {
 'p2': [(1355150022, 3.82), (1355150088, 4.24), (1355150154, 3.94), (1355150216, 3.87), (1355150287, 6.66)],
 'p1': [(1355150040, 7.9), (1355150110, 8.03), (1355150172, 8.41), (1355150234, 7.77)],
 'p3': [(1355150021, 1.82), (1355150082, 2.24), (1355150153, 3.33), (1355150217, 7.77), (1355150286, 6.66)],
}

I want this convert to list of lists:

ret = [
 ['time', 'p1', 'p2', 'p3'],
 [1355149980, '', 3.82, 1.82],
 [1355150040, 7.9, 4.24, 2.24],
 [1355150100, 8.03, 3.94, 3.33],
 [1355150160, 8.41, 3.87, 7.77],
 [1355150220, 7.77, '', ''],
 [1355150280, '', 6.66, 6.66]
]

I'm making conversion with that dirty definition:

def convert_parameters_dict(dict):
 tmp = {}
 for p in dict.keys():
 for l in dict[p]:
 t = l[0] - l[0] % 60
 try:
 tmp[t][p] = l[1]
 except KeyError:
 tmp[t] = {}
 tmp[t][p] = l[1]
 ret = []
 ret.append([ "time" ] + sorted(dict.keys()))
 for t, d in sorted(tmp.iteritems()):
 l = []
 for p in sorted(dict.keys()):
 try:
 l.append(d[p])
 except KeyError:
 l.append('')
 ret.append([t] + l)
 return ret

Maybe there is some much prettier way?

Answer 1

How about...

times = sorted(set(int(y[0] / 60) for x in d.values() for y in x))
table = [['time'] + sorted(d.keys())]
for time in times:
 table.append([time * 60])
 for heading in table[0][1:]:
 for v in d[heading]:
 if int(v[0] / 60) == time:
 table[-1].append(v[1])
 break
 else:
 table[-1].append('')
for row in table:
 print row

Or, less readable but probably faster if the dictionary is long (because it only loops through each list in the dictionary once):

times = sorted(set(int(v2[0] / 60) for v in d.values() for v2 in v))
empty_row = [''] * len(d)
table = ([['time'] + sorted(d.keys())] + 
 [[time * 60] + empty_row for time in times])
for n, key in enumerate(table[0][1:]):
 for v in d[key]:
 if int(v[0] / 60) in times:
 table[times.index(v[0] / 60) + 1][n + 1] = v[1]

• Mariusz Sawicki: Thank you all. Stuart, I'll use your second solution. Thanks.

Answer 2

Have a look at pandas:

import pandas as pd
d = {
 'p2': [(1355150022, 3.82), (1355150088, 4.24), (1355150154, 3.94), (1355150216, 3.87), (1355150287, 6.66)],
 'p1': [(1355150040, 7.9), (1355150110, 8.03), (1355150172, 8.41), (1355150234, 7.77)],
 'p3': [(1355150021, 1.82), (1355150082, 2.24), (1355150153, 3.33), (1355150217, 7.77), (1355150286, 6.66)],
} 
ret = pd.DataFrame({k:pd.Series({a-a%60:b for a,b in v}) for k,v in d.iteritems()})

returns

 p1 p2 p3
1355149980 NaN 3.82 1.82
1355150040 7.90 4.24 2.24
1355150100 8.03 3.94 3.33
1355150160 8.41 3.87 7.77
1355150220 7.77 NaN NaN
1355150280 NaN 6.66 6.66

It is not a list of lists, but using pandas you can do much more with this data (such as convert the unix timestamps to datetime objects with a single command).

• acjohnson55: Wow, now that is slick!