Optimization of average calculation

Question

I have a list of dictionaries, with keys 'a', 'n', 'o', 'u'. Is there a way to speed up this calculation, for instance with NumPy? There are tens of thousands of items in the list.

The data is drawn from a database, so I must live with that it's in the form of a list of dictionaries originally.

x = n = o = u = 0
for entry in indata:
 x += (entry['a']) * entry['n'] # n - number of data points
 n += entry['n']
 o += entry['o']
 u += entry['u']
 loops += 1
average = int(round(x / n)), n, o, u

• Oved D: what are you trying to calculate?
• Prof. Falken: @OvedD, an average of all 'a' values and a bunch of sums for the others.
• Reinderien: If it's in a database, calculate the average in the database. You haven't shown enough context for me to believe that any of this is called-for.
• Prof. Falken: @Reinderien, it's a decade later, so I don't remember, but you may be right.

Answer 1

I'm not sure if there really is a better way to do this. The best I could come up with is:

import itertools
from collections import Counter
def convDict(inDict):
 inDict['a'] = inDict['a'] * inDict['n']
 return Counter(inDict)
average = sum(itertools.imap(convDict, inData), Counter())
average['a'] = average['a'] / average['n']

But I'm still not sure if that is better than what you originally had.

Counter is a subclass of dict. You can get items from them the same way you get items from a normal dict. One of the most important differences is that the Counter will not raise an Exception if you try to select a non-existant item, it will instead return 0.

• Prof. Falken: Interesting but I don't see how I can replace my code with that. +1 though
• Matt: I realized after posting this that you wanted to do some multiplication before summing. Ill look into that
• Prof. Falken: Wouldn't that require to access the full list twice instead of once? In that case, it must be slower.
• Matt: @AmigableClarkKant Yeah, I just got done editing the question to state that. It is probably slower than what you already have. I'm not sure how much better Counter is (if any) than manually summing.