Memory Error Python Numpy Array

Question

I'm running into a memory error when trying to evaluate a function at a number of points and store in an array. I've read a few other posts about similar issues and I don't think that I am exceeding the memory limits of my system:

 total used free shared buff/cache available
Mem: 131898384 8964068 122074628 15128 859688 121905868
Swap: 14648316 1001760 13646556

and also

MemTotal: 131898384 kB
MemFree: 75056308 kB
MemAvailable: 74889296 kB

The error message is:

PDF= np.zeros([T,Y])
MemoryError

My code is:

Y=40000
T = 200000
tmin = 0
timestep = 10
tmax = timestep*T
TD_Psi = np.zeros([T,Y],'complex')
t = np.linspace(tmin, tmax, T)
for j in range(T):
 for i in range(M):
 TD_Psi[j] = TD_Psi[j] + c[i]*MO_basis[i]*np.exp(-evals[i]*t[j]*1j)
PDF= np.zeros([T,Y])
for time in range(T):
 PDF[time] = np.real(np.conjugate(TD_Psi[time])*TD_Psi[time])

I'm not sure if the problem is in my code or my system and what I can do to resolve this issue. It runs up until T = 150000 before giving me the error.

Thank you very much for your help.

• XBB: Oh sorry M refers to the number of basis functions in this case M = 30

Answer 1

np.zeros([T,Y],'complex') is 128 GB. np.zeros([T,Y]) is another 64 GB. You don't have 192 GB of RAM. You're out of memory.

Answer 2

Since you're running out of memory and the data cannot be stored in RAM, for a quick solution try increasing your virtual memory.

1. Open My Computer
2. Right click and select Properties
3. Go to Advanced System Settings
4. Click on Advance Tab
5. Click on settings under Performance
6. Click on Change under Advance Tab
7. Increase the memory size, that will increase virtual memory size.

Make sure to select as much as you need, it will convert your hard disk space to virtual memory, and your issue will be resolved now if your data fits into allocated memory.