numpy array filter and replace

Question

I have an array

a = np.array([1,2,3,4,np.nan])

I would like to replace anything which is less than 1.5 by np.nan, i.e. I would like

a = np.array([np.nan,2,3,4,np.nan])

how do I do that?

I did

 a[a<1.5] = np.nan

I got the following run time warning error in IPython (Py3.4) RuntimeWarning: invalid value encountered in less. Is this because my list had np.nan? Is there anything I can do to prevent this?

Also is there a way doing this inline without assigning? Instead of doing

a[a<1.5]=np.nan 
return a 

I can just do

 return a... 

where that .... is something that need filling in.

• Lost1: @BradSolomon i meant, I am writing this in the middle of a program. is there an inplace replacement function?

Answer 1

Is this [RuntimeWarning] because my list had np.nan?

Yes.

Is there anything I can do to prevent this?

In your case, this warning can be safely ignored. So that you don't accidentally suppress unrelated warnings, please don't put anything else inside the context manager besides the one line shown.

>>> import numpy as np
>>> a = np.array([1,2,3,4,np.nan])
>>> with np.errstate(invalid='ignore'):
... a[a<1.5] = np.nan
... 
>>> a
array([ nan, 2., 3., 4., nan])

This operates in-place, a copy is not created here. To return a copy, with the original a unmodified, prefer the masked array approach.

• MSeifert: I don't think it's actually about the in-place operation (well, a[a<1.5] = np.nan is also in-place) but about avoiding assignment so that it can be done in one-line.

Answer 2

Another option that gets you to your return statement as desired:

mask = ~np.isnan(a)
mask[mask] &= a[mask] < 1.5
return np.where(mask, np.nan, a)

Example:

def ma_lessthan(arr, num):
 mask = ~np.isnan(arr)
 mask[mask] &= arr[mask] < num
 return np.where(mask, np.nan, arr)
print(ma_lessthan(a, 1.5))
[ nan 2. 3. 4. nan]

mask credit to: @Jaime.

• wim: It should be mentioned: this creates a copy.
• wim: It is avoidable, and I showed how. Using a masked array approach is good too, mind you, I just thought this was worth mentioning because we often care about details of performance and memory usage when using numpy.