Remove empty strings from a list of strings

Question

I want to remove all empty strings from a list of strings in python.

My idea looks like this:

while '' in str_list:
 str_list.remove('')

Is there any more pythonic way to do this?

• aaronasterling: @Ivo, neither of those statements are true. You should never modify a list that your iterating over using for x in list If you are using a while loop then it's fine. the loop demonstrated will remove empty strings until there are no more empty strings and then stop. I actually hadn't even looked at the question (just the title) but I answered with the exact same loop as a possibility! If you don't want to use comprehensions or filters for sake of memory, it's a very pythonic solution.
• Eduard Luca: Still a very valid point to never change the list you're iterating over :)
• jfa: @EduardLuca if the point of iterating over a list is to change it, then that's the opposite of what you should do. You just have to be careful that you know that you do not cause an unexpected behavior by doing so.
• Camion: @EduardLuca, @JFA : The point is that he is NOT iterating over any list. He would if he had written something in the form for var in list:, but here, he has written while const in list:. which is not iterating over anything. it's just repeating the same code until a condition is false.
• Jacob Ward: You can use filter to remove the empty strings. The code should look something like this... data = list(filter(None, str_list))

Answer 1

I would use filter:

str_list = filter(None, str_list)
str_list = filter(bool, str_list)
str_list = filter(len, str_list)
str_list = filter(lambda item: item, str_list)

Python 3 returns an iterator from filter, so should be wrapped in a call to list()

str_list = list(filter(None, str_list))

• Humphrey Bogart: If you're that pressed for performance, itertool's ifilter is even faster—>>> timeit('filter(None, str_list)', 'str_list=["a"]*1000', number=100000) 2.3468542098999023; >>> timeit('itertools.ifilter(None, str_list)', 'str_list=["a"]*1000', number=100000) 0.04442191123962402.
• Humphrey Bogart: @cpburnz Very true. However, with ifilter results are evaluated lazily, not in one go—I'd argue that for most cases ifilter is better. Interesting that using filter is still faster than wrapping an ifilter in a list though.
• SnoringFrog: If you do this to a list of numbers, note that zeroes will also be removed (note: I only used the first 3 methods), so you'll need an alternate method.
• Tritium21: This focuses only on speed, not on how pythonic the solution is (the question that was asked). List Comprehensions are the pythonic solution, and filter should only be used if profiling has proven that the listcomp is a bottleneck.
• livibetter: @whoever-mentions-about-or-imply-Python-3, please just edit and update the answer. We were only discussing for the Python 2 when this question was asked, even Python 3 was released almost 2 years. But do update both Python 2 and 3 results.

Answer 2

Using a list comprehension is the most Pythonic way:

>>> strings = ["first", "", "second"]
>>> [x for x in strings if x]
['first', 'second']

If the list must be modified in-place, because there are other references which must see the updated data, then use a slice assignment:

strings[:] = [x for x in strings if x]

• Bond: I like this solution because it's easily adaptable. If I needed to remove not only empty strings but strings that are just whitespace, for example: [x for x in strings if x.strip()].
• Amar Kumar: [x for x in strings if x] This works fine but Please explain how this loop is working??
• Nat Riddle: @AmarKumar In Python, blank strings evaluate to false when announced in a Boolean context, like in if x. The brackets, for loop, and if clause combine to read "generate a list that consists of x for every element in strings if x actually contains something." @Ib33x Absolutely awesome work. This answer is certainly the most Pythonic.
• PatrickT: Nice. [x for x in strings if x.strip()] to remove strings of white space.
• ss301: this doesnt filter nan

Answer 3

filter actually has a special option for this:

filter(None, sequence)

It will filter out all elements that evaluate to False. No need to use an actual callable here such as bool, len and so on.

It's equally fast as map(bool, ...)

• kaleissin: This is a python idiom, in fact. It is also the only time I still use filter(), list comprehensions have taken over everywhere else.
• Martin CR: I find this easier to see the intention of the code, compared to a list comp

Answer 4

Sum up best answers:

1. Eliminate emtpties WITHOUT stripping:

That is, all-space strings are retained:

slist = list(filter(None, slist))

PROs:

• simplest;
• fastest (see benchmarks below).

2. To eliminate empties after stripping ...

2.a ... when strings do NOT contain spaces between words:

slist = ' '.join(slist).split()

PROs:

• small code
• fast (BUT not fastest with big datasets due to memory, contrary to what @paolo-melchiorre results)

2.b ... when strings contain spaces between words?

slist = list(filter(str.strip, slist))

PROs:

• fastest;
• understandability of the code.

Benchmarks on a 2018 machine:

## Build test-data
#
import random, string
nwords = 10000
maxlen = 30
null_ratio = 0.1
rnd = random.Random(0) # deterministic results
words = [' ' * rnd.randint(0, maxlen)
 if rnd.random() > (1 - null_ratio)
 else
 ''.join(random.choices(string.ascii_letters, k=rnd.randint(0, maxlen)))
 for _i in range(nwords)
 ]
## Test functions
#
def nostrip_filter(slist):
 return list(filter(None, slist))
def nostrip_comprehension(slist):
 return [s for s in slist if s]
def strip_filter(slist):
 return list(filter(str.strip, slist))
def strip_filter_map(slist): 
 return list(filter(None, map(str.strip, slist))) 
def strip_filter_comprehension(slist): # waste memory
 return list(filter(None, [s.strip() for s in slist]))
def strip_filter_generator(slist):
 return list(filter(None, (s.strip() for s in slist)))
def strip_join_split(slist): # words without(!) spaces
 return ' '.join(slist).split()
## Benchmarks
#
%timeit nostrip_filter(words)
142 μs ± 16.8 μs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
%timeit nostrip_comprehension(words)
263 μs ± 19.1 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit strip_filter(words)
653 μs ± 37.5 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit strip_filter_map(words)
642 μs ± 36 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit strip_filter_comprehension(words)
693 μs ± 42.2 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit strip_filter_generator(words)
750 μs ± 28.6 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit strip_join_split(words)
796 μs ± 103 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

• AMC: s and s.strip() can be simplified to just s.strip().
• ankostis: s and s.strip() is needed if we want to fully replicate filter(None, words), the accepted answer. I corrected x2 sample functions above and dropped x2 bad ones.

Answer 5

>>> lstr = ['hello', '', ' ', 'world', ' ']
>>> lstr
['hello', '', ' ', 'world', ' ']
>>> ' '.join(lstr).split()
['hello', 'world']
>>> filter(None, lstr)
['hello', ' ', 'world', ' ']

Compare time

>>> from timeit import timeit
>>> timeit('" ".join(lstr).split()', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
4.226747989654541
>>> timeit('filter(None, lstr)', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
3.0278358459472656

Notice that filter(None, lstr) does not remove empty strings with a space ' ', it only prunes away '' while ' '.join(lstr).split() removes both.

To use filter() with white space strings removed, it takes a lot more time:

>>> timeit('filter(None, [l.replace(" ", "") for l in lstr])', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
18.101892948150635

• Reihan_amn: it won't work if you have space among the string of a word. for example: ['hello world', ' ', 'hello', ' '] . >> ['helloworld', ' ', 'hello', ' '] do you have any other solution to keep spaces within an item in the list but removing others?
• AMC: Notice that filter(None, lstr) does not remove empty strings with a space ' ' Yeah, because that isn't an empty string.
• Abu Shoeb: Lifesaver 🤓 !!

Answer 6

Reply from @Ib33X is awesome. If you want to remove every empty string, after stripped. you need to use the strip method too. Otherwise, it will return the empty string too if it has white spaces. Like, " " will be valid too for that answer. So, can be achieved by.

strings = ["first", "", "second ", " "]
[x.strip() for x in strings if x.strip()]

The answer for this will be ["first", "second"].
If you want to use filter method instead, you can do like
list(filter(lambda item: item.strip(), strings)). This is give the same result.

• Fuad Ak: Can you explain this piece? normally, x.strip() returns False, we have the result but how this code works I didn't understand the logic.

Answer 7

Instead of if x, I would use if X != '' in order to just eliminate empty strings. Like this:

str_list = [x for x in str_list if x != '']

This will preserve None data type within your list. Also, in case your list has integers and 0 is one among them, it will also be preserved.

For example,

str_list = [None, '', 0, "Hi", '', "Hello"]
[x for x in str_list if x != '']
[None, 0, "Hi", "Hello"]

• Tritium21: If your lists have disparate types (except None), you may have a bigger problem.
• thiruvenkadam: What types? I tried with int and other numeric types, strings, lists, tupes, sets and None and no problems there. I could see that if there are any user defined types that do not support str method might give a problem. Should I be worried about any other?
• Tritium21: If you have a str_list = [None, '', 0, "Hi", '', "Hello"], it is a sign of a poorly designed application. You shouldn't have more than one interface (type) and None in the same list.
• thiruvenkadam: Retrieving data from db? list of arguments for a function while doing automated testing?
• Tritium21: Those are usually tuples.

Answer 8

You can use something like this

test_list = [i for i in test_list if i]

where test_list is list from which you want to remove empty element.

Answer 9

Depending on the size of your list, it may be most efficient if you use list.remove() rather than create a new list:

l = ["1", "", "3", ""]
while True:
 try:
 l.remove("")
 except ValueError:
 break

This has the advantage of not creating a new list, but the disadvantage of having to search from the beginning each time, although unlike using while '' in l as proposed above, it only requires searching once per occurrence of '' (there is certainly a way to keep the best of both methods, but it is more complicated).

• Krzysztof Karski: You can edit the list in place by doing ary[:] = [e for e in ary if e]. Much cleaner and doesn't use exceptions for control flow.
• Andrew Jaffe: Well, that's not really "in place" -- I'm pretty sure this creates a new list and just assigns it to the old one's name.
• wim: This performs very poorly as the tail of data gets shuffled around in memory on each remove. Better to remove all in one hit.

Answer 10

As reported by Aziz Alto filter(None, lstr) does not remove empty strings with a space ' ' but if you are sure lstr contains only string you can use filter(str.strip, lstr)

>>> lstr = ['hello', '', ' ', 'world', ' ']
>>> lstr
['hello', '', ' ', 'world', ' ']
>>> ' '.join(lstr).split()
['hello', 'world']
>>> filter(str.strip, lstr)
['hello', 'world']

Compare time on my pc

>>> from timeit import timeit
>>> timeit('" ".join(lstr).split()', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
3.356455087661743
>>> timeit('filter(str.strip, lstr)', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
5.276503801345825

The fastest solution to remove '' and empty strings with a space ' ' remains ' '.join(lstr).split().

As reported in a comment the situation is different if your strings contain spaces.

>>> lstr = ['hello', '', ' ', 'world', ' ', 'see you']
>>> lstr
['hello', '', ' ', 'world', ' ', 'see you']
>>> ' '.join(lstr).split()
['hello', 'world', 'see', 'you']
>>> filter(str.strip, lstr)
['hello', 'world', 'see you']

You can see that filter(str.strip, lstr) preserve strings with spaces on it but ' '.join(lstr).split() will split this strings.

• phillyslick: This only works if your strings do not contain spaces. Otherwise, you're splitting those strings as well.
• Paolo Melchiorre: @BenPolinsky as you reported join solution will split strings with space but filter will not. Thank you for you comment I improved my answer.

Answer 11

Keep in mind that if you want to keep the white spaces within a string, you may remove them unintentionally using some approaches. If you have this list

['hello world', ' ', '', 'hello'] what you may want ['hello world','hello']

first trim the list to convert any type of white space to empty string:

space_to_empty = [x.strip() for x in _text_list]

then remove empty string from them list

space_clean_list = [x for x in space_to_empty if x]

• AMC: if you want to keep the white spaces within a string, you may remove them unintentionally using some approaches. Like this approach, then?
• Muhammad Mehran Khan Attari: Thanks dude, it worked for me with a little change. i.e. space_clean_list = [x.strip() for x in y if x.strip()]

Answer 12

Use filter:

newlist=filter(lambda x: len(x)>0, oldlist) 

The drawbacks of using filter as pointed out is that it is slower than alternatives; also, lambda is usually costly.

Or you can go for the simplest and the most iterative of all:

# I am assuming listtext is the original list containing (possibly) empty items
for item in listtext:
 if item:
 newlist.append(str(item))
# You can remove str() based on the content of your original list

this is the most intuitive of the methods and does it in decent time.

• John Machin: Welcome to SO. You have not been ignored. You have not been attacked by an anynonmous downvoter. You have been given feedback. Amplifying: Your proposed first arg for filter is worse than lambda x: len(x) which is worse than lambda x : x which is the worst of the 4 solutions in the selected answer. Correct functioning is preferred, but not sufficient. Hover your cursor over the downvote button: it says "This answer is not useful".

Answer 13

match using a regular expression and a filter

lstr = ['hello', '', ' ', 'world', ' ']
r=re.compile('^[A-Za-z0-9]+')
results=list(filter(r.match,lstr))
print(results)

Answer 14

This is what worked for me. I had nan values as well since I created this list from a pandas dataframe column values.

[x for x in a if pd.notna(x) and x.strip()]