Pythonic way for double iteration (list all files with extensions)

Question

Given a base path and a list with extensions the task is to list all files:

Two of my solutions are:

from glob import glob
from os import path
EXTENSIONS = ['*.zip', '*.jar', '*.pdf']
DOC_PATH = '/path/to/files'
# Solution1:
files = [] 
for ext in EXTENSIONS:
 files.extend(glob(path.join(DOC_PATH, ext)))
# works but looks very clumsy
# Solution2:
files = reduce(lambda x,y: x+y,
 [glob(path.join(DOC_PATH, ext)) for ext in EXTENSIONS])
# Also functional but looks like a misuse of reduce

Have you got any other ideas?

• SylvainD: Not quite sure how idiomatic/efficient it is : sum((glob(path.join(DOC_PATH, ext)) for ext in EXTENSIONS), []) .

Answer 1

If you only need to iterate over them (once) and not an actual list, you could use itertools.chain and glob.iglob:

files = chain(*(iglob(path.join(DOC_PATH, ext)) for ext in EXTENSIONS))

If you do need an actual list, you can further call list(files), of course.

• ProfHase85: chain is quite a good idea

Answer 2

Using os.listdir and os.path.splitext:

import os
EXTENSIONS = 'zip jar pdf'.split()
EXTENSION_SET = set('.' + e for e in EXTENSIONS)
files = [f for f in os.listdir(DOC_PATH) if os.path.splitext(f)[1] in EXTENSION_SET]

Using os.listdir and re.search:

import os
import re
EXTENSIONS = 'zip jar pdf'.split()
EXTENSION_RE = re.compile(r'\.({})$'.format('|'.join(EXTENSIONS))
files = [f for f in os.listdir(DOC_PATH) if EXTENSION_RE.search(f)]
# or files = list(filter(EXTENSION_RE.search, os.listdir(DOC_PATH)))

The advantage of these approaches is that you only iterate over the files in the directory once (rather than once for each extension, as in the glob case).