Roughly removing 'duplicate' photos based on filename

Question

I quickly wanted a script that'll go through a lot of photos in a folder and only keep one photo per minute (gotta love burst shots on iPhone).

Fortunately, Dropbox names them with a timestamp upon import: I was thus allowed to base this one on the filename (an example filename is '2013-11-06 23.32.19 XXXXXX.jpg').

I was thinking of maybe moving these files to a 'duplicate' folder or something, just didn't have the time to figure it out for now. I only just started Python, and it was definitely fun to have such a tiny assignment done in under 10 minutes :)

Any comments on how I could've done this in a more pretty way is appreciated!

import os
directory = "{pathToPhotos}"
dt_counter = {}
for filename in os.listdir(directory):
 if not filename.endswith('.ini'):
 dt_rounded = filename[:16]
 if dt_rounded in dt_counter.keys():
 dt_counter[dt_rounded] += 1
 else:
 dt_counter[dt_rounded] = 1
 if dt_counter[dt_rounded] > 1:
 os.remove(os.path.join(directory, filename))

Additionally, I was thinking of expanding this due to the many 'duplicate' photos I have (same setting, different photo's) by grouping them in 10 second buckets and creating a GUI or so to show all alternatives and choose one to keep, any tips on where to start with this would also be greatly appreciated!

• Veedrac: FWIW, using pathlib makes the code a little nicer.

Answer 1

If you're only dealing with only .jpg files then it will be better to use the glob module:

import glob
for filepath in glob.iglob(os.path.join(directory), '*.jpg'):
 ...

Don't use dict.keys() to check for the existence of a key in dictionary, it is inefficient in Python 2(created a whole list first and then searches for key in that list) and requires an unnecessary call to .keys() in Python 3. Or even better use collections.Counter to keep counts of objects:

from collections import Counter
dt_counter = Counter()
for filepath in glob.iglob(os.path.join(directory), '*.jpg'):
 dt_rounded = os.path.split(filepath)[1][:16]
 dt_counter[dt_rounded] += 1
 if dt_counter[dt_rounded] > 1:
 # if `directory` is actually a `pathToPhotos` then you 
 # don't need to call os.path.join anymore
 os.remove(filepath)