File: flatten-itunes-2.py
#!/usr/bin/env python3
"""
==========================================================================
flatten-tunes - copy all music files in a folder tree to flat folders.
License: provided freely, but with no warranties of any kind.
Author/copyright: 2011-2018, M. Lutz (http://learning-python.com).
Version 2.2, Jul-2018:
Handle duplicates in the target flat folder better, by checking
same-named files for duplicate content, and skipping redundant
copies - they are retained in the 'from' tree, but not added to
the flat 'to' folder. As before, same-named files with differing
content are still added to the 'to' folder with unique filenames.
This was inspired by the tagpix.py photo-tree script's dup logic.
Also skip any ".*" hidden files (including Mac .DS_Store madness),
and offer to clean up prior run's 'to' folder, and make if needed.
This script is still safe to use, because it only copies files to
a new folder, and never deletes or changes original 'from' files.
Version 2.1: May-2018
Minor polishing: comments, inputs, and docs only.
Version 2.0, Nov-2011:
Normalize all iTunes-tree files into a uniform 4 subfolders per
device or folder, to better support differences when iTunes or
other music-file trees have diverged badly. Output appears in the
Flattened/* subfolders of the output folder, and collected MP3
files normally all show up in the Flattened/Playable folder.
This version also retains and copies every file in the iTunes tree:
any name duplicates are renamed with a numeric extension to make them
unique in 'to', and files in the original 'from' tree are never removed.
This version was also updated to run on both Python 2.X and 3.X.
Run the PP4E book's dirdiff.py to compare results if desired, and the
companion script rename-itunes.py to strip track-number prefixes in
music filenames for better sorting and comparing.
Note: despite its name, this can flatten _any_ folder tree, not just
an iTunes tree. Amazon MP3 zipfile downloads, for example, generate
similarly-nested trees. The "itunes" name is now historical artifact
only (systems that try hard to jail your media may be best avoided).
More details: learning-python.com/books/pp4e-updates.html#flatten.
Version 1.0, 2011 (original docs)
Flatten iTunes subfolder tree contents to store on a USB drive, for
copying onto devices where subdirectories don't work well (e.g., a
vehicle's harddrive). Code note: str.endswith() now allows a tuple
of strings to try, so we don't need to iterate over alternatives
with {any(filelower.endswith(x) for x in seq)}.
==========================================================================
EXAMPLE USAGE (Mac OS):
/MY-STUFF/Code$ python3 flatten-itunes-2.py
Input folder? (e.g., /Users/you/Music) /Users/mini/Desktop/test/new-may-2018
Output folder? (e.g., /Volumes/SSD/resolve-itunes) /Users/mini/Desktop/test
/MY-STUFF/Code$ ls /Users/Mini/Desktop/test
Flattened Flattened-results.txt new-may-2018
/MY-STUFF/Code$ ls /Users/Mini/Desktop/test/Flattened
Irrelevant Other Playable Protected
/MY-STUFF/Code$ ls /Users/Mini/Desktop/test/Flattened/Playable
01 - Brave New World.mp3
01 - Easy Come Easy Go.mp3
...
/MY-STUFF/Code$ python3 rename-itunes.py
Folder to scan? /Users/mini/Desktop/test/Flattened/Playable
Rename? "02 - Pacific Coast Highway.mp3" y
new name? ["Pacific Coast Highway.mp3"]
renamed: "02 - Pacific Coast Highway.mp3" -> "Pacific Coast Highway.mp3"
...
==========================================================================
"""
from __future__ import print_function # to run on python 2.X
import os, pprint, sys, shutil
if sys.version_info[0] == 2: input = raw_input # more 2.X compat
#-------------------------------------------------------------------------
# Get from/to dirs at console: absolute or relative paths
#-------------------------------------------------------------------------
if sys.platform.startswith('win'):
exfrom, exto = r'C:\Users\you\Itunes', r'F:\resolve-itunes'
else:
exfrom, exto = '/Users/you/Music', '/Volumes/SSD/resolve-itunes'
itunesRoot = input('Input folder? (e.g., %s) ' % exfrom) # music tree root
outputRoot = input('Output folder? (e.g., %s) ' % exto) # 4 output folders here
outputDir = os.path.join(outputRoot, 'Flattened')
# existing/valid dirs?
if (not os.path.isdir(itunesRoot)) or os.path.isfile(outputRoot):
print('Input or output folder paths are invalid: run cancelled.')
sys.exit(1)
#-------------------------------------------------------------------------
# Initialize file groupings (or use mimetypes?)
#-------------------------------------------------------------------------
categories = [
dict(name='Playable', exts=('.mp3', '.m4a')),
dict(name='Protected', exts=('.m4p',)),
dict(name='Irrelevant', exts=('.jpg', '.ini', '.xml')),
dict(name='Other', exts=None) ] # last is all other
for cat in categories:
cat['subdir'] = os.path.join(outputDir, cat['name'])
cat['members'] = []
cat['duplicates'] = 0
cat['dupnames'] = []
cat['skips'] = []
#-------------------------------------------------------------------------
# Make copy-tree dirs if needed
#-------------------------------------------------------------------------
if os.path.exists(outputRoot):
if input('Remove existing output folder? ').startswith(('y', 'Y')):
shutil.rmtree(outputRoot)
for dirpath in [cat['subdir'] for cat in categories]:
if not os.path.exists(dirpath):
os.makedirs(dirpath) # makes outputRoot+outputDir too
#-------------------------------------------------------------------------
# Copy one file to flat dir
#-------------------------------------------------------------------------
def noteAndCopy(category, dirfrom, filename):
"""
copy one file from itunes/music tree to normalized tree;
rename file with a numeric suffix is it's a duplicate, to
avoid overwriting the prior file (e.g. "track-N" generic
names for tracks in different subdirs, or same-named songs);
note the "while": though unlikely, the renamed name may also
be a duplicate from this or prior run; also note the simple
file reads and writes: read/write in chunks if files are very
large; TBD - dup numbers per filename instead of per category?
Jul-2018: skip a same-named file if its content is duplicate,
else users must find dup names and compare content manually;
note that checks files system to detect dups, not the former
{toname in category['members']} - otherwise, would apply to
new files only, and not detect dups from a prior run / tree;
also now numbers dups from 1 per file, for the same reasons;
also skip ".*" hidden files, including Mac .DS_Store madness;
"""
# skip hidden files
if filename[0] == '.':
print('Hidden file skipped:', filename)
return
copyfrom = os.path.join(dirfrom, filename)
copyto = os.path.join(category['subdir'], filename)
bytesfrom = open(copyfrom, 'rb').read()
toname = filename
numdups = 0
# name already copied in this run or earlier?
while os.path.exists(copyto):
# check for and skip name+content dups
if bytesfrom == open(copyto, 'rb').read():
print('***Duplicate name+content skipped: "%s"' % copyfrom)
return
else:
# retain and rename same-named files that differ
numdups += 1
basename, ext = os.path.splitext(filename)
suffix = '--%s' % numdups
toname = basename + suffix + ext
copyto = os.path.join(category['subdir'], toname)
# copy file to flat folder
if numdups > 0:
category['duplicates'] += 1
category['dupnames'].append(toname)
category['members'].append(toname)
open(copyto, 'wb').write(bytesfrom)
#-------------------------------------------------------------------------
# Walk the entire itunes/music tree
#-------------------------------------------------------------------------
for (dirHere, subsHere, filesHere) in os.walk(itunesRoot):
for file in filesHere:
for cat in categories[:-1]:
if file.lower().endswith(cat['exts']):
noteAndCopy(cat, dirHere, file)
break
else:
other = categories[-1]
noteAndCopy(other, dirHere, file)
#-------------------------------------------------------------------------
# Report results
#-------------------------------------------------------------------------
os.environ['PYTHONIOENCODING'] = 'utf-8' # filenames? too late?
resultsFile = outputRoot + os.sep + 'Flattened-results.txt' # same dir as outputs
sys.stdout = open(resultsFile, 'w') # send prints to a file
for category in categories:
print('*' * 80)
for key in ('name', 'exts', 'subdir', 'duplicates', 'members', 'dupnames'):
print(key, '=>', end=' ')
if key in ('members', 'dupnames'): print()
pprint.pprint(category[key])
print('=' * 80)
for cat in categories:
print('Total %s: %s' % (cat['name'].ljust(10), len(cat['members'])))
sys.stderr.write('See results in file: %s\n' % resultsFile)