Batch copy to multiple directories

Question

I have about 9000 files in a directory and I want to mv them into 90 files in 100 directories by filename order, ignoring any remainder. In Copy multiple files from filename X to filename Y?, Caleb wrote in a comment:

If the object is just to batch them, there is a MUCH easier way! ls | xargs -iX -n20 cp X target_folder/ would run cp on files in batches of 20 until they were all done.

So based off of using xargs, how could I switch the target_folder to create a new folder and loop the command 100 times?

• jasonwryan: U&L is a wiki, not a message board: please rewrite your question as a question, removing the reference to a community member and including a link to the post your reference. See the FAQ for help on asking questions

Answer 1

In bash, try the following code :

#!/bin/bash
c=0
for f; do
 if ! ((c % 100)); then
 folder=folder_$(printf "%03d\n" $c)
 mkdir -p $folder
 fi
 [[ -d "$f" ]] || mv "$f" "$folder"
 ((c++))
done

Run the script like that :

./script.bash *

Answer 2

That command doesn't look close to what you're trying to do. xargs can help, but it's cumbersome to use unless you know your file names do not contain any whitespace or quoting character. Here's a shell loop that dispatches files into newly-created subdirectories, 90 per directory.

set -- *
# Set args to "1ドル" "2ドル" ... "90ドル"
i=1 args=
while [ $i -le 90 ]; do
 slice="$slice \"\${$i}\""
 i=$((i+1))
done
# Move files 90 at a time
i=0
while [ $# -ge 90 ]; do
 mkdir part$i
 eval "mv $slice part$i"
 shift 90
done
# 0 to 89 files remain in the current directory

• James Youngman: xargs also has an option -0 which turns off the whitespace and quote processing. But I agree this kind of script is the best approach.
• daniel kullmann: @Gilles There's an error in the script: $slice should be $args

Answer 3

parallel supports running multiple commands with \; and {#} is the sequence number:

ls|parallel -n90 mkdir {#}\;mv {} {#}

Or if the filenames don't contain spaces, quotes, or backslashes:

i=1;while read l;do mkdir $i;mv $l $((i++));done< <(ls|xargs -n90)