Delete spaces, hyphens, and underscores in filenames?

Question

What is a good command to delete spaces, hyphens, and underscores from all files in a directory, or selected files?

I use the following command with Thunar Custom Actions to slugify filenames:

for file in %N; do mv "$file" "$(echo "$file" | tr -s ' ' | tr ' A-Z' '-a-z' | tr -s '-' | tr -c '[:alnum:][:cntrl:].' '-')"; done

But that command only replaces spaces with dashes/hyphens and lowercases capped characters.

I've used the following command in terminal to delete spaces from thousands of filenames in a folder, and it worked pretty fast:

 rename "s/ //g" *

Again, it only deletes spaces, and not hyphens/dashes and underscores as well.

Ideally I don't want any spaces, hyphens/dashes, and underscores in my filenames. And it would be great if the command could be used with Thunar Custom Actions on selected files.

• mdpc: I note one problem that a lot of the proposed solutions have, is not properly checking the existance of the "new" name before mv'ing the file. Not doing that could be the potential source of a lot of problems.
• whitewings: Is it possible to modify John1024's command to check for that?
• Sparhawk: @user8547 rename -i "s/[-_ ]//g" *
• whitewings: Thank you Sparhawk. Incidentally, for those interested in using this as a Thunar Custom Action, the command for Thunar is: for file in %N; do mv "$file" echo $file | sed -e 's/[ _-]//g'; done

Answer 1

The version of rename that comes with the perl package supports regular expressions:

rename "s/[-_ ]//g" *

Alternatively,

rename -i "s/[-_ ]//g" *

The -i flag will make rename use interactive mode, prompting if the target already exists, instead of silently overwriting.

Perl's rename is sometimes called prename.

Perl's rename versus util-linux's rename

On Debian-like systems, perl's rename seems to be the default and the above commands should just work.

On some distributions, the rename utility from util-linux is the default. This utility is completely incompatible with Perl's rename.

• All: First, check to see if Perl's rename is available under the name prename.

• Debian: Perl's rename should be the default. It is also available as prename. The rename executable, though, is under the control of /etc/alternatives and thus could have been altered to something different.

• archlinux: Run pacman -S perl-rename and the command is available as perl-rename. For a more convenient name, create an alias. (Hat tip: ChiseledAbs)

• Mac OSX According to this answer, rename can be installed on OSX using homebrew via:

  brew install rename 
  

• Direct Download: rename is also available from Perl Monks:

   wget 'http://www.perlmonks.org/?displaytype=displaycode;node_id=303814' -O rename
  

• Cristian Ciupitu: I think that depends on what rename are you talking about. The one from util-linux-2.24.2-1.fc20.x86_64 does not support regular expressions.
• John1024: @CristianCiupitu I just checked the man page for the version of rename that you found. Based on the arguments, the version of rename that the OP was using looks like the perl version and not the util-linux version.
• Cristian Ciupitu: For the record, this is the rename man page for the util-linux version. Anyway, besides that note, the important thing is that the OP got his answer (and you an upvote from me :-D).
• John1024: @CristianCiupitu Thanks for finding that. Back at you with a +1.
• ChiseledAbs: @John1024 archlinux, but i found out how, just go pacman -S perl-rename then I guess you can alias.

Answer 2

I would replace all those tr commands, with a sed substitution command, e.g.:

for file in %N; do 
 mv "$file" "$(echo "$file" | sed 's/[ _-]//g')"
done

Answer 3

Not counting mv, you don't really need an outside process for this at all - you can kind of just poof them.

ifsqz() ( LC_ALL=C sqz=1ドル
 isf() { [ -e "1ドル" ] || [ -L "1ドル" ] ; } 
 set -- * ; set -f
 for f do isf "$f" || break
 IFS=$sqz; set -- $f; IFS=
 isf "$*" || mv -- "$f" "$*"
 done
)

Still though, that means a mv invocation per file, and so probably rename is better. Though this should work given only a POSIX mv in $PATH and a POSIX shell.

So, I came up with a kind of crazy demo for this. The test set is generated like:

tee - - - - <<CGEN |\
dd cbs=90 conv=unblock |\
sed 'G;$!N'";s/^/touch -- '/;s/$/'/" |sh
$( #BEGIN CGEN
 LC_ALL=C
 i= n='"$((i=((i=i+1)==10||i==39||i==47)>0?(i+1):i))"'
 printf '%b -_ ---___' $(
 IFS=0; eval \
 printf '"\\\\%04o\\\\%04o "' "$(
 printf "$n"' "$i" '%s $(
 printf %.252d
#END
))"))
CGEN

In the first place I will be the first to acknowledge that the above command produces results that can be more easily obtained by other means. But other means would likely not demonstrate as well what might be done with $IFS and a little (sick?) imagination.

So the first bit is fairly straight-forward:

• tee pipes out 5 copies of its input - the heredocument called CGEN

• dd blocks its input by newlines at 90 bytes per block and pipes that to ...

• sed joins 2 of those blocks on two \newline characters, 'single-quotes the results, and prepends the string touch -- for every line cycle before piping out to ...

• sh which then executes all input as shell commands

The #CGEN bit though... Well, briefly...

• the bottom printf prints 252 0s

• the next from last receives 252 '' null-string arguments and for each prints the contents of $n followed by the string " $i "

• eval interprets the arguments of the next up printf before it prints the results of that interpretation as octal nums prepended by 2 backslashes a piece

• the last printf prints the byte values for those octals 2 at a time followed by the string -_ ---___ for each pair

• $n is initialized to an equation that will increment $i by one for every evaluation excepting that it skips the values 10, 39, or 47 - (which are \newline, 'single-quote, and /slash in ASCII decimal respectively)

The end result is a directory containing a lot of really ugly filenames containing every byte in my charset from 1 through 255 excepting the single-quote (only skipped to avoid one more sed s/// statement) and the /slash. Those filenames look like this:

(set -- *; printf '%s\n\n##############\n\n%s\n' "${9}" "${34}") | cat -A
 ---___ww -_ ---___xx -_ ---___yy -_ ---___zz -_ ---___{{ -_ ---___|| -_ ---$
$
___}} -_ ---___~~ -_ ---___^?^? -_ ---___M-^@M-^@ -_ ---___M-^AM-^A -_ ---___M-^BM-^B -_ ---___M-^CM-^C$
$
##############$
$
 -_ ---___M-ZM-Z -_ ---___M-[M-[ -_ ---___M-\M-\ -_ ---___M-]M-] -_ ---___M-^M-^ -_ ---___M-_M-_ -_$
$
---___M-`M-` -_ ---___M-aM-a -_ ---___M-bM-b -_ ---___M-cM-c -_ ---___M-dM-d -_ ---___M-eM-e -_ ---___$

Now I'll get some data on these files:

chksqz() ( LC_ALL=C sqz=1ドル
 set -- * ; set -f ; IFS= ; tc="$*"
 printf '#%s\n' \
 "There are $# files in this test directory." \
 "All filenames combined contain a total of ${#tc} bytes."
 IFS=$sqz ; set -- $* ; IFS= ; sc="$*" 
 printf "%s '$sqz'" \
 "#Of which ${#sc} bytes are not"\
 " and $((${#tc}-${#sc})) bytes are"
 set +f ; unset IFS
 printf ".\n#%s\n#Total:\t%d\n#Other:\t%d\n#'$sqz':\t%d\n" \
 "And to confirm these figures:" \
 $( printf %s * | wc -c 
 printf %s * | tr -d "$sqz" | wc -c
 printf %s * | tr -dc "$sqz" | wc -c
))
chksqz '_ -'

OUTPUT

#There are 101 files in this test directory.
#All filenames combined contain a total of 17744 bytes.
#Of which 2692 bytes are not '_ -' and 15052 bytes are '_ -'.
#And to confirm these figures:
#Total: 17744
#Other: 2692
#'_ -': 15052

Ok. Now finally, to action:

ifsqz '_ -'
chksqz '_ -'

OUTPUT

#There are 101 files in this test directory.
#All filenames combined contain a total of 2692 bytes.
#Of which 2692 bytes are not '_ -' and 0 bytes are '_ -'.
#And to confirm these figures:
#Total: 2692
#Other: 2692
#'_ -': 0

Success! You can see for yourself:

ls
????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
???????????????????????????
???????????????????????????
???????????????????????????
????????????????????????????
????????????????????????????
????????????????
??????????????????????
????????????????????????
??????????????????????????
??????????????????????????
??????????????????????????
??????????????????????????
???????????????????????????
???????????????????????????
???????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
??????????????????????????
????????????????????????
????????????????????
??????????????????
????????????????????????????
??
????????????????????????????
??????????????????????????
????????????????????????????
????????????????????????????
????????????????????!!""##
??????????????????!!""##$$
????????????????!!""##$$%%
????????????!!""##$$%%&&((
????????!!""??##$$%%&&(())
$$%%&&(())**??++,,..0011
%%&&(())**++??,,..00112233
&&(())**++,,??..0011223344
))**++,,..??0011223344556
**++,,..00??11223344556677
22334455667788??99::;;<<==>>
445566778899??::;;<<==>>??@@
5566778899::;;??<<==>>??@@AA
6778899::;;<<??==>>??@@AABB
8899::;;<<==??>>??@@AABBCCDD
\\]]^^``aa??bbccddeeffgghh
]]^^``aabbc??cddeeffgghhii
^^``aabbccdd??eeffgghhiijj
??@@AABBCCDDEE??FFGGHHIIJJKK
AABBCCDDEEFF??GGHHIIJJKKLLM
BBCCDDEEFFGG??HHIIJJKKLLMMNN
CCDDEEFFGGHHII??JJKKLLMMNNOO
EEFFGGHHIIJJ??KKLLMMNNOOPPQQ
ffgghhiijjkk??llmmnnooppqqrr
gghhiijjkkllmm??nnooppqqrrss
iijjkkllmmnn??ooppqqrrsstt
jjkkllmmnnoo??ppqqrrssttuuvv
kkllmmnnooppqq??rrssttuuvvww
LLMMNNOOPPQQRR??SSTTUUVVWWXX
MNNOOPPQQRRSS??TTUUVVWWXXYY
OOPPQQRRSSTT??UUVVWWXXYYZZ[[
PPQQRRSSTTUUVV??WWXXYYZZ[[\\
RRSSTTUUVVWW??XXYYZZ[[\\]]
ssttuuvvwwxx??yyzz{{||}}~~??
ttuuvvwwxxyyz??z{{||}}~~????
uuvvwwxxyyzz{{??||}}~~??????
wwxxyyzz{{||??}}~~??????????
xxyyzz{{||}}~~??????????????
YYZZ[[\\]]^^??``aabbccddee
ZZ[[\\]]^^``??aabbccddeeff

• mikeserv: @John1024 - what's really fun: set -- 'some arbitrary' args; eval printf '"%s\n"' "$(IFS=0; printf ' "$@" %s' $(printf %025d))"
• whitewings: Is mikeserv's command supposed to be pasted into the terminal as one line? Or is it supposed to be made into a bash script?
• Stéphane Chazelas: new="$(IFS=" -_"; printf %s 1ドル)" forks a subshell (except in ksh93) and has issues with tailing newlines. Another option is to use IFS=' -_'; set -- 1ドル; IFS=; new="$*" (and change your while loop to a for loop)
• Stéphane Chazelas: [ -e x ] will return false if x is a symlink to an non-existing or non-accessible file.
• countermode: Nice shell Kung-Fu!

Answer 4

if you have perl, you usually have rename. you can do:

> type rename
rename is /usr/bin/rename

and show how this script is written:

> cat /usr/bin/rename | head -n 5 #firt 5 lines for example
#!/usr/bin/perl -w
#
# This script was developed by Robin Barker ([email protected]),
# from Larry Wall's original script eg/rename from the perl source.
#

This script doesn't support -i flag (this is version in my system), but maybe yours supports. What about arguments. First is regular expressions with PCRE format, it works like filter, modify input name to output name. List of input names you give by asterisk '*'. for example, you do:

> cd /tmp
> rename 's/ //g' *

in real '*' can be expanded to:

> rename 's/ //g' file1 file2 file3 othe files found in current directory

When you have really big count files, you are in trap. shell will expand your line longer than system accepts. then you can do workaround using find or xargs. using 'find' is problem, because rename will be called many times equal to files count in directory. better use xargs with -r option. one rename call modify many files. for example:

> ls | xargs -r rename 's/ //g' #thats all, names will be appended at the end of this command.

last problem, what does it mean:

's/ //g'

this is regular expression for modify names. after first '/' is space. this is detected, and replaced by string after second '/'. But there is empty string ended with third '/', then space is replaced by nothing. Option 'g' makes this expression repetative. expression will walk for all name from begin to the end, and detects all spaces.

But what if you have tab character or other 'white' character? there is replacement for this '\s'. what other unneeded characters? simply add it to expression. All close with brackets, for example:

's/[\s_-]//g'

this is all. do you see similarity? I think you should read man perlrequick and man perlretut , this explain you (I hope) how regular expression works. you can use rename command in your own script if you need it.

Answer 5

The following sh shell loop will remove all spaces, underscores and dashes from the names of files in the current directory, taking care to not overwrite any existing files:

for f in *; do
 test -f "$f" || continue
 nf=$( echo "$f" | tr -d ' _-' )
 ! test -e "$nf" && echo mv "$f" "$nf"
done

For bash and ksh, and being slightly more verbose with the logic:

for f in *; do
 if [[ -f "$f" ]]; then
 nf=$( tr -d ' _-' <<<"$f" )
 if [[ ! -e "$nf" ]]; then
 echo mv "$f" "$nf"
 fi
 fi
done

Remove the echo when you're certain it does what you want it to do.

The tr command will delete (-d) any character in the given set of characters (' _-'). It's important to have the dash in the very start or end of the set, or it will be interpreted as a range of characters.