rm command in bash script does not work with variable

Question

I am doing a bash script to create a back-up of my mysql databases. Actually, I would like to delete the old ones.

So, I create my script with variable for maintability.

The whole script is actually working, the only one thing who does not work, is the delete of the old ones. I was thinking it would be the easiest part.

Anyway, here is my code, can someone telle me what's wrong ? And the rm command returns me : rm: impossible de supprimer « /path/to/backup/files/*.gz »: Aucun fichier ou dossier de ce type Which means rm: impossible to delete « /path/to/backup/files/*.gz »: no files or directory of this type

But what is really strange, first is that I found the script in a tutorial

Two, that if I launch myself the "rm /path/to/backup/files/*.gz" in a shell command, this is working and deleting all the .gz files (as excpected)

#!/bin/bash
USER="***"
PASSWORD="****"
OUTPUT="/path/to/backup/files"
rm "$OUTPUT/*.gz"
databases=`mysql --user=$USER --password=$PASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -v Database`
for db in $databases; do
 if [[ "$db" != "information_schema" ]] && [[ "$db" != _* ]] && [[ "$db" != "performance_schema" ]] ; then
 echo "Dumping database: $db"
 mysqldump --force --opt --user=$USER --password=$PASSWORD --databases $db > $OUTPUT/`date +%Y%m%d`.$db.sql
 gzip $OUTPUT/`date +%Y%m%d`.$db.sql
 fi
done

Thank you,

• bob_saginowski: try rm -rf "path"
• vekah: of course, this is not working. Option r is used of recursif (I dont want recursif, I want *, the wildcard, and is used to force)
• Dababi: try rm "$OUTPUT"/*.gz
• Jeff Schaller: The quotes in your rm command were preventing the expansion of the "*.gz" part; just quote $OUTPUT -> rm "$OUTPUT"/*.gz

Answer 1

The

rm "$OUTPUT/*.gz"

shell command line tells the shell to execute /bin/rm with with two arguments: rm and /path/to/backup/files/*.gz.

The space, " and $ are special characters in the shell language syntax. Space is used to delimit command arguments, " is used to quote other special characters (not all, $ for instance is still special) like that * above, and $ is used for some expansions (like the $OUTPUT parameter expansion that you're using here).

rm once started will try to remove that /path/to/backup/files/*.gz file. If that file doesn't exist (as is the case for you), it will report an error.

When you write:

rm /path/to/backup/files/*.gz

or

rm "$OUTPUT"/*.gz

Since * is not quoted this time, it triggers another special feature of the shell called globbing or filename generation or pathname expansion. The shell tries to expand the word that contains that * character to the list of file names that match the pattern.

So if /path/to/backup/files contains a a.gz and b.gz files, the shell will actually call rm with 3 arguments: rm, /path/to/backup/files/a.gz and /path/to/backup/files/b.gz which looks more like what you want here.

Note that $OUTPUT itself still needs to be quoted as otherwise it could end up being split if it contains characters of $IFS or also be subject to globbing if it contained any wildcard characters (like that * above).

It's also a good idea to get used to writing:

rm -- "$OUTPUT"/*.gz

Here $OUTPUT happens to start with /, so it's fine, but the day you change $OUTPUT to -foo- for instance, it will stop working as that -foo-/... would be taken by rm as options.

If the script is not meant to be interactive, you may want to add the -f option to rm. That will disable all user prompts, and remove the error if there's no matching file (many shells, when a glob has no match, pass the pattern as-is to the application, and rm -f doesn't complain when asked to remove a file that doesn't exist in the first place).

• miken32: To summarize: Only the variable needs to be quoted. Quoting the * stops it from globbing.

Answer 2

An alternative way is to combine rm with find and/or xargs . These are some alternatives:

find "$output" -name "*.gz" -type f -delete
find "$output" -name "*.gz" -type f -exec rm {} +
find "$output" -name "*.gz" -type f -print0 | xargs -0 rm

PS: -type f means to find regular files.

By default, find searches all subdirectories. You can limit the find operation to the current directory if required by using -maxdepth option:

find "$output" -maxdepth 1 -name "*.gz" -type f -exec rm {} +

If you need to keep working with rm and a variable, this worked for me in one line:

out="/home/gv/Desktop/PythonTests/appsfiles";rm "$out"/*.txt
rm: remove regular file '/home/gv/Desktop/PythonTests/appsfiles/a.txt'? y
rm: remove regular file '/home/gv/Desktop/PythonTests/appsfiles/a ver 1.txt'? y
rm: remove regular file '/home/gv/Desktop/PythonTests/appsfiles/b.txt'? y
rm: remove regular file '/home/gv/Desktop/PythonTests/appsfiles/c.txt'? y
rm: remove regular file '/home/gv/Desktop/PythonTests/appsfiles/d.txt'? y

• vekah: I have tried to add some slash at the end of my OUTPUT variable. This was not working, anyway the first find command works so thank you. I should think by myself to use find, but I am still confused with -exec (sometimes {}, sometimes [], sometimes \ ...)
• Jeff Schaller: rm works just fine with variables, actually
• miken32: rm $out/*.txt will give you unpleasant surprises if there are spaces in $out.

Answer 3

Try to get those files and then delete them:

ARCS=`ls $OUTPUT | grep "\.gz$"`
for _arch in $ARCS; do
 rm -f $_arch
done

• vekah: Please, try your code dude! This is full of errors, that I try to solve, but anyway this is not working. Actually, your _arch variable as "ARCS" value! GG
• vekah: rm: missing operand
• bob_saginowski: @vekah edited my answer; fixed typos ;)
• Zachary Brady: Why not to parse ls.