Bash: move files of specific pattern

Question

I write a script that iterates over a set of zip files, extracts them and moves only files that match a specific filename pattern to another folder.

The script:

#!/bin/bash
ARCHIVE_FILEMASK="????-??-??_E_MDT_0_0.7z"
FILEMASK="?????????_?????_A_?_????????????????-?????????"
extractDir=/path/to/dir
dest=/path/to/dest
for f in ${ARCHIVE_FILEMASK}
do
 7z e -aoa -o"${extractDir}" "$f"
 if [ $? != 0 ]
 then
 mv "${extractDir}/${FILEMASK}".xml "$dest"
 fi
done

When the script is executed an error occurs:

mv: cannot stat `/path/to/dir/?????????_?????_A_?_????????????????-?????????.xml': No such file or directory

I've already tried turning on nullglob and extglob but it didn't help. If I execute the command directly from the shell, it runs just fine. The answers from other posts with similar problems also didn't help.

edit: After Paweł Rumian mentioned possible problems with spaces in directory names, I tried AVJ's tip and added set +x before the mv command.

The result is:

+ mv '/path/to/dir/?????????_?????_A_?_????????????????-?????????.xml' /path/to/dest

So obviously I have to get rid of the single quotes now. Any idea how?

• gogoud: are you sure the script is running under bash? worth putting the bash shebang at the start: #!/bin/bash
• twenty7: Yes, I am. Its just a snippet of a bigger script. But I've put the shebang at the beginning of the snippet for clarity reasons now.

Answer 1

The usual advice to put double quotes around variable expansions is because $foo outside quotes means "take the value of foo, split it at whitespace1, and expand each resulting word as a wildcard pattern if it matches at least one file", rather than "take the value of foo". In particular, when the variable foo contains a file name, "$foo" is what you need. But here the variables ARCHIVE_FILEMASK and FILEMASK contain wildcard patterns. So you do in fact want to expand them when they're used, therefore you should use them unquoted. They will thus be interpreted as a whitespace-separated list of patterns; that doesn't make a difference here since you have no whitespace in these variables' value.

The variables f and extractDir contain file names, so they must remain double-quoted.

for f in ${ARCHIVE_FILEMASK}
do
 if 7z e -aoa -o"${extractDir}" "$f"; then
 mv "${extractDir}/"${FILEMASK}.xml "$dest"
 fi
done

Since you're using bash, you can call shopt -s nullglob to make the wildcard pattern "${extractDir}/"${FILEMASK}.xml expand to an empty list if it doesn't match any file. The mv command will complain that it's missing a file if you don't pass it any source file; you can solve that problem by invoking it only if there is at least one match.

shopt -s nullglob
for f in ${ARCHIVE_FILEMASK}
do
 if 7z e -aoa -o"${extractDir}" "$f"; then
 matches=("${extractDir}/"${FILEMASK}.xml)
 if ((${#matches[@]})); then
 mv "${matches[@]}" "$dest"
 fi
 fi
done

1 _{Or more generally as indicated by IFS.}

Answer 2

The solution was to remove the double qoutes. So now the move command looks like:

mv ${extractDir}/${FILEMASK}.xml "$dest"

• Paweł Rumian: Beware that this will lead you into deep problems when you will have space in directory name. I'd rather keep the quotes in place and dig for the solution of the real problem - which is the fact that the file in question is not found. Testing for exit status of 7z is not enough to ensure you have a file to move.
• Chris Davies: Use mv "${extractDir}"/${FILEMASK}.xml "$dest", which will protect $extractDir but allow $FILEMASK to be interpreted by the shell as a wildcard match. Provided you can guarantee that $FILEMASK itself does not contain spaces this will be safe.

Answer 3

Its because mv command is not able to find source file. There may be a special character or something in the name of source file.

Execute script by putting set -x before mv command and set +x after mv command.

It will show you exact command that is being executed during script execution. Check by executing that command directly from shell.

Answer 4

Do the .7z files contain more than one instance of the exact same filename? I presume not, because you're mv-ing them all to the same "$dest" directory.

In that case, you're better off having the mv command outside of the loop, partly because you'll only run one mv command total (rather than one mv per .7z file) but mostly because there's more likely to be at least one file matching your pattern. And you get to keep the double-quotes around the variables which is much safer if there are filenames with shell- or glob- significant characters in them.

If there are too many files extracted for a single mv command, you could use find, something like this:

find "$extractDir" -name "$FILEMASK.xml" -exec mv {} "$dest/" +

It's probably best to use find anyway - you never know in advance how many files will be extracted and this is usefully defensive programming. It also has the advantage of doing exactly nothing if there are no matching files found, so mv won't complain about missing files.