Script to record an inventory of a backup

Question

This is a script designed to make an inventory file which records the contents of a directory allowing verification that a backup has the same data. The same script can also

A full github repo is available at https://github.com/mikedlr/backup-tools including various test cases using python behave.

This is written as a shell script for reasonably rapid development and maxium ease of rapid portability. If it's useful for others I might rewrite in another language/other languages as well.

Comments appreciated especially for

• style
• safety / correctness
• portability

I have deleted a few irrelevant comments from the code to save your reading time. I'd happily make this portable to non bash shells but I think the use of arrays is pretty unavoidable.

#!/bin/bash
# inventory - record an inventory with checksums of the files in a directory
# Copyright (C) 2016 Michael De La Rue
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
TEMP=$(getopt -o ho:i:x:cv --long help,output:,input:,check,verbose -n 'inventory.sh' -- "$@" )
if [ $? != 0 ] ; then echo "Argument parsing fail; terminating..." >&2 ; exit 1 ; fi
eval set -- "$TEMP"
usage() { 
 cat <<EOF 
inventory - create or verify an inventory of a directory, typically for backup verification
 -h --help - output usage information
 -c --check - read inventory file and verify directory
 -v --verbose - verbose output
 -o --output <file> - output to <file>
 -x --exclude <file> - exclude filenames matching expressions in <file>
 -i --input <file> - use <file> for input (as an inventory file), together with -c
EOF
}
OFILE=""
IFILE=""
XFILE=""
CHECK="false"
VERBOSE="false"
while true ; do
 case "1ドル" in
 -h|--help) usage; exit 0;;
 -c|--check) CHECK="true" ; shift ;;
 -v|--verbose) VERBOSE="true" ; shift ;;
 -o|--output)
 if [ "" != "$OFILE" ]
 then
 echo "Only one output file allowed. Terminating" >&2
 exit 1
 fi
 OFILE=2ドル; shift 2 ;;
 -i|--input)
 if [ "" != "$IFILE" ]
 then
 echo "Only one input file allowed. Terminating" >&2
 exit 1
 fi
 IFILE=2ドル; shift 2 ;;
 -x|--exclude)
 if [ "" != "$XFILE" ]
 then
 echo "Only one exclude file currently allowed. Terminating" >&2
 exit 1
 fi
 XFILE=2ドル; shift 2 ;;
 --) shift ; break ;;
 *) echo "Internal error!" ; exit 1 ;;
 esac
done
if [ "" = "1ドル" ]
then
echo "must have at least one directory argument to run inventory on" >&2
exit 1
fi
if [ "false" = "$CHECK" ]
then
if [ "" != "$IFILE" ]
then
 echo "cannot give input file when generating inventory" >&2
 exit 1
fi
fi
check_inventory ()
{
local check_dir="1ドル"
if [ "false" = "$VERBOSE" ]
then
 SHAOPTS="--check --quiet"
else
 SHAOPTS="--check"
fi
if [ "" != "$OFILE" ]
then
 echo "cannot give output file when checking inventory" >&2
 exit 1
fi
( if [ "" = "$IFILE" ]
then
 head -n-2
else
 head -n-2 "$IFILE"
fi )| tail -n+2 | (cd "$check_dir"; sha384sum $SHAOPTS - )
SHASUMRES=$?
exit $SHASUMRES
}
create_inventory ()
{
local record_dir="1ドル"
set -e
local TEMPFILE
TEMPFILE=$(mktemp)
( echo inventoryfile-0 at "$(date --rfc-3339=seconds --utc)" directory: "$(readlink -f "1ドル")" ) > "$TEMPFILE"
( cd "$record_dir"
 { find . -type f "${FINDFILTER[@]}" -exec sha384sum {} + | sort -k 2
 echo -----------------------------------------------
 } >> "$TEMPFILE"
 #split to two lines to avoid reading and writing at the same time
 FOOT="inventory checksum $(sha384sum "$TEMPFILE" | sed 's/ .*//')"
 echo "$FOOT" >> "$TEMPFILE"
)
if [ "" = "$OFILE" ]
then
 cat "$TEMPFILE"
else
 mv "$TEMPFILE" "$OFILE"
fi
}
if [ "true" = "$CHECK" ]
then
check_inventory "1ドル"
fi
FINDFILTER=()
if [ "" != "$XFILE" ]
then
if [ ! -r "$XFILE" ] 
then
 echo "$XFILE no such file or directory" >&2
 exit 5
fi
while read line 
do 
 #TODO: comments
 if [ "" != "$line" ]
 then
 FINDFILTER=("${FINDFILTER[@]}" -not -path "$line")
 fi
done <"$XFILE"
fi
create_inventory "1ドル"

Answer 1

Use a fatal function

This kind of code appears in many places:

echo "Only one output file allowed. Terminating" >&2
exit 1

I suggest to introduce a fatal function:

fatal() {
 echo "$* Terminating" >&2
 exit 1
}

So that you can replace the example above with a simpler single line:

fatal "Only one output file allowed."

Checking for empty value

Instead of this:

if [ "" != "$IFILE" ]

A shorter way to accomplish the same:

if [ ! "$IFILE" ]

Appending to arrays

Instead of this:

FINDFILTER=("${FINDFILTER[@]}" -not -path "$line")

The correct way to append to an array is using the += operator:

FINDFILTER+=(-not -path "$line")

Pointless local variable

Instead of this:

SHASUMRES=$?
exit $SHASUMRES

You could skip the local variable:

exit $?

Too compact writing style

Instead of this:

if [ $? != 0 ] ; then echo "Argument parsing fail; terminating..." >&2 ; exit 1 ; fi

It will be much more readable to split to multiple lines:

if [ $? != 0 ]; then
 echo "Argument parsing fail; terminating..." >&2
 exit 1
fi

Inconsistent indentation

I see multiple indentation styles for nested code blocks (for example if-statements): indent with 2 spaces, 4 spaces, or none. I suggest to indent consistently (I like 4 spaces).

• Michael: I've accepted this as an answer since there's plenty for me to work on. Thanks @janos. Obviously any more answers appreciated.