Backup with Hanoi Strategy

Question

I want your suggestions about this shell script. I use it to backup some configurations on my home media server.

What improvements would you do? Keep in mind, I want to optimize space and speed and at the same time, backup the maximum configurations over time.

p.-s. For more informations about the hanoi scheme: https://en.wikipedia.org/wiki/Backup_rotation_scheme.

ideas :

• Only use the hanoi scheme when the limit space is reach by checking the space required by rsync.

#!/bin/bash
# ==================================================================================
# Description: BackUp script following a Hanoi scheme using rsync.
# Author: Hugo Lapointe Di Giacomo
# ==================================================================================
set -o errexit
set -o nounset
#set -o xtrace
# ----------------------------------------------------------------------------------
# Options
# ----------------------------------------------------------------------------------
setsMax=16 # ................................ Maximum number of sets bakcup.
cmdPrefix="" # .............................. "echo" if dry-run enable, "" otherwise.
srcDir="/appdata" # ......................... Source directory to backup.
dstDir="/mnt/backup/snapshots" # ............ Destination directory to backup in.
logDir="/mnt/backup/logs" # ................. Logs directory.
# ----------------------------------------------------------------------------------
# Constants
# ----------------------------------------------------------------------------------
SEC_PER_DAY=86400; # ........................ Number of seconds per day.
LATEST_SNAP="$dstDir/latest"; # ............. The "latest" snap.
LATEST_LOG="$logDir/latest"; # .............. The "latest" log.
DATE_FORMAT="%Y-%m-%d"; # ................... The format of a date.
HOUR_FORMAT="%H:%M"; # ...................... The format of a hour.
# ----------------------------------------------------------------------------------
# Show usage of this script.
# ----------------------------------------------------------------------------------
function showUsage() {
 echo "Usage: 0ドル [OPTIONS]..."
 echo ""
 echo "Options available:"
 echo " -h, --help Show usage of the script."
 echo " -m, --max Maximum sets of backup."
 echo " -s, --src Source directory to backup."
 echo " -d. --dst Destination directory to backup in."
 echo " -l, --log Log directory."
 echo " -r, --dry Enable dry-run."
}
# ----------------------------------------------------------------------------------
# Parse arguments.
# ----------------------------------------------------------------------------------
function parseArgs() {
 while [ $# -gt 0 ]; do
 case 1ドル in
 -h|--help)
 showUsage $@
 exit 0
 ;;
 -m|--max)
 setsMax=${2:-$setsMax}
 shift 2
 ;;
 -s|--src)
 if [ -d "2ドル" ]; then
 srcDir="2ドル"
 fi
 shift 2
 ;;
 -d|--dst)
 if [ -d "2ドル" ]; then
 dstDir="2ドル"
 fi
 shift 2
 ;;
 -l|--log)
 if [ -d "2ドル" ]; then
 logDir="2ドル"
 fi
 shift 2
 ;;
 -r|--dry)
 cmdPrefix="echo"
 shift 1
 ;;
 *)
 echo "Unknown option or value: 1ドル"
 showUsage $@
 exit 1
 ;;
 esac
 done
}
# ----------------------------------------------------------------------------------
# Create a backup with rsync.
# ----------------------------------------------------------------------------------
function createBackUp() { 
 snapDir="$dstDir/1ドル" # ................. Name of the snap dir.
 logFile="$logDir/1ドル.log" # ............. Name of the log file.
 RSYNC_CMD="rsync " # ................... Rsync Command.
 RSYNC_CMD+="--archive " # .............. Enable recursion and preserve infos.
 RSYNC_CMD+="--verbose " # .............. Increase amount of infos printed.
 RSYNC_CMD+="--human-readable " # ....... Output number in readable format.
 RSYNC_CMD+="--progress " # ............. Show progress during transfer.
 RSYNC_CMD+="--delete " # ............... Delete files from receiving side.
 RSYNC_CMD+="--link-dest=$LATEST_SNAP " # "latest" symbolic link to hardlink.
 RSYNC_CMD+="$srcDir " # ................ Source directory to backup.
 RSYNC_CMD+="$snapDir " # ............... Destination directory to backup in.
 # Create backup and save the output in a log file.
 $cmdPrefix $RSYNC_CMD 2>&1 | tee "$logFile"
}
# ----------------------------------------------------------------------------------
# Update the latest links.
# ----------------------------------------------------------------------------------
function updateLatestLinks() {
 $cmdPrefix rm -f "$LATEST_SNAP"
 $cmdPrefix ln -s "$snapDir" "$LATEST_SNAP"
 $cmdPrefix rm -f "$LATEST_LOG"
 $cmdPrefix ln -s "$logFile" "$LATEST_LOG"
}
# ----------------------------------------------------------------------------------
# Calculate the previous move in the cycle.
# https://en.wikipedia.org/wiki/Backup_rotation_scheme
# ----------------------------------------------------------------------------------
function calculateDaysFromBackupOfSameSet() {
 local todayInSec=$(date +%s)
 local todayInDay=$(($todayInSec / $SEC_PER_DAY))
 
 local daysToBackup=$((2 ** ($setsMax - 1))) 
 local daysElapsed=0
 local i=1
 while [ $i -lt $(($daysToBackup / 2)) ]; do
 local rotation=$(($todayInDay & $i))
 
 if [ $rotation -eq 0 ]; then
 daysElapsed=$(($i * 2))
 break
 fi
 daysElapsed=$(($daysToBackup / 2))
 i=$((2 * $i))
 done
 
 echo $daysElapsed
}
# ----------------------------------------------------------------------------------
# Main function.
# ----------------------------------------------------------------------------------
function main() {
 parseArgs $@
 
 local todayDatetime=$(date +$DATE_FORMAT.$HOUR_FORMAT)
 if createBackUp $todayDatetime; then
 updateLatestLinks
 
 local daysElapsed=$(calculateDaysFromBackupOfSameSet)
 local expiredDay=$(date -d "$daysElapsed days ago" +$DATE_FORMAT)
 $cmdPrefix rm -frv "$dstDir/$expiredDay."*
 fi
}
main $@

• Zeta: I have rolled back the last edit. Please do not update the code in your question to incorporate feedback from answers, doing so goes against the Question + Answer style of Code Review. This is not a forum where you should keep the most updated version in your question. Please see what you may and may not do after receiving answers .
• hlapointe: @Zeta, understood.
• Heslacher: I have rolled back your latest edits. Please do not update the code in your question to incorporate feedback from answers, doing so goes against the Question + Answer style of Code Review. This is not a forum where you should keep the most updated version in your question. Please see what you may and may not do after receiving answers .
• Heslacher: You may want to read codereview.meta.stackexchange.com/questions/1065/… as well.

Answer 1

Do not silently ignore invalid arguments

Your script allows to specify the source directory (and other locations). Here is the relevant part in function parseArgs():

 -s|--src)
 if [ -d "2ドル" ]; then
 srcDir="2ドル"
 fi
 shift 2
 ;;

If --src <sourceDir> is specified with an invalid source directory then this argument is (silently) ignored. The consequence is that even a simple typo

backup --src /my_importatn_data

causes the default directory to be backed up, and not /my_important_data.

Wrong arguments (here: an invalid or not existing directory) should print an error message (to the standard error) and terminate the shell script (with a non-zero exit code).

Answer 2

Double-quote variables used as command arguments

Although in many places in the posted code the variables used as command arguments are correctly double-quoted, there are more than a few exceptions, for example parseArgs $@, which should be parseArgs "$@" to preserve arguments with spaces.

It's a good habit to systematically double-quote variables used as command arguments.

Collect argument lists in arrays instead of strings

The createBackUp function collects arguments for the rsync command in a string and executes it. This will not work if some elements (here $srcDir, $snapDir, or $LATEST_SNAP) contain spaces.

You can make it safe by using an array instead:

createBackUp() { 
 snapDir="$dstDir/1ドル" # ................. Name of the snap dir.
 logFile="$logDir/1ドル.log" # ............. Name of the log file.
 RSYNC_CMD=("rsync") # ................... Rsync Command.
 RSYNC_CMD+=("--archive") # .............. Enable recursion and preserve infos.
 RSYNC_CMD+=("--verbose") # .............. Increase amount of infos printed.
 RSYNC_CMD+=("--human-readable") # ....... Output number in readable format.
 RSYNC_CMD+=("--progress") # ............. Show progress during transfer.
 RSYNC_CMD+=("--delete") # ............... Delete files from receiving side.
 RSYNC_CMD+=("--link-dest=$LATEST_SNAP") # "latest" symbolic link to hardlink.
 RSYNC_CMD+=("$srcDir") # ................ Source directory to backup.
 RSYNC_CMD+=("$snapDir") # ............... Destination directory to backup in.
 # Create backup and save the output in a log file.
 $cmdPrefix "${RSYNC_CMD[@]}" 2>&1 | tee "$logFile"
}

I also dropped the function keyword which is not recommended in Bash.

Simplify using arithmetic context

The calculateDaysFromBackupOfSameSet uses several arithmetic operations and conditions which could be simplified and made more readable using arithmetic context:

calculateDaysFromBackupOfSameSet() {
 local i todayInDay daysToBackup
 local todayInSec=$(date +%s)
 ((todayInDay = todayInSec / SEC_PER_DAY))
 ((daysToBackup = 2 ** (setsMax - 2)))
 local daysElapsed=daysToBackup
 for ((i = 1; i < daysToBackup; i *= 2)); do
 if ! ((todayInDay & i)); then
 ((daysElapsed = i * 2))
 break
 fi
 done
 echo "$daysElapsed"
}

I also eliminated some repeated daysToBackup / 2. I believe this is equivalent to the original, but I haven't tested it. Excuse me if I made an error, the point is more to demonstrate the power of the arithmetic context.