How to avoid race condition when using a lock-file to avoid two instances of a script running simultaneously?

Question

A typical approach to avoid two instances of the same script running simultaneously looks like this:

[ -f ".lock" ] && exit 1
touch .lock
# do something
rm .lock

Is there a better way to lock on files from a shell-script, avoiding a race condition? Must directories be used instead?

• Barry Kelly: Questions should be marked "[solved]" by selecting an answer, not by editing the title and adding "[solved]".
• n-alexander: problem is, I can't mark my own answer as "accepted"
• Barry Kelly: I don't think your answer is the best one, frankly :)

Answer 1

Yes, there is indeed a race condition in the sample script. You can use bash's noclobber option in order to get a failure in case of a race, when a different script sneaks in between the -f test and the touch.

The following is a sample code-snippet (inspired by this article) that illustrates the mechanism:

if (set -o noclobber; echo "$$" > "$lockfile") 2> /dev/null; 
then
 # This will cause the lock-file to be deleted in case of a
 # premature exit.
 trap 'rm -f "$lockfile"; exit $?' INT TERM EXIT
 # Critical Section: Here you'd place the code/commands you want
 # to be protected (i.e., not run in multiple processes at once).
 rm -f "$lockfile"
 trap - INT TERM EXIT
else
 echo "Failed to acquire lock-file: $lockfile." 
 echo "Held by process $(cat $lockfile)."
fi

• Chris W.: In case anyone else was wondering what the $$ does, it gives back the PID (process ID) of the running script.

Answer 2

Try flock command:

exec 200>"$LOCK_FILE"
flock -e -n 200 || exit 1

It will exit if the lock file is locked. It is atomic and it will work over recent version of NFS.

I did a test. I have created a counter file with 0 in it and executed the following in a loop on two servers simultaneously 500 times:

#!/bin/bash
exec 200>/nfs/mount/testlock
flock -e 200
NO=`cat /nfs/mount/counter`
echo "$NO"
let NO=NO+1
echo "$NO" > /nfs/mount/counter

One node was fighting with the other for the lock. When both runs finished the file content was 1000. I have tried multiple times and it always works!

Note: NFS client is RHEL 5.2 and server used is NetApp.

• Barry Kelly: Reasonably good coverage on flock: in my sampling, I have it on Cygwin and Linux, but not Solaris or Mac.
• Dr. Jan-Philip Gehrcke: Would you mind explaining the syntax a bit more? I am especially wondering about exec 200>"$LOCK_FILE". Gonna figure this out with the man pages, but your answer would be much better if it explained what these lines do.
• system PAUSE: Bash 3.2 manual, section 3.6: "Redirections using file descriptors greater than 9 should be used with care, as they may conflict with file descriptors the shell uses internally." Intense use of FD 200 (such as 500 times in a row from multiple processes) could cause problems, no?

Answer 3

Lock your script (against parallel run)

http://wiki.bash-hackers.org/howto/mutex

FYI.

Answer 4

Creating a directory is atomic, and fails if it already exists if you don't use -p, so

 mkdir $lockName || exit 1

works.

...but use flock instead.

Answer 5

seems like I've found an easier solution: man lockfile

• Barry Kelly: Note that lockfile is not portable - it might not available; it's part of procmail (AFAIK).
• Barry Kelly: The shell is more portable than procmail, in so far as you're more likely to have bash than procmail. I have bash on Solaris, Linux, Mac and Windows here. lockfile isn't on any of them.
• juancn: I wrote a bash script using noclobber that mimics some of the behavior of Procmail's lockfile. It should be portable, and you can find it here: codng.com/2011/05/file-locks-in-bash.html