Testing an infinite loop in the background within Docker?

Question

I want to run a background queue working in my docker image:

php artisan queue:work --daemon --sleep=1 --tries=3 &

Immediately after that it starts Apache httpd running a PHP laravel app. The laravel app sends push notifications out to Redis. The background queue worker collects the messages from Redis and sends them via a push service.

That works and logs out every second. In case the background command crashes I would like it to restart. According to this answer https://unix.stackexchange.com/a/223780/72040 I can run a command infinitely in the background with something like:

while true; do php $QUEUE_WORK; done &

To test this I ran my container with docker run then did docker exec -it to login and see:

UID PID PPID C STIME TTY TIME CMD
100000 1 0 0 19:55 ? 00:00:00 httpd -D FOREGROUND
100000 21 1 0 19:55 ? 00:00:00 /bin/sh -e /tmp/scripts/run
100000 22 21 1 19:55 ? 00:00:01 php artisan queue:work --sleep=1 --tries=3
100000 43 1 0 19:55 ? 00:00:00 /usr/bin/cat
100000 50 1 0 19:55 ? 00:00:00 /usr/bin/cat
100000 51 1 0 19:55 ? 00:00:00 /usr/bin/cat
100000 52 1 0 19:55 ? 00:00:00 /usr/bin/cat
100000 53 1 0 19:55 ? 00:00:00 httpd -D FOREGROUND
...
100000 130 0 1 19:57 pts/0 00:00:00 /bin/bash
100000 144 130 0 19:57 pts/0 00:00:00 ps -efww

I then run kill 22 see output on the docker run of:

/tmp/scripts/run: line 10: 22 Killed php $QUEUE_WORK

The loop doesn't keep the process up. I have double checked that the loop is the code being run in the image. If I have the loop run a "echo x: sleep 1" it runs okay.

Why doesn't the loop replace the command when I kill it?

NOTE: The application is deployed as multiple pods on Kubernetes with monitoring and automated restarts if the httpd health check URL returns an error or time-out.

I don't want to deploy the queue workers as a separate pod or as a sidecar container I want to run it as a background process in the httpd container so that it shares the application code with httpd with zero config.

I am not interested in running process monitors or other tools or technology to "keep alive".

My question is why does the Bash loop exit on kill of the process it is running?

• Jesse Chisholm: re: why didn't the while catch it? the while loop itself does not put the php command in a subshell, so the pid for the php command is effectively the pid for the while command as well. You could address this by forcing the php command into a subshell with while true; do (php args ...); done.

Answer 1

Setup

I set up the following Dockerfile:

$ more Dockerfile
From centos
ADD run.sh /tmp/run.sh
RUN chmod +x /tmp/run.sh
ENTRYPOINT ["/tmp/run.sh"]

Setup a script, run.sh:

$ cat run.sh
while true; do sleep 15 ; echo "background"; done &
while true; do sleep 12 ; echo "foreground"; done

Built it:

$ docker build -t sleeper .
Sending build context to Docker daemon 7.791 MB
Step 1/4 : FROM centos
 ---> 49f7960eb7e4
Step 2/4 : ADD run.sh /tmp/run.sh
 ---> b4099de53780
Removing intermediate container 1ce8e3a1dac5
Step 3/4 : RUN chmod +x /tmp/run.sh
 ---> Running in e410429a6cba
 ---> 06789467e636
Removing intermediate container e410429a6cba
Step 4/4 : ENTRYPOINT /tmp/run.sh
 ---> Running in ad8b847b505f
 ---> a2415df63f99
Removing intermediate container ad8b847b505f
Successfully built a2415df63f99

Example

Then started it up:

$ docker run -dit sleeper
28c19c338e6e6177529cf989f42c7f14b17f1c705a61f5244d5350f0ab8f8364

Then it runs repeatedly showing:

foreground 
background 
foreground 
background

Now when I watch it I can see that it's staying up, in spite of the sleep commands eventually "dying":

$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
28c19c338e6e sleeper "/tmp/run.sh" About a minute ago Up About a minute focused_lumiere

From above we can see that this container has been up in excess of 1 minute. Here's what the ps auxf looks like of the inside of the container:

$ ps auxf
...
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 24 0.0 0.1 11828 2964 pts/1 Ss 16:00 0:00 /bin/bash
root 58 0.0 0.1 51712 3432 pts/1 R+ 16:01 0:00 \_ ps auxf
root 1 0.0 0.1 11692 2572 pts/0 Ss+ 15:58 0:00 /bin/bash /tmp/
root 5 0.0 0.1 11696 2272 pts/0 S+ 15:58 0:00 /bin/bash /tmp/
root 56 0.0 0.0 4368 636 pts/0 S+ 16:01 0:00 \_ sleep 15
root 57 0.0 0.0 4368 664 pts/0 S+ 16:01 0:00 sleep 12

Now if we kill the background sleep 15 like so:

$ kill 56

And run another docker ps:

$ docker ps
...
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 24 0.0 0.1 11828 2964 pts/1 Ss 16:00 0:00 /bin/bash
root 60 0.0 0.1 51712 3344 pts/1 R+ 16:01 0:00 \_ ps auxf
root 1 0.0 0.1 11692 2572 pts/0 Ss+ 15:58 0:00 /bin/bash /tmp/
root 5 0.0 0.1 11696 2272 pts/0 S+ 15:58 0:00 /bin/bash /tmp/
root 59 0.0 0.0 4368 636 pts/0 S+ 16:01 0:00 \_ sleep 15
root 57 0.0 0.0 4368 664 pts/0 S+ 16:01 0:00 sleep 12

We can see that the while loop that's guarding our background sleep 15 process has done its job and restarted another sleep 15.

• simbo1905: i have accepted your answer as it shows my code will work in prod if the queue worker decides to die a natural death, i don't now why the php behaves differently. yet i have run out of curiosity for the time being. thanks.