Poor Man's Service

Ever want to run a permanent restarting service on a Unix machine as a non-root user?

I often use a screen + cron recipe to run long-lived processes where cron checks to see if it's running, restarting it if it isn't, and screen provides a handy detached interface to stdin/stdout etc.

So, let's call this service "service1" and make cron run a script every five minutes to check that it's OK...

 0,5,10,15,20,25,30,35,40,45,50,55 * * * *       /home/michael/service1/check

The "check" script looks for the associated named screen process that holds the service, in this case it's an Edgewall Trac server. If not found it starts it. Everything gets logged along with timestamps so we have a record if it ever goes down.

#!/bin/bash
# Make script crontab friendly:
# ==> Change to directory where this script lives.
cd $(dirname $0)

LOGFILE=service1.log
# want to redirect own stdout and stderr to logfile

exec >> $LOGFILE

echo `date`
echo "search for service1 server process..."
echo `ps auxww | grep "SCREEN -dmS service1" | grep -v grep`

if ! ps auxww | grep 'SCREEN -dmS service1' | grep -v grep > /dev/null
then
    echo "*** WARNING *** seems not to be running -- starting service"
    # do something smart here - launch it in detached screen
    /usr/bin/screen -dmS service1 /usr/bin/tracd --port 8000 \
        --basic-auth=tractest,/home/michael/tractest/.htpasswd,/home/michael/tractest \
        --basic-auth=swimtracproj,/home/michael/swimtrac/.htpasswd,/home/michael/swimtrac/swimtracproj \
        --basic-auth=rcvsim-proj,/home/michael/trac-rcvsim/.htpasswd,/home/michael/trac-rcvsim/rcvsim-proj \
	/home/michael/tractest \
	/home/michael/swimtrac/swimtracproj \
	/home/michael/trac-rcvsim/rcvsim-proj
else
    echo "seems to be running"
fi

Update --Michael Erskine (talk) 12:28, 29 February 2016 (UTC)

I found that on one of my remote servers that the exit status of the ps piped to grep was coming out wrong and I got multiple screen instances started. I introduced a better check that captures the output of the grep command and checks if for zeroness (in /usr/bin/test terms -- see http://www.tldp.org/LDP/abs/html/testconstructs.html#TTESTREF). This also means that the ps command is only run once rather than twice (I never liked that inefficiency!). The service name, i.e. the screen session name, has also been turned into a variable. In the following example we expect a screen session named "poornode" to exist. This service just runs a tail -f on its own logfile (getting a bit meta here!)

#!/bin/bash
# Make script crontab friendly:
# ==> Change to directory where this script lives.
cd $(dirname $0)

LOGFILE=cronservcheck.log
POORSERVICE=poornode
SERVICECMD="tail -f cronservcheck.log"

# want to redirect own stdout and stderr to logfile
exec >> $LOGFILE

echo `date`
echo "search for $POORSERVICE server process..."

PSOUT=`ps auxww | grep "SCREEN -dmS $POORSERVICE" | grep -v grep`

echo "output of ps is..."
echo $PSOUT

if [ -z "$PSOUT" ]
then
    echo "*** WARNING *** seems not to be running -- starting poor service"
    # do something smart here - launch it in detached screen
    /usr/bin/screen -dmS $POORSERVICE $SERVICECMD
else
    echo "seems to be running"
fi
echo "------------------------------------------------"