18 March, 2008

Using DJ Bernstein's daemontools

I use DJ Bernstein's daemontools to monitor Barnyard, making sure the barnyard process will restart if it dies for any reason. Barnyard is an output spooler for Snort and is probably the the least stable of all the software that is used when running Sguil. When Barnyard encounters errors and exits, it needs to be restarted.

Daemontools is useful because it will watch a process and restart it when needed. For anyone that has used other DJ Bernstein software like djbdns or qmail, you may also have used daemontools. I think daemontools has a reputation as difficult to install and configure, but I've used it on a number of systems with barnyard or djbdns without any major issues. (As for qmail, I prefer postfix).

Here is how I installed it, which only has one small change from the install instructions.

mkdir -p /package
chmod 1755 /package
tar xzvpf install/daemontools-0.76.tar.gz -C /package/
cd /package/admin/daemontools-0.76/
Before running the install script, note the "errno" section on DJ Bernstein's Unix portability notes. On Linux, since I'm installing from source I need to replace one line in the src/error.h file, as shown in this patch snippet.
-extern int errno;
+#include <errno.h>
After changing error.h, I can run the installer.
./package/install
I configure daemontools to work with barnyard.
mkdir /etc/barnyard
vim /etc/barnyard/run
The "run" file simply is a script that runs barnyard. For example, the contents of mine:
#!/bin/sh
exec /bin/barnyard -c /etc/snort/barnyard.conf -d /nsm -f unified.log \
-w /nsm/waldo.file -a /nsm/by_archive
Next, I link the new barnyard directory to make it a subdirectory of daemontool's service directory.
ln -s /etc/barnyard /service/barnyard
When installing, daemontools automatically adds this entry to /etc/inittab:
SV:123456:respawn:/command/svscanboot
svscanboot starts a svscan process that then starts a supervise process for each subdirectory, which in this case would only be the barnyard directory. I can have the inittab file re-parsed with telinit q after daemontools is installed rather than rebooting.

If the barnyard process dies, daemontools will automatically try and restart it based on the contents of the "run" file.

Now, even if I kill the barnyard process on purpose then it will be restarted automatically. If I need to manage the process, I can use the svc command. For instance, to send barnyard a HUP or a KILL:
svc -h /service/barnyard
svc -k /service/barnyard
To add another process for daemontools to manage, just create a directory, create a run file, then link the new directory to daemontools' service directory.
mkdir /etc/someprocess
vim /etc/someprocess/run
ln -s /etc/someprocess /service/someprocess

1 comment:

  1. Nice example. I installed the daemontools-run package from the Debian repository and simply followed your instructions. Worked great. The only modification I made was that the Debian package creates the service directory in /etc/service, so all I did is change the link. Good stuff.

    ReplyDelete