Snort Performance and Memory Map Pcap on RHEL

I previously wrote about installing Phil Wood's memory map enabled libpcap as an academic exercise on my home network. As Victor Julien pointed out to me, Snort in inline mode will be using ip_queue rather than the libpcap interface to the kernel. However, I have actually been using mmap libpcap with Snort in IDS mode for quite a while to help reduce packet loss monitoring fairly high bandwidth connections. In testing, I have been able to consistently run Snort in IDS mode and a flow_depth of zero without packet loss on a link doing up to 80Mb/s. Admittedly, this is on pretty high end hardware but it is much better performance than Snort with the regular libpcap. By tuning Snort, like setting flow_depth to something more sensible than zero and disabling rules that are bad for performance, it is easy to see how Snort could be used to monitor even busier links.

There are other ways to increase Snort performance, too. One is using PF_RING. Richard Fifarek has a good write-up about setting up PF_RING with Red Hat Enterprise Linux 4. Maybe I'll get around to testing PF_RING under the same circumstances as I use mmap libpcap, but I'd love to see comments from those that have done it already. I have seen a few comparisons, but not recently. The documentation for PACKET_MMAP, which can be found in the Linux kernel source, has more to say about packet capture performance.

It's fine to use PACKET_MMAP to improve the performance of the capture process, but it isn't everything. At least, if you are capturing at high speeds (this is relative to the cpu speed), you should check if the device driver of your network interface card supports some sort of interrupt load mitigation or (even better) if it supports NAPI, also make sure it is enabled.

I haven't had a chance to play with NAPI yet, but for anyone that is interested there is a NAPI_HOWTO.txt, also in the documentation section of the Linux kernel source.

One last way to improve performance is to use an architecture-specific compiler when building Snort. Although I'm not sure about other architectures, using the Intel compiler for some Intel hardware was mentioned as one of the best ways to improve performance when someone asked a SourceFire employee at Shmoocon a couple years ago about improving Snort performance.

Installing the modified libpcap and reinstalling all the software that depends on it is fairly simple on RHEL. In this case, I'm using RHEL5. Before installing the new libpcap, I stopped any processes that depended on Red Hat's official libpcap package. I removed the official Red Hat libpcap package and any other RPMs that depended on it.

rpm -e libpcap tcpdump

Then I installed the new libpcap.

tar xvzf libpcap-0.9x.20070528.tar.gz
cd libpcap--0.9x.20070528
./configure
make
make install

Now I can go ahead and reinstall the other software using the new libpcap. Just for kicks, the following installs libnet so I can enable flex-resp support even though I don't consider flex-resp very useful in production. Better to just put the box inline if you really need to stop traffic. TCP resets or ICMP unreachable messages leave something to be desired.

cd /usr/src/Libnet-1.0.2a
./configure --prefix=/usr/local/libnet
make
make install

cd /usr/src/tcpdump-3.9.8/
make clean
./configure
make
make install

The following configure statement for Snort should work with 2.6.1.5 or 2.8.0, but I think the --enable-stream4udp is actually now a default for 2.8.0 while it was not for 2.6.x.x.

cd /usr/src/snort-2.8.0
make clean
./configure --without-mysql --without-postgresql --without-oracle
--without-odbc --enable-dynamicplugin --enable-flexresp --enable-stream4udp
--enable-perfprofiling --with-libnet-libraries=/usr/local/libnet/lib
--with-libnet-includes=/usr/local/libnet/include
make
make install

I'm using sancp for session data:

cd /usr/src/sancp-1.6.1/
make clean
make
./install.sh

Finally, I need to set PCAP_FRAMES to a value that the system can use. Finding the maximum can be trial and error, but here are some examples how to set it. One is to set it for each individual application at startup. For instance, sancp doesn't seem to like to use PCAP_FRAMES, so I put the following in the sancpd init script:

export PCAP_FRAMES=0

On the other hand, Snort in IDS mode or packet logging mode can benefit a lot from setting PCAP_FRAMES. For instance,

export PCAP_FRAMES=65535

As I said, it may take experimentation to find the maximum value or the value that results in the best performance. For something like tcpdump, you might want to create an alias something like:

alias tcpdump='PCAP_FRAMES=65535 tcpdump'

Tweet