Transparent Bridging, MMAP pcap, and Snort Inline

I use Snort and the Sguil Analyst Console for NSM, but there is always room to experiment and/or improve. Up to this point, I have used Snort either on mirrored (SPAN) ports or with a network TAP, both common configurations. After finally getting a second-hand CPU and motherboard to replace a dead CPU, I had a spare system to set up Snort inline at home.

The upgrade from Snort 2.4.x to 2.6.x.x was quite taxing on performance, so I decided it was also time to play with Phil Wood's MMAPped libpcap. The modified libpcap will make drastically fewer system calls when compared to the standard libpcap sniffing on a busy network. Although my home network certainly isn't high bandwidth, I wanted the experience of setting up Snort with the Phil Wood's modified version of libpcap. Since I actually did all this many months ago and am just now posting about it, I can say that I have seen a huge performance improvement when going from the standard libpcap to Phil Wood's libpcap in high bandwidth environments.

How would I implement Snort Inline on a home network? The two choices were to replace one of my routers with a BSD or Linux system configured as a router, or to set up Linux as a transparent bridge. For those that prefer FreeBSD, you would have to configure it as a router since the bridge code in FreeBSD does not support the ipfw divert socket. I am not familiar enough with other BSD versions to say whether their bridge code is the same or not.

I much preferred a bridge rather than a router since it would avoid the time-consuming process of reconfiguring my network topology. This meant that I had to use Linux, and my distribution in this case is Slackware-current. The process should not be much different for any distribution.

Installing Software

Because plugging in an untested bridge between my LAN and the Internet could interrupt my connection, I decided it would be easiest to get and install all the software prior to configuring the bridge and putting it inline.

My first step was to install the modified libpcap, which needs either flex and bison, or yacc. This was essentially a freshly built Slackware system and I didn't have them installed, so I used swaret to install the packages.

swaret --install flex
swaret --install bison

I was now ready to install libpcap.

cd /usr/src/
wget http://public.lanl.gov/cpw/libpcap-current.tar.gz
tar xvzf libpcap-current.tar.gz
ln -s libpcap-0.9.20060417 libpcap
cd libpcap
./configure
make
make install

Snort will need the header files from libpcap and the install did not copy them anywhere, so I manually copied the files to /usr/include/. Another option would be to create a link to the files in the include directory.

cp /usr/src/libpcap/pcap.h /usr/include/
cp /usr/src/libpcap/pcap-bpf.h /usr/include/
cp /usr/src/libpcap/pcap-namedb.h /usr/include/

Because this is a modified libpcap, all the software that depends on libpcap must also be compiled against the version I just installed. I will definitely be using tcpdump when I test the bridge.

wget http://www.tcpdump.org/daily/tcpdump-current.tar.gz
tar xvzf tcpdump-current.tar.gz
ln -s tcpdump-2007.01.07 tcpdump
cd tcpdump
./configure
make
make install

Now I could test that tcpdump worked with the PCAP_FRAMES option available because of the modified libpcap. For some reason, perhaps because of my kernel version, PCAP_FRAMES=max did not work but I was able to use it by manually setting the value. I was able to bump the value of PCAP_FRAMES quite high, above 300000, before it resulted in errors. I have yet to determine what that really means for performance. Here are two commands I used to show that the newly installed tcpdump worked with the modified libpcap.

PCAP_FRAMES=65535 PCAP_VERBOSE=1 PCAP_TO_MS=0 PCAP_PERIOD=10000 /usr/local/sbin/tcpdump \
-i eth0 -s 1514 -w /dev/null -c 100

PCAP_FRAMES=65535 /usr/local/sbin/tcpdump -v -i eth0

Snort needs libnet-1.0.2a when configured with --enable-inline, so I had to install libnet first.

tar xvzf libnet-1.0.2a
cd libnet-1.0.2a
./configure
make
make install

Finally, install Snort 2.6.x.x. (Note: Since this document was written quite a while ago, there are quite a few newer versions of Snort available). Another alternative to using the --enable-inline option with mainline Snort is to download snort_inline, which is maintained by William Metcalf and Victor Julien. There are a number of added features and conveniences when using snort_inline, as highlighted on Victor's blog. However, I used mainline Snort in the following example.

tar xvzf snort-2.6.1.2.tar.gz
cd snort-2.6.1.2
./configure --enable-dynamicplugin --enable-inline
make
make install

I tested snort with -V to check that it would start and was compiled to work inline. It shows that Snort was configured with the inline option.

snort -V

 ,,_     -*> Snort! <*-
o"  )~   Version 2.6.1.2 (Build 34) inline
 ''''    By Martin Roesch & The Snort Team: http://www.snort.org/team.html
         (C) Copyright 1998-2006 Sourcefire Inc., et al. 

Configuring Linux for Transparent Bridging

Configuring the bridging is fairly simple. The only module I had to manually insert was ip_queue. Other modules that may be needed are ip_tables, iptable_filter and bridge.

In this case, eth0 is my separate management interface, I named the bridge interface bridge0, and the physical interfaces joining bridge0 were eth1 and eth2. The bridge device can be configured with an IP address if you do not want to use a separate NIC for management. In either case, make sure to secure the management NIC on your Snort box, for example limiting connections to the management IP so only source IP addresses in your private IP space are allowed to connect. Here, I created the bridge interface, added eth1 and eth2 to it, and brought them up:

/sbin/brctl addbr bridge0
/sbin/brctl addif bridge0 eth1
/sbin/brctl addif bridge0 eth2
/sbin/ifconfig eth1 up
/sbin/ifconfig eth2 up
/sbin/ifconfig bridge0 up

After some iptables configuration, bridge0 should work. Assuming the system is dedicated to being a bridge running Snort Inline, the only addition necessary to make bridging work is the following:

iptables -I FORWARD -o bridge0 -j ACCEPT

Now bridge0 should be ready to pass packets.

Once the bridge is connected, iptables can be used to show packet statistics and confirm that the bridge is forwarding. You can also use tcpdump -v -i bridge0 to confirm that traffic is passing if you skip ahead to installing libpcap and tcpdump prior to plugging in the bridge.

iptables -vL
--snip--
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source    destination
527  168K ACCEPT     all  --  any    bridge0 anywhere  anywhere
--snip--

The bridge is seeing packets! tcpdump showed that they were more than just broadcast packets as I accessed the Internet through the bridge.

PCAP_FRAMES=65535 /usr/local/sbin/tcpdump -v -i bridge0
tcpdump: WARNING: bridge0: no IPv4 address assigned
tcpdump: listening on bridge0, link-type EN10MB (Ethernet), capture
--snip--
57 packets captured
57 packets received by filter
0 packets dropped by kernel 

Testing Snort Inline

Finally, I'm ready to configure and test Snort Inline. I am not going to cover Snort configuration, but I did write one test rule to put in local.rules and disabled all the other rule sets in snort.conf. The following rule should drop any outbound connection on the HTTP_PORT set in snort.conf.

drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Test rule outbound HTTP"; \
classtype:misc-activity; sid: 3000000;)

Now I run Snort. The -v will print packets to the console so I can confirm that snort is seeing the traffic and the -Q tells snort to accept input from the iptables queue.

/usr/local/bin/snort -Qvc /etc/snort/snort.conf -l /data

Note that I forgot to add the PCAP_FRAMES value. In production on a busy network, I would add it permanently to my environmental variables and/or to my init scripts for Snort so it would always take advantage of Phil Wood's libpcap.

I add the necessary rule to iptables. I don't want to risk losing all connections by queueing everything, so I just queue port 80. This is what the line in my iptables-save file looks like:

-A FORWARD -i bridge0 -p tcp -m tcp --dport 80 -m state --state NEW -j QUEUE

I try to browse to a web site and get the following alert in /data/alert:

[**] [1:300000:0] Test rule outbound HTTP [**]
[Classification: Misc activity] [Priority: 3]
01/11-22:22:24.458394 xx.xx.xx.xx:60813 -> 66.35.250.209:80
TCP TTL:63 TOS:0x0 ID:52831 IpLen:20 DgmLen:60 DF
******S* Seq: 0xE79DA525  Ack: 0x0  Win: 0x16D0  TcpLen: 40
TCP Options (5) => MSS: 1460 SackOK TS: 604992578 0 NOP WS: 2

Everything is working. Now I have to get everything ready for production by fully configuring and tuning Snort. Once that is done, I will queue all traffic except port 22. Queuing port 22 could result in not being able to connect using SSH if the Snort process were to die or had to be restarted. If snort-inline is not running, all queued traffic is effectively dropped since Snort is required to pass the traffic from the queue back to iptables.

Tweet