30 November, 2007

A "for" loop with tcpdump

This is the kind of post to which most people with Linux or Unix experience will say "Duh!", or even offer better ways to do what I'm doing. It's just a short shell script I use to loop through all the packet capture files on a Sguil sensor looking for specific host(s) and/or port(s). Sometimes it is easier to run a script like this and then SCP the files from the sensor to a local system instead of querying and opening the packet captures one by one using the Sguil client. This also allows more leisurely analysis without worrying about the log_packets script deleting any of the data I want to see.

Nigel Houghton mentioned that any shell script of more than 10 or 20 lines should be rewritten in Perl. Despite his advice, I do plan to switch some of my shell scripts to Perl when I get the chance. ;)

The sensorname variable works for me because my sensor uses the hostname as the sensor name. If my sensor name didn't match the actual hostname or I had multiple sensors on one host then I would have to change the variable.

The BPF expression can be changed as needed. Maybe a better way would be to set it to a command line argument so you don't have to edit the file every time you run the script. The following example uses a Google IP address and port 80 for the BPF. I most often use this script to simply grab all traffic associated with one or more IP addresses.

It may also be worth noting that I run the script as a user that has permissions to the packet captures.

#!/bin/sh

sensorname="`hostname`"
bpfexpression="host 64.233.167.99 and port 80"
outdir=/home/nr/scriptdump

if [ ! -d $outdir]; then
mkdir $outdir
fi

# For each dir in the dailylogs dir
for i in $( ls /nsm/$sensorname/dailylogs/ ); do
# For each file in dailylogs/$i dir
for j in $( ls /nsm/$sensorname/dailylogs/$i ); do
# Run tcpdump and look for the host
tcpdump -r /nsm/$sensorname/dailylogs/$i/$j -w $outdir/$j.pcap $bpfexpression
done
done

# For each pcap
cd $outdir
for file in $( ls *.pcap ); do
# If file has size of 24, it has no data so rm file
if [ "`ls -l $file | awk '{print $5}'`" = "24" ]; then
rm -f "$file"
fi
done
One of the things I like about posting even a simple script like this is that it makes me really think about how it could be improved. For instance, it might be nice to add a variable for the write directory so it is easier to change where the output files go.

Also, in Sguil the full content packet captures use the Unix time in the file names. If you ever had a copy of the file where the timestamp wasn't preserved, you could still find it by looking at the epoch time in the file name. For instance, with a file named "snort.log.1194141602", the following would convert the epoch time to local time:
$ date -d @1194141602
Sat Nov 3 22:00:02 EDT 2007

29 November, 2007

Origin of Eating Security

When I started this blog, I couldn't really think of a name for the blog. I figured that maybe someone else could do the job for me, so I searched the Internet for "blog name generator". I didn't really find anything useful, so then I tried searching for a band name generator. I plugged the word "security" into the first hit, and that's how I ended up Eating Security.

16 November, 2007

Bleeding Edge Founder Steps Down

The founder of Bleeding Edge Threats, Matt Jonkman, has stepped down. I don't think this could be considered anything but bad news for the project. Apparently the project now belongs to Sensory Networks, the main sponsor of the last 12 months. I believe the rules are BSD licensed, so while they are currently free, who knows what will happen in the future.

I'm sure Matt Jonkman will continue to do some interesting things in the future, and it sounds like he hopes to share with the rest of us again.

01 November, 2007

Snort Performance and Memory Map Pcap on RHEL

I previously wrote about installing Phil Wood's memory map enabled libpcap as an academic exercise on my home network. As Victor Julien pointed out to me, Snort in inline mode will be using ip_queue rather than the libpcap interface to the kernel. However, I have actually been using mmap libpcap with Snort in IDS mode for quite a while to help reduce packet loss monitoring fairly high bandwidth connections. In testing, I have been able to consistently run Snort in IDS mode and a flow_depth of zero without packet loss on a link doing up to 80Mb/s. Admittedly, this is on pretty high end hardware but it is much better performance than Snort with the regular libpcap. By tuning Snort, like setting flow_depth to something more sensible than zero and disabling rules that are bad for performance, it is easy to see how Snort could be used to monitor even busier links.

There are other ways to increase Snort performance, too. One is using PF_RING. Richard Fifarek has a good write-up about setting up PF_RING with Red Hat Enterprise Linux 4. Maybe I'll get around to testing PF_RING under the same circumstances as I use mmap libpcap, but I'd love to see comments from those that have done it already. I have seen a few comparisons, but not recently. The documentation for PACKET_MMAP, which can be found in the Linux kernel source, has more to say about packet capture performance.

It's fine to use PACKET_MMAP to improve the performance of the capture process, but it isn't everything. At least, if you are capturing at high speeds (this is relative to the cpu speed), you should check if the device driver of your network interface card supports some sort of interrupt load mitigation or (even better) if it supports NAPI, also make sure it is enabled.
I haven't had a chance to play with NAPI yet, but for anyone that is interested there is a NAPI_HOWTO.txt, also in the documentation section of the Linux kernel source.

One last way to improve performance is to use an architecture-specific compiler when building Snort. Although I'm not sure about other architectures, using the Intel compiler for some Intel hardware was mentioned as one of the best ways to improve performance when someone asked a SourceFire employee at Shmoocon a couple years ago about improving Snort performance.

Installing the modified libpcap and reinstalling all the software that depends on it is fairly simple on RHEL. In this case, I'm using RHEL5. Before installing the new libpcap, I stopped any processes that depended on Red Hat's official libpcap package. I removed the official Red Hat libpcap package and any other RPMs that depended on it.
rpm -e libpcap tcpdump
Then I installed the new libpcap.
tar xvzf libpcap-0.9x.20070528.tar.gz
cd libpcap--0.9x.20070528
./configure
make
make install
Now I can go ahead and reinstall the other software using the new libpcap. Just for kicks, the following installs libnet so I can enable flex-resp support even though I don't consider flex-resp very useful in production. Better to just put the box inline if you really need to stop traffic. TCP resets or ICMP unreachable messages leave something to be desired.
cd /usr/src/Libnet-1.0.2a
./configure --prefix=/usr/local/libnet
make
make install

cd /usr/src/tcpdump-3.9.8/
make clean
./configure
make
make install
The following configure statement for Snort should work with 2.6.1.5 or 2.8.0, but I think the --enable-stream4udp is actually now a default for 2.8.0 while it was not for 2.6.x.x.
cd /usr/src/snort-2.8.0
make clean
./configure --without-mysql --without-postgresql --without-oracle
--without-odbc --enable-dynamicplugin --enable-flexresp --enable-stream4udp
--enable-perfprofiling --with-libnet-libraries=/usr/local/libnet/lib
--with-libnet-includes=/usr/local/libnet/include
make
make install
I'm using sancp for session data:
cd /usr/src/sancp-1.6.1/
make clean
make
./install.sh
Finally, I need to set PCAP_FRAMES to a value that the system can use. Finding the maximum can be trial and error, but here are some examples how to set it. One is to set it for each individual application at startup. For instance, sancp doesn't seem to like to use PCAP_FRAMES, so I put the following in the sancpd init script:
export PCAP_FRAMES=0
On the other hand, Snort in IDS mode or packet logging mode can benefit a lot from setting PCAP_FRAMES. For instance,
export PCAP_FRAMES=65535
As I said, it may take experimentation to find the maximum value or the value that results in the best performance. For something like tcpdump, you might want to create an alias something like:
alias tcpdump='PCAP_FRAMES=65535 tcpdump'