23 May, 2008

Snort 2.8.1 changes and upgrading

Snort 2.8.1 has been out since April, so this post is a little late. I wanted to upgrade some Red Hat/CentOS systems from Snort to 2.8.1. When I write RHEL or Red Hat, it will include CentOS since what is applies to one should apply to the other.

I quickly found that the upgrade on RHEL4 was not exactly straightforward because the version of pcre used on RHEL4 is pcre 4.5, which is years old. Snort 2.8.1 requires at least pcre 6.0. For reference, RHEL5 is using pcre 6.6.

PCRE Changes

I had assumed an upgrade from to 2.8.1 was minor, but the pcre change indicated there were definitely significant changes between versions. A few of the pcre changes made it into the Snort ChangeLog, and there is more in the Snort manual. I also had a brief discussion about the pcre changes with a few SourceFire developers that work on Snort and some other highly technical Snort users.

The most significant changes seems to be adding limits to pcre matching to help prevent performance problems and denial of service through pcre overload. At regular expression compile time, there will be a maximum number of state changes that can be passed and on evaluation libpcre will only change states that many times. There is a global config option that sets the maximum number of state changes and the limit can also be disabled per regular expression if needed.

The related configuration options for snort.conf are pcre_match_limit:

Restricts the amount of backtracking a given PCRE option. For example, it will limit the number of nested repeats within a pattern. A value of -1 allows for unlimited PCRE, up to the PCRE library compiled limit (around 10 million). A value of 0 results in no PCRE evaluation. The snort default value is 1500.
and pcre_match_limit_recursion:
Restricts the amount of stack used by a given PCRE option. A value of -1 allows for unlimited PCRE, up to the PCRE library compiled limit (around 10 million). A value of 0 results in no PCRE evaluation. The snort default value is 1500. This option is only useful if the value is less than the pcre_match_limit
The main discussion between the SourceFire folks and everyone else was whether it was wise to have the limits turned on by default and where the default of 1500 came from. I think leaving the pcre limits on by default makes sense because those that don't fiddle much with the Snort configuration probably need the protection of the limits more than someone that would notice when Snort performance was suffering.

The argument against having the limits on by default is that it could make certain rules ineffective. By cutting short the pcre, the rule may not trigger on traffic that should cause a Snort alert. I suspect that not many rules will hit the default pcre limit.

Other Changes Included in Snort 2.8.1

From the release notes:
[*] New Additions
* Target-Based support to allow rules to use an attribute table
describing services running on various hosts on the network.
Eliminates reliance on port-based rules.

* Support for GRE encapsulation for both IPv4 & IPv6.

* Support for IP over IP tunneling for both IPv4 & IPv6.

* SSL preprocessor to allow ability to not inspect encrypted traffic.

* Ability to read mulitple PCAPs from the command line.
The SSL/TLS preprocessor helps performance by allowing Snort to ignore encrypted traffic rather than inspecting it. I haven't looked at the target-based support yet, but it definitely sounds interesting.

I also noticed something from 2007-12-10 in the ChangeLog:
 * configure.in:
Add check for Phil Woods pcap so that pcap stats are computed
correctly. Thanks to John Hally for bringing this to our
That should be good for those running Phil Wood's pcap who may not have been seeing accurate statistics about packet loss.

A last note of interest about Snort 2.8.1 is that it fixes a vulnerability related to improper reassembly of fragmented IP packets.


Upgrading on RHEL5 was pretty simple, but upgrading on RHEL4 required downloading the source RPM for the pcre included with RHEL5 and building a RPM for RHEL4.

First, I installed compilers and the rpm-build package and the RPM source package for pcre-6.6. (Note that I'm using CentOS4 in the example, but the procedures for Red Hat should be nearly identical except for URLs). Next I built the pcre-6.6 RPMs and installed both pcre and pcre-devel.
# up2date -i cpp gcc gcc-c++ rpm-build
# rpm -Uvh http://isoredirect.centos.org/centos/5/updates/SRPMS/pcre-6.6-2.el5_1.7.src.rpm
$ rpmbuild -bb /usr/src/redhat/SPECS/pcre.spec
# rpm -U /usr/src/redhat/RPMS/i386/pcre-*rpm
# rpm -q pcre pcre-devel
Now I can configure, make and make install Snort as usual. The snort.conf file will also need to be updated to reflect new options like the SSL/TLS preprocessor and non-default pcre checks. Note that the README.ssl also recommends changes to the stream5 preprocessor based on the SSL configuration.
$ ./configure --enable-dynamicplugin --enable-stream4udp --enable-perfprofiling
$ make
# make install
If you are building on a production box instead of a development box, some would recommend removing the compilers afterwards.

Here is an example of a SSL/TLS and stream5 configuration using default ports. By adding the default SSL/TLS ports to stream5_tcp settings, you also have to enumerate the default stream5_tcp ports or they will not be included. I bolded the default SSL/TLS ports, and the rest are default stream5_tcp ports:
preprocessor stream5_global: track_tcp yes, track_udp yes, track_icmp yes
preprocessor stream5_tcp: policy first, ports both 21 23 25 42 53 80 \
110 111 135 136 137 139 143 443 445 465 513 563 636 \
989 992 993 994 995 1433 1521 3306
preprocessor stream5_udp
preprocessor stream5_icmp
preprocessor ssl: \
I have tested to make sure Snort runs with this configuration for these preprocessors, but don't just copy it without checking my work. It is simply an example after reading the related README documentation. Note that the Stream 5 TCP Policy Reassembly Ports output to your message log when starting Snort will be truncated after the first 20 ports but all the ports listed will be included by Stream 5.

16 May, 2008

Query sguildb for alert name then do name lookup

I wrote a Perl script to query for a specific alert name and then get the NetBIOS or DNS name of the systems that triggered the alert. This script is useful to me mainly as an automated reporting tool. An example would be finding a list of systems associated with a specific spyware alert that is auto-categorized within Sguil. The alert will never show up in the RealTime alerts but I use the script to get a daily email on the systems triggering the alert.

Some alerts aren't important enough to require a real-time response but may need remediation or at least to be included in statistics. I don't know if this would be useful for others as it is written, but parts of it might be.

As always with Perl, I welcome suggestions for improvement since I'm by no means an expert. (By the way, does anyone else have problems with Blogger formatting when using <pre> tags?)

# by nr
# 2008-05-11
# Script to query db for a specific alert
# and do name lookup based on source IP address in results

use strict;

# Requires MySQL, netbios and DNS modules
use Mysql;
use Net::NBName;
use Net::DNS;

my $host = "localhost";
my $database = "sguildb";
my $tablename = "event";
my $user = "sguil";
my $pw = "PASSWORD";
my $alert = 'ALERT NAME Here';

# Set the query
my $sql_query = "SELECT INET_NTOA(src_ip) as src_ip,count(signature) as count FROM $tablename \
WHERE $tablename.timestamp > DATE_SUB(UTC_DATE(),INTERVAL 24 HOUR) AND $tablename.signature \
= '$alert' GROUP BY src_ip";

# perl mysql connect()
my $sql_connect = Mysql->connect($host, $database, $user, $pw);

print "\"$alert\" alerts in the past 24 hours: \n\n";
print "Count IP Address Hostname\n\n";

my $execute = $sql_connect->query($sql_query); # Run query

# Fetch query results and loop
while (my @result = $execute->fetchrow) {
my @hostname; # 1st element of this array is used later for name queries
my $ipresult = $result[0]; # Set IP using query result
my $count = $result[1]; # Set alert count using query result
my $nb_query = Net::NBName->new; # Setup new netbios query
my $nb = $nb_query->node_status($result[0]);
# If there is an answer to netbios query
if ($nb) {
my $nbresults = $nb->as_string; # Get query result
# Split at < will make $hostname[0] the netbios name
# Is there a better way to do this using a substitution?
@hostname = split /</, $nbresults;
} else {
# Do a reverse DNS lookup if no netbios response
# May want to add checks to make sure external IPs are ignored
my $res = Net::DNS::Resolver->new;
my $namequery = $res->query("$result[0]","PTR");
if ($namequery) {
my $dnsname = ($namequery->answer)[0];
$hostname[0] = $dnsname->rdatastr;
} else {
$hostname[0] = "UNKNOWN"; # If no reverse DNS result
format STDOUT =
@>>> @<<<<<<<<<<<<<< @*
$count, $ipresult, $hostname[0]