09 May, 2009

Extracting emails from archived Sguil transcripts

Here is a Perl script I wrote to extract emails and attachments from archived Sguil transcripts. It's useful for grabbing suspicious attachments for analysis.

In Sguil, whenever you view a transcript it will archive the packet capture on the Sguil server. You can then easily use that packet capture to pull out data with tools like tcpxtract or tcpflow along with Perl's MIME::Parser in this case. The MIME::Parser code is modified from David Bianco's blog.

As always with Perl or other scripts, I welcome constructive feedback. The first regular expression is fairly long and may scroll off the page, so make sure you get it all if you copy it.

#!/usr/bin/perl

# by nr
# 2009-05-04
# A perl script to read tcpflow output files of SMTP traffic.
# Written to run against a pcap archived by Sguil after viewing the transcript.
# 2009-05-07
# Updated to use David Bianco's code with MIME::Parser.
# http://blog.vorant.com/2006/06/extracting-email-attachements-from.html

use strict;
use MIME::Parser;

my $fileName; # var for tcpflow output file that we need to read
my $outputDir = "/var/tmp"; # directory for email+attachments output

if (@ARGV != 1) {
print "\nOnly one argument allowed. Usage:\n";
die "./emailDecode.pl /path/archive/192.168.1.13\:62313_192.168.1.8\:25-6.raw\n\n";
}

$ARGV[0] =~ m
/.+\/(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}):(\d{1,5})_(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}):(25)-\d{1,3}\.raw/
or die "\nIncorrect file name format or dst port is not equal to 25. Try again.\n\n";

system("tcpflow -r $ARGV[0]"); # run tcpflow w/argument for path to sguil pcap

my $srcPort = sprintf("%05d", $2); # pad srcPort with zeros
my $dstPort = sprintf("%05d", $4); # pad dstPort with zeros

# Put the octest and ports into array to manipulate into tcpflow fileName
my @octet = split(/\./, "$1\." . "$srcPort\." . "$3\." . "$dstPort");

foreach my $octet(@octet) {
my $octetLength = length($octet); # get string length
if ($octetLength < 5) { # if not a port number
$octet = sprintf("%03d", $octet); # pad with zeros
}
$fileName = $fileName . "$octet\."; # concatenate into fileName
}

$fileName =~ s/(.+\d{5})\.(.+\d{5})\./$1-$2/; # replace middle dot with hyphen
my $unusedFile = "$2-$1"; # this is the other tcpflow output file

# open the file and put it in array
open INFILE, "<$fileName" or die "Unable to open $fileName $!\n";
my @email = <INFILE>;
close INFILE;

my $count = 0;
# skip extra data at beginning
foreach my $email(@email) {
if ($email =~ m/^Received:/i) {
last;
}
else {
delete @email[$count];
$count ++;
}
}

my $parser = new MIME::Parser;
$parser->output_under("$outputDir");
my $entity = $parser->parse_data(\@email); # parse the tcpflow data
$entity->dump_skeleton; # be verbose when dumping

unlink($fileName, $unusedFile); # delete tcpflow output files

07 May, 2009

Do we need anti-virus software?

My friend Richard has a good post about Verizon's 2009 Data Breach Report. One of his last comments really struck me since it is something I have seen firsthand again and again.

Most companies are probably relying on their anti-virus software to save them. This is too bad, because the explosion in customized malware means it probably won't.
Anti-virus software just does not work against most recent malware. The table from the Verizon report shows a drastic upswing in customized malware and my experience tells me that doesn't tell half the story. Even only small changes will often evade anti-virus software.

I'm not saying anything new here. Anyone that does penetration tests, reverse engineers malware, writes exploits, or is involved with information security in a number of ways already knows that anti-virus software is terrible at detecting new malware. I have even written about it before and pointed out that more subtle methods of exploitation aren't always necessary because of the effectiveness of commodity malware.

My question is, do we really need anti-virus software?

When you take into account the amount of resources spent running anti-virus in the enterprise, is it a good investment in risk reduction? We pay for hours worked to setup the anti-virus infrastructure, update, and troubleshoot. If you are in an enterprise, you're paying for the software, not using a free alternative. You're probably paying for support and also paying for hardware.

What does it get you? I find malware on a weekly basis, sometimes daily, that is not detected by the major vendors. I submit the malware to some of these vendors and places like VirusTotal, but the responses from anti-virus vendors are inconsistent at best. Even after definitions are updated, I'll then run across malware that is obviously just an altered version of the previous but is once again not detected.

I don't pretend to have the answers, but I do wonder if all the resources spent on anti-virus by a business, particularly large or enterprise businesses, might be better spent somewhere else. Is it really worth tens or hundreds of thousands of dollars in software, hours, and hardware to make sure old malware is detected? If not, how much is it worth? Does the occasional quick response to emerging malware make it more worthwhile? If you have enough influence on the vendor, does being able to contact them directly to help protect against a specific attack make it more valuable?

Anti-virus software is too ingrained in corporate culture to think it is realistic that companies will stop using it altogether, but we need to keep asking these types of questions.

28 April, 2009

Building an IR Team: People

For some time I have been thinking about a series of posts about building an incident response team. I started in security as part of a very small Computer Incident Response Team (CIRT) that handled network security monitoring (NSM) and the ensuing security incidents. Although we were small, we had a very good core of people that helped us succeed and grow, probably well beyond anything we had imagined. We grew from a handful of people to four or five times our original size. While there were undoubtedly setbacks, we constantly got better and more efficient as the team grew.

As the first in this series, I definitely want to concentrate on people. I don't care what fancy tools, enormous budget, buy-in from management, or whatever else you have. If you don't have the right people, you'll have serious problems succeeding. Most of this is probably not unique to a response team, information security, or information technology.

Hiring
Of course, hiring is where it all starts. What do you look for in a candidate for an incident response team? Here are some of the things I look for.

  • Initiative: The last thing I want is someone that constantly needs hand-holding. Certainly people need help sometimes, and sharing knowledge and mentoring are huge, but you have to be able to work through the bumps and find solutions. A NSM operation or CIRT is not a help desk. Although you can have standard procedures, you have to be flexible, adapt, do a lot of research, and teach yourself whenever possible.
  • Drive: Most people who are successful in security seem to think of it as more than a job. They spend free time hacking away at tools, breaking things, fixing things, researching, reading, and more. I don't believe this kind of drive has to be all-consuming, because I certainly have plenty of outside interests. However, generally speaking there is plenty of time to be interested in information security outside of work while still having a life. I, and undoubtedly many successful security professionals, enjoy spending time reading, playing with new tools, and more. Finding this type of person is not actually difficult, but it can take some patience. Local security groups or mailing lists are examples of places to look for analysts to add to a team. Even if they have little work experience, by going to a group meeting or subscribing to mailing lists, they are already demonstrating some drive and initiative.
  • Communication skills: Although this may be more important for a senior analyst, being able to write and speak well is crucial. Knowing your audience is one of the most important skills. For instance, if you are writing a review of a recent incident that includes lessons learned, the end product will be different depending whether the review is for management or the incident responders on the team. Documentation, training, and reporting are other examples where good writing and speaking skills are important. I think good communication skills are underrated by many people in the field and IT in general, but the higher you look the better the chance you will find someone that realizes the importance of effective communication.
  • Background: Most of the successful NSM analysts and incident responders I know have a background in one or more of three core areas; networking, programming, or system administration. A person from each background will often have different strengths, so understanding the likely strengths of each background can go a long way toward filling a missing need on the team. You do not have to come from one of these backgrounds, it is just relatively common for the good analysts I know to have backgrounds in these areas.
  • The wrong candidate in the wrong position: Do not be scared to turn down people that are wrong for the job. That seems obvious, but it is worth emphasizing. Along the same lines, if someone is not working out, take steps to correct the problems if possible, but do not be afraid to get rid of a person that is not right for the job. Try to understand exactly what you are looking for and where in the organization the person is most likely to excel.
Experience versus Potential
When filling a senior position, experience is definitely important. However, when filling a junior position I think automatically giving a lot of weight to information security experience can be a mistake. The last thing I want to do is hire someone who has experience but is overly reliant on technology rather than critical thinking skills. I don't mean to denigrate or automatically discount junior analysts that have experience, I just mean that I'd rather have someone with a lot of potential that needs a little more training in the beginning than what some would call a "scope dope", someone whose experience is looking at IDS alerts and taking them at face value with little correlation or investigation. If you have both experience and potential, great!

Training
Information security covers a huge range of topics, requires a wide range of skills, and changes quickly. Good analysts will want training, and if you don't give it to them you will wind up with a bunch of people that don't care about increasing their knowledge and skills as the ones that do want to learn look for greener pastures.

There are many different types of training in addition to what most people think of first, which is usually formal classes. Senior analysts mentoring junior analysts can be one of the most useful types of training because it is very adaptable and can be very environment-specific. "Brown-bag" sessions where people get together over lunch and analyze a recent incident or learn how to more efficiently investigate incidents can also work well. Finally, when someone researches and learns new things on one's own or with coworkers as mentioned previously, that is also an excellent form of training. Load up a lab, attack it, and look at the traffic and resulting state of the systems.

Finally, do not forget about both undergraduate and graduate degrees. Though you may not consider them training, most people want to have the option open to either finish a degree or get an advanced degree in their off hours. There are a huge number of ways to provide training.

People versus Technology
Analysts are not the only ones that can overly rely on technology. Management will often take the stance that paying a bunch of money for tools and subscriptions means two things. One, that the systems must be what they need and will do all the work for them. Two, that the money means that the selling company has the systems optimally designed and configured for your environment. Just because you pay five or six digits for an IPS, IDS, anomaly detection, or forensics tools does not mean that you can presume a corresponding decrease in the amount you need to spend on people. Any tool is worthless without the right people using it.

Turnover, Retention, Mobility, and Having Fun
Creating and continuing a successful response team is to make sure the people you want to keep remain happy. There are a lot of things you need to retain the right people, including competitive pay, decent benefits, a chance for promotion, and a good work environment. Honestly, I think work environment is probably the most important factor. I know many analysts I have worked with receive offers of more money, but a good work environment has usually kept them from leaving. My boss has always said that the right environment is worth $X dollars to him, and I feel the same way. Effective and enjoyable coworkers, management that listens, and all the little things are not worth giving up without substantial reasons. Some opportunities are impossible to pass up, but having an enjoyable work environment and management that "gets it" goes a long way towards reducing turnover.

Bottom Line
I believe getting a good group assembled is the most important thing to have an effective response team. Obviously, I kept the focus of this post relatively broad. I would love to see comments with additional input. I hope to post additional material about building a response team in the near future, possibly covering organizing the team, dealing with growth, and a few other topics.

10 April, 2009

Upgrading to Snort 2.8.4

Hopefully, everyone running Snort has been paying attention and noticed that Snort 2.8.4 was released on 07 April and the corresponding new rule sets were released on 08 April. If you pay for Snort rules, the new netbios.rules will not work with Snort versions prior to 2.8.4, so you need to upgrade.

Upgrading was mostly painless just to get Snort running, though the dcerpc2 preprocessor settings definitely may require tweaking beyond the defaults. Generally, when I'm upgrading and there is something like a new preprocessor, I will start by getting Snort upgraded and successfully running with the defaults for the preprocessor, then will tune the settings as needed after I'm sure the new version is working.

The first thing to do after downloading and extracting is read the RELEASENOTES and also any applicable READMEs. Since the dcerpc2 preprocessor is new, the README.dcerpc2 may be of particular interest. A lot of people don't realize how much documentation is included.

$ wget http://www.snort.org/dl/snort-2.8.4.tar.gz
$ tar xzf snort-2.8.4.tar.gz
$ cd snort-2.8.4/doc
$ ls
AUTHORS README.alert_order README.ppm
BUGS README.asn1 README.sfportscan
CREDITS README.csv README.ssh
INSTALL README.database README.ssl
Makefile.am README.dcerpc README.stream5
Makefile.in README.dcerpc2 README.tag
NEWS README.decode* README.thresholding
PROBLEMS README.decoder_preproc_rules README.variables
README README.dns README.wireless
README.ARUBA README.event_queue TODO
README.FLEXRESP README.flowbits USAGE
README.FLEXRESP2 README.frag3 WISHLIST
README.INLINE README.ftptelnet faq.pdf
README.PLUGINS README.gre faq.tex
README.PerfProfiling README.http_inspect generators
README.SMTP README.ipip snort_manual.pdf
README.UNSOCK README.ipv6 snort_manual.tex
README.WIN32 README.pcap_readmode snort_schema_v106.pdf
The following is the default configuration as listed in the README.dcerpc2.
preprocessor dcerpc2_server: default, policy WinXP, \
detect [smb [139,445], tcp 135, udp 135, rpc-over-http-server 593], \
autodetect [tcp 1025:, udp 1025:, rpc-over-http-server 1025:], \
smb_max_chain 3
For some environments, it may be useful to turn off the preprocessor alerts or turn them off for specific systems.
preprocessor dcerpc2: memcap 102400, events none
Or:
preprocessor dcerpc2_server: default, policy WinXP, \
detect [smb [139,445], tcp 135, udp 135, rpc-over-http-server 593], \
autodetect [tcp 1025:, udp 1025:, rpc-over-http-server 1025:], \
smb_max_chain 3
preprocessor dcerpc2_server: net $SCANNERS, detect none
No matter what, the dcerpc configuration should be removed from the snort.conf and replaced with a dcerpc2 configuration for Snort 2.8.4.

After that, I'll configure and install Snort.
$  ./configure --enable-dynamicplugin --enable-inline --enable-perfprofiling
$ make
$ sudo /etc/rc.d/rc.snort-inline stop
$ sudo make install
$ which snort
/usr/local/bin/snort
$ snort -V

,,_ -*> Snort! <*-
o" )~ Version 2.8.4 (Build 26) inline
'''' By Martin Roesch & The Snort Team: http://www.snort.org/team.html
Copyright (C) 1998-2009 Sourcefire, Inc., et al.
Using PCRE version: 7.7 2008-05-07

$ sudo /etc/rc.d/rc.snort-inline start
Starting snort-inline
Initializing Inline mode
building cached socket reset packets
Although the old netbios.rules will still work with the new version, it's also time to update to the latest rules to take advantage of the smaller number of netbios.rules. Note that this only applies for subscription rules since the free ones won't reflect the changes for 30 days.

Some of the rules in the new netbios.rules file have SIDs that were unchanged, so I decided to re-enable any of those that I had previously disabled in my oinkmaster.conf. I can always disable them again, but with the new preprocessor and possible changes to the remaining SIDs, I decided it was best to reevaluate them.

Since I'm also using precompiled shared object rules, I also update from the ones compiled for 2.8.3.2 to the ones compiled for 2.8.4. I keep the rule stubs in a directory along with symlinks to the corresponding SO files, so I simply remove the symlink and recreate it to the new SO rules.
$ rm /etc/snort/so_rules/*so
$ ln -s /etc/snort/so_rules/precompiled/CentOS-5.0/i386/2.8.4/*so /etc/snort/so_rules/

09 April, 2009

Watching the watchers

The Washington Post had an article on the Federal Page about falsified security clearance checks.

Half a dozen investigators conducting security-clearance checks for the federal government have been accused of lying in the reports they submitted to the Office of Personnel Management, which handles about 90 percent of the background inquiries for more than 100 agencies.

Federal authorities said they do not think that anyone who did not deserve a job or security clearance received one or that investigators intentionally helped people slip through the screening. Instead, law enforcement officials said, the investigators lied about interviews they never conducted because they were overworked, cutting corners, trying to impress their bosses or, in the case of one contractor, seeking to earn more money by racing through the checks.

This is not particularly surprising, especially when you consider that the article reports a 22 percent increase in security checks since 2006. Presumably, the personnel and budget allocations did not get a corresponding 22 percent increase. Even before 2006, there were plenty of reports about the backlog for security investigations. Factor in the increase in investigations and you have a situation ripe for abuse through shortcuts.

The problem shows the importance of management and quality assurance. Through the fairly simple process of sending follow-up questionnaires to 20 percent of those that were interviewed, OPM was able to identify falsified interviews and investigations. GAO also has determined that 90 percent of a sampling of reports were missing at least one required document. These seem like fairly easy and accurate ways to determine whether the investigations were being completed properly.

Fixing the problem is another matter, but you can't fix a problem until you've identified it. Prosecuting the responsible investigators, reviewing allocation of funds and personnel, and changing compensation structure are all just a few things that may help reduce the amount of fraud.

The personnel problem immediately jumps out when you see the numbers in the article, which lists 1380 staff investigators and 5400 contractors doing 2 million investigations last year. You have to assume that even the most basic clearance requires a minimum of a credit check, criminal background check, employment history check, and some type of interview. I don't know how 292 investigations per investigator per year could possibly be exhaustive enough to provide the needed information.

It also seems to demonstrate that the number of people requiring clearances is excessive, which is really a whole different discussion.

26 March, 2009

Slackware-current updates and Nvidia driver

I recently updated a desktop running Slackware-current to the latest packages released up to 24-Mar-2009. There were a few minor issues. The first is that, if using slackpkg to upgrade, I had to upgrade the findutils package before everything else or slackpkg would no longer work properly.

# slackpkg update
# slackpkg upgrade findutils
# slackpkg upgrade-all
The second was that I couldn't install the Nvidia driver for the new 2.6.28.8 kernel package because it was failing.
ERROR: The 'cc' sanity check failed:

The C compiler 'cc' does not appear able to
create executables. Please make sure you have
your Linux distribution's gcc and libc development
packages installed.
It turns out that gcc-4.3.3 in slackware-current depends on mpfr.
# slackpkg info mpfr

PACKAGE NAME: mpfr-2.3.1-i486-1.tgz
PACKAGE LOCATION: ./slackware/l
PACKAGE SIZE (compressed): 348 K
PACKAGE SIZE (uncompressed): 930 K
PACKAGE DESCRIPTION:
mpfr: mpfr (Multiple-Precision Floating-Point Reliable Library)
mpfr:
mpfr: The MPFR library is a C library for multiple-precision floating-point
mpfr: computations with exact rounding (also called correct rounding).
mpfr: It is based on the GMP multiple-precision library.
mpfr: The main goal of MPFR is to provide a library for multiple-precision
mpfr: floating-point computation which is both efficient and has
mpfr: well-defined semantics. It copies the good ideas from the
mpfr: ANSI/IEEE-754 standard for double-precision floating-point arithmetic
mpfr: (53-bit mantissa).
mpfr:

# slackpkg install mpfr
After installing mpfr, I was able to compile the Nvidia module for the running kernel.

Finally, kdeinit4 was failing with an error about loading the libstreamanalyzer.so.0 and libqimageblitz.so shared libraries. I fixed this by installing strigi and qimageblitz.
# slackpkg info strigi

PACKAGE NAME: strigi-0.6.3-i486-1.tgz
PACKAGE LOCATION: ./slackware/l
PACKAGE SIZE (compressed): 904 K
PACKAGE SIZE (uncompressed): 2570 K
PACKAGE DESCRIPTION:
strigi: strigi (fast and light desktop search engine)
strigi:
strigi: Strigi is a fast and light desktop search engine. It can handle a
strigi: large range of file formats such as emails, office documents, media
strigi: files, and file archives. It can index files that are embedded in
strigi: other files. This means email attachments and files in zip files
strigi: are searchable as if they were normal files on your harddisk.
strigi:
strigi: Homepage: http://strigi.sourceforge.net/
strigi:

# slackpkg info qimageblitz

PACKAGE NAME: qimageblitz-r900905-i486-1.tgz
PACKAGE LOCATION: ./slackware/l
PACKAGE SIZE (compressed): 82 K
PACKAGE SIZE (uncompressed): 240 K
PACKAGE DESCRIPTION:
qimageblitz: QImageBlitz (Graphical effect and filter library for KDE4)
qimageblitz:
qimageblitz: Blitz is a graphical effect and filter library for KDE4.0 that
qimageblitz: contains many improvements over KDE 3.x's kdefx library
qimageblitz: including bugfixes, memory and speed improvements, and MMX/SSE
qimageblitz: support.
qimageblitz:

# slackpkg install strigi qimageblitz
After those minor issues, I had KDE4 up and running with the latest Nvidia driver.

12 February, 2009

Snort 2.8.4 has a new DCE/RPC preprocessor

According to the VRT blog, Snort 2.8.4 is going to have a new DCE/RPC 2 preprocessor. The README.dcerpc2 was not included in the first release candidate for 2.8.4, but a follow-up on the snort-users list included it. The README covers a lot of information, and is definitely useful to understand the changes and configuration options.

These changes look to be an improvement, but they also have the potential to cause a lot of work and heartache in the short term for Emerging Threats and Snort users. From the VRT blog:

This preprocessor handles all the decoding functions that were previously taken care of using rules and flowbits in a lot of those rules. The upshot is that the number of netbios rules released for any vulnerability that can be exploited over dcerpc is going to be reduced greatly. The number of netbios rules previously released is also going to be reduced in a similar manner.

The downside is that this functionality is only available in Snort 2.8.4 with the dcerpc2 preprocessor. There is no backwards compatibility. Also, a number of netbios rules will be deleted and replaced.
Snort users are going to have to make sure to properly configure the new version. I know from the snort-users mailing list that far too many people use old versions of Snort, so this causes them problems. Realistically, there is usually no reason to use older versions rather than the latest stable release.

Emerging Threats distributes quite a few NetBIOS rules, so I'm sure the new preprocessor will also have an effect on Emerging Threats rules. I seriously doubt that either VRT or Emerging Threats wants to maintain a set of rules for 2.8.4 and above, plus another set for older versions. If I'm interpreting Nigel's blog post correctly, it seems that VRT is going to force the issue by only issuing new and updated NetBIOS rules for 2.8.4 and above. Assuming the improvements in the preprocessor are as stated, I think that is the right choice, but lots of users are going to complain.