September 02, 2010, 7:33 AM — by Brent Huston, Microsolved, Inc. - For this week's tool review, we're looking at Splunk. Splunk is a log collection engine at heart, but it's really more than that. Think of it as search engine for your IT infrastructure. Splunk will actually collect and index anything you can throw at it, and this is what made me want to explore it.
Setting up your Splunk server is easy, there are installers for every major OS. Run the installer and visit the web front end, and you are in business. Set up any collection sources you need, I started off with syslog. I started a listener in Splunk, and then forwarded my sources to Splunk (I used syslog-ng for this). Splunk will also easily do WMI polling, monitoring local files, change monitoring, or run scripts to generate any data you want. Some data sources require running Splunk as an agent, but it goes easy on system resources as the GUI is turned off. Installing agents is exactly the same process — you just disable the GUI when you're finished setting up; however you can still control Splunk through the command line.
Splunk can also run addons, in the form of apps. These are plugins that are designed to take and display certain information. There are quite a few, provided both by the Splunk team and also some created by third parties. I found the system monitoring tools to be very helpful. There are scripts for both Windows and Unix. In this instance, it does require running clients on the system. There are also apps designed for Blue Coat, Cisco Security and more.
In my time using Splunk, I've found it to be a great tool for watching logs for security issues (brute forcing ssh accounts for example), it was also useful in fine tuning my egress filtering, as I could instantly see what was being blocked by the firewall, and of course the system monitoring aspects are useful. It could find a home in any organization, and it plays nice with other tools or could happily be your main log aggregation system.
Splunk comes in two flavors, free and professional. There's not a great difference between them. The biggest difference is that with the free version Splunk is limited to 500MB of indexing per day, which proves to be more than enough for most small businesses, and testing for larger environments. Stepping up to the professional version is a lot easier on the pockets than might be expected, only about $3,000.
Splunk 4 Review [State of Security]
Brent Huston is CEO and Security Evangelist for Microsolved, Inc., a leading provider of security assessments and penetration testing, as well as, the developer of the HoneyPoint line of security software products.
