Windows tip: Sizing your event logs

The Windows Event Logs are a good thing -- they're a primary source you can mine when troubleshooting server problems or monitoring server security. Because of this, it's a good idea to collect as much information as possible in these logs in order to provide you with a historical baseline of your server's operations to help you troubleshoot difficult issues, and also to create an audit trail for security purposes. The trouble is, can you have too much of a good thing?

In Windows Server 2003, the theoretical maximum size for a single event log is 4 GB. For various technical considerations however, you really can't use logs that large, and for all practical purposes the maximum size for all of your event logs combined shouldn't exceed around 300 MB. With Windows Server 2008 however the problem of sizing event logs becomes even harder since the only limit on the size of your event logs now is how much available disk space you have to store them. The temptation therefore, especially on x64 servers with tons of memory, is to create really large event logs in the gigabyte range.

Don't do it. The problem is that if you let your event logs grow that large, they become difficult to use since each time you try and filter for a specific type of event, the filter must examine in detail every single event contained in your log. This can slow down filtering and grouping operations considerably, making it frustrating to mine your logs for useful information you need.

Archiving your event logs regularly can of course help as it will prevent your log files from growing too large. But the problem here is that if you archive them too often, you'll end up with a lot of separate archives to span your query across when you're trying to troubleshoot an issue or piece together an audit trail.

The answer? Balance. Usability suffers when event logs grow too large because filtering operations take longer. But usability also suffers when event logs are archived too frequently because "event log fragmentation" makes mining your logs more difficult. Unfortunately there's no one-size-fits-all solution to this conundrum, so the best approach is simply to try different size limits and archiving settings until you find what works best for your environment. Remember though that using an event log aggregating tool doesn't mean this problem goes away since the same filtering issue applies to logs analyzed using this tool, and you've still got to archive aggregated logs otherwise you'll end up drowning in data.

As always, email me with your favorite shortcuts, tips and tricks and I'll mention your suggestions in a future column.