Unix Tip: Scanning your messages file for warnings: Take two

Send in your Unix questions today! |
See additional Unix tips and tricks



Several weeks ago, I encouraged readers to automate the extraction of warning messages from their /var/adm/messages (or /var/log/messages) files and provided a script to do just that. In this article, we will look at a re-implementation of that script.



The new version of this script was written by Jared Still, Certifiable Oracle DBA and Part Time Perl Evangelist and author of "Perl for Oracle DBAs". Jared's rewrite of the look4warnings script illustrates how elegantly Perl can be used to extract and summarize the contents of log files.



The first commands in the script rewrite are, more or less, the same as in the original version. We assign our messages file to a variable ($msgs) and attempt to open the file. If we run into problems, we exit, displaying a message like this generated by the die command:

Cannot open /var/log/messages - No such file or directory

The script, then, begins with a comment and the lines to set up our input file.

#!/usr/bin/perl -w

# look through messages file for warnings, show summaries
$msgs="/var/log/messages";

open (MSGS,"<$msgs") || die "Cannot open $msgs - $!\n";

The extraction of all lines containing the word "warning", on the other hand, is accomplished far more simply and elegantly than in the original version. Notice that we accomplished this using a single command.

# get all warnings
my @warnings=grep(/warning/i,<MSGS>);

At this point, every line in the messages file that contains the word "warning" in any combination or uppercase and lowercase letters has been added to the @warnings array. This includes duplicate warnings with the same or different timestamps. If a particular warning has arrived once a day for a week, for example, that warning will be saved in the array seven times.

The following command then strips the time stamps off of each line with the map command. This particular map command is preserving in each element of the array the text beginning with the word "warning" (or "WARNING", "Warning" and so on) as indicated by the "i" following the pattern we are matching.

Specifically, warning matches "warning", \s* matches some amount of white space, (.*) matches anything (i.e., the rest of the line) and "i" says to ignore case.

Once this command is run, we have the same number of elements in the @warnings array, but each has been shortened, the time stamp having been stripped.

# get string without timestamp
@warnings = map( /warning:\s*(.*)/i, @warnings);

Perl's map function is perfect for simplifying potentially repetitive operations, such as capitalizing strings of text or selecting some portion of each line.

If no warnings were found in the messages file, we will issue a message to this effect and exit. We exit with a return code of 0 since no error is indicated by a messages file that doesn't contains warnings.

unless (@warnings) {
        print "No warnings found in $msgs\n"; 
        exit 0;
}

Next, we do the trickier part. We set %warncount up as an empty hash and then use it to accumulate a count of each warning message. $warncount{$_}++ is a count of the current warning message, incremented each time a particular warning is encountered as we move through the array.

# create hash with count
%warncount=();
%warnings = map{$_,$warncount{$_}++} @warnings;

foreach my $warning ( sort keys %warnings ) {
        print "warning: $warning: ", $warnings{$warning}+1,"\n"; 
}

print "\nunique warning messages: $#warnings\n";

#!/usr/bin/perl -w

# look through messages file for warnings, show summaries
$msgs="/var/log/messages";

open (MSGS,"<$msgs") || die "Cannot open $msgs - $!\n";

# get all warnings
my @warnings=grep(/warning/i,<MSGS>);

# get string without timestamp
@warnings = map( /warning:\s*(.*)/i, @warnings);

unless (@warnings) {
        print "No warnings found in $msgs\n";
        exit 0;
}

# create hash with count
%warncount=();
%warnings = map{$_,$warncount{$_}++} @warnings;

foreach my $warning ( sort keys %warnings ) {
        print "warning: $warning: ", $warnings{$warning}+1,"\n";
}