Unix Tip: Local message processing with syslog

The syslog deamon (syslogd) on Unix systems provides message logging for other services so that each service doesn't have to duplicate the same basic functionality to manage logging for itself. The messages issued and their severity level depends on the applications, but where these messages are logged and how they are filtered when using the services of syslogd depends on how syslog is configured. The basic format of a line in syslog's configuration file specifies a message type and how messages matching the type will be handled.

For example, a line such as the one below tells syslogd to send informational messages from the line printer to the lpr.log file:

  lpr.info                                /var/log/lpr.log

This line doesn't just influence informational (low severity) messages, however. It also determines how lpr messages with higher severity levels will be logged.

One important thing to understand about syslog message before we get any further is that there is a clearly defined "level" for each message, the lower the numerical severity value, the more critical the message. An emergency message, for example, is level 0. This is the most critical message category. Debugging messages, on the other hand, are level 7, the lowest message level. The table below shows all 8 severity levels.

Message Type 	Description 			Syslog Message 	Severity Level
emergencies 	System unusable 			LOG_EMERG 	0
alerts 		Immediate action needed 	         LOG_ALERT 	1
critical 		Critical conditions 		LOG_CRIT 	         2
errors 		Error conditions 			LOG_ERR 	         3
warnings 		Warning conditions 		LOG_WARNING       4
notifications 	Normal but significant 		LOG_NOTICE 	5
informational 	Informational messagesonly 	         LOG_INFO 	         6
debugging 	Debugging messages 	         LOG_DEBUG 	7

If you elect to have all information messages for lpr logged, therefore, you will also log notifications, warnings, errors, critical messages, alerts and emergencies. If you elect to log only emergencies, on the other hand, none of
the lower level messages will be logged.

Message types in the /etc/syslog.conf file are composed of two parts. The first part specifies the service or "facility" that is generating the messages and the second part is the level or severity of the message.

Facilities correspond to values specified in the openlog and syslog library routines, so you can't just make them up on the fly. However, besides the message types that correspond to familiar facilities -- such as mail, news, user, uucp, cron and so on, the local0 through local7 facilities are defined so that you can add custom message types for your own processing and configure syslogd to send them to whatever log file works best for you.

If a message facility is specified in the syslog.conf file with an asterisk (e.g., *.emerg), this means that all emergency messages will be selected for the specified treatment.

Any time you make a change to the syslog.conf file, you have to restart syslogd or send it a hangup. On some systems, you can do this with the "pkill -HUP syslogd" command.

Systems may log messages to a remote system's log files if the "loghost" variable is used in the syslog.conf file is used and refers to a remote system. Some sysadmins like to use this option because it keeps them from having to log into each system they manage to view log messages. Others prefer to have messages stay on the systems on which they were generated.

If you want to add local messages, you can to select the message facility you want to use (e.g., local0) and then make sure that your application sends it messages using with syslog calls to that facility.

Add a line such as this to /etc/syslog.conf

local0.err 					/var/log/testing

Once you've restart syslogd, you can test whether your new messaging facility is working using the logger command. If you're using local0, for example, the bash loop included in the script below will attempt to log local0.debug through local0.emerg messages. If your syslog.conf line looks like the one shown above, you should notice four new messages in your /var/log/testing file.

#!/usr/bin/bash
  # logmsgs: test syslog local0 processing

for i in debug info notice warn err crit alert emerg
do
    logger -t $i -p local0.$i $i
done

The output below shows the messages that would be generated by this script:

# ./logmsgs
  # tail -4 /var/log/testing
  Jan 10 12:01:54 boson err: [ID 702911 local0.error] err
  Jan 10 12:01:54 boson crit: [ID 702911 local0.crit] crit
  Jan 10 12:01:54 boson alert: [ID 702911 local0.alert] alert
  Jan 10 12:01:54 boson emerg: [ID 702911 local0.emerg] emerg
  

Don't be surprised that some of the messages will also be displayed on your screen. This is the normal system response for the higher severity messages.

Jan 10 12:01:54 boson alert: [ID 702911 local0.alert] alert
Jan 10 12:01:54 boson emerg: [ID 702911 local0.emerg] emerg
Message from syslogd@boson at Thu Jan 10 12:01:54 2008 ...
boson emerg: [ID1002911 local0.emerg] emerg