Unix Wizdumb: Clean up empty "in use" files with "cat /dev/null"

This kind of problem can occur because processes can create and hold an open file descriptor, preventing the file's inode from being relinquished and the associated disk space freed up. If you can't kill the process, you might be stuck until you can -- even if that means putting up with scrolling errors that make it nearly impossible to type commands on the console.

A better way to proceed when you notice a file is hogging disk space is to cat /dev/null to the file. This will generally free the space immediately while not requiring the process to be killed and restarted.

The syslog file, for example, is held open by the syslog daemon, syslogd. If this file gets so large that the /var partition on your system is choking, you won't have much luck releasing the space it occupies with an rm command, but you can always cat /dev/null to the file to regain the space. Other files that might benefit from the same procedure include auth_log, messages, sulog and wtmpx.

If prior syslog files (i.e., syslog.0 through syslog.7) exist and are correspondingly large, you should clearly start with these files. Since they are historical "rolled over" files, syslogd is no longer using them and they can be removed without side effects.

# lsof syslog
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF  NODE NAME
syslogd  137 root    8w  VREG  85,44        0 18059 syslog

Here we have an example of removing a wtmpx file that has been ignored and has grown far too big over the span of several years. Notice that nearly 300 MB is recovered. That's a lot in a file system that's only 1 GB in size.

# lsof wtmpx
COMMAND PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
utmpd   235 root    3r  VREG  85,44     1488 7760 wtmpx
# df -k .
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/md/dsk/d44      1021735  607960  352471    64%    /var
# cat /dev/null > wtmpx
# df -k .
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/md/dsk/d44      1021735  323688  636743    34%    /var

Now let's do it the wrong way on a similar system. This time we remove the file and no change in the available (or used) space is evident.

# lsof wtmpx
COMMAND PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
utmpd   230 root    3r  VREG  85,44   493644 7760 wtmpx
# df -k .
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/md/dsk/d44      1021735  311676  648755    33%    /var
# rm wtmpx
# df -k .
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/md/dsk/d44      1021735  311676  648755    33%    /var

If a file isn't being held open by a process, the lsof command will not generate any output as shown in the example below.

# lsof /var/log/syslog.?
# lsof /var/adm/lastlog
# ls /var/adm/mess*
/var/adm/messages    /var/adm/messages.1  /var/adm/messages.3
/var/adm/messages.0  /var/adm/messages.2
# lsof /var/adm/mess*
COMMAND PID USER   FD   TYPE DEVICE SIZE/OFF  NODE NAME
syslogd 241 root    9w  VREG   32,0        0 14659 /var/adm/messages

Of all the files above, only the current messages file is open. The others can be removed with impunity. The messages files is better removed using /dev/null.