January 04, 2001, 12:00 AM — It's a new year and time for a fresh start -- out with the old and in
with the new. Well, that's certainly appropriate in my case. My home
directory and email has been completely wiped out. Hacker attack? No,
this was an inside job and a much bigger threat than malicious hackers.
The culprit? Complacency. Like the unshod sons of shoemaker's, my own
systems suffered the neglect I would never tolerate at a client site.
A head crash on my mail and home directory server destroyed all data on
the disk. Backups? Sure, everyone does backups. How many people verify
that the backup tapes are actually good? I do, at least when someone is
paying me. I learned the value of testing years ago, when a site I
worked on discovered that the 6250 dpi tape backups were unreadable.
My own systems? Er...well, there wasn't going to be a user screaming at
me, so it didn't seem that important.
To make matters worse, I usually maintain a quick online backup
contingency by creating duplicate filesystems on alternate disk drives
and having a nightly cron job copy all modified files. Saves having to
mount tapes when a user is in a hurry. Note: the filesystem is
mounted "read-only" for normal usage. My contingency, of course, is no
protection against someone who can become root, but is there to protect
the average user from accidentally overwriting the backup file:
1. create duplicate filesystem
2. mount /backup/whatever
3. copy all data to backup partition
cd /whatever; find . -print | cpio - pdmv /backup/whatever
4. umount /backup/whatever
5. create vfstab entry to mount /backup/whatever read-only:
/backup/whatever ufs 2 yes ro
6. mount /backup/whatever
7. create script to copy modified files on a daily basis. Something
/usr/sbin/mount -o rw /backup/whatever
/bin/cd /whatever ; /usr/bin/find . -mount -mtime -1 -print|
/usr/bin/cpio -pdmv /backup/whatever
8. make crontab entry to run the above script.
Crude as it is, the above procedure has helped me numerous times.
Particularly when I receive frantic calls from users who have
accidentally blown away files like, for example, the home page for
their Web site. It saved me the hassle of trying to talk them through
finding the tape, mounting the tape, postitioning the tape to the
correct spot, and restoring the file. What a time saver!
Unfortunatly, in my case, we sort of skipped step 8. Oops.
I could look at the bright side: the instant "urban renewal" of my home
directory and mail files is a great way to recover disk space (and
spare me having to decide what I want to keep).