It's a new year and time for a fresh start -- out with the old and in
with the new. Well, that's certainly appropriate in my case. My home
directory and email has been completely wiped out. Hacker attack? No,
this was an inside job and a much bigger threat than malicious hackers.
The culprit? Complacency. Like the unshod sons of shoemaker's, my own
systems suffered the neglect I would never tolerate at a client site.
A head crash on my mail and home directory server destroyed all data on
the disk. Backups? Sure, everyone does backups. How many people verify
that the backup tapes are actually good? I do, at least when someone is
paying me. I learned the value of testing years ago, when a site I
worked on discovered that the 6250 dpi tape backups were unreadable.
My own systems? Er...well, there wasn't going to be a user screaming at
me, so it didn't seem that important.
To make matters worse, I usually maintain a quick online backup
contingency by creating duplicate filesystems on alternate disk drives
and having a nightly cron job copy all modified files. Saves having to
mount tapes when a user is in a hurry. Note: the filesystem is
mounted "read-only" for normal usage. My contingency, of course, is no
protection against someone who can become root, but is there to protect
the average user from accidentally overwriting the backup file:
1. create duplicate filesystem
2. mount /backup/whatever
3. copy all data to backup partition
cd /whatever; find . -print | cpio - pdmv /backup/whatever
4. umount /backup/whatever
5. create vfstab entry to mount /backup/whatever read-only:
/backup/whatever ufs 2 yes ro
6. mount /backup/whatever
7. create script to copy modified files on a daily basis. Something
/usr/sbin/mount -o rw /backup/whatever
/bin/cd /whatever ; /usr/bin/find . -mount -mtime -1 -print|
/usr/bin/cpio -pdmv /backup/whatever
8. make crontab entry to run the above script.
Crude as it is, the above procedure has helped me numerous times.
Particularly when I receive frantic calls from users who have
accidentally blown away files like, for example, the home page for
their Web site. It saved me the hassle of trying to talk them through
finding the tape, mounting the tape, postitioning the tape to the
correct spot, and restoring the file. What a time saver!
Unfortunatly, in my case, we sort of skipped step 8. Oops.
I could look at the bright side: the instant "urban renewal" of my home
directory and mail files is a great way to recover disk space (and
spare me having to decide what I want to keep). Unfortunately, I
realised too late that I had slipped into some bad habits by keeping a
lot of my articles in my home directory -- some of which were never
published. I also lost a Resource list I was building for my Web site,
not to mention all my mail. Nothing I can't re-do, but it's
inconvenient, unnecessary, and I knew better.
Over the past year, we helped a small business recover their financial
records from a PC that had scrambled file tables. Fortunately for them,
a product available from Ontrack Data Recovery, Tiramisu, was able to
recover the data. Ontrack presently offers EasyRecovery for this
We proceeded to instruct the business owners on the importance of
regular backups. Isn't it ironic?
There's a certain complacency regarding security in one's own home. Can
you imagine a member of your household announcing a fire drill? You
would probably question his or her sanity (or, at least level of
inebriation). More and more people are working from home these days,
away from the behind-the-scenes efforts of a system administration
staff. How many of them do regular backups or store the media offsite?
On a personal note: I've enjoyed reading and responding to all the mail
sent to me by readers. I've learned a lot from some of the mail sent to
me - and now regret the loss. If you have sent me mail and I haven't
responded, please re-send. I'll be in my lab, doing backups.