Unix Tips: Advice from Unix experts
Tips and tricks for every-day Unix administration tasks and some clever ways to approach more challenging problems. Send your comments or questions to submissions@itworld.com.
Terminating unattended processes (03/27/2008)
Easy way to terminate processes left running after users log off a system. Read more.
More on bc (03/19/2008)
The bc utility is not just a command line tool. You can also write numerically intensive scripts in bc. The tool incorporates enough syntax to assign values to variables, write and call functions, collect responses from users and print annotated results. Read more.
Using bc for big calculations (03/12/2008)
For sysadmins who want to perform simple calculations on the command line, there's a considerably more useful tool than expr. Try bc. Like expr, the bc command allows you to do simple math. Here are some examples. Read more.
Burning bootable DVDs (03/05/2008)
One of the messages that you don't want to see when getting ready to upgrade a system to Solaris 10 is "can't open boot device". The problems this kind of error suggests, such as bad CD/DVD reader or improper device alias, can be troublesome. So, when I ran into this problem on a Friday afternoon, I was not amused. I had tried various boot commands, but got the same basic response each time... Read more.
Terminating options processing with -- (02/28/2008)
To understand how this works, think back to how the shell processes what it sees on the command line. If we type "ls -l a*", for example, the shell translates a* into a list of all files that begin with the letter "a" and then makes the appropriate system calls to provide a detailed listing of those files, complete with owners, sizes and most recently changed dates and times. Read more.
More on filenames (02/19/2008)
Another and simpler way to display file names that include unprintable characters is to use the -b option available with the ls command on some versions of Unix (e.g., Solaris). Read more.
Stripping file extensions with the basename command (02/14/2008)
I recently discovered a new trick for using basename to extract filenames from a full pathnames. Read more.
How-to rename an oddball file (02/06/2008)
Every now and then I come across a file that just doesn't display properly in a file listing. In fact, it can be a little tricky to determine the name of a file that contains odd characters. You might think a file's name is "myfile.txt" only to find out that it's really "myfile.txt " (note the extra blanks). Here's a way to name a file with an odd name. Read more.
Reassembling very large "split" filesync (01/30/2008)
One of the unwritten rules of computing says that, as various storage media grow in size, so do the files that are written to them. I recently found myself staring at two files, each roughly 4 GB in size, that needed to be combined into one before the file could be decompressed and its contents extracted. What I was expecting to work with was a large cpio archive ... Read more.
Experimenting with rsync (01/24/2008)
This article shows how to create a directory with a number of files, copy the directory with an assortment of rsync (and other) commands, take a closer look at some of rsync's command options and then use rsync to verify that the replicated directory is intact. Read more.
Shuffling file systems with rsync (01/17/2008)
I recently found myself needing to move several very large file systems into roomier data quarters on a disk array and began thinking about the most efficient and reliable way to get the job done. Read more.
Local message processing with syslog (01/10/2008)
The syslog deamon (syslogd) on Unix systems provides message logging for other services so that each service doesn't have to duplicate the same basic functionality to manage logging for itself. The messages issued and their severity level depends on the applications, but where these messages are logged and how they are filtered when using the services of syslogd depends on how syslog is configured. The basic format of a line in syslog's configuration file specifies a message type and how messages matching the type will be handled. Read more.
Reboot mystery solved (12/27/2007)
If you remember a column that I wrote back in November when I was baffled by a periodic reboot of a Windows box that I manage and trying my hand at various DOS commands to determine when and why the system was rebooting, you might be interested in knowing the cause of the problem.Read more.
The crossover LAN (12/20/2007)
You have probably heard that a two-node LAN can be configured by stringing a crossover cable between the network ports on two systems. Not only is this true, but it's not all that hard to do provided you have a few tools, a length of CAT 5 cable and a couple RJ45 connectors. Using a crossover cable has one decided advantage over a traditional network hookup -- it avoids the requirement that a hub be used to connect the two systems, saving both money and rack space. So, let's examine what is needed to create your own crossover cable. Read more.
Using VNC to access remote systems (12/13/2007)
Similar to Remote Desktop, VNC (Virtual Network Computing) provides its users with access to remote systems' displays from their desks. Read more.
File typing with Magic (12/03/2007)
Several readers suggest the /etc/magic file and the file signatures that it provides to identify file types regardless of how the files are named -- even, in fact, when no file extensions are used.Read more.
File extensions and you (11/29/2007)
It's good to have a resource available to help you identify the type of unusual files when you run across them or to learn a little more about the format of a familiar file type. Read more.
Tracking down a reboot using DOS commands (11/21/2007)
When one of only a handful of Windows servers that I manage dropped off my radar for a while, I wasn't sure what to think. I had been running an application, for which this system was the server and it stopped responding. When I tried reconnecting, it was clearly not available. The system was rebooting, but it took me a while to figure that out. How do I go about determining if a Windows server has rebooted?
Read more.
Creating the chrooted file system (11/13/2007)
There are a quite a few steps involved in setting up a chrooted environment. One of the first things you must do is create a directory for your chrooted file system. Read more.
Creating a chrooted FTP account (11/08/2007)
Often referred to as a "chroot jail" or a "padded cell" because users cannot escape from the directories into which they are put when they log in, the basic concept of a "chroot" (change root) configuration is that a user's working view of the system's file system is dramatically modified so that only some particular directory and its contents is visible. However, in order to be useful, each chrooted environment should contain many of the same directories you would expect to find in the root file system on a typical Unix system. Read more.
Unix File System Names (11/01/2007)
One of the questions that Unix users are sometimes reluctant to ask is how all the directory names -- like /etc, /var and /usr/local -- came about in the first place. Why did we adopt the particular names we use today instead of names that might make more sense to the casual user? What do these file system and directory names tell us about the intended content? Read more.
Why I Gmail (10/25/2007)
I recently closed an email account that I had been using since I moved to the Delmarva peninsula nearly seven years ago. I have switched to Gmail. Had the account I closed not been a local provider for most of the years that I was a customer and had I not put a good deal of value of electronic stability (I'd been sandra@ that provider for many years), I would never have hung on so long. Gmail is everything I need and then some. Read more.
Book Review: Head First SQL (10/18/2007)
Several weeks ago, I encouraged readers to automate the extraction of warning messages from their /var/adm/messages (or /var/log/messages) files and provided a script to do just that. In this article, we will look at a re-implementation of that script. The new version of this script was written by Jared Still, Certifiable Oracle DBA and Part Time Perl Evangelist and author of "Perl for Oracle DBAs". Read more.
Scanning your messages file for warnings: Take two (10/10/2007)
Several weeks ago, I encouraged readers to automate the extraction of warning messages from their /var/adm/messages (or /var/log/messages) files and provided a script to do just that. In this article, we will look at a re-implementation of that script. The new version of this script was written by Jared Still, Certifiable Oracle DBA and Part Time Perl Evangelist and author of "Perl for Oracle DBAs". Read more.
Book Review: The Art &038; Science of CSS (10/04/2007)
For those of us who work primarily on the technical side of web site management, there is usually a clear divide between the design of web sites and the various forms of programming that allow that design to be expressed. The people who worry about font pixelation and those who concern themselves with the content of error_log files rarely talk shop at the water cooler, never mind work together to ensure that the web sites they jointly manage both adhere to current technical standards and promote a look and feel which is both functional and elegant. This where "The Art & Science of CSS" has important lessons to teach.
Read more.
Recovering from a fatal error: Lost libgen.so.1 (9/24/2007)
Unix vets know that it is possible to recover from fatal errors. A recent example is when a sysadmin accidentally zeroed out a file named /usr/lib/libgen.so.1. This file is used for string pattern matching and pathname manipulation. When you don't have this shared library on a system, you will get errors such as this.
Read more.
Scanning your messages file for warnings (9/20/2007)
Too often we sysadmins look through our messages files only when something has gone wrong on our systems and we need some clues that might help us piece together what has happened. Routine scanning or, better yet, intelligent extraction and summarization of important notices and warnings can help alert us to problems when they make their first appearance and sometimes avoid the worst repercussions of a problem on your systems. This article looks at a script that scans for warning messages, collects the text and frequency of appearance of these warnings and presents this information in digested form to the user.
Read more.
Replacing batteries on a Sun StorEdge disk array (9/09/2007)
I had only recently come to notice that the batteries had expired. Using the sccli (StorEdge controller command line interface) command, I had noted the expiration dates were well enough in the past that replacing them as soon as possible was a good idea. When I checked the batteries, the battery status information looked like this.
Read more.
Looking at soft partitions (9/04/2007)
Soft partitions provide a way to squeeze more than seven partitions onto a single disk on a Solaris system. First introduced into Solaris as a patch for Solstice DiskSuite on Solaris 8 and then bundled into Solaris 9 as a feature of Solaris Volume Manager, soft partitions provide a way to make more flexible use of the increasingly large disks showing up on Solaris servers. Prior to soft partitions, the maximum of seven partitions often had sysadmins weighing tradeoffs when setting up their systems. Which file systems demand separate slices and which can share disk space without running into problems? With soft partitions, sysadmins can pretty much set up as many partitions on a disk as they care to configure. If you are looking at the soft partitioning on a Solaris server for the first time, you may find the information concerning the configuration of soft partitions a little hard to digest.
Read more.
Using netgroups to share files (8/29/2007)
Solaris and other SVR4 versions of Unix provide file sharing through
NFS and the /etc/dfs/dfstab file. The syntax of this particular file
permits you to restrict file system sharing to a limited number of
hosts or to share file systems with no restrictions whatsoever.
Restricting file systems to a select set of hosts is particularly
important if you are permitting the client hosts to access the file
systems read/write and even more important if root on the clients will
have the authority of root on your NFS server. You can specify the hosts that are permitted to mount file systems by
stringing them together in a colon-separated format with the share
command, such as this command which can be issued on the command line
or included in the dfstab file for a more or less permanent setup.
Read more.
Viewing device aliases on the command line (8/21/2007)
Ever wish you could take a look at the device aliases on a Solaris
system without having to descend all the way down to the ok prompt?
Well, you can. By issuing the proper version of the prtconf (print
system configuration) command, you can include a list of device
aliases in your prtconf output.
Read more.
Redirecting standard error in Perl (8/15/2007)
There are numerous ways to redirect standard error in a Perl script.
You can redirect the output from one particular command, you can
combine standard error with standard out so that the two are handled
together or you can send all standard error to the bit bucket. None
of these techniques depends on what the person running the script
does. Instead, they can all be set up in Perl scripts. Here's how this works.
Read more.
A SCSI mystery: Come and gone (8/02/2007)
Late Friday evening, one of the servers I manage suddenly started having what appeared to be
serious disk problems. Its NFS clients were collecting "NFS server not
responding still trying" errors. The system itself was unable to
process share and unshare commands. My initial fear was that the
system had suffered a disk crash and would have to be rebuilt from
spare parts and backups before the staff's arrival on Monday morning.
The system was still running, but having serious problems running
ordinary commands and the console was filling up with messages like
these.
Read more.
Taking advantage of trust in a script (7/26/2007)
If you Google the terms "ssh" and "password", you will come up with a
generous number of hits on how to set up SSH so that you can log in to
a system or run commands remotely on that system without entering a
password. While representing something of a security risk (anyone who
breaks into your account on one system can access your account on the
other as well), this configuration is extremely handy if you need to
collect data from the system using a script. For example, you could
type "ssh bandana uptime" to find out how long bandana has been
running since its last reboot.
Detecting whether a command run via ssh within a script required entry
of a password or made use of public key authentication, on the other
hand, is more than a little tricky.
Read more.
Use your Unix scripting skills to write a batch file (7/19/2007)
Attaching a remote drive or directory on a Windows box is
called "drive mapping" though the process is essentially the same as
what we call mounting in the Unix world. While I rarely spend any
significant time working on the Windows end of a Samba connection, I
recently had an opportunity to assist a user who was tired of
repeatedly having to manually connect directories from a couple of
Solaris servers equipped with Samba for sharing directories with
select Windows desktops.
The solution was to whip up a batch file that mapped the drives on
login. Batch files, though far less sophisticated than their Unix
shell script counterparts, nonetheless have many of the important
features that I have become accustomed to working with on my Unix
systems. These include such things as redirection, testable return
codes, the ability to test for the existence of a file and the
equivalent of /dev/null for discarding command output, errors and all.
Read more.
Finding idle users (7/12/2007)
I used to rely on the "finger -i" command to tell me how many users
were logged into systems I managed and how long each of them had been
idle. If I needed to reboot a system and the only users logged in at
the time were both unresponsive and long idle, I felt justified in
proceeding with the reboot. In recent years, however, I've seen too
many instances in which finger doesn't tell me the whole story. The
difference between the output of the who command and the output of
finger shows me that the who command often reports more logged in
users than does finger. Here's an example. On this particular system, the who command shows
that we have eleven logins from seven systems.
Read more.
Trimming inboxes (7/05/2007)
If you want to remove mail that arrived in previous years from root's
inbox or the inbox of a user who doesn't want to clean up years' worth
of mail himself, here's a simple Perl script to help you do just that.
I call it "trimbox".
Read more.
Port scanning with nmap (6/26/2007)
A number of weeks ago, I provided a script that uses
nmap to quickly scan a system or subnet for activity on a particular
port. Useful for detecting, say, web servers or Oracle installations,
the script runs very quickly and provides a useful table of host
names, IP addresses and the name of the service being sought. In this
week's column, we look a little more deeply into nmap, what it can do
for you and some of the scans that it can perform.
Read more.
Cleaning up DNS (6/21/2007)
I recently had an occasion to help a friend update her DNS
configuration on a system responsible for a small test domain. Since
she seldom touches the zone files on the server, never mind the main
configuration file, I suggested that we pay attention to the messages
that the process writes to the messages file to highlight any
configuration problems that existed in the long neglected named.conf
file. While we didn't run into many problems and the DNS server was working,
we noted a number of minor configuration problems -- some resulting in
errors and others generating warnings -- which we systematically
identified and then resolved.
Read more.
Clearing the screen (6/12/2007)
Every now and then I come across a system which, when I issue a
request to clear the screen (i.e., the clear command), I get something
like this instead:
oopsy# clear ld.so.1: clear: fatal: libncurses.so.5: open failed: No such file or directory Killed
Often, on encountering this problem, I simply growl, press enter a
couple dozen times and continue working. When I'm in a
particularly "fixit" mood, on the other hand, I figure I shouldn't be
putting up with annoying configuration problems and decide to break my
train of thought just long enough to tackle this distracting problem
head-on. Here's what you can do when you get an error such as this.
Read more.
Using fuser to determine why a file system is busy (6/07/2007)
We have looked at the fuser command before to display information
about what users or processes are keeping file systems busy. This
command can save you a lot of time tracking when you need to quickly
determine how a file system or a particular file is being used,
particularly when you cannot unmount a file system due to current
usage. Today, we will look at a script that provides more information on the
processes keeping a file system busy by using fuser to supply process
IDs to ps and ps to then display additional information about each
process.
Read more.
Generating random data (5/31/2007)
There are several ways to conveniently generate random data on Unix
systems. One popular way is to make use of Perl's rand function.
Another is to employ the /dev/random or /dev/urandom pseudodevice to
generate a string of the required length. Both tools easily produce
the required data, but take a bit of finessing to get just what you
need. Perl's rand function without a parameter, for example, will
generate a number between 0 and 1. You might use it like this in a
script.
Read more.
Finding services on a Subnet, part 2 (5/24/2007)
Last week column, we looked at how nmap, the well known port mapper,
can be used to help monitor applications and services on a network or
subnet with very little overhead and surprisingly little effort. In
this week's column, the findAppl script has been enhanced at a
reader's request to include the system name for each line of
output.
Read more.
Finding services on a subnet (5/17/2007)
If you have ever needed to survey a large group of systems to find out
which of them supported some particular service, such as ftp, telnet,
ssh or some other particular application, you have probably thought of
numerous ways to query the systems for the required information and
display it in some usable fashion. Many methods of obtaining
information from servers, however, require some sort of login or a
remote shell request that either takes more time than you want to
spend or requires you to configure some sort of trust on the part of
the systems with the information for the system on which it is being
collected. In today's column, we will look at a way to find out about
services running on systems without setting up any access ahead of
time.
Read more.
Sendmail gets smarter (5/10/2007)
Before sendmail 8.13, sendmail servers were generally set up in one of
two ways. Either their configurations were based on the main.cf file
and they used DNS MX records to determine the identities (and IP
addresses) of hosts to which mail was to be delivered or their
configuration was based on the subsidiary.cf file and they were set up
to relay mail through a system with more knowledge of the outside
world (in particular, access to a real world DNS server) than systems
requiring the relay service. These two basic configurations have basically merged in version 18.3
of the sendmail software with the addition of a new configuration
option -- the FallbackSmartHost.
Read more.
Finding text in context -- again (5/03/2007)
If you need to see a few lines above and below the matching line of
grep check out these two very nice tools to help you pull text from
files while retaining enough of the context to make good use of the
text you've located.
Read more.
Power outage 2007 (4/26/2007)
During the booting of Solaris, both the / and /usr file systems are
mounted read-only and then later, before the boot process is fully
complete, remounted read-write. This is all part of the normal boot
process and no reason for concern. Disruptions to the normal boot
sequence, however, can leave you with a file system that is mounted
read-only and requiring repair with no obvious way to fix the
problem. Do situations such as this require you to boot from CD, DVD
or from a boot server before you can edit the files or scripts that
are sabotaging your boot process? No, you can make use of the mount
command's handy remount option and save yourself a lot of time and
trouble. Let's say that you are booting a system and you run into these errors:
Read more.
Taking notes the easy way (4/19/2007)
If you ever find yourself cutting and pasting Unix commands and their
output into a file in order to create a record of some anomaly you
have encountered on a system you manage or to document a procedure so
that someone else can take charge while you are on vacation, you may
have overlooked a handy command that takes all the fuss out of
recording the content of your online session -- the script command.
Read more.
Rejecting email from outside (4/12/2007)
If you would like to generate email on a server, but don't want to put
up with email from the outside, you can configure sendmail to do this
for you.
Read more.
Mail loops back to me (4/05/2007)
Many systems administrators have run into errors in their syslog files
that complain that mail is looping back, suggesting a possible MX
problem. The common cause of this problem is that a server is
receiving email for a domain that it doesn't recognize as its own.
Then, when the server looks up the MX address for the intended target
(in order to send the mail on its way), it notices that the MX record
is one that identifies the mail exchanger as the system itself.
Read more.
The language of power (3/29/2007)
If you have any role in dealing with the power provided to the systems
you manage, it's useful to be familiar with the terminology used to
describe power and power devices such as UPS systems. Let's examine
some of the terminology that you are likely to hear when people are
talking about power provided to computer systems.
Read more.
Last workday of the month: A retake (3/15/2007)
One of my articles last month described an approach to determining
whether some particular weekday happens to be the last workday of the
month. While no one took me up on the challenge to turn this script
into a densely packed on-liner, one reader offered a version that has
a particularly interesting advantage. Instead of being implemented as
code which can be inserted into any script in which you want to
execute certain commands only on the last workday of the month, this
script is a standalone that you would use to invoke other scripts or
specific commands when it's the appropriate time for them to run --
when it's the last workday of the month.
Read more.
Book Review: MySQL Cookbook, 2nd Edition by Paul DuBois (3/8/2007)
MySQL is one of the most popular open source databases. With an
estimated 10 million active installations and more then 50,000
downloads every day, it runs on nearly every Unix platform and Windows
as well. Along with this popularity, however, comes a host of
questions from users on how to best accomplish any of a myriad of
tasks. That's where MySQL Cookbook comes in. Whether you're new to
MySQL or a long-time user, you will find numerous "tricks" that you
can put to immediate use.
Read more.
Core dumps for dummies (3/1/2007)
If once every several years, you have problems with a process and end
up with a troublesome core dump, you might be able to extract some
useful information from the file before you delete it from your
system. In this week's column, we look at an extremely simple script
intended to help you analyze core dumps without requiring you to
remember the relevant commands.
Read more.
Examining TimeZone data in Solaris (2/22/2007)
In last week's column, I encouraged anyone who hasn't already patched
their OSes for the imminent daylight savings time changes. This week,
we look at a command for displaying timezone data and verifying the
date on which the switch to DST will be made.
Read more.
Wake up and patch the OSes (2/15/2007)
Time is running out for anyone planning to ready their systems for the
biggest time-related change since Y2K -- the 2007 Daylight Savings Time
change. As Americans turn their clocks ahead one hour this year --
three weeks earlier than previously, systems administrators will be
well advised to ensure their systems will do the same. Timezone
configurations on systems from Solaris and Linux to Windows will
require updates if not already updated through a patch, OS upgrade or
service pack, to accommodate the new dates for switching to and then
back off daylight savings.
Read more.
Tracking down disk usage (2/8/2007)
Whenever you run across a root file system that's 100% full, the first
thing you are likely to do is ascertain which of many directories are
actually stored, rather than simply mounted, within /. Variations of
the df and du command are likely to come in handy. However, the
process of detailing how disk space is being used and identifying
files or subdirectories that can be removed often takes a lot longer
than most of us would like. In this week's column, we'll look at a simple script that can help you
determine which directories in / are abnormally large.
Read more.
Last workday of the month (2/1/2007)
In a column published last year, we looked at various ways to
write scripts that would only run on the last day of the month.
Determining whether an arbitrary day is the last day of the month can
be done in a number of ways, using the cal and/or the date command.
For example, if tomorrow is the first, then today must be the last day
of the month. This gets us around the complexity of months with 28,
30 or 31 days. When a reader recently asked how to write a script that would only run
on the last workday of the month, a new twist was added to the old
problem. A day is the last workday of a month only if it's a weekday
and the following weekday falls in the next calender month. So, I
came up with two approaches to solving the new challenge.
Read more.
Shells and search order (1/25/2007)
It's easy to take the Unix command line for granted. That is, it's
easy to get used to typing in commands and getting responses without
thinking very much about how the system determines what command to
run. Some of the commands we type are binary files that are part of
the OS, some are scripts, some are shell built-ins and some are
aliases that are configured into our accounts. Still others may be
shell routines. The order that Unix systems use in evaluating the
commands we enter is not solely dependent on our PATH variables.
Instead, the search order follows a predefined order. If you happen to
have a script that has the same name as a Unix command, an alias and a
shell routine or built-in, for example, how do you know which one you
will end up executing when you press the return key?
Read more.
NFS mounts and Solaris 10 problem (1/18/2007)
I hadn't seen a stale NFS file handle for more than a decade, so this
mount error surprised me.
Read more.
Recovering control of a firmware password protected system (1/11/2007)
Firmware passwords can be a very useful security measure when you need
to keep systems stable in spite of the fact that they may be exposed
to unauthorized individuals, but they can also present a time-
consuming challenge if you acquire such a system and don't know the
password that was set or if you forget. Having recently purchased some
used Sun systems off eBay, I quickly discovered that one of the
systems had been configured with a firmware password that the seller
didn't know, so I was forced to find a way to overcome this security
setting before I could attempt to take control of the system and
reinstall it.
Read more.
Firmware passwords and the OpenBoot prompt (1/4/2007)
A firmware password secures a system against unauthorized changes at
the ok prompt. If you don't have a firmware password set, anyone with
access to your system's console can modify your OpenBoot parameters.
If you do have a firmware password set, be careful not to forget it or
even you may not be able to boot your system. With a restrictive
security setting, accessing a system can be very difficult.
Read more.
Problems with inetd (12/21/2006)
I ran into a problem recently with one of my Solaris 9 servers. For
some reason which I have yet to pinpoint, any lines that I add to
the /etc/inet/inetd.conf file are removed from the file almost as soon
as I finish adding them. Well, almost any lines. Since I generally
precede any additions to system configuration files with comments that
describe what the lines are for, I did so with these recently added
lines as well. When the file was modified, I noticed that only my
comments remained in the file of the lines that I had inserted.
What was the culprit?
Read more.
Adding lines to the ends of files (12/14/2006)
Let's look at a quick, clever way to add lines to the ends of files on
a Unix system. This little Unix trick can be very useful if you are
distributing installation directions to customers and want to limit
the possible mistakes that they can make in updating important system
files. The basis of this trick is the Unix here document -- a special
form of I/O redirection that allows you to insert the content to be
added between the redirect command and a special marker that is
recognized as the end marker for the inserted text.
Read more.
Sharing DVD drives (12/07/2006)
While only the newer Sun systems and higher-end PCs today are equipped
with DVD drives, application software is beginning to arrive on DVDs
because of the tremendous increase in storage capacity. So what do you do
if you need to install software that is only available on a DVD on a system which has only a CD drive? One answer
is to mount and share the DVD from a system with a DVD reader.
Read more.
Coping with spam (11/30/2006)
The primary differentiator in email clients today has to do with how effectively
the email client detects and removes spam. What does it take for an email client
to recognize spam and why can't these floods of innocuous emails be
recognized and diverted from my inbox?
Read more.
Creating CDs for Unix or Windows (11/23/2006)
Preparing a CD that displays proper file names whether it is mounted
on a Unix system or on a Windows box requires little more than the
proper set of arguments to the mkisofs command and software that knows
how to work with ISO images. This looks at a script that prepares a multi-use ISO image from files in a selected
directory.
Read more.
Comparing files with checksums (11/16/2006)
Unix systems provide numerous ways to compare files. The most common
way to verify that you have received or downloaded the proper file is
to compute a checksum and compare it against one computed by a
reliable source. MD5 is frequently used to compute checksums because
it is computationally unlikely that two different files will ever have
the same checksum. Similar commands, such as sum and cksum, also
compute checksums but not with as much reliability. Let's look at
several checksums and see why.
Read more.
Automating Analog (11/09/2006)
For those of us who use Analog routinely to understand how well our
web sites are doing, a little automation goes a long way. To prepare
monthly reports, for example, you will want to rotate your log files,
keeping each month's log data separately. Whether you retain old log
files or only the Analog reports, you can set up for your sites in a
way that lends itself to month by month comparisons.
Read more.
Filename completion (11/02/2006)
A number of readers have recently asked how they set up and use
filename completion in a variety of Unix shells. Since file
completion does not work the same in every shell, we will examine how
you can use this great time saving feature in csh (tcsh), bash and ksh.
Read more.
Compiler problems on Solaris 10 (10/26/2006)
I recently decided that it was time that I learn a little C++. After
all, I studied Computer Science back when Unix was a toddler and now
have a daughter who is taking Math and Computer Science classes at
UCLA. So, I started with a typical "hello, world" kind of program and
tried to compile it on my newly installed Solaris 10 system. But,
instead of an easy compilation of my embarrassingly simple program, I
ran into a series of errors ...
Read more.
Installing Solaris 10 with self-prepared CDs (10/19/2006)
Installing Solaris 10 from a set of CDs that I burned myself turned out
to take four times as long as I had expected it would and to be a
surprisingly annoying process. The first and most time-wasting problem
turned out to be how troublesome it was to feed the CDs into the
system at the right time.
Read more.
The directory that wasn't (10/12/2006)
I recently found myself with a directory that didn't contain the
standard "." and ".." directories that would have properly tied it
into the file system. To fix the immediate problem, I moved the
troublesome directory (using mv) and created a new one. When I tried
to remove the (then renamed) directory, I found that I couldn't.
First, I tried the obvious. I tried removing the directory with rm,
rmdir, rm -f and rm -rf. None of these commands worked for me. Here's
the process I took to remedy the problem.
Making a device alias for your root mirror (10/05/2006)
Once you have mirrored your root partition, you may still need to
create a device alias for the mirror so that you can boot from it if
the primary mirror ever fails. To prepare for this, you should first
generate a long listing of the partition on which the mirror resides.
It will look something like this.
Read more.
Mirroring your root partition with Solaris Volume Manager (09/28/2006)
Solaris Volume Manager can make easy work of mirroring your root file
system, but you have to use the right commands in the right sequence
to make easy work of this task. In this week's column, we'll run
through each of the commands required to mirror root and show how you
can check on what is happening in each step.
Read more.
Using pscp to copy files securely (09/14/2006)
Many people who use PuTTY to make secure connections between Windows
desktops and Unix servers are unaware that PuTTY also provides a tool
for securely moving files between the two platforms. The reason for
this oversight is that, while PuTTY provides a terminal-like window on
its users desktops, the pscp command (PuTTY's scp command) that PuTTY
provides for moving files has to be run from the command line.
If you use PuTTY on a Windows box to connect to your Unix systems,
here's what you have to do to use the pscp command.
Read more.
The perfect crash: Planning after the disaster (09/07/2006)
Sadly, it is in the aftermath of a big network crash that some of us
do our clearest thinking about what we should have done before the
disaster struck. How do we get our systems back online with some semblance of control and efficiency
when we're not sure what we're starting with? First, let's look at one scenario that could leave us in the dark.
Read more.
Managing line termination differences in text files (08/31/2006)
Most every Unix sysadmin has run smack into line ending
incompatibilities from time to time. The most common problem is the
appearance of ^M characters at the ends of lines in text files that
were built for or on Windows systems. Text files often end up with the
pesky ^M characters when they're transferred from one system to
another using scp or ftp in binary (byte-by-byte) mode instead of
ASCII mode.
Read more.
More on Associative Arrays (08/24/2006)
An associative array, when implemented in Perl, has come to be known
as a "hash" -- a word that is also used to describe the digested value
(i.e., the "message digest") generated from a longer piece of text and
used to ensure that the text has not been altered (if a message before
and after transmission results in the same hash, the text can assume
to be unaltered).
Read more.
Using Associative Arrays in the Korn Shell and Perl (08/17/2006)
An associative array is an array which uses strings as indices instead
of integers. To see how associative arrays work, we're going to look
at both the Korn shell and Perl, though only the newest version of the
Korn shell (referred to as "korn93") supports associative arrays.
Read more.
Using Indexed Arrays in the Korn Shell (08/10/2006)
One of the most convenient ways to manipulate information in scripts
is to store it in an array. Arrays facilitate looping through lists
of related values, keeping track of an ever changing number of items
and relating descriptive text with the items that they describe. As a
very simple example, consider the colors of a rainbow. You can always
loop through a list of rainbow colors like this:
Read more.
IPv6 Basics (08/01/2006)
IPv6 is coming. It's coming slowly, due to technologies like NAT and
the advent of private address spaces like 10.0.0.0/8 and
192.168.0.0/16 that have provided breathing room for IPv4 in spite of
the rapidly depleting IPv4 address space. Still, it is coming. And
it's different enough from IPv4 that preparing for its arrival is a
very good idea. In this week's column, we'll take a look at some of
the fundamental differences between the two protocols and see what
IPv6 addresses look like.
Read more.
Viewing library dependencies with ldd (07/26/2006)
The use of dynamically linked libraries makes a lot of sense when it comes to keeping system binaries small, but can generate some head-scratching problems when you are trying to install or run software and the required libraries appear to be missing. Let's take a quick look at what dynamic libraries are and how you can work around some of the common problems you might run into that involve their use.
Read more.
Using fuser to Identify Users and Processes (07/19/2006)
The fuser (pronounced "ef-user") command is a very handy command for determining who is currently using a particular file or directory. If one user can't access a file because another user has it locked in some way, the fuser command can help you determine who that user is so that you can decide how to resolve the apparent conflict.
Read more.
Solaris and Controller Numbering (07/12/2006)
Ever wonder how Solaris comes up with disk controller numbers? Why are the disks on one particular controller called /dev/dsk/c0... and those attached to another controller /dev/dsk/c1... And, if you were to add another controller or replace one of the originals, would you be confident that your original file systems would still mount with the same /dev addresses that they had used for mounting previously?
Read more.
Killing Idle Logins with idled (07/05/2006)
In last week's column, we looked at a couple of ways that you can force an automatic logout of a user's session when he has been idle for too long. In particular, we looked at the TMOUT variable available in some shells and a script that uses the "who -u" command to determine how long a login session has been idle and kill to terminate sessions.
Read more.
Killing Idle Logins (06/29/2006)
It is not at all unusual for systems administrators to get a little antsy when users' login sessions sit idle for hours or days. Not only can login session consume resources, tie up software licenses and prevent file systems from being unmounted but, since we generally can't see what is going on at the user end of these sessions, we don't know whether these users could provide opportunities for unauthorized individuals to execute commands and access data under the guise of authorized users. A user who leaves for lunch without logging out or securing his login sessions by locking his screen, for example, might be giving someone else a chance to run commands using his account.
Read more.
Working with Symbolic Links (06/22/2006)
For most Unix users, symbolic links are obvious and natural -- a means to make connections that span file systems and avoid the need to keep duplicates of files in multiple file system locations. Symbolic links are often used by many system administrators to recall previous locations of data and applications. If the log files for a web application were previously stored in /usr/logs, for example, and have been moved to /opt/logs, a system administrator might create a symbolic link reminding themselves and others that the files have moved (/usr/logs -> /opt/logs) with a command like this:
Read more.
Reading and Writing with File Descriptors, Part 2 (06/15/2006)
Last week's column looked at how we can read from and write to files by associated the files with user-defined file descriptors. This week, we take this idea a bit further.
Read more.
Reading and Writing with File Descriptors (06/08/2006)
Have you ever wanted to read a file one line at a time in a shell script and found the task to be a lot more trouble than you ever imagined? If you use a "for line in `cat file`" loop, for example, each pass through the loop is going to set your $line variable to, not the next line in the file, but the next word in the current line. One handy way around this problem is to assign a file descriptor to your input file and then use that file descriptor to read the file line by line. Here's how it works.
Read more.
Zones in Solaris (06/01/2006)
Without a doubt, one of the most interesting features of Solaris 10 is its ability to establish numerous virtual systems on a single Solaris box -- a concept known as "zones". This radically innovative new release of Solaris gives us the ability to create these isolated operating environments (virtual "instances" of Solaris) and to assign them to various projects or groups of users as if they were completely separate computers with separate file systems, users and applications. In the process of setting up a zone, we might allocate some system resources -- CPUs, memory and other devices -- to the exclusive use of a particular zone or have all zones share most system resources. While there are some "clues" -- some ways that a savvy user might detect that he is working in a zone and not on a dedicated computer, the differences are subtle and the experience of working in a zone parallels in all important ways the experience of working on any Solaris system.
Read more.
Script Debugging (05/25/2006)
The error messages that are displayed when a script malfunctions range from effective to practically useless -- especially when the scripts themselves are long, complicated and heavily parameterized. One of the ways that you can get more information on script failures is to use the -x argument in your shebang line.
Read more.
What Becomes of a URL? (05/18/2006)
For many people who use the web frequently, the URL is still something of a mystery. What each portion of this odd-looking address actually means and how it relates to the page that eventually appears in their browsers is anything but obvious. In this week's column, we are going to dissect a URL and describe how each portion is processed and packaged in the course of an everyday page request.
Read more.
Working Late or Working Smart? (05/10/2006)
I have now been a Unix systems administrator for more than 20 years and, when I got a copy of the book "Time Management for Systems Administrators" by Thomas A. Limoncelli, I was doubtful that it would really change the way in which I work, especially over the long haul. I had, in fact, put a chapter on time management in one of my own books and feel that I'm somewhat anal about automating routine tasks and documenting difficult or time-consuming processes so that no one has to spend a lot of time rediscovering something that I've already figured out. Even so, I'm a ten-hour-a-day, five-day-a-week kind of sysadmin and I'm the first to admit that I have far too little leisure time and almost never have an entire day in which there is nothing that I am required to do (i.e., my definition of a day off). This little book called me to the carpet for buying into the frequently-interrupted, something's always breaking, there's-not-enough-of-me-to-go-around frenzy that I see so many of my fellow systems administrators taking on as a way of life. It reminded me that even I can, in fact, get more done in less time and honestly lay claim to the free time that I have rightly earned.
Read more.
Favorite Tricks, Take Two (05/03/2006)
Several weeks ago, I shared some of my favorite Unix "tricks" and invited readers to tell me about some of theirs. Quite a few people responded, so this week's column reflects some of the submitted favorites.
Read more.
Sending HTML Email (04/27/2006)
A common task of sysadmins is to generate email messages and send them out to a group of people. Reminders about upcoming system downtime or tips to help users make better use of their Unix accounts might be sent out routinely. In this week's column, we look at a bare bones script for sending out email in HTML format. First, we'll start off our script with a typical Perl shebang line, assign the location of the sendmail executable to $sendmail and the sender's email address to $sender.
Read more.
Using set to Assign Arguments (04/20/2006)
The set command is a very basic shell command, used to assign values to
variables for any kind of manipulation, but it can also be used to
assign each piece of text in a line of text to a separate variable,
facilitating subsequent processing. For example, if I want to extract
yesterday's high and low temperatures from my daily temperatures file, I
could do something like this.
Read more.
Favorite Unix Tricks (04/12/2006)
Whether you're looking to save on keystrokes, avoid errors or simply show off how terse and clever you can be, Unix systems offer a lot of options for smart use of the command line. In this week's column, we'll look at a few of my current favorite command line tricks.
Read more.
Is this the Last Day of the Month? (04/06/2006)
One of the issues that has long irritated Unix users when setting up cron jobs is how to set up a job to run on the last day of the month. Unless one wants to set up a separate cron job for every month (and this still leaves the problem of leap years), it isn't at all straightforward how to attack this seemingly ordinary problem with an ordinary solution. However, there are some reliable ways to code around the problem. One long winded but workable way to determine the last day of any month in any year is to string together some well known Unix commands. Read more.
Building and Using Analog on Solaris (03/30/2006)
Analog is a free web traffic analysis tool that prepares reports on
activity on your web sites, including graphs that summarize hourly,
daily, file size file type, visiting site, return codes and numerous
other statistics that illustrate how your web sites are being used. I
recently compiled and deployed Analog on a couple of Solaris 9
servers. This column is a how-to on building Analog and a quick
introduction to how it works.
Read more.
Creating Unix CDs on a Windows Box (03/23/2006)
I recently had to cut, on my laptop, some CDs of software to be loaded
on several Solaris boxes at a remote site and found myself running up
against incompatibilities between the Rock Ridge and Joliet extensions
to the ISO 9660 International standard which defines a file system for
CD ROMs. Specifically, I needed the files written to the CD in such a
way that they would have the proper Unix long file names when mounted
on the target systems. This would ensure that a scripted and heavily
tested install procedure would work correctly.
Read more.
Passwords: Baffle the Bad Guys, Not Your Users (03/16/2006)
Not long ago, I received an announcement about a new book on constructing good passwords. My first reaction was an incredulous "An entire book on PASSWORDS?". I have written columns on passwords, good and bad, and on password aging, but the idea of someone writing more than a hundred pages or more on the subject of passwords baffled me. So I got myself a copy.
Read more.
Displaying File Attributes with the Perl stat Command (03/09/2006)
A lot of information is available about individual files on a Unix system. For example, the ls -l command will display the permissions matrix and ls -i will display a file's inode. But, if we want to list all three of the file's dates (atime, ctime and mtime) or retrieve one of the attributes in a script, one of the versatile tools is a command included in Perl -- the stat command.
Read more.
Introducing TCP Wrappers (03/03/2006)
If you have never secured a Unix server with TCP Wrappers, you might be very surprised at how easily this can be done, especially on systems such as newer Linux servers on which the wrapper service is built in. The basic intent behind TCP Wrappers is to provide a security layer for services that don't have their own. An ftp daemon, for example, will generally accept connections from anyone with a username and password. With TCP Wrappers, however, you can restrict ftp to certain hosts or networks. In fact, any network service that is usually wide open with respect to who can connect to it can be restricted instead to a single system or a group of systems by running the service through TCP Wrappers. The wrapper service will then determine who is allowed and who is denied access based on how the service is configured.
Read more.
Using stty to Your Advantage (02/23/2006)
The stty (set the options for a terminal) command can be very useful once you get the knack of using it. Stty can give you quick manual control over the characteristics of your terminal. If ever you have found yourself working on a terminal in which the terminal settings did not match your actual terminal, you will know how annoying this situation can be. It is often experienced when you attempt to backspace to fix a typo on the command line and, instead of seeing the cursor move backwards on the line erasing characters, you see an increasingly long string of ^H or ^? characters appearing at the end of your command line.
Read more.
Using Lynx to Monitor a Web Site (02/16/2006)
Last week's column looked at sulog -- the file that collects information on switch user activity on many Unix systems -- and presented a simple Perl script for summarizing su activity. In this week's column, we are going to look at a version of the script that only reports on switches to the root account and one that summarizes switch user activity by month, allowing you to focus on recent activity instead of summaries that go back as many years as the sulog has been in use.
Read more.
Using Lynx to Capture Data from Web Sites (02/09/2006)
Last week's column looked at sulog -- the file that collects information on switch user activity on many Unix systems -- and presented a simple Perl script for summarizing su activity. In this week's column, we are going to look at a version of the script that only reports on switches to the root account and one that summarizes switch user activity by month, allowing you to focus on recent activity instead of summaries that go back as many years as the sulog has been in use.
Read more.
Tracking su Activity, Part 2 (02/02/2006)
Last week's column looked at sulog -- the file that collects information on switch user activity on many Unix systems -- and presented a simple Perl script for summarizing su activity. In this week's column, we are going to look at a version of the script that only reports on switches to the root account and one that summarizes switch user activity by month, allowing you to focus on recent activity instead of summaries that go back as many years as the sulog has been in use.
Read more.
Tracking su Activity (01/26/2006)
One of the things that every Unix systems administrator needs to know to properly manage a Unix system is who else is using the powers of root. Without that knowledge, it is not possible to be confident that the system's configuration will remain stable, that the cause of problems can be tied down to specific actions or who else might be given root access. It's also essential to know who is trying, but failing, to use the root account and who is using another user's account. Since one of the essential rules of any good Unix security policy is that no one knows another user's password, we probably want to know when this policy is being violated. If we can't be sure who is using each login, accountability on our systems in non-existent.
Read more.
Know Your Network Interface (01/19/2006)
Unix sysadmins routinely use the "ifconfig -a" command to list network parameters associated with their systems. In particular, they use this command to determine the IP address, netmask and broadcast address used by a particular interface. But there's a lot more information available in the "ifconfig -a" output. Let's examine some of the other information provided and see what it tells us.
Read more.
New Year's Resolutions for Unix Sysadmins (01/12/2006)
New Years celebrations have been going on for as long as 4,000 years. Some historians date them back to the ancient Babylonians welcoming the return of Spring. And New Years resolutions have probably been made throughout these years. While their success rate does not seem to have improved over the millennia, the practice has still not lost its appeal and the beginning of each new year is a time when many people will be thinking about what they want to improve both in their personal lives and in their jobs. So, what are some likely resolutions for Unix sysadmins? Here are a dozen worth considering.
Read more.
Troubleshooting Oracle Connections (01/05/2006)
I often find myself needing to verify whether an Oracle database is working. Sometimes a process running on one server extracts data from a database on another server and, if the process fails, testing the connection to the database is a good starting point from which to determine what went wrong. I don't have a lot of Oracle tricks up my sleeve, but a small amount of information about accessing Oracle goes a long way when troubleshooting potential problems.
Read more.
TCP/IP: The Guide (12/22/2005)
The thickest book on my shelf is no longer O'Reilly's sendmail (Second Edition) nor is it Learning Java (Third Edition). No, these hefty (nearly 2" thick) books have been outsized by a newcomer -- a book titled "The TCP/IP Guide" and subtitled "A Comprehensive, Illustrated Internet Protocols Reference". Written by Charles M. Kozierok (the force behind The PC Guide) and published by No Starch Press earlier this year, The TCP/IP Guide is 2.25" thick (without the covers). With more than 1,500 pages and 88 chapters, it includes all of the topics you would expect to find in a book about TCP/IP -- chapters, for example, on the OSI reference model, protocol layers, routing protocols and support protocols such as ICMP. It also offers material on a wide variety of application protocols such as DHCP, NFS, DNS, BOOTP and HTTP, adding meaning to the word "comprehensive". And these chapters don't give you just a gentle introduction to each of these protocols. Instead, they provide a thorough coverage with plenty of tables and diagrams. DHCP alone takes up four chapters. HTTP takes up five (with a sixth chapter introducing the Web).
Read more.
Capturing a Login Session (12/15/2005)
One of my newer users asked me just yesterday how he might record a login session in order to review the commands he entered and the output he generated at a later time. I pointed out two ways of doing what he wanted: 1) he could use the Unix "script" command to start and stop recording of all input and output from the command line during his Unix login session, or 2) he could use the facilities of the PuTTY tool, which he launches from his Windows desktop to access any of our Unix servers, to capture the entirety of his login session. In today's column, we'll look at these two ways of capturing session data and the data thus captured.
Read more.
Scrubbing Disks, Part 2 (12/08/2005)
In last week's column - Scrubbing Disks, Part 1, we asked a couple questions: "Why should you sanitize disks before letting them out of your sight?" and "How should you do it?". Given the availability of cheap data recovery tools and the ever-increasing population of hackers and identity thieves, releasing readable data into the used computer market can involve considerable risk -- a risk that far too many individuals and companies take on a routine basis.
Read more.
Scrubbing Disks, Part 1 (12/01/2005)
Retiring a disk that can't keep up with your growing mass of data might seem a routine task. After all, ten years ago, a big disk might have held 1 GB of data. Today, a big disk is 100 GB or larger. While retiring small disks may be routine, however, what you do with these disks before you send them on to the computer junkyard is not. Depending not only on whether you scrub your disks but how you scrub them, data may still be readable. In fact, numerous companies offering data recovery services routinely read data from disks which have been erased or damaged. And they work with nearly every type of disk and every popular operating system -- even Unix. How, then, should you erase a disk that you are retiring from service? What tools and techniques are available and at what cost and level of difficulty? Who can read data that might remain on a presumably "erased" disk and at what cost to them?
Read more.
Stuck in nslookup? (11/24/2005)
How many times have you had a user call to complain that his system was acting very strangely only to find that he had used nslookup and failed to exit properly? Typing "quit" instead of "exit", after all, is going to leave nslookup attempting to resolve the word "quit" as though it were a hostname and, before it has determined that the particular hostname cannot be resolved, your user may be already be madly typing commands at the command line and wondering why he isn't getting the responses that he expects. And, when the system finally does respond, it will probably be responding with something that looks like this...
Read more.
Setting Up apropos and whatis (11/17/2005)
One of the more frustrating aspects of learning Unix, or just about any computer language for that matter, is trying to recall the names of unfamiliar commands. This is especially annoying when the command names have little to do with their function. How, for example, would a new Unix user remember the name "awk"? Unix systems, however, have a couple of very useful commands that can help these users find the commands that they're looking for and figure out what other commands are supposed to do before they try them out -- apropos and whatis.
Read more.
Archiving User Accounts (11/10/2005)
One of the routine responsibilities of Unix sysadmins is to manage user accounts -- creating accounts when new people join the organization, configuring accounts to make it easier for these people to make use of application software or build software of their own and closing them when people leave the organization.
Read more.
Cobbling a Book Together: Safari U and You (11/03/2005)
Before we move on to another topic, there are a few more things that we can do to improve our password management scripts. For one thing, we can modify our daysleft.pl script so that it does not need to be run from a particular directory in order to locate the secondary caldate.pl script. For another, we can generate email notifications to users with passwords that are soon to expire instead of leaving the job of notifying users to the sysadmins. Last, we can modify the format of our displayed date to make it more international. Let's take a look at how these changes can be implemented.
Read more.
Password Aging, Part 4 (10/27/2005)
Before we move on to another topic, there are a few more things that we can do to improve our password management scripts. For one thing, we can modify our daysleft.pl script so that it does not need to be run from a particular directory in order to locate the secondary caldate.pl script. For another, we can generate email notifications to users with passwords that are soon to expire instead of leaving the job of notifying users to the sysadmins. Last, we can modify the format of our displayed date to make it more international. Let's take a look at how these changes can be implemented.
Read more.
Password Aging, Part 3 (10/20/2005)
In Parts 1 and 2 of "Password Aging", we looked at how many Unix systems (in particular, Solaris) manage the process of aging users' passwords and requiring them to be changed periodically. In particular, we looked at the password aging fields in the /etc/shadow file and a script that displays the current day in the days-since-Jan-01-1970 (or "Unix Time") format used to record the day that a user's password was last changed. In today's column, we're going to look at an efficient Perl command for expressing the current day in this format and a script that you can use to 1) list all users on a system whose passwords will soon be expiring and 2) print how many days are remaining along with the calendar date on which the password will expire.
Read more.
Password Aging, Part 2 (10/13/2005)
If you're starting with a group of users who have been active for a long time and not had their passwords aged, how should you go about introducing password aging? To start, you might first take a look at the dates on which your users' passwords were last changed. To view the dates by themselves, you might use a command such as this (run as root)...
Read more.
Password Aging, Part 1 (10/06/2005)
In this week's column, we look at the various fields in the shadow file that govern password aging and suggest settings that might give you the right balance between user convenience and good password security.
Read more.
Report Phishing Attempts (09/28/2005)
By the third time I'd received the same phishing attempt purporting to be a warning from PayPal, I was irritated enough to do something about it. A couple search terms later, Google had provided me with a link to something called the Phish Report Network -- a site established to address the growing problem of phishing.
Read more.
Tossing Email Addressed to Unknown Users (09/22/2005)
One of the great annoyances of this Age of Spam is how many system cycles are wasted every day by mail servers attempting to notify fictitious users that the spam they sent, so often addressed to randomly-generated usernames or to users who no longer have accounts on the system, could not be delivered. These messages generally wind up in your /var/spool/mqueue directory and there they languish for several days before sendmail gives us its valiant attempt to "do the right thing" and deletes them.
Read more.
The Ins and Outs of .rhosts (09/13/2005)
One of the ways of setting up inter-host "trust" between Unix systems is to set up ~/.rhosts or /etc/hosts.equiv files that allow everyone or only select users from other, presumably secure, systems to log in or issue remote shell (rsh) commands without providing passwords. While this technique is generally frowned upon these days because of the potential security breaches to which it is vulnerable, it is still used, particularly within relatively isolated networks. In today's column, we're going to look at how these two rsh configuration files provide password-free access to systems, consider the security issues that surround rsh and take a look at a problem that you might run into when setting up the rsh files. Read more.
Turning on SAR (09/06/2005)
Sar, the system activity reporter, is a tool for monitoring critical aspects of system performance -- such as CPU usage, memory usage and paging activity -- that exists on a number of Unix platforms. On Solaris systems, it is installed as an optional package (SUNWaccu). While sar provides invaluable information about how a system is performing, one of its main claims to fame is that it incurs a negligible performance hit on the server being monitored. Read more.
Learning In-Place, Learning Perl (09/01/2005)
Years ago, I was able to get employers to send me off for an occasional week-long class. They were usually Sun Ed classes in California and were almost always very good. In addition, getting away from my desk and my kids for a week of learning and quiet evenings was good for me. But I had a couple problems with these week-long classes so far from home. One was that eight hours of sitting in a classroom five days in a row was intense. By the end of the week, I'd be having a hard time staying in my chair and paying attention. I would be anxious to get back to the office and try out my newly acquired skills on some real problems. The other was that week-long classes, especially those in remote parts of the country, cost my employer quite a bit of money. And in lean years, when the budget tightened and money for training dried up, they were out of the question.Read more.
How to make Unix look like DOS (08/16/2005)
Before we even get started with this topic, let me acknowledge all of you readers out there who are wondering why I didn't call this week's column "Shoving Technology Backwards". Granted, few of us Unix sysadmins would actually strive to cripple our Unix systems by making them appear too much like DOS. On the other hand, for the same reason we might try to make our DOS command prompts look and act Unix-like (basically, user familiarity), we might take pity on long-time DOS users who are still uncomfortable in Unix and give them some DOS-like commands. Read more.
How to Make DOS Look Even More Like Unix (8/10/2005)
One of the other problems that I ran into when trying to make my DOS prompt behave more like my Unix systems was how to invoke my carefully crafted macros every time I opened a new DOS command window. When I put the commands to change my prompt and invoke my macros in a batch file and then ensured that this batch file was on my search path, I could reduce the effort that I had to make to running the batch file every time I opened a command prompt. I've since learned that it is possible to have my batch file invoked automatically whenever I open a DOS window (i.e., run cmd.exe). So, in this week's column, we're going to examine how this can be done. Read more.
How to Make DOS Look More Like Unix (08/03/2005)
Two weeks ago, in this column, we looked at some macros and DOS commands that can make your time at the DOS command prompt more like the Unix environment that most of us know and love. In this week's column, we're going to push this topic a little further by delving into some additional tools and commands that you can use to make DOS a bit more hospitable. First, we'll look at some ways to expand on your use of macros. Read more.
How to make DOS look like Unix (7/19/2005)
Ok, well maybe the subject line of this column is stretching it just a wee bit. There's no way that you can emulate more than a small fraction of Unix power at the DOS Command Prompt. But DOS has grown quite a bit since the days when it was an operating system in its own right. With pipes and redirection, a pile of commands that are roughly equivalent to their Unix counterparts (dir is to ls what type is to cat) and a little finesse, you can make your time working in DOS seem a little less like a detour through the dark ages and a little more like home. Read more.
Making better use of exit codes (7/13/2005)
Some of the worst-written scripts that I have encountered perform some action on behalf of the person running the script and then blithely continue on without ever checking whether the action taken was successful. For example, a script might echo issue a sequence of "installing ..." messages without ever checking whether the various packages that the script was intended to install were available for installing or whether the installations completed successfully. Others exit with a non-zero status code whenever something goes wrong, but then ignore the resultant status code when one script calls another. In fact, I have seen scripts which issue "successfully installed" messages in spite of the fact that the software installation was anything but successful. Read more.
What kind of file is this? (7/6/05)
One of the big differences between Unix and Windows systems is how the two systems classify and recognize different types of files. While Unix systems place less significance on file extensions than their Windows counterparts, file extensions, they can still play an important role -- such as identifying a file as a gzipped tar file or a pdf. But even when some type of file manager is used, the action that takes place when a user double clicks on an icon may have less to do with the file name than with the contents of the file.
Read more.
Review: Opengear 8-port Console Server (6/1/05)
After writing a column two months ago about how wonderfully convenient it is to use console servers to manage otherwise "headless" servers -- and from as much distance as you care to put between yourself and the servers to be managed, I couldn't resist an opportunity to evaluate a line of console servers that cost less than half what the others that I've priced are selling for. And, given that I'm working in a lab that is already equipped with a number of console servers, I felt compelled to question, in addition to the things you'd expect me to evaluate -- such as how easily a unit is to configure and how reliable it is -- how much additional work would be involved in managing two competing brands of console servers in the same lab. After all, I have said numerous times that it's easier to manage three hundred systems that are all basically the same than half a dozen which are different. So, I had to consider whether the added burden of managing two sets of devices that provide the same basic service would outweigh the cost savings. Read more.
A Survey Generator (6/23/05)
In this week's column, we're going to look at a bash script that generates survey forms. I use this script to generate quizzes for one of the classes that I teach at a local college. My students can take the quiz online and have the results emailed to me, but the same tool can be used whenever you want to ask true/false, multiple choice and short answer questions on the web. Read more.
Finding Text in Context (6/19/2005)
Unix tools are great for finding particular text in an input stream and for selecting a portion of each line to display. Like selecting rows and columns from a table in a relational database, commands like grep and awk allow us to trim huge amounts of data down to just what we want to see. In fact, the ability to pipe commands together and get a useful answer in a single line of commands continues to be one of the great appeals of our favorite operating system (in all its many flavors).
Read more.
Building Apache with a script (6/8/05)
Systems administrators have been building Apache servers from source for decades. For most of them, the routine is the same -- download a compressed tar file, extract its contents and run through the steps of configure, make, make install and make clean. Sometimes, however, additional modules are required or additional packages need to be installed before the new server can be installed. In these cases, the installation procedure is a bit more complicated. And, with these complications comes the risk that the server might not be built the same way if it needs to be installed on multiple machines. To facilitate the process of installing a new Apache server consistently across a group of systems, a sysadmin might build the new software on one box, tar up or otherwise bundle all the files that comprise the new package and install it on any number of other servers. Another method, and the one we will look at in this column, is to put all of the build commands in a script and transport it along with the source code and auxiliary files to any server needing to be upgraded. Read more.
Book Review: Silence on the Wire (6/1/05)
In today's column, we're going to take a look at a fascinating book entitled "Silence on the Wire: a Field Guide to Passive Reconnaissance and Indirect Attacks" by Michal Zalewski and published this year by No Starch Press. This book takes a very unusual approach in explaining the nature of a type of threat that most of us would probably never have normally considered; that is -- how much information can be gleaned about our systems, our businesses and ourselves by a very patient individual who is doing little more than paying attention to subtle "leaks" of information that can be extracted from everyday system communications.
Read more.
Setting Up RAID Volumes with Solaris Volume Manager, Part 2 (5/25/05)
In last week's column, we examined the process of setting up RAID volumes on a Solaris 9 server. The process is surprisingly straightforward when there are no file systems on the target disks or partitions. When file systems occupy the target space, they have to be backed up and then reloaded for RAID 5. For mirrored volumes, on the other hand, you can set up your volume in such a way as to preserve the initial partition's contents. In this column, we'll look at how these things are done. Let's says that we want to turn the /var partition on our new Solaris 9 server into a RAID 5 volume. One way to back up its contents is to use a ufsdump command to copy the file system to another location on the disk. A command such as this will copy /var quite expediently (you need to create /var-bkp first). Read more.
Setting up RAID volumes with Solaris Volume Manager (5/18/05)
Imagine you've just received shipment of a new Sun server, pre-installed with Solaris 9 and ready for setup as a web server for your small company. Now imagine that you want to take advantage of Solaris Volume Manager (previously called DiskSuite) to configure your new server to be resistant to disk failures. How do go about configuring the system and how do you set up its disks to provide fault resistent data storage? Read more.
Copying files from here to there (5/11/05)
There's more than one way to copy a file from one place to another whether the destination is on the same system or across the expanse of the Internet. Whether you're replicating a directory structure or backing up a single file for safe keeping, your choices of how to move the bits around include quite a few options.
Read more.
Running SSH on a non-standard port (05/05/2005)
If shutting off telnet access and insisting that all system-to-system connections use ssh isn't enough to toughen your system's hide, here's another way to make your servers just a little more difficult to access -- run ssh on a non-standard port. Unless unwelcome users are pointing port scanners at the system to detect active ports, they are not likely to figure out why they're not able to log in. While there are, of course, many ways to prevent normal users from logging into a system, this is one which provides a way to reduce access to a system while changing almost nothing about its configuration.
Read more.
Cool Shell Tricks (4/28/05)
Having become accustomed to using bash on nearly all of the systems that I manage, I find myself feeling irritated when I have to revert to a using a shell that lacks some of my favorite features or implements them differently enough that I have to work at making them work for me. In some shells, in fact, I find myself typing a command several times before I can get it right, each time giving up and starting over by typing a control-C, my dependence on keyboard shortcuts surprising me. The difference between these shells and those that make me feel like as if my fingers are dancing over the keyboard all revolve around two specific shell features -- command completion and an easy-to-use history mechanism.
Read more.
Making special files (4/19/05)
From time to time, someone asks me to locate an especially large file that he or she can use in testing. Sometimes, a large file may be needed to test throughput or ensure that some script or command doesn't fail to work when a target file larger than some number of gigabytes is presented to it. Finding an adequately large file on a system can be anything from very easy to a significant challenge, depending on the particular system and how it is used. Creating a large file -- even one that is many gigabytes in size, on the other hand just takes time. In today's column, we're going to look at the various commands for creating files of different kinds.
Read more.
Managing Library Paths with crle (04/14/2005)
Definitely NOT one of the first dozen Unix commands that a person learns, the crle command is so infrequently discussed in the hallways of software development companies that even I don't know how to pronounce it. Is it "see are elle ee" (phonetically "si ar el i") or am I supposed to say "curly"? After all, Unix offers few clues in such matters. We have the likes of vi ("vee eye" and awk (which rhymes with "talk") as two of many examples illustrating that there is no general pronunciation rule that governs our favorite OS. But, fortunately, since I don't have to read this column to you, let's move on to the more important issue -- what does this command do and how can you use it in managing your systems?
Read more.
What is a console server? (04/07/2005)
I recently had a chance to set up and then work with a console server. This was a first for me and so useful a device that I've become convinced that any systems administrator with more than a dozen systems to manage should know something about how console servers work and the service they can provide.
Read more.
Looking for last logins(03/31/2005)
I recently checked the status of a seemingly dormant account on one of my servers using the last command and got the following response...
Read more.
Passwords fail to update (03/24/2005)
One troubleshooting tale doesn't usually fall on the heels of another, but this one fell into my lap. A systems administrator at another site and long-time e-chum had run into some bizarre password problem in which one of his users was complaining that he had several different passwords on the LAN and couldn't manage to synchronize them. This might not seem like such an usual situation, except that this fellow sysadmin is using NIS on a fairly small network and has been managing this network solo since last summer.
Read more.
Can't open/chown savefile (03/17/2005)
Today's column is another troubleshooting tale - a real story from the pits of systems administration. And, although the solution to the particular problem encountered turned out to be trivial, the process of tracking it down involved some interesting turns and insights into little known details about the mail command works on a Unix system.
Read more.
Monitoring non-critical processes (03/10/2005)
The script we're going to look at in this week's column is a low-profile process watcher. On a configured interval, it determines whether the process being monitored is running or not and it writes records to a log file. However, unlike many process monitoring scripts, it only writes to the log file if the process was previously running and has stopped or was not previously running and has started. So, the log file ends up containing records that show when the process started and stopped, adjusted to the precision of the configured interval -- records such as these...
Read more.
Quick fixes for Sendmail security (03/03/2005)
As one of the most enduring Unix applications and one of the most ubiquitous, sendmail has been the target of numerous attacks over the years -- from buffer overflows to denials of service. And, even with constantly improved security, sendmail can be prodded into a state of improved security through a number of quick fixes that any sysadmin can make to reduce the vulnerability of their sendmail servers.
Read more.
Troubleshooting: Interface down (02/24/2005)
There are numerous problems that can interfere with your access to a system -- everything from the system being down to problems with name servers and routers. In today's troubleshooting tale, we're going to look at a system with a downed network interface and try to unravel what went wrong.
Read more.
Capturing shutdown time (02/17/2005)
There are many ways to determine the time that a Unix system last booted. Among these are boot messages that appear in the system log file (whether /var/adm/messages, /var/log/system.log or a similarly named file) -- messages such as this line from a Solaris /var/adm/messages file...
Read more.
Zipping Your Way to Free Space: Part 3 (02/10/2005)
In the last two columns, we contrasted a number of compression tools that you might use to free up disk space on your systems. In comparing compression utilities, the most important issues to consider are speed, the compression ratio (how much space you can expect to save), portability and reliability. Which of these factors ranks highest in your list depends on your application, but the compression tool that you use routinely should work well on most of the files that you need to compress, giving you reasonably good file size reduction and acceptable performance.
Read more.
Zipping your way to free space: Part 2 (02/03/2005)
Last week, we examined a variety of compress utilities and considered some of the criteria with which you might rank their effectiveness in your environment -- criteria such as the compression ratio and whether the utilities might allow you to transfer files between Unix and non-Unix platforms. This week, we're going to compare the speed and effectiveness of the same five compression utilities and look at a script that you can use to run the same tests on your files. The script will provide measurements both on how long each compression takes to run and on how much the target files are compressed.
Read more.
Zipping Your Way to Free Space: Part 1 (01/27/2005)
One good way to save disk space is to compress large files that are used only now and then or that are maintained on your servers solely as archives. These files can use up a LOT of free space and there's little, if any, advantage to keeping them uncompressed. In fact, the only advantage that I can think of is that you'll never have to worry if you have enough space to uncompress them when the time comes -- not much of a justification for anything but the most critical files.
Read more.
Am I interactive?: Take three (01/20/2005)
If there's one thing that every Unix systems administrator knows, it's that there's always more than one way to get something done. Some ways of solving a problem are more efficient than others. Other ways of solving a problem are easier to understand or extend to similar, but not identical, situations. But there are always multiple solutions to a task.
Read more.
Am I being run interactively? (1/12/05)
In one of last month's columns, we looked at a method for determining whether a process was being run by cron. In that column, we used the ptree command, which shows a process' lineage in an indented display like that shown below, to determine whether the "ancester" of the process is the cron process. In this week's column, we'll examine an even simpler way to determine whether a process is being run interactively -- by using the tty command. Read more.
Multiple personality scripts (01/06/2005)
There are several ways that a single file on a Unix system can be referenced by more than one name. By creating a symbolic link, for example, you might refer to a file both by the name of the file and by the name of the link. By creating an alias, on the other hand, you can assign an alternate name to a file and, also manage to avoid a lot of typing -- especially if you tend to run the command with the same complex parameter string. Each of these methods of assigning multiple system "identities" to a file has some advantages. You can establish names that are easier to remember than the names of the original files, you can reduce your typing and you can avoid pre-pending path names for executables which are not in your search path. An extremely useful third option is to use hard links to give a file multiple file system identities. If the script behaves differently in some way, depending on the name used to invoke it, you can create a single script that supports any number of different but related services, depending on how it is called. Let's look at an extremely simple example.
Read more.
Calculating overall disk space (12/23/04)
In this week's column, we're going to examine a bash script for calculating the overall disk space on a Solaris system. This script will use thr prtvtoc command and a bash array to organize and manipulate information about the geometry for each of the system's disks.
Read more.
Checking on memory with the memconf utility (12/16/04)
While there are a number of ways to determine how much memory a system has installed, very few tell you succinctly how that memory is arranged and whether or not you have available sockets when you're thinking of upgrading.
Read more.
Am I being run by cron? (12/09/2004)
Have you ever wanted to force a script to act differently when it is invoked by cron? If the script is run interactively, for example, you might want the script to prompt the user for some information while, if the script is run through cron, you might instead want to run with a set of defaults. Read more.
Getting an entry with getent (09/23/2004)
While there are numerous ways to look up the IP address associated with a particular hostname, most of these methods imply that you have some knowledge about the sources that the system you are troubleshooting will be using. Thus, if you grep on a hostname in the /etc/hosts file, you may find an IP address for that system, but it may not be the one that the local system is using. Why? Because, until you look a little further -- at the /etc/nsswitch.conf file -- you won't know whether the entries in the /etc/hosts file take precedence over other sources of such information, such as DNS, NIS+ and LDAP. Similarly, if you use nslookup to find an IP address for the host in question, you have the same basic issue. Without looking at the /etc/nsswitch.conf file, you won't know whether the information retrieved is the same information that the local system will be using. Here's how the genent command can help.
Read more.
Troubleshooting - Trying to ssh to a remote server (11/24/2004)
Are you trying to ssh to a remote server only to find yourself staring at a "connection refused" message? Attempts to telnet as root fail with the expected "not on system console" message? Not to be discouraged by this small setback, you telnet into the box as yourself (shuddering all the while). Once logged into the box, you check on the status of sshd. But, to your surprise, not only is sshd not running, but it refuses to start.
Read more.
Printing in columns (11/18/2004)
Regardless of how sophisticated Unix has become, one of its ever-endearing qualities is the ease with which the command line can be used to select, manipulate and display data. Clever little languages like sed and awk continue to remind why it was so much fun to sit down with the early O'Reilly books, trying each new little "trick" that they taught. Here's a look at a couple of tools for creating columnar output from a list of strings -- a simple task, but one that I hate to do "by hand".
Read more.
Debugging tales: SSH command failure (11/11/2004)
In a typical work week, a Unix systems administrator is likely to have at least one small mystery to solve -- one "huh?", one "that doesn't make any sense" or one "I've never seen this before". This column follows a Unix expert's train of thought as they poked through one such small mystery.
Read more.
Mount Options for UFS (11/04/2004)
While not heavily used, options available for UFS file systems can dramatically improve file system performance and stablility. This article looks at how some of these options affect the way file systems operate and why you might consider using them.
Read more.
Setting up a Unix system to be an NTP client is generally straightforward. On those Unix platforms that include NTP in the default configuration, you may only need to make a few adjustments to NTP's default configuration file for the system to sync up to a reference time server(s) and keep good time.
Read more.
Scrounging for Disk Space (10/21/2004)
Server disks are getting larger all of the time, but this simple fact doesn't seem to be making much of a difference in the system administrator's day-to-day routine. OUr disks are still filling up faster than most of us can trim them down. So, when we just need to free up a chunk of disk space in a hurry on some particular file system, what should we do?
Read more.
NTP Commands (10/14/2004)
NTP clearly has considerable advantages over the date and rdate commands. For one, NTP provides a way to synchronize system clocks with unusual precision and to make required adjustments smoothly, avoiding the jumps in time that commands such as rdate often cause. Let's take a closer look.
Read more.
For some of us, the word "synchronize" brings up memories of the characters on Mission Impossible synchronizing their watches so that they can complete a seemingly impossible mission with split-second accuracy. While time synchronization on computer systems seems far less dramatic, it is no less critical. Time differences between systems running a complex application can lead to some very troublesome problems -- such as when these time differences make transactions appear to have happened out of order. To avoid this kind of problem, a number of solutions for keeping system clocks in synch, have been devised. NTP is the larges scale and most effective of these.
Read more.
Fragmentation and Unix file systems (09/30/2004)
People who have had bad experiences on Windows disks that performed poorly after they had become very fragmented often wonder if they need to apply some kind of periodic analysis and defragmentation on their Unix systems as they transition into a Unix admin or user role. For the most part, the answer is no. But, rather than give a quick yes/no answer, let's take a look at what fragmentation, and why Unix file systems are not as prone to fragmentation as their Windows cousins and what you need to do when a Unix file system is fragmented.
Read more.
If you have tried to give away any files lately on any of your Unix systems (i.e., without first becoming root), you may have been surprised to learn that you can't. On most Unix systems today, any non-root user is likely to get errors such as these when he or she tries to change ownership on a file.
Read more.