Tips on good shell programming practices

October 17, 2001 —

Once upon a time, Unix had only one shell, the Bourne shell, and when a
script was written, the shell read the script and executed the
commands. Then another shell appeared, and another. Each shell had its
own syntax and some, like the C shell, were very different from the
original. This meant that if a script took advantage of the features of
one shell or another, it had to be run using that shell. Instead of
typing:

doit

The user had to know to type:

/bin/ksh doit

or:

/bin/csh doit

To remedy this, a clever change was made to the Unix kernel -- now a
script can be written beginning with a hash-bang (#!) combination on
the first line, followed by a shell that executes the script. As an example, take a look at the following script, named doit:

#! /bin/ksh
#
# do some script here
#

In this example, the kernel reads in the script doit, sees the hash-bang, and continues reading the rest of the line, where it finds /bin/ksh.
The kernel then starts the Korn shell with doit as an argument and
feeds it the script, as if the following command had been issued:

/bin/ksh doit

When /bin/ksh begins reading in the script, it sees the hash-bang in the first line as a comment (because it starts with
a hash) and ignores it. To be run, the full path to the shell is required, as the
kernel does not search your PATH variable. The hash-bang handler in the
kernel does more than just run an alternate shell; it actually takes
the argument following the hash-bang and uses it as a command, then
adds the name of the file as an argument to that command.

You could start a Perl script named doperl by using the hash-bang:

#! /bin/perl

# do some perl script here

If you begin by typing doperl, the kernel spots the hash-bang,
extracts the /bin/perl command, then runs it as if you had typed:

/bin/perl doperl

There are two mechanisms in play that allow this to work. The first is
the kernel interpretation of the hash-bang; the second is that
Perl sees the first line as a comment and ignores it. This technique
will not work for scripting languages that fail to treat lines starting
with a hash as a comment; in those cases, it will most likely cause an error. You needn't limit your use of this method to running scripts either, although that is where it's most useful.

The following script, named helpme, types itself to the terminal when
you enter the command helpme:

#! /bin/cat
vi     unix editor
man    manual pages
sh     Bourne Shell
ksh    Korn Shell
csh    C Shell
bash   Bourne Again Shell

This kernel trick will execute one argument after the name of the
command. To hide the first line, change the file to use more by
starting at line 2, but be sure to use the correct path:

#! /bin/more +2
vi     unix editor
man    manual pages
sh     Bourne Shell
ksh    Korn Shell
csh    C Shell
bash   Bourne Again Shell

Typing helpme as a command causes the kernel to convert this to:

/bin/more +2 helpme

Everything from line 2 onward is displayed:

helpme
vi     unix editor
man    manual pages
sh     Bourne Shell
ksh    Korn Shell
csh    C Shell
bash   Bourne Again Shell
etc.

You can also use this technique to create apparently useless scripts, such as a file that
removes itself:

#! /bin/rm

If you named this file flagged, running it would cause the command
to be issued as if you had typed:

/bin/rm flagged

You could use this in a script to indicate that you are running something,
then execute the script to remove it:

#! /bin/ksh
# first refuse to run if the flagged file exists

if [-f flagged ]
then
    exit
fi

# create the flag file

echo "#! /bin/rm" >flagged
chmod a+x flagged

# do some logic here

# unflag the process by executing the flag file

flagged

Before you begin building long commands with this technique, keep in mind that systems often have an upper limit (typically 32 characters) on the length of the code in the #! line.

Testing command line arguments and usage

When you write a shell script, arguments are commonly needed for it to
function properly. In order to ensure that those arguments make sense, it's
often necessary to validate them.

Testing for enough arguments is the easiest method of validation. For
example, if you've created a shell script that requires two file names to
operate, test for at least two arguments on the command line. To do
this in the Bourne and Korn shells, check the value of $# -- a variable
that contains the count of arguments, other than the command itself. It
is also good practice to include a message detailing the reasons why the command
failed; this is usually created in a usage function.

The script twofiles below tests for two arguments on the command line:

#! /bin/ksh

# twofile script handles two files named on the command line

#  a usage function to display help for the hapless user

usage ()
{
     echo "twofiles"
     echo      "usage: twofiles file1 file2"
     echo      "Processes two files"
}

# test if we have two arguments on the command line
if [ $# != 2 ]
then
    usage
    exit
fi

# we are ok at this point so continue processing here

A safer practice is to validate as much as you can before running your
execution. The following version of twofiles checks the argument count and tests both files. If file 1 doesn't exist (if [ 1 ! -f $1 ]) an error message is set up, a usage is displayed, and the program exits. The same is done for file 2:

#! /bin/ksh

# twofile script handles two files named on the command line

#  a usage function to display help for the hapless user

# plus an additional error message if it has been filled in

usage ()
{
     echo "twofiles"
     echo      "usage: twofiles file1 file2"
     echo      "Processes two files"
     echo " "
     echo $errmsg
}

# test if we have two arguments on the command line
if [ $# != 2 ]
then
    usage
    exit
fi

# test if file one exists and send an additional error message
# to usage if not found

if [ ! -f $1 ]
then
    errmsg=${1}":File Not Found"
    usage
    exit
fi

# same for file two

if [ ! -f $2 ]
then
    errmsg=${2}":File Not Found"
    usage
    exit
fi


# we are ok at this point so continue processing here

Note that in the Korn shell you can also use the double bracket test
syntax, which is faster. The single bracket test actually calls a
program named test to test the values, while the double bracket test
is built into the Korn shell and does not have to call a separate
program.

The double bracket test will not work in the Bourne shell:

if [[ $# != 2 ]]

or

if [[ ! -f $1 ]]

or
if [[ ! -f $2 ]]

This thorough validation can prevent later errors in the program logic
when a file is suddenly found missing. Consider it good programming
practice.

Unix Insider