October 01, 2001, 3:59 PM — One of the mysteries of Unix (aside from Unix itself) is the command line, filled as it is with strange and cryptic characters. Now let's see, do I need a dot between two backslashes, or should it be a backwards quote followed by a hyphen?
One thing that will help sort out what is actually happening in a command line, and maybe even help you construct one of your own, is an understanding of how the command line is interpreted.
The command line is actually the input to the shell program. The shell program (sh, ksh, csh, or any other variant) reads the input line and untangles it before it attempts to execute the command. The sequence of steps the program goes through to untangle a command provides interesting insight into shell programming. By studying it you're sure to learn some new tricks.
We will cover these pieces in more detail in a moment, but first let's take a look at the sequence of evaluation of a command line:
- History substitution (except for the Bourne shell)
- Splitting words, including special characters
- Updating the history list (except for the Bourne shell)
- Interpreting single and double quotes
- Alias substitution (except for the Bourne shell)
- Redirection of input and output (
< >and|) - Variable substitution (variables starting with
$) - Command substitution (commands inside back quotes)
- File name expansion (file name wild cards)
You will note that the Bourne shell is missing the steps to do with history and alias substitution. These are not included in the Bourne shell.
History substitution
If you have history set up in the Korn shell (ksh), C shell (csh), or any similar shell, command lines are saved in a history file before they are executed. You can review your previous commands by typing:
$ history
The list of commands is preceded by a number, as in:
13 ls *.txt 14 cd $HOME 15 ls *.log
In Korn shell you can usually recover a history command by typing r followed by the number. For example, typing r 13 in the example above would repeat the command ls *.txt.
In the C shell, use an exclamation point and no space instead of an r: !13.
When processing a command line, the shell first checks for these command substitutions, looks them up in the history file, recovers each command, then creates a new command line with each. There is much more to history than these simple steps, but we'll save that for a separate article.
Splitting words
The next step is to separate the words and special characters into words. A
word is basically a token that is recognized by the shell program as an element of a command. For example, the following command does a long listing of the current directory and searches for mjb in any line of the directory information.
ls -l|grep mjb
The words in the command are ls, -l, |, grep, and mjb. A word can also be a quoted string. In the following command, a long directory
listing is searched for files created on "Sep 07."
ls -l|grep "Sep 07"
In this case the words are ls, -l, |, grep, and Sep 07. Note that Sep 07 is treated as one word because it was quoted in the command.
Update the history list
Once the words are identified, the command is written to the end of the history file. (Assuming that you're using history.)
Single and double quotes
Where a word has been surrounded by double or single quotes, the word is tagged so that variable expansion either does or does not occur within the quotes. Variables that are surrounded by single quotes will be left as is, and variables in double quotes will be expanded. To see this difference for yourself, enter the following commands with no quotes, single quotes, and double quotes.
