Purging the process

RELATED TOPICS

Unix Insider –

Redirection allows a user to redirect output that would normally go to the screen and instead send it to a file or another process. Input that normally comes from the keyboard can be redirected to come from a file or another process.

When a typical Unix utility starts up, three files are automatically opened for you inside of it. These files are given file descriptor numbers inside the program -- 0, 1, and 2 -- but they're more commonly known as stdin (standard in -- file descriptor: 0), stdout (standard out -- file descriptor: 1) and stderr (standard error -- file descriptor: 2). When the program starts, default assignments for these files are made to

<font face="Courier">/dev/tty</font>
, which is the device name for your terminal. The stdin file is assigned to the keyboard of your terminal, while stdout and stderr are assigned to its screen.

Let's start with a simple example using

<font face="Courier">grep</font>
. Type a
<font face="Courier">grep</font>
command to find lines containing the word hello, then type the following lines at your terminal. At the end of each line press Enter to move down to the next line. Watch what happens as you type
<font face="Courier">say hello</font>
.

<font face="Courier">
$ grep "hello"
Now is the time
for every good person to
say hello.
</font>

The screen repeats the last line.

<font face="Courier">
$ grep "hello"
Now is the time
for every good person to
say hello.
say hello.
</font>

Hold down the Control key and press D to end the input to

<font face="Courier">grep</font>
. Control-D is an end-of-file marker and can be entered as a keystroke to stop any utility that is taking its input from the keyboard.

The

<font face="Courier">grep "hello"</font>
line is a command to search standard input for lines containing hello and echo any such line found to standard output. The Unix console automatically echoes anything you type, so the three lines appear on the screen as you type them. Then
<font face="Courier">grep</font>
hits a line containing hello and decides to output it to standard out, and
<font face="Courier">say hello</font>
appears on the screen a second time. The second appearance is the output from
<font face="Courier">grep</font>
.

Standard output can be redirected to a file using the right angle bracket (

<font face="Courier">></font>
) as shown in the example below. The same
<font face="Courier">grep</font>
command is redirected to send its output to a file named
<font face="Courier">junk.txt</font>
. The
<font face="Courier">say hello</font>
line doesn't appear a second time because it's been directed to the
<font face="Courier">junk.txt</font>
file. After the user presses Control-D,
<font face="Courier">cat</font>
is used to display the contents of
<font face="Courier">junk.txt</font>
, which contains
<font face="Courier">grep</font>
's single output line.

<font face="Courier">
$ grep "hello" >junk.txt
Now is the time
for every good person to
say hello.
(type control-D here)
$ cat junk.txt
say hello.
$
</font>

Standard input can be redirected to come from a file by using the left angle bracket (

<font face="Courier"><</font>
). In order to demonstrate this, we need a file that can be used for input. Use vi to create the following sample file and save it as
<font face="Courier">hello.txt</font>
.

<font face="Courier">
Now is the time
for every good person to
say hello.
</font>

When you type the following command, notice that the output from

<font face="Courier">grep</font>
is the single
<font face="Courier">say hello</font>
. Because input is being drawn from a file, you don't need to use Control-D to stop the process.

<font face="Courier">
$ grep "hello" <hello.txt
say hello.
</font>

Both standard input and output are redirected in the following example. Once

<font face="Courier">grep</font>
starts up, it takes its input from
<font face="Courier">hello.txt</font>
and outputs the result to
<font face="Courier">junk.txt</font>
. There is no output on the screen, but you can use
<font face="Courier">cat</font>
to display
<font face="Courier">junk.txt</font>
and verify the contents.

<font face="Courier">
$ grep "hello" <hello.txt>junk.txt
$ cat junk.txt
say hello.
$
</font>

If a redirection to an output file encounters a file that already exists, that file is destroyed and a new one, containing the new output, is created, assuming the user has appropriate permissions to delete and create a new file. You can confirm this by using the previous example to search for a different line of text. In this example, the earlier version of

<font face="Courier">junk.txt</font>
has been replaced with the new output from
<font face="Courier">grep</font>
, the single line
<font face="Courier">Now is the time</font>

<font face="Courier">
$ grep "Now" <hello.txt >junk.txt
$ cat junk.txt
Now is the time
$
</font>

There is a convention used in Unix programs which dictates that, if a file is expected as input to a program but no file is named on the command line, standard input is used. Because

<font face="Courier">grep</font>
is designed to search for a string in a file, or files, it uses a command-line syntax that lets you name a file on the command line, and the input redirection symbol is not needed. Internally,
<font face="Courier">grep</font>
checks if a file is named on the command line and opens and uses it. If no file name is found, standard input is used. The following command lines for
<font face="Courier">grep</font>
have the identical effect.

Internally, the first command reassigns

<font face="Courier">hello.txt</font>
to standard input and uses it for input; the second command opens
<font face="Courier">hello.txt</font>
as a file and uses it for input.
<font face="Courier">grep</font>
doesn't expect an output file to be named on the command line. To get the output into a file, you must use output redirection. It doesn't hurt to redirect
<font face="Courier">grep</font>
input, but in the case of
<font face="Courier">grep</font>
, the redirection is already taken care of for you on the command line.

<font face="Courier">
$ grep "Now" <hello.txt >junk.txt
$ grep "Now" hello.txt >junk.txt
</font>

If you want to preserve the existing output file and append new information to it, use a double right angle bracket (

<font face="Courier">>></font>
). The following example uses
<font face="Courier">echo</font>
, which normally outputs to the screen, to create the
<font face="Courier">hello.txt</font>
file without using an editor. The output of the
<font face="Courier">echo</font>
command is redirected into the file, and two more lines are appended to it.

<font face="Courier">
$ echo "Now is the time" >hello.txt
$ echo "for every good person to" >>hello.txt
$ echo "say hello." >>hello.txt
$ cat hello.txt
Now is the time
for every good person to
say hello.
$
</font>

Pipes are created as a means of taking the output of one program and using it as the input to another. The pipe symbol (

<font face="Courier">|</font>
) is used as a connector between the two programs. In the following example, look at the first part of the command up to the first pipe symbol. The
<font face="Courier">cat</font>
command normally outputs to the screen; in this case, however, the output has been sent into a pipe. On the righthand side of the pipe, this output becomes the input to
<font face="Courier">grep "hello"</font>
. The output from
<font face="Courier">grep "hello"</font>
is in turn sent into another pipe. On the right side of that pipe, the output is used as standard input to a
<font face="Courier">sed</font>
command that searches for hello and replaces it with bye. The final result is redirected to a file named
<font face="Courier">result.txt</font>
which
<font face="Courier">cat</font>
displays on the screen as
<font face="Courier">say bye</font>
.

<font face="Courier">
$cat hello.txt | grep "hello" | sed -e "s/hello/bye/" > result.txt
$cat result.txt
say bye.
$
</font>

If this were broken down step by step using simple redirection, you would need several commands, as well as the final

<font face="Courier">rm</font>
steps to clean up the intermediate work files that were created.

<font face="Courier">
$cat hello.txt >wrk1.txt
$ grep "hello" <wrk1.txt >wrk2.txt
$ sed -e "s/hello/bye/" <wrk2.txt >result.txt
$cat result.txt
say bye.
$rm wrk1.txt wrk2.txt
</font>

The initial step of getting

<font face="Courier">hello.txt</font>
into the
<font face="Courier">grep</font>
command could also be done in several other ways. Two examples are shown below. The first redirects input to
<font face="Courier">grep</font>
from
<font face="Courier">hello.txt</font>
on the lefthand side of the pipe; the second puts parentheses around the
<font face="Courier">grep</font>
and
<font face="Courier">sed</font>
commands, groups them as a subprocess, then redirects input and output to the grouped process.

<font face="Courier">
$ grep "hello" < hello.txt | sed -e "s/hello/bye/" > result.txt
$( grep "hello" | sed -e "s/hello/bye/" ) < hello.txt > result.txt
$
</font>

Redirecting standard error output

So far I've only shown you how to pipe and redirect standard output, but it's frequently useful to do something with error output. In the following example,

<font face="Courier">find</font>
is being used to search the entire system (starting at
<font face="Courier">/</font>
) for files with a
<font face="Courier">.txt</font>
extension. Whenever one is found, its full directory entry is placed in a file named
<font face="Courier">textfiles</font>
. The example below shows sample error messages that are generated when
<font face="Courier">find</font>
attempts to access an unavailable directory.

<font face="Courier">
$ find / -name *.txt -exec ls -l {} \; >textfiles
find: /some/directory: Permission denied
find: /another/one: Permission denied
$
</font>

The error messages can be suppressed by redirecting them to

<font face="Courier">/dev/null</font>
, which is a special device that can be thought of as a wastebasket for bytes written to it on output. Everything that goes to
<font face="Courier">/dev/null</font>
disappears. To redirect standard error, use a right angle bracket preceded by a 2, which is the file descriptor number for standard error. If you don't care about error messages, send them to the
<font face="Courier">/dev/null</font>
byte bucket.

<font face="Courier">
$ find / -name *.txt -exec ls -l {} \; 2>/dev/null >textfiles
$
</font>

The following command combines redirection and pipes to extract and bring a full list of all

<font face="Courier">.txt</font>
files sorted in order by the third field in the
<font face="Courier">ls -l</font>
directory entry, the owner's name.

<font face="Courier">
$ find / -name *.txt -exec ls -l {} \; 2>/dev/null |sort -k 3 >textfiles
$
</font>

Shell scripts can also redirect their output, so the above command could be put into a shell script without redirection, but the output can be redirected when the command is executed.

<font face="Courier">
#!/usr/bin/sh
# usertexts
#    outputs a listing of texts files on the system, ordered by owner id

find / -name *.txt -exec ls -l {} \; 2>/dev/null |sort -k 3
</font>

This shell's script could be executed with the output redirection done at the shell script level.

<font face="Courier">
$ usertexts >textfiles
$
</font>

Pipes and redirection can be combined to create very powerful tools that start a text stream and then apply different tools to that stream, filtering it as it passes through different processes.

Next month, I'll take a look at more advanced uses of pipes and redirection.

RELATED TOPICS
Infographic: Starting salaries for computer science grads
View Comments
You Might Like
Join the discussion
Be the first to comment on this article. Our Commenting Policies