Unix: When pipes get names

Unix pipes are wonderful because they keep you from having to write intermediate command output to disk (relatively slow) and you don’t need to clean up temporary files afterwards. Once you get the knack, you can string commands together and get a lot of work done with a single line of commands. But there are two types of pipes that you can use when working on a Unix system – regular, unnamed or anonymous pipes and named pipes. These two types of pipes share some advantages, but are used and implemented very differently.

The more common type of pipe allows you to take the output of one command, say ps –ef, and pass it to another command, say grep, to be processed further. Just stick a | between the commands and, voila, you have a pipe and probably some useful output. In fact, you can string together commands and pipes until you run out of things to do with the data you are manipulating. I have seen some clever one-liners with three or four pipes. These pipes exist inside the Unix kernel.

Named pipes, like their unnamed counterparts, allow separate processes to communicate, but they do so by establishing a presence in the file system. They are sometimes referred to as FIFOs. FIFO stands for “first in, first out” and, as you might suspect, these pipes work like a line at the supermarket. If you get in line first, you should be the first to push your shopping cart out to the parking lot. But, unlike unnamed pipes, these pipes can be viewed with the ls command and are created with the mkfifo command.

$ mkfifo mypipe
$ ls -l mypipe
prw-r----- 1 shs geeks 0 Sep 29  2013 mypipe

Notice that the file type is represented by the letter “p” and that the file has no apparent content. Permissions, as with other files, depend on your umask settings and determine who can read or write to your pipe.

Want to try a simple example? Open two ssh sessions to a system. In one, create your pipe and then send some command output to it.

$ mkfifo mypipe
$ cal > mypipe

Don’t worry that your command just seems to hang.

In the other, read from the pipe.

$ cat < mypipe
   September 2013
Su Mo Tu We Th Fr Sa
 1  2  3  4  5  6  7
 8  9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30

What should pop out of this simple demonstration is how your pipe is able to allow processes running in separate sessions to communicate. Depending on permissions, separate processes run by separate users are just as easy. And, of course, these pipes are reusable. They’ll still be sitting in your file system after your commands have completed.

Run this looping script, sending its output to your pipe.

#!/bin/bash

while true
do
    echo "I'm still running"
    sleep 10
done

$ ./loop > mypipe

In the second login session, read from your pipe:

$ cat < mypipe
I’m still running
$

As soon as you ^c in the first window, you should see your session go back to the prompt.

I'm still running
I'm still running
$

Depending on what you end up doing with your pipe, you could get a “broken pipe” message in the second window. This means that the input side of the pipe went away too quickly

You can also write scripts or run commands that wait for output from a named before taking the next step – as in this example. In one session, do this:

$ if read line <mypipe; then
> echo this is a test
> fi

Again, the session seems to freeze. But then send something through the pipe from the other session and you’ll be back at the prompt:

$ echo hello > mypipe

Check on your first window again and you should see something like this:

$ if read line <mypipe; then
> echo this is a test
> fi
this is a test

Named pipes are especially useful if many processes are going to read from and write to your pipes or when processes need to send a lot of information, especially a variety of data, to other processes. You can look for named pipes on your systems using a command such as this:

$ find / -type p -print 2> /dev/null

Don't be surprised if you find only a handful. Named pipes can be extremely handy, but they're not heavily used on most Unix systems.

pipes-Danndalf.jpg