Unix: When pipes don't make sense

While pipes are one of the things that make Unix such a powerful and versatile operating system, they don't always make sense. Many people still fall into the "useless use of a pipe" habit while still others use pipes in situations in which pipes simply don't function at all. Knowing when to pipe and when not to pipe remains a sign of Unix mastery. If you want to look Unix savvy, use pipes only when they provide a critical service and never when they're superfluous or downright wrong.


It's been nearly twenty years since I first came across the Useless Use of Cat (UUOC) awards. Unix notable and Perl disciple Randall Schwartz had begun handing out these embarrassing awards around 1995 to people who used commands such as "cat myfile | head -3" when they could more easily have used "head -3 myfile". Wasting system cycles and spawning unneeded processes is not a Big Sin when it comes to Unix, but it's clearly both wasteful and, well, indicative of users who are either falling into a bad habit or just not paying attention.

The worse offense, using pipes when they simply don't work, is generally a sign that someone is still trying to understand how the Unix command line works. The fact is that some Unix commands are engineered to read command output that is piped to them while others are not. Recognizing the difference takes time and a bit of patience, but pipe mastery is well worth the investment.

For the UUOC transgression, the basic thing to keep in mind is that many commands can work directly with files as easily as they can work with content that is piped to them. You can sort a file with sort myfile as easily as you can do the same thing with cat myfile | sort. And, if minimizing your typing is a sign of Unix wisdom, well then, the first command is going to keep your fingers from tapping any more keys than necessary so bring on the smarts. In fact, there may be very few justifiable uses for commands that begin with cat myfile |. One example is when you want to do something like this -- though I can't say I've ever seen anyone actually doing this:

$ cat myfile | tee file1 file2 file3

In this command, we get a chance to view the contents of a file and redirect its content to several other files in one command. A more useful example of the "cat file |" command prefix might be this:

$ cat myfile | mailx -s "data file 12" recip11@mysite.org

Here we're sending the content of a file to someone's inbox. This kind of command proves useful in many situations, especially when collecting and emailing system performance or status information to yourself or a group of admins.

Most other uses of cat myfile | are probably not going to prove very clever, especially if you're sending the output to head, tail, wc, grep, awk or other commands that are well prepared to read files without intervening pipes. An exercise that I like to give to my Intro Unix students is to have them rewrite commands such as cat /etc/passwd | grep $USER: without the pipes. "Sure" I tell them, "pipes are wonderful things. But that doesn't mean you need to use them every chance you get".

Of course, explaining to new Unix users when pipes work and when they don't is much harder than having them rewrite a list of sub-optimal commands.

Photo Credit: 

flickr / Beige Alert

Join us:






Operating SystemsWhite Papers & Webcasts

See more White Papers | Webcasts

Answers - Powered by ITworld

ITworld Answers helps you solve problems and share expertise. Ask a question or take a crack at answering the new questions below.

Ask a Question