topics that matter; ideas worth sharing

share a tip, submit a link, add something new

Implementing filters and pipes

March 30, 2001, 10:53 AM —  Unix Insider — 

A filter is a program that accepts input, transforms it, and outputs the transformed data. Filters are closely associated with several ideas basic to Unix: standard input, standard output, input/output redirection, and pipes.


Standard input and output



Standard input and output refer to the default places from which a program will take input and to which it will write output. The standard input (stdin) for a program running interactively at the command line is the keyboard; the standard output (stdout) is the terminal screen.


Redirection



With input/output redirection, a program can take input or send output someplace other than standard input or output -- to or from a file, for instance. Redirection of stdin is accomplished using the < symbol, and redirection of stdout by the > symbol. For example:


$ ls > list


redirects the output of the ls command, which would normally go into a file called list. Similarly,


$ cat < list


redirects the input for cat, which, in the absence of a filename, would be expected to come from the file list. So we print the contents of that file on screen.


Pipes



Pipes connect programs through I/O redirection and are denoted by the | symbol. For example:


$ ls | less


is a common way of comfortably viewing the output from a directory listing where there are more files than will fit on the screen.


grep



The principle of grep is very simple: search the input for a pattern, and then output that pattern. Here's an example:


$ grep 'Linus Torvalds' *


This searches all the files in the current directory for the name Linus Torvalds.


Various command-line switches can modify grep's behavior. For example, if we aren't sure about case, we can write:


$ grep -y 'linus torvalds'


The -y switch tells grep to match without considering case. If you use uppercase letters in the pattern, however, they'll still match only uppercase. (This is broken in GNU grep, which ignores case when given the -y switch -- that's what the -i switch is for).


Given even this much grep, it's easy to construct a practical application. Store name and address details in a file and you've got a searchable address book.


$ grep -y [search arg] ~/lib/phone-book


Put the command above in a text file called filename and make it executable:


$ chmod +x filename


But suppose we want to find all the occurrences of a text string that could be a reference to Linus Torvalds. Searching for Linus Torvalds won't find Linus or Torvalds individually. We need a way of saying, "This or this or this." Here's where egrep (extended grep) comes in. This handy program modifies standard grep to provide just such a syntax.


$ egrep 'Linus Torvalds|L\. Torvalds|L\. T\.|Mr\. Torvalds'


This will find most ways of naming the inventor of Linux. Note the backslashes to escape the full stop; because that's a special character in regular

I like it!
Post a comment
The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
Resources
White Paper

Symantec Backup Exec 12 and Backup Exec System Recovery 8 deliver industry leading Windows data protection and system recovery. Download this whitepaper to find out the top reasons to upgrade and how to get continuous data protection and complete system recovery.

Webcast

Data and system loss — from a hard drive failure, malicious attack, natural disaster, or simple human error — can happen anytime. Don’t leave your business vulnerable. Make sure you have a secure recovery strategy in place. Symantec's latest backup and system recovery technology can efficiently restore critical applications, individual emails and documents and even restore your entire system in minutes in the event of a loss.

White Paper

Businesses face a growing challenge to ensure that the IT environment is properly protected. Backup Exec 12 integrates with other applications in the Symantec family of products, to complement your current data protection strategy, keep your data securely backed up and make it recoverable when you need it most.

Free stuff
Featured Sponsor

Get a broad understanding of important regulations and how you can make sure your site is in adherence.





Learn how VeriSign SGC-enabled SSL Certificates can help improve site security and customer confidence in the free white paper, "How to Offer the Strongest SSL Encryption." In this paper you will learn the differences between weak and strong encryption and what they mean for your site's performance.

Get VeriSign's free white paper: "The Latest Advancements in SSL Technology" and learn about the benefits of strong SSL encryption, Extended Validation (EV) SSL and security trust marks and what these SSL offerings can do for your site.

Now with Extended Validation (EV) SSL available from VeriSign, you can show your customers that they can trust your site. Learn about EV SSL benefits in this free VeriSign white paper.

More Resources