March 10, 2010, 3:45 PM — Even after decades of using Unix on thousands of systems, I find that it's still fun to discover various convolutions of sed and awk commands to perform command line wizardry. There's a lot more to each of these tools than those uses I make of these commands on a routine basis. Let's take a look at some one-liners you might not yet have tried.
One of my long-standing "tricks" for awk is using $NF to print the last field on every line. Since NF represents the number of fields on a line (e.g., 6), $NF represents the value of that last field (e.g., $6). Printing the last field of every line, therefore, might look like this:
$ awk '{print $NF}' myfile
or this:
$ awk -F: '{print $NF}' /etc/passwd
depending on whether the file is white space delimited or not.
You can get the length of every line in a text file by using awk's length command. For example, this command prints the length of every line in the /etc/motd file:
$ awk '{print length($0)}' /etc/motd
You can compute the square root of every fourth field in a file of numbers like so:
$ awk '{print sqrt($4)}' nums
You can display select lines from a chosen file.
$ awk '/nobody/ { print $0 }' /etc/passwd
You can generate an alphabetical listing of your users with an added sort command.
$ awk -F: '{ print $1 }' /etc/passwd | sort
You can count the lines in a file. In this example, we're using the END (only perform this after all lines have been processed) feature of awk to print the line number of the last line in the file.
$ awk 'END { print NR }' myfile
If you want to know how many words are in every line of a file, try this:
$ awk '{print NF}' myfile
You can add line numbers to every line in a file.
$ awk '{print NR, $0}' myfile
Sed, which I often find myself using only for those dead easy text substitutes such as sed "s/this/that/g" myfile also has some more interesting applications.
You can double space a file (appends a newline to every displayed line). Use 'G;G' (don't forget the quotes!) for triple spacing, 'G;G;G' for quadruple and so on.
$ sed G myfile
You can remove double spacing from a file.
$ sed 'n;d'
I've used sed to replace only the first instance of a string or every instance of a string in each line of text, but you can also use it to replace a specific instance -- for example, only the third or only the seventh instance.
$ sed 's/this/that/3'
You can also change multiple strings in a single sed command using semicolons like so:
$ sed 's/this/that/g;s/you/me/g;s/him/her/g' myfile
The sed and awk commands are far more versatile than I had remembered them being and could provide some useful aliases for you to toss into your startup files.
