sed: not just for pipes and one-liners

August 26, 2012, 7:03 PM — Over the years, the primary way that I've used sed is to change text on the fly. There are many occassions in which I want to change one string to another the first or every time that it appears in a file or command output.

$ sed 's/2011/2012/' new
$ sed s/tomorrow/today/ msg2
$ sed 's/,/:/g' < file

sed is, after all, a stream editor and it does this kind of thing exceptionally well and blindingly fast.

The sed command is, however, also very good for a number of other tasks associated with selecting text from a file. For example, you can select the line in a text file by using a combination of head and tail commands, but this takes a little thought:

$ head -11 myfile | tail -1

Alternately, you can use a sed command and easily pick out the 11th line. Say we have a list of tasks and we want to see just the eleventh task in the list. Use sed like this:

$ sed -n '11 p' tasks
11:  Get the sailboat back in the water

If you want to pick 100 lines from the middle of a file with millions of lines, the difference in how much less time it takes to do the job with sed could be considerable.

You can also do the same thing like this:

$ sed -n 11p tasks
11:  Get the sailboat back in the water

sed can remove a line from a file as well. Once the sailboat is actually back in the water, you might want to remove the 11th line in your tasks list like this (but keep in mind that sed is identifying the line by number not content):

$ sed -i '11 d' tasks

or

$ sed -i 11d tasks

The -i in this command instructs sed to do its editing in place -- in other words, change the source file. Where the "p" above means "print", the "d" here means "delete". If you want to remove a range of lines rather than a single line, you can specify a range of lines with an expression like 11,13 (lines 11 through 13) or even 11,$ (the eleventh line through to the last line in the file). This command will leave only the first ten lines in your tasks list:

$ sed -i '11,$ d' tasks

The $ in "11,$" stands for the last line in the file.

You can use a similar syntax if you want to display those lines instead of removing them from the file:

$ sed -n '11,$ p' tasks
11:  Get the sailboat back in the water
12:  Drywall the living room
13:  Rebuild the front porch
14:  Donate the llamas to a petting zoo
15:  Take a vacation
16:  Get a new job
17:  Sell the sailboat for a big profit
18:  Hang a photo of the sailboat in the basement of the new home
19:  Live happily ever after

This command will only display the lines. It won't affect the original file.