IT management/strategy

Unix tricks - Useful but unusual

April 1, 2009, 10:33 AM — 

Let's explore a few unusual Unix tricks that you might not have seen before.

First, let's look at the paste command. It's more or less the opposite of the cut command which you have probably seen or used to slice some text from its input. The paste command, on the other hand, splices together input from multiple files, lining the text from each source in a side-by-side fashion. Let's look at an example.

If you have three files, each with a small amount of text on each line, for example, you can paste the files together like this:

boson> paste file1 file2 file3
alligators       1,345   xenops
bears    2,890   yaks
cats     3,154   zebras

The output of paste will be more or less columnar, depending on the width of each file's data. Of course, if you want to stash the output in a fourth file, you only have to redirect it.

boson> paste file1 file2 file3 > file4

Another useful "trick" is to make use of the options that come with the cat command to display unusual data in files. The -v option, for example, will display unprintable characters. If you have octal 13 characters in your file, for example, cat -v will display them as ^K characters and octal 7s as ^G.

boson> cat -v file4
A hen is only an egg's way of making another egg.
        Samuel Butler
^G^G^G^G^G
-----
boson> cat file4
A hen is only an egg's way of making another egg.
        Samuel Butler

-----

They would otherwise appear as a blank line at the end of the file.

If you use a -e (-ve on Solaris), line endings will be displayed with a $ as shown here:

boson> cat -ve file4
A hen is only an egg's way of making another egg.$
        Samuel Butler$
^G^G^G^G^G$

A -t (-vt on Solaris) will display tabs as ^I characters.

boson> cat -vt file4
A hen is only an egg's way of making another egg.
^ISamuel Butler
^G^G^G^G^G

And, of course, you can combine these options if you like:

boson> cat -vet file4
A hen is only an egg's way of making another egg.$
^ISamuel Butler$
^G^G^G^G^G$
-----$

One more. There are numerous ways to omit blank lines from text displays or to remove them from files. I've often used the beginning of line and end of line markers together like this "^$" to indicate a line with no content.

grep -v "^$" file
grep -v "^$" file > newfile

An even easier way to do the same thing is to match on any character like this:

cat file | grep .
cat file | grep . > newfile

Better yet, you can avoid the pipe and save a few CPU cycles:

grep . file

The "." will match on any character (but not linefeeds), so any line containing any type of content will be displayed. Plus, if you only want to see lines with more than one character or at least three characters, you can expand your matching string to "..", "..." and so on.

Sign up for ITworld's Daily newsletter
Follow ITworld on Twitter @IT_world

I like it!
Comments

pipe nightmares

So many people type

cat file | grep Whatever

and I never really got it - why do all that extra typing when you could just say

grep Whatever file

???
| reply

Unix Tricks

It just goes to show you that in Unix, there is more than one way to skin a "cat".
| reply

and inefficient

If you do this in a shell script, you'll be repeatedly starting an extra process for no reason at all.
| reply
peer-to-peer

Esther Schindler
If the comments are ugly, the code is ugly

claird
SVG a graphics format for 21st century

pasmith
Take Chrome OS for a test spin

Sandra Henry-Stocker
Solaris Tip: Have Your Files Changed Since Installation?

sjvn
64-bits of protection?

jfruh
Android fragments vs. the iPhone monolith

mikelgan
What Gizmodo missed about the Pro WX Wireless USB disk drive

 

Sidekick: The Good News & the Bad News
Either way you look at it Microsoft Data Center management did not follow standards or best practices in this failure. In which case it makes me wonder more about the outsourcing of corporate data much less personal data.
- mburton325

Join the conversation here

The Daily Tip

The Daily TipQuick, practical advice for IT pros. Made fresh daily.

Hot tips:

Want to cash in on your IT savvy? Send your tip to tips@itworld.com. If we post it, we'll send you a $25 Amazon e-gift card.

Newsletters

Subscribe to ITWORLD TODAY and receive the latest IT news and analysis.

I would like to receive offers via email from ITworld partners.
By clicking submit you agree to the terms and conditions outlined in ITworld's privacy policy.
Featured Sponsor

AISO founders envisioned a Web hosting company that was environmentally friendly. While the company employed energy-efficient innovations like solar panels, its infrastructure produced unacceptable power and cooling requirements. Find out how AISO leveraged AMD technology to overcome their challenge in this case study white paper.

In this whitepaper, Scalar explores the opportunity to change the landscape with respect to mission critical databases built around Oracle. Leveraging technologies such as Linux, high-end commodity processing power and Oracle RAC technology to architect, design, build and maintain database infrastructure that delivers maximum availability, reliability and performance at a fraction of traditional cost.

On a typical day, weather.com, the Web site for The Weather Channel in Atlanta, serves up between 15 million and 20 million page views. But in September 2004, when back-to-back hurricanes ransacked Florida, the peak traffic on one day more than tripled: over 70 million page views by more than 7 million unique visitors. Read the full success story now.

Marketplace