Managing terabytes of data

Be the first to comment | 1I like it!
April 23, 2001, 12:01 PM —  Computerworld — 

How much data do large corporations manage? Tons of it. Referring to "tons" of data may be intuitive for paper records, but it's an unusual way to describe computer-stored information, which is usually measured by character counts and file sizes. Still, using ton may give an added sense of how much data a terabyte is. To be sure, measuring data by the ton isn't definitive because a disk drive's weight doesn't vary significantly over a wide range of storage capacities, but it's a handy starting point. A common 8GB hard drive weighs a little more than 1 lb. Figure that the weight of a shared enclosure, power supply and electronics will roughly double the drive's weight, and we can say that 8TB of data is approximately equivalent to 1 ton. That much storage is cumbersome and ungainly.

How does an enterprise deal gracefully and effectively with such unwieldy mountains of information? We asked four data-intensive companies -- Aetna Inc., The Boeing Co., Atos Origin and AT&T Corp. -- to tell us about the problems they faced in managing massive data stores, and how they solved them. For each company, the data is a significant corporate asset resulting from huge investments of time and effort. The data is also the source of many trials and tribulations for the employees who keep vigilant watch over it.

While these companies say that good tools are important for managing terabytes of information, their IT and database administrators also agree that having a clear and comprehensive perspective on the data, via both logical and physical views, is even more critical. Security, data integrity and data availability aren't trivial concerns, they point out, and giving users easy access to the data is a never-ending job.

Insuring a Healthy 21.8 Tons

On a daily basis, Renee Zaugg, operations manager in the operational services central support area at Aetna, is responsible for 21.8 tons of data (174.6TB). She says 119.2TB reside on mainframe-connected disk drives, while the remaining 55.4TB sit on disks attached to midrange computers running IBM's AIX or Sun Microsystems Inc.'s Solaris. Almost all of this data is located in the company's headquarters in Hartford, Conn. Most of the information is in relational databases, handled by IBM's DB2 Universal Database (Versions 6 and 7 for OS/390), DB2 for AIX, Oracle8 on Solaris and Sybase Inc.'s Adaptive Server 12 on Solaris. To make matters even more interesting, Zaugg adds, outside customers have access to about 20TB of the information. Four interconnected data centers containing 14 mainframes and more than 1,000 midrange servers process the data. It takes more than 4,100 direct-access storage devices to hold Aetna's key databases.

Tips for Managing Large Data Stores

Be selective in how you implement HSM. Instead of blindly giving all your data to a robotic HSM process, analyze and classify your company's data usage to know how often the data is reused and

I like it!
Post a comment
The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
Free books

Essential JavaFX
Get started building rich Web apps quickly with an introduction to the power of JavaFX key features -- scene node graphs, nodes as components, the coordinate system, layout options, colors and gradients, custom classes with inheritance, animation, binding, and event handlers.Enter now!

The Nomadic Developer
Consulting can be hugely rewarding, but it's easy to fail if you are unprepared. To succeed, you need a mentor who knows the lay of the land. Aaron Erickson is your mentor, and this is your guidebook. Enter now!

Featured Sponsor

AISO founders envisioned a Web hosting company that was environmentally friendly. While the company employed energy-efficient innovations like solar panels, its infrastructure produced unacceptable power and cooling requirements. Find out how AISO leveraged AMD technology to overcome their challenge in this case study white paper.

In this whitepaper, Scalar explores the opportunity to change the landscape with respect to mission critical databases built around Oracle. Leveraging technologies such as Linux, high-end commodity processing power and Oracle RAC technology to architect, design, build and maintain database infrastructure that delivers maximum availability, reliability and performance at a fraction of traditional cost.

On a typical day, weather.com, the Web site for The Weather Channel in Atlanta, serves up between 15 million and 20 million page views. But in September 2004, when back-to-back hurricanes ransacked Florida, the peak traffic on one day more than tripled: over 70 million page views by more than 7 million unique visitors. Read the full success story now.

Marketplace