Measuring how fast is really, really fast
In computing, there are two perennial testing issues. They address speed and compatibility. When you consider how many facets there are to a modern PC and its communications infrastructure, testing performance and compatibility can be a wickedly difficult science.
In the early days of IBM clones, compatibility tests boiled down to whether or not a PC could run Lotus 1-2-3 out of the box and support Microsoft's Flight Simulator. System performance was measured by obscure but discrete metrics such as how many Fibonacci numbers could be calculated.
As the PC benchmarking game evolved, several trade magazines attempted to develop hardware and network performance measurements while vendors took a crack at their own measurements. But because most were developed in a vacuum, few standard test methodologies existed.
While performance and compatibility measurements at the desktop are important, networking PCs now also raises questions about system throughput and elasticity from end to end. There's good news and bad news about systems testing. Real-world performance numbers have traditionally been hard to come by. However, recent benchmarking efforts do seem to provide performance numbers that are much closer to the real world because they use tunable simulations rather than pristine -- even bleached -- platforms.
Two prominent (although largely vendor-driven) organizations exist: Standard Performance Evaluation Corporation (SPEC) and Benchmark and Performance Corp. (BAPCo).
SPEC provides a number of tests that simulate characteristics ranging from battery consumption to the number of Java browser transactions that can be performed per unit of time. SPEC Web99 and SPEC JBB2000 are two examples of platform simulators that measure the madness that network and Web operations centers go through.
While SPEC derives measurements from simulations, BAPCo uses real-world applications that have been "frozen together" in amalgamations to measure desktop characteristics and Internet content creation via its SYSmark2000 tool. A user's Internet experience is now measured by BAPCo's WebMark2001, which can be useful in the quest to measure the ever-important impatience point of Web performance.
I'd be remiss in talking about benchmarking without mentioning NetBench, ServerBench and WebBench from eTestingLabs. These tools, while not as honed as those from BAPCo and SPEC, deliver meaningful but confined analysis of their respective targets.
All transactional benchmarks today have a staggering variety of tuning options. The number that's often proffered as a vendor-competitive metric may have an enormous amount of confining characteristics used in testing to deliver the optimal result. The devil is in the details, and most benchmarks now include full disclosure statements. When determining whether or not The Number has any meaning in your network, make sure you read all the fine print.
» posted by ITworld staff
Network World
Symantec Backup Exec 12 and Backup Exec System Recovery 8 deliver industry leading Windows data protection and system recovery. Download this whitepaper to find out the top reasons to upgrade and how to get continuous data protection and complete system recovery.
Data and system loss — from a hard drive failure, malicious attack, natural disaster, or simple human error — can happen anytime. Don’t leave your business vulnerable. Make sure you have a secure recovery strategy in place. Symantec's latest backup and system recovery technology can efficiently restore critical applications, individual emails and documents and even restore your entire system in minutes in the event of a loss.
Businesses face a growing challenge to ensure that the IT environment is properly protected. Backup Exec 12 integrates with other applications in the Symantec family of products, to complement your current data protection strategy, keep your data securely backed up and make it recoverable when you need it most.
