Security

What to Look in a Test Automation Product: Features or Benefits

July 20, 2009, 11:10 AM — 

Almost all the same benefits apply to almost any automation, whether it is vulnerability testing by security experts, integration testing by large enterprises, load testing by tier-1 carriers, or acceptance tests of outsourced development. I am not sure if this is useful for you, but if it is: Please give a thumbs-up and I will know that at least some people have troubles with automation.

Test Automation

Manual testing and test scripts are often painful to develop and use, but basically still are the de-facto in the industry. A manual test, or self-developed test scripts doesn't require that much investments in tools, only in time and people. But when time, and people are exactly what you do not have, then the natural move is towards test automation.

When you start looking at test automation products, you can easily get distracted by the technological features in the collateral and related white-papers. Model-based! TTCN! XML syntax notation! Dynamic message sequences! 100% test coverage! I want that!

The leap from manual tests or simple scripts into fully automated test suites makes all self-respecting test engineers just simply excited. It is difficult to see the limits to the things where test automation could be used. But hold on! Are you thinking straight? Why was it that you were looking for test automation in the first place?

To remind you, there are three main things why people look for test automation:

  • cheaper test execution,
  • less resources in development and maintenance, and
  • better test coverage.

Cheaper, Faster, More... Through Automation

Most test automation tools can be categorized in two groups:

  • Capture-playback tools
  • Test modeling tools 

A capture-playback tool lets you define your own test sequences, or if you look at it another way, it requires you to build the entire test setup just to produce a valid, conformant test sequence. It will then record that and let you re-run the test. Test automation people in this camp are known to say "If you need to do it more than twice, automate".

The challenge in this type of automation is that the tests are only as good as the capture you take to "seed" the tests. The test automation vendor takes no responsibility for the quality of tests. And you still need to build the "old fashion way" for the first test execution, and then the tool just automates the re-execution of the tests.

Sign up for ITworld's Daily newsletter
Follow ITworld on Twitter @IT_world

I like it!
Comments

Do something - a step in the right direction

The most important choice is to make a choice. Even if you decide to change your mind later on, make a start now.

It's like all of the tools available to help with software development - some are great, some are useless, and most are in between. But having any tool is better than having no tool at all.

Once you have a tool and play with it for a while, you'll begin to see the benefits, and you'll have a better understanding for where you want to go next. You can't get this perspective just from evaluating and deliberating. You have to pick one and try it.

So jump in - tools are great, there are lots of them out there and the more you try the better you'll understand your own specific problem space.

Thanks for the article!

Mr. Hericus
www.ZedBuildsAndBugs.com
| reply
peer-to-peer

Esther Schindler
If the comments are ugly, the code is ugly

claird
SVG a graphics format for 21st century

pasmith
Take Chrome OS for a test spin

Sandra Henry-Stocker
Solaris Tip: Have Your Files Changed Since Installation?

sjvn
64-bits of protection?

jfruh
Android fragments vs. the iPhone monolith

mikelgan
What Gizmodo missed about the Pro WX Wireless USB disk drive

 

Sidekick: The Good News & the Bad News
Either way you look at it Microsoft Data Center management did not follow standards or best practices in this failure. In which case it makes me wonder more about the outsourcing of corporate data much less personal data.
- mburton325

Join the conversation here

The Daily Tip

The Daily TipQuick, practical advice for IT pros. Made fresh daily.

Hot tips:

Want to cash in on your IT savvy? Send your tip to tips@itworld.com. If we post it, we'll send you a $25 Amazon e-gift card.

Newsletters

Subscribe to ITWORLD TODAY and receive the latest IT news and analysis.

I would like to receive offers via email from ITworld partners.
By clicking submit you agree to the terms and conditions outlined in ITworld's privacy policy.
Featured Sponsor

AISO founders envisioned a Web hosting company that was environmentally friendly. While the company employed energy-efficient innovations like solar panels, its infrastructure produced unacceptable power and cooling requirements. Find out how AISO leveraged AMD technology to overcome their challenge in this case study white paper.

In this whitepaper, Scalar explores the opportunity to change the landscape with respect to mission critical databases built around Oracle. Leveraging technologies such as Linux, high-end commodity processing power and Oracle RAC technology to architect, design, build and maintain database infrastructure that delivers maximum availability, reliability and performance at a fraction of traditional cost.

On a typical day, weather.com, the Web site for The Weather Channel in Atlanta, serves up between 15 million and 20 million page views. But in September 2004, when back-to-back hurricanes ransacked Florida, the peak traffic on one day more than tripled: over 70 million page views by more than 7 million unique visitors. Read the full success story now.

Marketplace