Testing GUI applications, part 1

By Cameron Laird and Kathryn Soraiz, Unix Insider |  Development, GUI Add a new comment

What programs can test graphical applications? As consultants, we recognize that a large part of our business is decoding what clients really want, but we are wary of this particular question, as it has a history of obscuring true goals. This two-part series will explore the quality assurance (QA) of programs with GUIs.

First, we need a brief theory of quality. There's an extensive amount of literature on quality, much of it applicable to computing. In fact, quality assurance is one of the few well-grounded domains of computing, in the sense that it has yielded replicable experimental data. Evidence about the effectiveness of object orientation or the correct philosophy for chip sets (CISC versus RISC?) remains ambiguous and generally open to interpretation. Quality researchers, though, proudly point to the principles they've established beyond reasonable doubt:

  • Quality is built-in, not tested-in
  • Tests can only prove a product wrong, not right
  • It's far less expensive to detect faults earlier rather than later
  • Individuals and organizations generate characteristic levels of errors that they reproduce in subsequent trials

What's the significance for the construction of a high-quality GUI application? Well, quality matters. In general, it matters more than any testing program bandaged on a project at the last minute. Does your project have the resources to manage a QA phase that only takes action when the product is almost ready? Then it has more than enough resources to document requirements explicitly, design the application carefully, and engineer the software throughout development. We've never seen a case where investing in end-phase testing and beta programs pays off more than it does with those kinds of up-front engineering.

Even when the above commandments are observed, there's generally more important work for a QA staff than the often disorganized grunt work traditionally assigned them. This month, we'll describe the activities a successful QA department undertakes as part of a development project. Next month, we'll look at specific testing frameworks for common GUI applications.

QA's contribution

Too many organizations try to paste QA onto the end of their development process. When they think they're nearly ready to go to market, they toss their newest employees, and sometimes others who don't fit in anywhere else, in a room, stir in a poorly defined beta program, and ask for a miracle. This is the intellectual equivalent of solving security for a rock concert by hiring a biker gang on the final weekend before the show. Both are recipes for spilt blood.

Real quality requires involvement from a QA engineer from the start. A good product has sound finances, a manufacturing plan, and so on, from the start, to focus on achievable objectives that meet customer needs. It also has a QA plan from the beginning.

From the first planning session, a good QA engineer asks, "What does it mean to have an 'adequate' response? Does the screen need to flash within three seconds? Thirty? Do you realize that writing custom widgets rather than using the native ones has a history of soaking up weeks of tinkering and just confuses end users? Will the documentation we're planning satisfy all three of the market segments we're targeting? Why have we budgeted for a smaller customer support staff, on a proportionate basis, than our competitors?"

The first job is to assess feasibility. A plan that calls for even elite coders to turn out hundreds of line of source daily, with all but one in a thousand flawless, is unrealistic.

In the best projects, documentation and testing protocols are written at the beginning and are constantly available to guide source development. While this is one of the precepts of extreme programming (XP), it's a principle that was known long before XP's current popularity. It's certainly standard procedure in more mature domains of engineering, including aerospace.

Perhaps most indispensably, QA engineers hunt for testability.

Testing only proves programs wrong, not right

Testing never proves a program right. That proposition is well-known dogma in QA circles, and neglected or not even understood outside them. Here's a stark example. Can you test your way to confidence that:


     0.1 * (a + b) = (0.1 * a) + (0.1 * b)
   

How many values of a and b would you have to try to make you feel good about the formula?

    Add a comment

    Post a comment using one of these accounts
    Or join now
    At least 6 characters

    Note: Comment will appear soon after you have activated your account.
    Obscene/spam comments will be removed and accounts suspended.
    The information you submit is subject to our Privacy Policy and Terms of Service.

    ITworld LIVE

    DevelopmentWhite Papers & Webcasts

    White Paper

    HP NonStop SQL Fundamentals whitepaper

    This whitepaper offers a detailed look into the fundamentals of HP NonStop SQL solutions. See how this system delivers unprecedented levels of application availability with fail-safe data integrity and meets the needs of enterprises with large-scale business critical applications.

    White Paper

    Nebraska Medical Center case study

    See how the Nebraska Medical Center implemented a SQL solution to make information more readily available to streamline operations, improve patient care and facilitate medical research with an enterprise solution running on HP NonStop servers.

    White Paper

    Concepts of NonStop SQL/MX

    For DBAs and developers who are familiar with Oracle solutions and want to learn about NonStop SQL/MX, this whitepaper provides an overview of the similarities and differences between the two products-with a specific focus on implementation.

    White Paper

    6 Things Your CIO Needs to Know About Requirements

    If your organization is not predictably successful on technology projects, there is likely an issue in requirements. CIOs must take action and own requirements maturity improvement. There are 6 main things a CIO must know about requirements.

    Webcast On Demand

    User Experience Monitoring

    In this webinar, you will learn hints & tips for improving end-user response times from Forrester Research analyst, Jean-Pierre Garbani.

    Sponsor: Nimsoft

    See more White Papers | Webcasts

    Ask a question

    Ask a Question