July 10, 2013, 6:00 AM —
Image credit: flickr/Bill Bradford
If you develop software for any length of time, you’ll inevitably run into two things: bugs and end users. Unfortunately, no matter how carefully you build your software - and how thoroughly you test it - those two things will, sooner or later, find each other. When they do, your job will be much easier if those users can send you useful reports about those bugs.
The problem is, users often don’t write bug reports that provide developers with all of the information needed to reproduce - and, ultimately, fix - the bug. But how can developers get users to write useful bug reports? Over the years, programmers have come up with a number of different approaches to solving this problem. They tend to fall into one of the following categories:
Train users to write useful bug reports
Bad bug reports, some believe, are just the results of users not being properly trained in how to write good ones. There are any number of guidelines for writing good bug reports that are available to be shared with users. If users are too busy to study these, or if they find these directives to be too difficult to understand (or too long to read), then developers just need to specify - in simple, clear language - what they need from users.
We've had success by making it very clear what information we need. They can't know what information we need, so we need to tell them. It's a hard problem, but not unsolvable.”
You can ask easy to understand and easy to answer questions to users to expect useful reports.
Automate the bug reporting process
Some developers feel that training users may never work, so it’s best to make the job of bug reporting as easy as possible by automating the process. This means creating or using bug reporting functionality that automatically collects URLs, screenshots, state variables, and any other available data associated with the user’s actions and the results. This approach ensures that developers at least get some basic diagnostic information as part of the report.
Make those people a browser extension that captures a screenshot, lets them paint a big red rectangle, a comment field, then annotates with things like all cookies for current page, browser history (on the current site), user email address, … Then teach them "something goes wrong, push the button".
It's called a "report a bug" menu item that automatically compiles as much data as you can think of that might help....
Put 'report bug' as an option in the help menu. And make sure your bug-reporting mechanism is the best-tested portion of the entire piece of software.
Talk to users to get the information you need
Even with bug report automation, users will usually still be needed to provide additional information on what they did and what happened. Rather than expecting users to know what information or data are needed, some developers realize the advantage of simply talking to users, preferably face-to-face. Like any good relationship, communication is key!
The solution... is to go visit the users and have a conversation. Understand how they view your software and you can then understand their explanation of what it did wrong. Stop expecting detailed bug reports and start understanding users.
In my opinion, what's more important is using the bug to establish meaningful on going contact with the user.
They'll write bad bug reports anyway. What you typically need to do is communicate with the users somehow.
Designate a go-between to collect bug reports
Some companies are able to facilitate bug reporting by have a designated third party collect the required information from users and report it to the developers. This approach, of course, depends on the company having the resources to dedicate to this role.
I think what you need is someone to be the go between. Get a tester to receive those emails, recreate the issue, then file a bug report.
I've had success by educating a small group of business users to serve as triagers of bug reports - it's a lot easier to have them enforce your rules .... or have them validate the issue on their own before an engineer even looks at it.
Our company has also instituted a layer of BA's that sit between the devs and the clients and have thankfully done a fantastic job of training the clients what they need to provide when they submit a bug report.
Stop trying to change users; it's hopeless
Like with grief, the final stage (for some) of trying to get users to write better bug reports is acceptance that it's a hopeless task. They stop expecting users to consistently provide useful reports. Rather than fight against it, they simply soldier on and do the best with what they get.
It's not their job.... If you want good bug reports, employ professionals to find your bugs.
Some of the worst offenders have been other developers! If peer developers cannot do it, what makes anyone think that the regular user community will step up?
...send them all to reeducation camps. Try not to stress about it, it's hopeless.
All of these approaches, I think, have something to offer, though I suspect it’s a problem that developers will be discussing forever.
What about you? Do you have any special methods or techniques for extracting useful bug reports from your users? Let’s hear them in the comments.
Read more of Phil Johnson's #Tech blog and follow the latest IT news at ITworld. Follow Phil on Twitter at @itwphiljohnson. For the latest IT news, analysis and how-tos, follow ITworld on Twitter and Facebook.