How to use agile development to avoid project failures
The now-notorious rollout of the Patient Protection and Affordable Care Act health insurance exchange website, healthcare.gov, brought the issue of software product failure to national attention. While it's one of the more notorious IT software product failures in recent memory, it's far from the only one. These failures happen every day, to businesses in all types of industries around the world.
A Failure of IT and Business to Communicate
What's behind these failures? Tony Orlando, executive vice president of sales and marketing at Fairfax, Virginia-based software and solutions delivery firm 3Pillar Global, cites the disconnect between business unit stakeholders -- sales, marketing, finance, and the like -- and IT.
"One of the reasons we're facing these IT project challenges, these failures of new initiatives, is that what the business side and the IT side want to deliver don't sync," Orlando says.
What you get is a company that can't see the larger picture -- that with the maturation of the digital economy, customers expect a product or a service to deliver an experience. That user experience needs to be a component of and a reflection of your overall business strategy; you can't continue to look at technology or business or end user experiences separately," Orlando says.
It was out of these seemingly contradictory goals that the agile development framework was born, and its principles are even more important in today's business climate (see below).
Click for larger imageInside the Agile Framework: Courtesy of The Agile Alliance
Your business is only as agile as its software. Being able to respond quickly to customer demand while simultaneously meeting business needs requires that software developers have the freedom to build software quickly and efficiently, to exacting end-user standards, and that corporate stakeholders have visibility into the development process, Orlando says.
"Developers want to write good, clean code. The sales and marketing and finance guys want to see the products succeed. Customers demand a good user experience. How can you make sure all these things happen, and that you're not in the headlines when something goes awry?" Orlando says.
You must shift your focus from a project focus to a product focus, and put an emphasis on collaboration and communication between lines-of-business and departments that may seem at odds, Orlando says.
Before agile, says Dan Klaussen, director of product management at 3Pillar Global, explains that most software projects were built following some form of "waterfall" style process. This meant trying to come up with every requirement a product might possibly need to meet before starting to build it.
"While it's great to have well-defined requirements, far too often they [were] written in a kind of vacuum where it's not really possible to anticipate the messiness of the real world," Klaussen says.
"Users don't actually behave the way they say they will, software challenges are sometimes harder than expected, competitors change and shift in unexpected ways, management support ebbs and flows. Agile came about as an answer to the notion that there had to be a way to allow for some flexibility in the software development process," Orlando says. Or, as he explains, "The main idea is to iterate, quickly and repeatedly, to keep up with the market. Get your idea in a minimal form out to the market quickly, and drive customer feedback so you can iterate and adapt that product in real-time to customers' needs," Orlando says.
There are great risks involved in developing a new product in the digital, always-connected, social-media-heavy environment of today, says Orlando. The margin for error is razor-thin, and if something goes awry, a 24-hour news cycle will make sure your business takes a very public whipping. The potential for failure is driving down business's risk tolerance, says 3Pillar Global's Klaussen, and forcing them to consider new ways to make sure products are successful.
"Business owners are not willing to, or even able to, fund a project with a low likelihood of achieving the desired business outcomes," Klaussen says.
"These project sponsors want to be involved in a way that allows them to touch, feel and test as early as possible to validate that their product is heading in the right direction," Klaussen says.
And the movement toward an agile framework has been prompted by developers themselves, says Klaussen. There is nothing worse for a developer than writing thousands of lines of code that nobody will ever use, he says. It's especially depressing for developers who can see that the product they're working on doesn't meet the objectives (business or consumer) while they're writing them, he says.
"But by putting their code in front of real users and stakeholders on a regular basis -- typically every other week -- and getting direct, honest feedback that adds value, well, that is very motivating," Klaussen says. "Now they know exactly how to delight their users."
The agile framework emphasizes flexibility, constant iteration and above all else, collaboration, according to the Agile Alliance, an organization devoted to supporting developers and stakeholders who apply agile development principles.
"Quite simply, the agile process is designed to allow key stakeholders to continually validate that a project is aligned with and meeting their organization's business objectives," says Klaussen. "It's the notion that software you build is never finished and never static," he says.
And the agile framework forces collaboration, says Brian Bargmann, program manager with ESPN. (Bargmann says his views on the subject are his own and do not necessarily represent or reflect those of ESPN.)
"You've got business and IT working together, collaborating, and that's what will make the difference," Bargmann says. "At first, everyone feels like they're 'forced' to work together, but once it becomes apparent that the goal for both teams is continual improvement, that's when you start to see the value," he says.
With an agile methodology, if products are headed in the wrong direction, problems will be caught early, says Klaussen, and the methodology is designed to allow for changes in the overall product roadmap without a lot of waste -- in terms of capital, resources, and time. Those are real benefits that are seen and felt by all stakeholders -- the business side, the developers, and the customers, he says.
"The agile process encourages the team to build the project in a way that can touched, felt, played with, and tested on a very frequent basis. It means building the product in slices based on demonstrable value, like end-user functionality, that often require a little bit of every part of the solution working together," says Klaussen, so developer teams often are integrating front-end, middleware and back-end components frequently to ensure functionality rather than trying to cram all the pieces together at the last minute.
"What this eliminates," says ESPN's Bargmann, "is the situation where your developer teams have estimated, say, a nine-month project window, and then they realize they can't get it all done, or they have to go back and integrate other features, or fix bugs. So they come back to the business stakeholders and go, 'Sorry, it's gonna take an extra couple weeks,'" he says.
With Agile, Klaussen says, "You have the vision, you create the plan. But when reality shows up, when feedback needs to be incorporated, when business objectives change, then following a methodology that accounts for this enables the team to be more resilient and the delivered outcome to drive actual business value," he says.
While in traditional software development frameworks, incorporating feedback into the product roadmap was seen as a failure, within Agile, it's seen as a success, Klaussen says.
"This is a success because the team learned they didn't have it quite right and were able to fix it quickly. Most other methodologies treat this is a failure. It wasn't spec'd right, and then the blame game ensues. But there is a human element in listening, spec'ing and implementing a solution. And the more the team creates a loop that checks that they are hearing and creating meaningful functionality, the faster they learn how to get it right," Klaussen says, and thus, the greater value to the business.
Going Agile Isn't the Whole AnswerOf course, 'going agile' in name alone won't change much, says Bargmann. You have to make at least a partial leap to a true, product- and business-outcome-focused mindset, he says.
"Any development team can say, 'Look, I can give you a quick hack that'll meet a business need,' but what does that really get you if you're not focused on a holistic business outcome," Bargmann says.
"What that translates to is the old developer analogy of the 'squirrel burger.' If your customer says, 'I need a burger. Right now,' and the developers say, 'Well, that's going to take a certain amount of time to raise the meat, grind it, flatten it into a patty, cook it &' but the customer says, 'Just give me anything,' then you end up with a 'squirrel burger.' It's some meat slapped between two slices of bread -- it's 'something,' but it's not what you need. It's not what you asked for, and it certainly doesn't meet the business or the customer needs," Bargmann says.
Going agile for its own sake won't change the business outcome, says Klaussen, if you're not addressing the collaboration and iteration parts of the problem. "It turns out that being successful with a smaller amount of functionality is much better than failing with a grand plan. The extra time it takes to rework the failed vision will take longer, and be a much more painful road for everyone involved," he says.
Sharon Florentine covers IT careers and data center topics for CIO.com. Follow Sharon on Twitter @MyShar0na. Email her at firstname.lastname@example.org Follow everything from CIO.com on Twitter @CIOonline and on Facebook.
Read more about it organization in CIO's IT Organization Drilldown.