August 22, 2012, 3:00 PM — The FBI issued a press release at the end of July that hardly sent a ripple. You probably didn't notice--which is ironic, considering the release caps more than a decade of hard work and more than $1 billion in taxpayer dollars spent. The end of a story so fraught with miscommunication, incompetence and failed oversight that the entire debacle would have been a shoo-in for the most flagrant waste of taxpayer dollars since, well, ever--except that the project was turned around, put back on track and is now live thanks to the fact that it was rescued by following an agile software development methodology.
Wait, agile rescued a huge money-pit fiasco of a government project? You mean, iterative, skunkworks, put-the-customer-on-the-team, forget-the-plan agile? You betcha. Agile turned out to be the hero in the tights and cape, coming to save the day.
Not only that, the U.S. Government is serious about Agile. Not only is agile part of Federal CIO Steven VanRoekel's "Future First" initiative, but the Government Accountability Office (GAO) just issued a report on the federal government's use of agile.
The primary lesson? Agile actually works. It's not perfect, and many people in government and in the contractor community still struggle with it, but it's succeeding where it counts--enabling the rollout of large-scale IT projects that are on time, on budget and actually do what stakeholders want them to do. Imagine that.
Case Management System Replaced Decades of Paper Files
The FBI project in question is the Sentinel case management system, which finally moves the law enforcement bureau off J. Edgar Hoover-era paper files. The initial motivation for Sentinel arose in part from 9/11, when the FBI couldn't even email photos of the hijackers securely and had to resort to sending faxes or overnighting CD-ROMs. In response, Congress budgeted $170 million for the Virtual Case File initiative, and the FBI selected SAIC as the primary contractor.
By 2004, the project had died an early death, the victim of mismanaged requirements analysis, abysmally poor design, ineffective development and flawed project management. SAIC's 730,000 lines of code never worked properly, leading the FBI to scrap the entire project. By 2006, the total cost had ballooned to $600 million, and all the FBI had to show for all that money was bupkus.
Enter Sentinel. This time it was Lockheed Martin's turn. The first two phases were budgeted at $306 million, but by August 2010 the project was only half done and $100 million over budget. So the FBI asked MITRE, a federally-funded research organization, to estimate what it would cost to finish Sentinel. The number was $351 million--on top of the $405 million already spent, not to mention the $600 million the FBI had dropped on VCF.
With $100 million here and another $100 million there, pretty soon we're talking real money. Something had to give.
Interview: FBI CIO's Mission Is to Modernize
In December 2008, FBI director Robert Mueller tapped Lehman Brothers veteran Chad Fulgham to oversee Sentinel. By September 2010, Fulgham had wrested Sentinel out of Lockheed Martin's hands to manage it internally as an agile development project. He switched to an iterative Scrum approach that leveraged two-week sprints, a core scrum best practice.
Fulgham's team got the project under control, came in under the revamped $451 million budget and, though it missed its initial September 2011 deadline, credits agile for getting the project done.
GAO: FBI, Other Agencies Show That Agile Works
For developers and other technology professionals well-versed in agile approaches, this stunning turnaround is certainly pleasing, but not surprising. After all, the point of agile is to focus software development projects on positive outcomes.
From the viewpoint of enterprise CIOs, on the other hand, this result should come as an eye-opener. From the executive's perspective, agile has a decidedly mixed reputation as lacking project management and accountability, being more appropriate for small, skunkworks-type projects. Basically, it's seen as little more than a fad that requires onerous organizational change to have any hope of real success.
If it were a solitary example, then the Sentinel success story might be written off as a flash in the pan, an example of exceptional competence on the part of Fulgham and his team. That points to another common criticism of agile: Success, it is said, requires exceptional teams.
The GAO report belies this conclusion. It profiles agile initiatives from the Department of Defense, NASA, the US Patent and Trademark Office, the VA and the IRS. While the GAO's analysis uncovered a number of challenges, they also delineated 32 effective agile practices and approaches--effective in the agile sense that they led to working software that met stakeholder requirements for capabilities, quality, time and budget.
The billion-dollar question: What's their secret? How can an organization as large and bureaucratic as the U.S. government, burdened by a labyrinthine procurement infrastructure and a history of large-scale IT project failures, manage to achieve such notable success with agile?
There are a number of reasons, of course, but the GAO report's first recommendation provides some much-needed insight: "Strive to be more agile, rather than simply following agile methods and steps." In other words, dont just use agile, be agile. Adoption of the philosophy of agile is more important than following the specific steps of agile.