Report cites FBI's 'weaknesses' on Agile development

Add the FBI to the list of federal government agencies that are reportedly failing to correctly implement Agile software development practices -- a move mandated by the Obama administration.

Last month FierceGovernmentIT reported an independent investigation that concluded there were numerous "high-level concerns" and "weaknesses" in the FBI's plan -- called the Path Forward -- to complete an oft-troubled case management system named Sentinel.

The report, commissioned by the FBI, was conducted by the Software Engineering Institute at Carnegie Mellon University (SEI) and was obtained by FierceGovernmentIT through a Freedom of Information Act request.


Related coverage:

The good, the bad and the ugly of agile programming


The FierceGovernmentIT article was published near the same time that other articles reported problems with the Veterans Administration implementation of Agile methodologies.

The SEI report specifically cited problems with the FBI's implementation of commonly accepted Agile practices. Some of these problems included:

  • Size of teams. While Agile typically calls for small teams of five to 10 members, the FBI development and testing teams consist of about 25 people each, with only one ScrumMaster.
  • Role sharing. While Agile typically calls for all roles to be shared and team members to be in the same location, the FBI's plan includes separate development and testing teams, with members located in different locations.
  • Not using Agile techniques. The report said "we heard comparatively little discussion of such Agile techniques as continuous integration, sprint retrospectives, user involvement, story development, team-based incentives or automated testing."
  • Management. While Agile teams are supposed to be self-managed and self-organized, interviews with team members "seemed to assume that many decisions would be deferred to management." The report also listed one of the "high-level concerns" with the plan was undefined details and that "the program manager will apparently be making most of the decisions about those undefined details, instead of those decision[s] becoming the shared responsibility of the scrum team[s]."

Other concerns with the correct implementation of Agile centered on a non-Agile "fixed time/fixed scope approach" instead of a more flexible schedule, a "separate QA and verification and validation team" and lack of product owner or "user community at large" involvement, among others.

The report also listed several strengths in the plan, such as:

  • Clearly defined overall objectives, including cost and schedule
  • Clearly defined roles and responsibilities and use of a common document repository
  • An intention to use existing assets
  • Co-location of the development team in the Washington FBI facility

However, the report's first listed "high-level concern" was that the plan meant "an enormous change in direction for the FBI." It also noted the FBI's tradition of a rigorous and exacting culture may make it difficult or impossible for team members to "rapidly acquire so different a perspective on their work" as is required in Agile development.

And even though the SEI cited numerous weaknesses that should be addressed and remedied, it said, "We cannot go so far as to predict that, should these weaknesses be remedied, the Path Forward project will succeed."

About the Author

David Ramel is features editor at MSDN Magazine.

Reader Comments

Wed, Mar 23, 2011 Chicago, IL

The previous two comments seem to come from individuals unfamiliar with agile methodologies. While no methodology that I know of can account for the "ineptitude of critical members on the project teams," stating that agile development fails to address "excessive bureaucracy, lack of adequate involvement by key personnel..." is actually an admission that you have no clue what agile development is all about. Stop trying to bash the Obama administration for attempting to modernize governmental agencies and learn the subject matter at hand.

Tue, Feb 8, 2011

is this a failure to develop a quality product? or a failure to follow some kind of prescribed "agile manifesto" created to keep a group of businesses earning rediculous amounts of money while making ZERO contribution to the success of the contract? if it's the latter -- the government shouldn't care. product quality, on-time-delivery, and within budget is ALL that counts. nothing else really matters. let's not add additional costs to fill some consultant's pocket.

Wed, Jan 12, 2011 Michael D. Long Knoxville, TN

The mandate for the use of Agile by the Obama administration is an oxymoronic move, as it is direct contradiction to the mandate to use Firm Fixed-Price contracting "to the maximum extent practical." Agile methodology lends itself to Time & Materials contracts, but also has the potential pitfall (actually a high likelihood in Federal IT projects) for cost overruns due to lack of or inadequate participation by Federal managers who are the business owners knowledgeable of the driving requirements. If the Obama administration is looking to Agile methodology to reduce cost overruns and the high failure rate of Federal software development projects, then it demonstrates immaturity. While looking at the world with a childlike mind may be a blessing to artists and fools with no responsibility, it certainly does not serve the public interest. The taxpayer spends a sizeable fortune with the Carnegie Mellon Software Engineering Institute on the Capability Maturity Model Integration in the effort to reduce project failure rates; this has been ineffective, but has had the effect of increasing overall costs to the government. The problem with Agile, CMMI, and other approaches used by the government in attempts to control costs and improve the success rate for project completions is that these fail to address the root causes - excessive bureaucracy, lack of adequate involvement by key personnel, and ineptitude of critical members on the project teams. BTW, how does the team that put together Recovery.gov stack up on the Agile assessment? It consisted of 10 teams of 20 to 25 members tasked with getting the new site up and running in 10 weeks and cost $18M. From the outside looking in this certainly looks like a case of "Don't do what I do; do what I say!"

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above