Industry Insight

Why Agile can work for complex systems like

Agile software development has taken some hits in the recent discussions of’s failures. But implications that, because agile development discourages “big design up front,” it is inappropriate for complex systems paint an incorrect picture.

These misconceptions about agile methodology — that it‘s unstructured, lacks  documentation or only works for small projects — couldn’t be further from the truth. Some people have a “developers gone wild” image in their minds, despite the fact that research has clearly shown that the success of agile projects is three times that of waterfall projects. 

Other views

Lesson from A launch is no time for a beta test

The past month of problems and de-bugging on the Affordable Care Act portal has not been so bad, if you think of it as a beta. Read more.

Media got it wrong: failed despite agile practices

Developer documentation shows agile development practices were used in building the site. Here's where that could have hurt. Read more.

So how would a complex project be tackled in an agile manner?

Complex projects must overcome significant external factors and challenging functionality. Government systems, such as, face complexity on a large scale: new laws, politics, multi-agency and multi-organizational dependencies, capability on a massive scale, new teams. The building of a new system must be the product of successfully navigating all these issues.

That navigation cannot be accomplished, in total, by upfront requirements definition and system design.  The complexities noted above require a method that also allows a progressive discovery of project scope and dependencies, along with identification and mitigation of key risks.  While some challenges can be solved in advance, many require the actual problems to arise before solutions can be discovered, circumvented or negotiated with external parties.

For large systems — ones which require multiyear efforts — an agile approach must address all of these significant complexities and dependencies. Yet, success is not solely based on process. Team members must have skills in agile methods, demonstrated by successful track records delivering working systems.

Those skilled agile teams often use the Unified Process (UP). Agile and UP are, in many ways, the perfect marriage — both are based on highly iterative problem solving and software delivery. While agile practices focus on adapting to change and new information, UP sets up a four-phase approach that organizes agile activities for large-scale development. While agile practices foster problem solving, collaboration and adaptation on a small and iterative scale, the four phases of UP help navigate the complexity of a large-scale project.

1. Inception: The depth and breadth of the project are established — the business case, potential solution approaches, key risks and identification of a core team.

2. Elaboration: A core team is staffed, significant requirements are characterized and the technical approach and key risks are determined by building initial software to validate key assumptions and risks. A development plan sets up the next phase.

3. Construction: The significant portion of the system is iteratively built, tested and delivered. Software is fielded, typically with overlap to the transition phase.

4. Transition: Software is delivered to the target user base, again iteratively, validating functionality and solving issues that can only be resolved through actual use.

By focusing on solving actual problems through iteratively building and delivering software, skilled agile teams can surface, negotiate and resolve complex challenges and external dependencies. The ultimate measure of progress is working software. Fielding the most challenging aspects of a system early and performing break-fix iterations along the way are keys to success. In this regard, agile development would not have contributed to the launch of an unproven system – rather it would have surfaced those key issues far sooner in the creation of the system.

Reader Comments

Thu, Jan 22, 2015

The author is clearly focused on addressing agile as a method for attacking complex problems. He simply states that UP provides a framework for tackling more complex systems. He doesn't say it is easy, or it solves or problems, or how to address security, or any other specific issue. Obviously no brief article with one core point would cover that. In this regard, the notion of "iteration 0" and get coding is clearly debunked - there are well known ways to engage a complex system with agile. And the Unified Process provides that. This does not address the need for skilled people, how to tackle congressional mandates, et cetera.

Thu, Feb 27, 2014

Does Agile include QA? If so the title needs a fix....

Mon, Nov 4, 2013

The problems faced by ACA because of this thinking:"Software is delivered to the target user base, again iteratively, validating functionality and solving issues that can only be resolved through actual use." user base will use the software they are not part of system development/testing process.

Mon, Nov 4, 2013

The writer who suggested the 10 steps must believe the a well defined process, followed step by step, is the solution. In my opinion, this writer has never been in charge of delivering on time and on budget. The recommendations appear to be from a consultant who studies, records, reports and recommends - then let's someone else worry about delivery. I am astounded at the number of articles written about what went wrong, all written by people who have no knowledge of the details. The article about agile and UP being good approaches is very sound, as has been documented in numerous 'studies' on the success of this approach vs. waterfall. However, to say that approach would have worked for ACA is speculative - albeit, likely to be accurate.

Mon, Nov 4, 2013

In response to the first posting, I believe the person making the comment failed to understand a basic point. Software doesn't have to be delivered (exposed to the public) to do iterative development and testing. With that assumption the article's author is dead on. Iterative is the way to go...

Show All Comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above