CyberEye

Blog archive
HealthCaregov test checklist

Lesson from HealthCare.gov: A launch is no time for a beta test

So, just how bad was the roll-out of the Affordable Care Act portal, HealthCare.gov?  It depends.

As a production launch, it was really bad. Failures on the site frustrated users and burned up a lot of goodwill that the administration could ill afford to lose.

Other views

Media got it wrong: HealthCare.gov failed despite agile practices

Developer documentation shows agile development practices were used in building the site. Here's where that could have hurt. Read more.

Why Agile can work for complex systems like HeathCare.gov

Development of large, complex systems requires a method that also allows a progressive discovery of project scope and dependencies, along with identification and mitigation of key risks. Read more.

As a beta test, it still was pretty bad; but that’s what beta tests are for. There is no disgrace in finding problems in an application during testing. “This is pretty standard,” said David Lindsay, senior security product manager at Coverity, a development testing company. The failure with HealthCare.gov is that the beta test started Oct. 1 rather than two months earlier.

Any large application project requires testing throughout the planning and development phases. This begins with the individual components as they are developed and continues as they are assembled into working parts. “There is still going to be a need for production testing,” when the entire application is completed, Lindsay said. “A final pass before things go out the door.”

There is a paradox in large project developments, however. The more complex the application, the more testing it needs. But the more complex it is, the later all of the parts come together and the less time there is for final testing. And it is not enough to squirt bits at the software on a laboratory testbed. It should be exposed to real users who can put it through its paces to expose the unintended consequences of development decisions. This is the beta test, the last stop before production release and the last chance to fix problems before you disappoint users and delight antagonists.

HealthCare.gov certainly qualifies as a complex application. It serves as a gateway for 36 state programs and must securely access sensitive information in multiple state and federal databases. The results will determine whether and what kind of health care millions of people have access to, so the stakes are high.

Doing a beta test for a large-scale public Web application is not simple, but it can be quietly and slowly rolled out in a number of locations. Given the current projected fix date of Nov. 30 for the site, this testing phase should have begun no later than Aug. 1 in order to be ready for production by Oct. 1. But the current fix is being done on a crash basis, which is not a good way to develop and fix software. So a July date for beta release would have made better sense.

This would not have been simple to do. All of the states involved and the insurance companies offering coverage would have needed to have their parts ready three months early. That kind of scheduling has to be built into the project from the beginning, not brought up as an afterthought. Synchronizing the release date with the launch date is a recipe for failure, and that is what happened with HealthCare.gov.

There was time to do it right. The Affordable Care Act was passed in early 2010. The task was complicated, however, by opponents of the program who helped to talk many states out of offering their own healthcare exchanges, making the federal portal that much more complex. Those opponents share much of the blame for the botched launch of the site. But that does not excuse those who planned and developed the system from doing it right.

Posted by William Jackson on Nov 01, 2013 at 6:20 AM


Featured

  • Telecommunications
    Stock photo ID: 658810513 By asharkyu

    GSA extends EIS deadline to 2023

    Agencies are getting up to three more years on existing telecom contracts before having to shift to the $50 billion Enterprise Infrastructure Solutions vehicle.

  • Workforce
    Shutterstock image ID: 569172169 By Zenzen

    OMB looks to retrain feds to fill cyber needs

    The federal government is taking steps to fill high-demand, skills-gap positions in tech by retraining employees already working within agencies without a cyber or IT background.

  • Acquisition
    GSA Headquarters (Photo by Rena Schild/Shutterstock)

    GSA to consolidate multiple award schedules

    The General Services Administration plans to consolidate dozens of its buying schedules across product areas including IT and services to reduce duplication.

Stay Connected

Sign up for our newsletter.

I agree to this site's Privacy Policy.