Energy contract stirs conflict

Deal raises questions about evalution process, Common Criteria's value

Common Criteria: less than perfect

The Energy Department's decision to make Common Criteria Level 3 certification a requirement for asset vulnerability management software is consistent with federal policy, but some experts question its real value.

The Committee on National Security Systems updated its National Information Assurance Acquisition Policy to require that as of July 1, 2002, all commercial-off-the-shelf information assurance IT products used on national security systems be evaluated and validated under at least one of three standards:

  • Common Criteria, as established by the International Common Criteria for Information Security Technology Evaluation Mutual Recognition Arrangement

  • The National Information Assurance Partnership Evaluation and Validation Program

  • The National Institute of Standards and Technology Federal Information Processing Standard validation program.

This is not an absolute, however. The number of products that have undergone evaluation is still small, and growing slower than the flood of new products and technologies. So CNSS left an out: Agencies can elect to defer compliance for most products they buy. The vendor has to agree to get its product certified, but can sell it to the agency in the meantime.

An agency is gambling that the product will pass certification, said a member of CNSS, who asked to speak on background because the conversation had not been approved by the agency's public affairs office.

But other issues with Common Criteria might also lead to an agency electing to defer certification, the committee member said, pointing to a recent Government Accountability Office report, which found that the process did not necessarily improve security.

'There's very little confidence in the certifications coming out of some of these labs,' the committee member said.

And there are other drawbacks.

'Once certified software is installed, when you patch it, it's no longer certified' because it's not exactly the same product, the CNSS member said. 'Certification is really not cost-effective from an agency standpoint.'

The GAO report also pointed out that many agencies are buying older versions of software'which are certified'rather than the latest, most advanced versions, which are not yet certified.

This is the case with DOE and its purchase of the Citadel software, in which the department opted for Version 3.5 (certified) over Version 4.1 (in the process of being certified). But the department will have to pay for the upgrade once the certification is obtained.

DEAL? NO DEAL: The department's decision to acquire asset security management software began early last year, when Bruce Brody, then DOE's associate CIO for cybersecurity, approached the Defense Information Systems Agency about the possibility of piggybacking its order on I-Assure. But the arrangement didn't work out.

Ricky Carioti

Problems with large government IT contracts grab headlines every day, but small awards can be just as much trouble. Just ask the Energy Department.

A recent award, to a software supplier whose offer was priced substantially higher than two competitors and rated lower on technical merits, is one example.

On the surface, this is a 'nuisance story,' as one public relations executive described it to GCN. The contract is small, relative to DOE's size. None of the companies involved are household names. The two government executives at DOE involved through most of the process have retired from public service. There's no indication of financial impropriety.

Yet the implied disregard for 'best value' raises questions about how DOE reached its decision and at the very least renews debate about the validity of Common Criteria certification, which the department cites as the deciding factor in its decision.

The controversy arose after Citadel Security Software Inc. of Dallas in March received an initial $5.2 million order to install its Hercules asset security management application on DOE computers, with the upside potential for the order to total $14 million. The order came after months of product evaluations, demonstrations, meetings, e-mails and phone calls between DOE and vendors.

Losers upset

But an erratic chain of events prior to the order left the losing vendors, Secure Elements Inc. of Herndon, Va., and BigFix Inc. of Emeryville, Calif., angry and frustrated, and wondering how the process went awry, especially in an era of the Darlene Druyun procurement scandal at DOD, GSA purchasing irregularities and numerous other headline-making federal procurement issues.

Energy officials defended their decision, saying the award hinged on Citadel's product having Common Criteria evaluation assurance Level 3 certification. While the other competitors were in the process of gaining certification, only Citadel met the department's requirements immediately, they said.

'The Office of the CIO under the leadership of Rose Parkes made a comparison of the three highest rated products in the [market] survey,' Energy press secretary Craig Stevens said in a written response. 'The comparison determined that the only acceptable solution that met the requirements was [Citadel's].'

The pricing disparity between Citadel and its competitors isn't insignificant. An executive with one of the losing bidders said that in the course of responding to DOE's various requests for information, his company submitted a bid of $2.2 million to provide its software to the entire department. An executive with the second unsuccessful vendor declined to give a specific figure but said that his company's bid, too, was 'substantially less.'

Steve Solomon, Citadel's chief executive officer, defended his company's selection by Energy, pointing out that the company holds contracts with other large agencies.

'We were awarded a contract through DOD for worldwide [deployment], for VA worldwide. They are very successful deployments,' he said.

As for the price differences among bids, Solomon was dismissive.

'Do I think our technology is expensive? No, I actually think it's cheap,' he said. 'There are ... other solutions out there that are system management solutions, probably 500 or 600 times our price when we do 80 percent of what they do.'

Technical issues

Price, of course, is just one criterion'the technical quality of the solution is another. And this is another area where the DOE award seems odd to some procurement experts and professionals close to the deal.

The department's decision to acquire asset security management software began early last year, when Bruce Brody, then DOE's associate CIO for cybersecurity, approached the Defense Information Systems Agency about the possibility of piggybacking its order on I-Assure, a seven-year, indefinite delivery/indefinite quantity, multiple-award contract. But the arrangement didn't work out.

Then, in the spring of 2005, DOE received an unsolicited proposal from Secure Elements, one of Citadel's competitors, for the same type of solution. So in June, DOE asked Mitre Corp. of McLean, Va., to conduct a market survey of possible suppliers. The survey requested information on a wide range of technical, pricing, management and operational issues. Of more than 100 questions, one asked whether the companies' products had Common Criteria certification.

Based on those results, the DOE Office of the Associate CIO for Operations invited the top three'Citadel, Secure Elements and BigFix'to submit their products for a technical evaluation in September.

Brody, who was planning to retire from DOE in December, bowed out of the procurement proceedings to avoid any potential conflict of interest.

DOE used 173 criteria to judge the three products. No. 25 on the list asked about Common Criteria certification.

Common Criteria certification is a requirement for applications on classified networks, most of which are in the Defense Department.

It is not currently a blanket requirement for civilian agencies' systems, but it is a requirement for systems designated as 'national security systems.'

That appelation fits about 45 percent of DOE's systems, including the National Nuclear Security Administration, a key component of the department.

The Government Accountability Office recently issued a report critical of the certification process, because'among other problems'it takes so long for companies to obtain, and it does not necessarily guarantee an improvement in security for government agencies.

Other experts also question its usefulness. Alan Paller, a leading security expert with the SANS Institute in Bethesda, Md., called the certification process a paper-pushing exercise that has a lot of support in the international community, because labs in other countries can bring in hard currency from U.S. firms trying to get certified.

Still, Energy required it, and Citadel had its certificate, while the other two companies' products were undergoing evaluation.

According to multiple sources, DOE's evaluators found that all three products met requirements. BigFix's solution was top-rated, Secure Elements' was second, and Citadel's product was third.

New questions

About a month later, a DOE procurement officer sent an e-mail to all three companies asking additional questions about their products'their scalability, including references for customers who could verify deployment for 250,000 or more endpoints; whether they held certificates for Windows Server 2003 and Microsoft Gold partners; copies of their Common Criteria Level 3 certification reports; and documentation showing product support for labeling assets in accordance with Federal Information Processing Standard 199.

The procurement officer provided a two-day window for responses; only Citadel could comply with all four questions.

'This is where a flag popped up for me in this,' said James Kane, president and CEO of the Systems and Software Consortium Inc. of Herndon, Va., a nonprofit organization dedicated to developing tools and methods for improving software and systems development. 'It's a way that I can put questions to companies that create a competitive advantage for the firm that I want to select, or conversely, a competitive disadvantage for the companies I don't want.'

Meanwhile, Energy was apparently willing to overlook the Common Criteria issue when parts of the department were hit by a spyware infestation. The department asked BigFix to help out on an emergency basis; the company 'loaned' DOE a fairly large number of copies of its software for a 'field evaluation,' which allowed the department to get the problem fixed. In January, DOE gave BigFix a purchase order for 10,000 copies to use at the department's headquarters.

The procurement took another odd turn when in March, RS Information Systems Inc., a major DOE IT contractor, entered the picture and issued a subcontract to Citadel for $5.2 million, with a total potential worth of $14 million, for up to 500,000 copies of Hercules V.3.5.

Before the subcontract award, there was no indication that RSIS would play a role in the decision. It had been DOE that approached DISA, that asked Mitre to conduct the market survey, that conducted the technical evaluations, that sent the follow-up questions and that called in BigFix.

'Given that RSIS is a major IT solutions provider for DOE and has used more than 40 subcontractors over the past three years at DOE, it was natural that the department would turn to us to handle the award,' RSIS spokesman Jim Amanna said in a written statement.

Why'd they do that?

The entire chain of events raises a number of questions:
  • If Common Criteria certification was always a requirement, why go through a lengthy evaluation of all three vendors' products?

'We knew all three had put in for ... certification and we didn't want to have a situation where we picked based on [that], announce it on a Tuesday and the certification [is] issued to one of them on Wednesday,' said a DOE official familiar with the process.

Steve Schooner, an associate law professor and co-director of the government procurement law program at George Washington University, said that if Stevens' assertion was correct that Common Criteria was mandatory, the department could have simply ended the whole procurement process and issued a sole-source contract.

'If there's only one firm available [to meet your needs], you can buy sole source,' Schooner said. 'There's a statutory exemption' for it.

n If the certification was a requirement, why did DOE turn to BigFix for help with its spyware problem?

While the infestation might not have attacked DOE's national security systems, Citadel's software should also have been able to address the problem, and DOE would have demonstrated its commitment to Common Criteria and to trying to establish a departmentwide platform for asset vulnerability management.

DOE's Stevens said the decision was made by officials at the associate CIO's office to use BigFix, and that the department is not trying to mandate to all its components which solution to use.

'However, DOE has standardized on the use of [Citadel's] Hercules for the aggregation of asset inventory information from Hercules and other similar products,' Stevens said.

n Why did DOE acquire Hercules Version 3.5 instead of 4.1 (the most current version on the market)?

A DOE official close to the deal said it was because the new version does not have Common Criteria certification and the older one does; the department will upgrade when the certification is obtained, he said.

There are provisions in the requirements for national security systems to allow agencies to designate products with 'deferred certifications' that are in the works. That same deferral also could have been applied to either BigFix's or Secure Elements' products. And the likelihood that DOE will have to pay more to upgrade to Version 4.1 when the certification is complete raises further questions about its higher initial cost.

n If RSIS is a major contractor to DOE, why did the company not conduct the market research and the product evaluations? Why did DOE incur that burden and expense? Or, having done the work, why did DOE not issue a formal request for proposal or a task order, but rather request that RSIS issue a subcontract?

'The department has decided there is a requirement that every entity in the department have asset management, vulnerability management, configuration management, to ensure proper configurations [and] sufficient patching,' said DOE's Stevens.

He said repeatedly that the Common Criteria certification was always a requirement and that Citadel had the only product that met that requirement.

Stevens said that Rose Parkes, at that time CIO at DOE, was trying to make it easy for people to comply with the minimum mandatory requirements for Common Criteria.
Parkes announced in November that she would be retiring at the beginning of January. Last month she declined to discuss the Citadel subcontract with GCN.

Stevens said that because the Citadel product was acquired by RSIS, 'the ultimate decision was actually made by RSIS. However, that decision was made following extensive coordination with DOE staff,' and RSIS had to get the consent of DOE's procurement officer before issuing the subcontract.

RSIS' Amanna, however, said in a written statement that DOE determined that the Common Criteria Level 3 certification was a requirement and that, therefore, only Citadel's product was qualified.

One advantage to a subcontract could be that it can't be protested, said Schooner and other executives knowledgeable about contracting regulations.

'I can't imagine how DOE could think they're going to hold the prime contractor responsible for all of this,' Schooner said.

Whether these responses satisfactorily answer the questions remains to be seen, but the deal is complete. Citadel has the business, and Secure Elements and Big Fix are left to wonder what happened on what should have been a fairly small-ticket, straightforward procurement.

Reader Comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above