Beat the Clock
Wondering aloud. Remember those Defense Department systems that,
by mistake or false reporting, got listed as year 2000-ready when they were not?
The DOD inspector general uncovered the discrepancy in an audit and recommended changes
to correct weaknesses in DODs standard for certifying systems for readiness. DOD
should independently verify and validate each systems readiness, the IG said.
DODs problem, in some ways, is everybodys problem where year 2000 testing
is concerned. How do you know when you are finished?
Phil Carrai, president of McCabe & Associates Inc., a software engineering company
in Columbia, Md., said the answers he hears to that question are far more subjective than
they should be.
Carrais mission is to bring more objectivity to year 2000 certification
procedures. His company does it through code coverage and analysis techniques.
Rigorous metrics. DOD relies on metrics from code
coverage and code analysis tools to develop and test its missile-tracking systems and
defense logistics applications. The same metrics could apply in testing mission-critical
information systems for year 2000 readiness, Carrai said.
The metrics can be as simple as certifying that you have found all source code dates
impacted by the century change, fixed all the dates and tested them while the system clock
was advanced in some cases and not advanced in others.
A regression test suite is inadequate for certifying a system as year 2000-ready,
because it misses a large percentage of the code changed in a year 2000 project, Carrai
Regression suites typically test less than 20 percent of the date logic in an
application, he said, whereas code coverage and analysis tests produce more objective
metrics for certifying a system as 2000-ready.
I will say, with bias, that the lack of code coverage and code analysis is a big
weakness of most year 2000 certification plans, Carrai said.
Have auditors, will travel. McCabe & Associates
recently took its Visual 2000 code coverage and analysis tool on the road, doing
spot-check audits of mission-critical government systems. McCabe prices its audit services
at $50,000 for analyzing 250,000 lines of Cobol, C/C++, Visual Basic, Fortran, Ada, 370
Assembler, Pascal or Model 204 source code.
An audit typically lasts a month, Carrai said, and ends when the audited agency is
satisfied with the quality and completeness of its year 2000 testing.
Carrai said he sees companies much larger than his getting interested in year 2000
auditing. Computer Associates International Inc., Compuware Corp. of Farmington Hills,
Mich., and IBM Corp. are talking about code coverage and code analysis more than
they ever have before, he said.