NT is Posix-compliant, GSBCA decides; ruling raises questions
All operating systems certified as Posix-compliant ought to be
equal. But some, as George Orwell might say, are more equal than others.
A recent ruling by the General Services Administration Board of Contract Appeals
affirmed the Posix compliance of Microsoft Corp.'s Windows NT operating system.
Several months ago, the National Institute of Standards and Technology certified NT as
complying with the Posix standard, Federal Information Processing Standard 151-2 for open
operating systems. However, C3 Corp. and Tisoft Inc. protested loudly when Unisys Corp.
bid NT on the Coast Guard's Standard Workstation III procurement and won.
C3 and Tisoft insisted NT doesn't comply with all the mandatory requirements for Posix.
The protesters were overruled. GSBCA said NT "is either compliant or met the
government's requirements for this contract,' said Steve Meltzer, director of GSA's
Federal Computer Acquisition Center, which made the award to Unisys.
The decision has raised questions about the role of Posix certification in federal
procurements, as new operating systems like NT challenge the scope of NIST's test suite.
Posix, a portable interface, defines the interactions between an operating system and its
applications.
It's widely considered to be nearly synonymous with Unix, but now that linkage has been
severed. "When people specify Posix, they need to be very specific,' said Martha
Gray, a computer scientist who administers NIST's Posix testing program.
NT might meet the technical specifications for Posix compliance, she said, but it
doesn't support optional Posix features such as graphical terminal interfaces, modem
controls, mountable file systems and appropriate privileges.
Gray said she's worried that many procurement managers don't understand all these
ramifications when they specify the level of Posix compliance for certain tasks.
In the Coast Guard workstation protest, C3 and Tisoft complained that NT didn't pass
the main Posix tests cleanly. This too is a hazy area.
NT contains what is known as a Posix subsystem. Is that actually part of the operating
system, or is it separate? GSBCA says it's included. NIST doesn't attempt to define what
an OS must include.
"If you run the test suite and pass everything, you're in business,' said Roger
Martin, NIST's chief of systems and software technology. "But you might get what's
called a false negative.'
A false negative, as opposed to an actual failure, occurs when the NIST test suite
doesn't clear a tested system in every area and can't indicate plainly what the problem
is.
One reason: The standard itself might not be measuring the right things for every
system. In that case, NIST can issue what's known as a resolve test code. It's not fair to
say the OS had "a true pass,' Martin said. "I think the correct wording would
be, "This test did not fail.' '
For example, say a system is being tested for black-and-white output. NIST's testers
might report failure if it outputs shades of gray, because the test suite doesn't
recognize grayscale as a form of black and white.
The testers then could pass the product with a resolve test code that competing vendors
could use to challenge the legitimacy of the certification. Such situations are difficult
to settle, because the distinctions are minute.