NIST working on new method for finding software bugs
Connecting state and local government leaders
Researchers a the National Institute of Standards and Technology and the University of Texas at Arlington hope to release for beta testing next month a tool to help spot possible problems in complex software.
Researchers a the National Institute of Standards and Technology and the University of Texas at Arlington hope to release for beta testing next month a tool to help spot possible problems in complex software.
FireEye will generate tables of tests to look for adverse reactions that can cause applications to crash. Because crashes can be caused by unexpected interactions between large numbers of configurations, testing possible configurations can be prohibitively costly and time consuming. The project has reduced the number of parameters that need to be tested to a manageable level, and FireEye will calculate which possible combinations need to be tested for an application.
'We have advanced the mathematical computational part as far as it needs to go,' said Raghu Kacker, a mathematical statistician at NIST. 'The bottleneck is how to integrate this tool into a push-button system' for testing specific applications.
NIST plans initially to focus on firewalls and access control tools because scientists there are familiar with those applications. They hope beta testers in government and industry can help with integrating the tool to test a variety of larger systems.
Researchers expect to release the initial production version of FireEye within a few months of its beta release. They intend to eventually make FireEye source code available as open-source software.
The work was spurred by studies begun in the 1990s on why software crashes. A majority of failures in applications tested were caused by the interaction of two or three different settings, or parameters. But some failures were cased by the interaction of as many as six parameters. Interaction of two variables can be tested for in a process called pair-wise testing. But pair-wise testing can be expensive and difficult and does not cover more than two variables at a time.
'We're not saying that is an absolute number,' said Richard Kuhn, computer scientist at NIST. 'But we can safely say two things. Pair-wise testing isn't enough, and a small number of interactions need to be tested.'
The goal of the FireEye program is to advance the technology from pair-wise to multiway testing, in this case, as many as six variables.
Effective, efficient testing for such problems could help improve the security and reliability of software, but NIST computer scientist Paul Black warned that finding bugs is not the same as fixing them. There are many reasons why a known problem might not be fixed. Fixing one bug might create a bigger problem, or there might not be money available in a project budget to make the fix.
'We don't want to oversell it,' Black said. FireEye will be a tool, not a panacea.
NEXT STORY: NSA certifies General Dynamics network encryptor