Online Extra: 'Bake-offs' help agencies rate data-mining tools
Some agencies evaluating data-mining tools like to use what the Homeland Security Department's Lee Holcomb calls application bake-offs to find the best app for the task at hand.
'You build a database with known facts in it,' the DHS chief technology officer said. 'Then you hit the database with any tool that wants to play.'
The databases might range in size up to a couple of terabytes, Holcomb said. The test database might contain eight items of relevant data for the tools to seek, he said.
Each vendor typically gets a set period of time to analyze the database. At the end of the exercise, an agency evaluates how many of the hidden data gems the tools found and how many false hits were generated.
Bake-offs can help analyze the speed and precision of comparable data-mining programs, Holcomb said.
'Wilson P. Dizard III