Online Extra: 'Bake-offs' help agencies rate data-mining tools

Some agencies evaluating data-mining tools like to use what the Homeland Security Department's Lee Holcomb calls application bake-offs to find the best app for the task at hand.

'You build a database with known facts in it,' the DHS chief technology officer said. 'Then you hit the database with any tool that wants to play.'

The databases might range in size up to a couple of terabytes, Holcomb said. The test database might contain eight items of relevant data for the tools to seek, he said.

Each vendor typically gets a set period of time to analyze the database. At the end of the exercise, an agency evaluates how many of the hidden data gems the tools found and how many false hits were generated.

Bake-offs can help analyze the speed and precision of comparable data-mining programs, Holcomb said.

'Wilson P. Dizard III


  • business meeting (Monkey Business Images/

    Civic tech volunteers help states with legacy systems

    As COVID-19 exposed vulnerabilities in state and local government IT systems, the newly formed U.S. Digital Response stepped in to help. Its successes offer insight into existing barriers and the future of the civic tech movement.

  • data analytics (

    More visible data helps drive DOD decision-making

    CDOs in the Defense Department are opening up their data to take advantage of artificial intelligence and machine learning tools that help surface insights and improve decision-making.

Stay Connected