Cracking big data for $3K and geek cred
- By William Jackson
- Oct 03, 2012
The Energy Department and the National Science Foundation have announced a series of competitions through the NASA Tournament Lab (NTL) to create applications that could integrate and make use of big data repositories now siloed in individual agencies.
"They are looking for new tools and methodologies to help agencies use this big data," in the areas of health, energy and Earth sciences, said NTL director Jason Crusan.
The first of four anticipated competitions in the Big Data Challenge is conceptual, seeking ideas about how data could be used, and is expected to last about two weeks. Subsequent competitions will drill down into the analysis and software needed to realize the ideas.
Competitors will be asked first to imagine tools and techniques for unlocking the value hidden in discrete information domains, and then describing how they may be shared as universal, cross-agency solutions. The goal eventually is to create mobile applications that can integrate across agency repositories.
The purse for the initial competition will be about $3,000 split at least three ways. But the competitors are not in it just for the money, organizers said. They get to build their reputations and flesh out their resumes, giving them an opportunity to advance their careers. The quality of ideas is not directly correlated to their cost, Crusan said.
"We get unsolicited ideas all the time," he said. "It doesn’t require a high incentive to get high quality ideas."
The NASA Tournament Lab was launched in 2010 as an online platform for contests between independent programmers who compete to create software and solve computational problems and other technology challenges. It is a collaboration with the Harvard Business School and TopCoder, an online company that brings together customers with problems to solve with a virtual community of more than 320,000 programmers around the world who compete to solve these problems for cash prizes.
The Big Data Challenge is being sponsored by the Networking and Information Technology Research and Development program, which hosts the Big Data Steering Group, a multi-agency group co-chaired by NSF and the National Institutes of Health.
The initial competition will draw from Top Coder developers who focus on graphic display of data as well as the analysis of it, said Andy LaMora, who runs TopCoder’s government practice.
"We’ll be running contests that appeal to both sides of the shop," he said. "We’re looking for ideas from several difference sources."
"Big data is characterized not only by the enormous volume or the velocity of its generation but also by the heterogeneity, diversity and complexity of the data," said NSF’s Suzi Iacono, co-chair of the Big Data Senior Steering Group. "There are enormous opportunities to extract knowledge from these large-scale diverse data sets, to provide powerful new approaches to drive discovery and decision-making and to make increasingly accurate predictions."