Credit: Falko Kuester, California Institute for Telecommunications and Information Technology (Calit2), University of California, San Diego

Cracking big data for $3K and geek cred

The Energy Department and the National Science Foundation have announced a series of competitions through the NASA Tournament Lab (NTL) to create applications that could integrate and make use of big data repositories now siloed in individual agencies.

"They are looking for new tools and methodologies to help agencies use this big data," in the areas of health, energy and Earth sciences, said NTL director Jason Crusan.

The first of four anticipated competitions in the Big Data Challenge is conceptual, seeking ideas about how data could be used, and is expected to last about two weeks. Subsequent competitions will drill down into the analysis and software needed to realize the ideas.

Competitors will be asked first to imagine tools and techniques for unlocking the value hidden in discrete information domains, and then describing how they may be shared as universal, cross-agency solutions. The goal eventually is to create mobile applications that can integrate across agency repositories.

The purse for the initial competition will be about $3,000 split at least three ways. But the competitors are not in it just for the money, organizers said. They get to build their reputations and flesh out their resumes, giving them an opportunity to advance their careers. The quality of ideas is not directly correlated to their cost, Crusan said.

"We get unsolicited ideas all the time," he said. "It doesn’t require a high incentive to get high quality ideas."

The NASA Tournament Lab was launched in 2010 as an online platform for contests between independent programmers who compete to create software and solve computational problems and other technology challenges. It is a collaboration with the Harvard Business School and TopCoder, an online company that brings together customers with problems to solve with a virtual community of more than 320,000 programmers around the world who compete to solve these problems for cash prizes.

The Big Data Challenge is being sponsored by the Networking and Information Technology Research and Development program, which hosts the Big Data Steering Group, a multi-agency group co-chaired by NSF and the National Institutes of Health.

The initial competition will draw from Top Coder developers who focus on graphic display of data as well as the analysis of it, said Andy LaMora, who runs TopCoder’s government practice.

"We’ll be running contests that appeal to both sides of the shop," he said. "We’re looking for ideas from several difference sources."

"Big data is characterized not only by the enormous volume or the velocity of its generation but also by the heterogeneity, diversity and complexity of the data," said NSF’s Suzi Iacono, co-chair of the Big Data Senior Steering Group. "There are enormous opportunities to extract knowledge from these large-scale diverse data sets, to provide powerful new approaches to drive discovery and decision-making and to make increasingly accurate predictions."

About the Author

William Jackson is a Maryland-based freelance writer.


  • Records management: Look beyond the NARA mandates

    Pandemic tests electronic records management

    Between the rush enable more virtual collaboration, stalled digitization of archived records and managing records that reside in datasets, records management executives are sorting through new challenges.

  • boy learning at home (Travelpixs/

    Tucson’s community wireless bridges the digital divide

    The city built cell sites at government-owned facilities such as fire departments and libraries that were already connected to Tucson’s existing fiber backbone.

Stay Connected