Counting to infinity
- By Kevin McCaney
- Jun 15, 2008
University and industry researchers are embarking on the seemingly Sisyphean task of determining how much information there is in the world.
In an ambitious three-year project, researchers at the University of California at San Diego, Massachusetts Institute of Technology and University of California at Berkeley will work with several information technology companies toward a result that will likely be instantly obsolete, considering the exponential growth of digital information. Nevertheless, the idea of the study is compelling.
For one thing, researchers say quantifying the amount of data isn't as important as trying to understand what we do with it. 'At the end of the day, we are studying how information works,' said James Short of UC San Diego, co leader of the project with Roger Bohn. They'll have plenty of information to study.
Similar past efforts offer a clue about what to expect. In 1997, Michael Lesk of Rutgers University studied information sources in relation to storage sales and estimated a few thousand petabytes in the world.
In 2003, Peter Lyman and Hal Varian, two professors at UC Berkeley, looked at information produced in a single year and came up with 1.5 billion gigabytes worth ' about 1,500 petabytes, or 250M per person ' counting print, film, optical and magnetic formats.
And earlier this year, IDC and EMC estimated what they called the digital universe ' information in digital form, discounting print and other media ' at 281 billion gigabytes, or about 45G per person.
The report said the digital universe is growing by about 60 percent a year and by 2011 will be 10 times the size it was in 2006, equivalent to its own Big Bang.
And as with the physical universe, the expansion shows no signs of stopping. Too much dark matter, as it were, is feeding the expansion.
Kevin McCaney is a former editor of Defense Systems and GCN.