Keys to big data in the brain, not the computer, former NSA exec says

Data visualization tools are critical to the success of big data analytics. But achieving that success will require more than people with engineering skills; it will require input from cultural anthropologists and social scientists, a former National Security Agency executive told an audience in the Washington, D.C. metro area.

Basically, data analytics comes down to how humans process information, Richard Schaeffer, former information assurance director for NSA and head of the consulting firm Riverbank Associates, said April 26 in a keynote address at SAP National Security Services’ Big Data Day in McLean, Va.

Big data involves datasets that grow so large they become awkward to work with using traditional database management tools. Organizations that handle large volumes of data such as the intelligence community and scientific researchers know how to capture, collect and store large datasets. They are also learning how to more effectively index data, Schaeffer said. 


Related coverage:

Big data's big question: Where are the data scientists?


The problem areas are processing, analytics and visualization. In fact, visualization of big data, rendering it into graphical means for analysis, might be the most significant problem organizations will face in the future, Schaeffer said.

The intelligence community handles lots of unstructured data voice, video, text and that is becoming the norm for many organizations now. And they wonder how to process that type of information, he said.

“We haven’t seen real innovation in visualization” tools to aid in the processing and analysis of information, Schaeffer said. “Visualization and analytics will require cultural anthropologists and social scientists," he said, adding that before the Sept. 11 attacks he had not heard of cultural anthropologists. But afterward, he added, they became an integral part of intelligence analysis.

Schaeffer described an opportunity he had recently to take part in simulation training for flying an F-35 fighter plane. The amount and types of information young pilots have to process while flying an F-35 at Mach 2 speeds are mind-boggling, he said.

They are ingesting, processing and probably mentally storing the information momentarily, while flying an airplane at twice the speed of sound. “That is the type of thinking we have to bring into visualization,” he said.

So studying how human beings process information and make it actionable is an untapped arena, Schaeffer said.

“The next couple of years we will see unimaginable breakthroughs in analytics and visualization tools, enabling real decision-making” and ultimately improving the quality of life of every citizen on the planet, he said. 

About the Author

Rutrell Yasin is is a freelance technology writer for GCN.

Reader Comments

Fri, Jun 15, 2012 Denver

"Not many people can learn to fly an F-35 at Mach 2. Could that mean that not many people would have the mental capacity for processing Big Data?" That questions begs the issue of training up a generation of students capable of handling Big Data. Are our schools set up to handle the challenge?

Fri, Apr 27, 2012 SoutheastUS

Not many people can learn to fly an F-35 at Mach 2. Could that mean that not many people would have the mental capacity for processing Big Data? Admitedly, the physical stress components of processing Big Data are much less stringent than flying the F-35. But the mental gymnastics will still require spatial reasoning to visualize Big Data in a form that would help aid understanding relationships of types and classifications of data taken in context with analysis.

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above