Companies, agencies, still struggling with big data's learning curve

Most organizations need to address a range of issues before launching big data projects, starting with getting a handle on what big data actually is, according to a report released by the Computing Technology Industry Association.

CompTIA’s Big Data Insights and Opportunities study finds low levels of familiarity with big data among IT and business executives surveyed. Only 37 percent of them  report being very familiar with the concept of big data.

Approximately one in five businesses claim to have a big data initiative underway, while 36 percent plan to start one in the next 12 months, according to CompTIA officials, who conducted online surveys of 500 IT and business executives and 435 executives in U.S. IT firms in July 2012.

Related coverage:

Big data system ready to handle fed intell, surveillance sensor data 

NASCIO: Big data is a big deal

Big data is defined as a volume, velocity, and variety of data that exceeds an organization’s storage or compute capacity for accurate and timely decision making.

“As expected for an emerging technology with an evolving definition, many executives are still moving along the big data learning curve,” said Tim Herbert, vice president of research with CompTIA, a non-profit association for the IT industry. 

Although government IT executives accounted for less than 5 percent of those surveyed by CompTIA, other studies indicate that government executives are struggling with some of the same issues related to using big data for real-time decision-making as their private-sector counterparts.

While agencies are looking to leverage big data, most lack the data storage, computational power and personnel they need to take advantage of opportunity, according to a study released in May 2012 by MeriTalk, which was sponsored by NetApp. The report, “The Big Data Gap,” revealed that federal IT professionals believe big data can improve government but that the its real potential is locked away in unused or inaccessible data.

Federal IT professionals agree there are many benefits to big data, but the technology and applications needed to successfully leverage big data are still emerging, the MeriTalk report stated. Sixty percent of civilian agencies and 42 percent of Defense Department and intelligence agencies said they are just now learning about big data and how it can work for their agency.

Accounts of large retailers, government agencies, health care providers, utility companies and other organizations doing impressive things with big data offer a glimpse of the potential of this trend. But the reality is that most organizations have far more basic data-related needs, said CompTIA’s Herbert.

“Not every business will need a big data strategy,” Herbert said. “But just about every business will need to effectively aggregate, store, manage and analyze the data they do have, regardless of its volume, velocity or variety,” he said.

Slightly more than one-third of respondents surveyed by CompTIA say they are exactly or very close to where they want to be in managing and using data. In relatively few areas do businesses report proficiency – just 20 percent of respondents say they are currently doing well at analyzing Web traffic patterns; 15 percent at measuring e-mail marketing campaign effectiveness; and 12 percent at social media monitoring.

“Basic work needs to be done before many companies are ready for a big data initiative,” Herbert noted. “Many companies are still struggling with analytics, storage, backup and business continuity.”

The CompTIA study reveals a number of other data-related challenges:

• Nearly three in four companies report a high or moderately high degree of data silos within their organization, making it difficult to view data holistically.

• Many do not have a complete or accurate understanding of their data profile, especially as it relates to unstructured data, such as audio and video files and social streams of data.

• Just one in three companies have a comprehensive business continuity and disaster recovery plan in place.

As companies work to improve data utilization and move closer to realizing the possibilities of big data, they also will have to contend with the possibility of gaps in workforce skills, the report states.

Herbert noted that big data initiatives often require bringing together technical, analytical, statistical and creative thinking skills. Unfortunately, these skill sets have been somewhat compartmentalized, a situation many companies will need to address, he said.

Because of the complexities with many aspects of big data, many businesses will also turn to outside experts for assistance in meeting their objectives. Over the next two years, 28 percent of respondents indicated they plan to contract with an IT solution provider or vendor for services ranging from cloud storage integration, data center management or predictive analytics application deployment, according to the report.

The complete report is available at no cost to CompTIA members who can access the file at www.CompTIA.org or by contacting research@comptia.org.