Reality Check

Blog archive
Leaky bucket in a data center

Does the 'big data' hype hold water?

“Big data” is a marketer’s dream phrase – short, pithy and chock full of enough futuristic implications to make “Minority Report” seem dated. Of course, big data is not a single product or technique. Instead, it’s an umbrella term for a swath of technologies intended to solve that ever-increasing and almost cliché problem of “drowning in data yet starved for information.” But umbrella terms are notorious for being abused because the people  who use them adhere to Humpty Dumpty’s advice: “When I use a word ... it means just what I choose it to mean — neither more nor less.”

Let’s examine three ramifications of the confusion caused by big data hype:

1. Big data becomes a sales slogan. Let the Ballyhoo Boys loose because there is a new game in town!  And you best hop on the bandwagon now before it’s too late ... This is a limited time offer. Just follow the pointy-haired boss in this amazing Dilbert cartoon. Big data is the latest silver bullet, so every snake-oil salesman is out in force. Get in line!

2. Big data becomes a threat. As the “new, new” thing, big data will replace everything you’ve ever done or thought about doing with data. What, you don’t know about Hadoop, Avro, Hive or Mahout? Are you living under a rock? If you are a traditional data management, data integration or data warehousing company you better pack your bags now ...  And it’s not just you, traditional data management vendors, who should be scared. So should you IT customers. What? You don’t know if you even have big data? Maybe you have what we’ll call “dark data.” Is that a physics thing like dark matter? Of course, the implied threat is that if you don’t have big data – maybe you should. That way your executives can’t ask, “Everybody’s doing it ... why aren’t we?”

3. Big data becomes a unicorn. Everyone talks about it, but no one has actually seen one. But given that I am currently working on a big data project, I can assuage your fears and tell you that “Yes, Virginia, big data does exist.” However, I am working on a very straightforward big data project – processing huge volumes of sensor data. That is big data’s No. 1 use case. If you don’t have that use case, you may have to scare up some (creepy music cues up) … dark data.

Of course, the reality should be none of the above. Each point above has both an element of satire and a kernel of truth. The common thread running through them is that over-hyping or twisting an umbrella term breeds confusion. Let’s keep it simple folks – big data is mostly about Handling Huge Volume (let’s call that the HHV principle).  Is your data HUGE?  If your database administrator has not yet said, “You’re gonna need a bigger boat,” then you are fine.

Michael C. Daconta ( is the Vice President of Advanced Technology at InCadence Strategic Solutions and the former Metadata Program Manager for the Homeland Security Department. His new book is entitled, The Great Cloud Migration: Your Roadmap to Cloud Computing, Big Data and Linked Data.

Posted by Michael C. Daconta on Oct 23, 2013 at 12:12 PM


  • FCW Perspectives
    tech process (pkproject/

    Understanding the obstacles to automation

    As RPA moves from buzzword to practical applications, agency leaders say it’s forcing broader discussions about business operations

  • Federal 100 Awards
    Federal 100 logo

    Fed 100 nominations are now open

    Help us identify this year's outstanding individuals in federal IT.

Stay Connected

Sign up for our newsletter.

I agree to this site's Privacy Policy.