GCN LAB IMPRESSIONS

Right on bin Laden, analyst would use tweets to predict future

In a somewhat Orwellian Big Brother announcement, a company called Dataminr says it can predict the future, sort of, using only raw Twitter feeds.

As proof, it’s offering up a press release it sent out to its clients announcing Osama bin Laden’s death four hours before the president announced the news to the world.

Of course we know now that by that time, bin Laden had already been killed, but it remained a high security secret until President Obama announced it. Apparently Dataminr cut through the secrecy and let its clients know the news early.


Related stories:

Don't look now, but everybody (CIA, DHS, etc.) is watching

5 thieves, 5 cities, 12 hours: Can Twitter catch them?


The company said its clients, whose identities it’s keeping secret, are financial services companies that could act on information before it’s widely available, thus turning a potentially tidy profit before the markets, and the rest of us, can react. It also said it has some government clients, which it also refused to name.

Given the constraints that Dataminr was working under previously, it’s impressive that it figured out what happened in the bin Laden take down before anyone else. Apparently the program scans the Twitter universe looking for connecting threads and pieces them together to form its predictive picture.

Again, this means the data already has to be known, or parts of the secret have to be known, by someone before Dataminr can aggregate it. In the case of bin Laden’s death, it apparently pooled 19 tweets from around the world and predicted that bin Laden was dead.

The remarkable thing is that Dataminr had no formal relationship with Twitter, so it was an outsider with no more access to the social networking site than anyone else. Now that is about to change as the company announced a partnership with Twitter that will make the entire Twitter datastream available to the Dataminr program.

Assuming Dataminr can actually process every Twitter feed on the planet, it could improve its predictive capabilities to unheard of levels.

This is fascinating, but also a little scary. Dataminr won’t say who its clients are, and if they are like most companies trying to turn a profit, they probably don’t care. But I wonder if this precision in prediction can rise to the level of insider trading, letting firms with deep pockets and access to the Dataminr feed reap huge profits by acting on information hours before anyone else.

I also wonder how great a benefit it would be to government. On the one hand, government is often in the business of keeping the very secrets that Dataminr seems able to root out. On the other, a variety of agencies, including those in the intelligence community, have taken to monitoring Twitter and other socials media platforms for the same kind of information — early warnings on global trends, possible terrorist activities or popular uprisings. Maybe those agencies are on Dataminr’s secret client list too.

Then again, perhaps Dataminr isn’t as powerful as the company claims. Finding out one government secret, which is essentially what they did, a few hours before an announcement doesn’t make them omnipotent.

(And this kind of thing can go wrong. On the night of Jan. 21, news flew around Twitter that former Penn State football coach Joe Paterno had died, after a student news organization reported it. He did die a short while later, but as the tweets started flowing, he was still alive.)

Still, one wonders that, if they could perform that feat with the trickle of information they could analyze from the outside, what might be possible when connected to the entire Twitter firehose. I just hope it’s not the public that ends up getting wet.

About the Author

John Breeden II is a freelance technology writer for GCN.

Reader Comments

Wed, Apr 11, 2012 SoutheastUS

Still, knowing something before it is generally announced can be valuable to some people in some instances. For example, knowing the general tenor of earnings reports several hours or days prior to the inevitable pre-announcement leakage could be advantageous for investors. If it can be "discovered" from publicly available sources like Twitter feeds, then it is not "insider trading".

Wed, Apr 11, 2012

Lerning that bin Laden was dead WAS NOT predicting anything. Saying that the SEALS will be going after bin Laden four before they did would be more of a prediction.

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above