'Data' and 'insight' are not the same thing

We’re increasingly discovering that, just as “intelligent” is not the same as “smart,” “data” doesn’t necessarily mean “insight.”

An example of this emerged just this week, when The Intercept revealed that a brief secret report from the British Security Service MI5 in 2010 warned the agency had begun gathering more data than it could possibly analyse.

“This creates a real risk of ‘intelligence failure’ i.e. from the Service being unable to access potentially life-saving intelligence from data that it has already collected,” the report stated.

Surveillance agencies are not the only ones that have found themselves drowning in data, though. Writing in The New York Times last month, economists Alex Peysakhovich — who’s also a data scientist at Facebook — and Seth Stephens-Davidowitz describe how our growing obsession with big data carries the risk of us paying less attention to what they call “small data.”

“If you’re trying to build a self-driving car or detect whether a picture has a cat in it, big data is amazing,” they wrote. “The problem is this: The things we can measure are never exactly what we care about. Just trying to get a single, easy-to-measure number higher and higher (or lower and lower) doesn’t actually help us make the right choice. For this reason, the key question isn’t ‘What did I measure?’ but ‘What did I miss?'”

Another recent article, this one in The Washington Post, described how some technology companies are actually beginning to get rid of data rather than trying to collect, store and sift through it all. Why? Because some firms have started seeing a greater downside to holding too much data… especially if that data is potentially of interest to criminals or government agencies.

Even IBM — which just unveiled its new “Data Science Experience” to help data scientists tease more meaning out of disparate sets of data — says it has come to realise that data science is more than just numbers, bits and bytes: it’s “an act of cutting away the meaningless, and finding humanity in a series of digits.”

So what does this mean for an enterprise seeking to make the best use of its customer and business data? It suggests that any new data-gathering or analytics tools need to be deployed only when there’s a very good reason for it… and for most businesses, the greatest reason for that is better serving and satisfying their customers. If you’re not aiming for that goal, you need to question what data you’re focusing on and why.