Big Data vs. Black Magic

Big data is the new black. Ask anyone. They say that the future of your business, our business or indeed any business lies in our ability to collect, keep and comprehend screeds and screeds of seemingly unrelated data points.

Way back in 2008 Chris Anderson (smart dude and editor of Wired magazine) took it even further. He argued that The Data Deluge Makes the Scientific Method Obsolete. But what does all that mean? What’s in it for marketing? And are we really staring down the barrel of a robot-inspired future inferred by the actions of the past? I’m not so sure.

Big data is just data. Lots of it.

Wikipedia defines Big Data as a broad term for data sets so large or complex that traditional data-processing applications are inadequate. In other words, more information than any normal computer can even begin to deal with. Think of it like the ocean. Vast, deep and interesting with hidden cities and murky monsters lurking beneath the surface. That being the case, most marketing conversations around Big Data are actually only talking about normal data. More like a rock pool. Small, contained, easily explorable. But before we explore that, what’s data anyway?

Data is information collected about stuff we do.

That’s the gist of it. You sign up for a prize draw, you might leave your email address. You go for a walk with a FitBit and someone knows how far you went. Everything you click on the internet drops cookies to a server to track your progress and help someone sell you something you might like. Lots of stuff is being tracked, logged and stored every day. IBMreckons it’s about 2.5 quintillion bytes of data daily. And it’s growing. Fast.

Data infers the future from a story of the past.

So data is just data. And big data is lots of it. Why are people so excited about it? Essentially, the power of data lies in our ability to model trends and infer future behaviour from the actions of the past. It’s crystal ball stuff. People who drink Lion Red probably like beer. Chances are they’ll drink another one… you can see why marketers like the theory.

Magic is something else entirely.

Magic lives in the moments that change things. Newton’s falling apple. Fleming’s ‘accidental’ discovery of penicillin. The bathtub exclamation of Eureka as Archimedes put two and toe together to posit the theory of displacement. Magic can’t be inferred from the past. It’s a moment born of when trillions of insignificant data points collide together to make something new. And that’s creativity.

What's magic got to do with marketing?

Let’s be honest, most of what we do in marketing is about turning the wheels of what we’ve always done. The smart application of data (big or otherwise) is critically important to help drive results.  Specialists become specialised in pushing the needle past yesterday to create new profit for tomorrow. But magic is where marketing changes the status quo.

Data defines yesterday. Magic imagines the future.

Ask anyone, from the data scientists in our biggest corporates to the GCSB to Chris Anderson and his theory about the end of theory – they’ll all tell you that carefully mined data can signpost future intent. And it can. You need some pretty grunty computers and some very clever computer wranglers, but you can use certainly use data as a crystal ball – to a point. Trouble is: any future predicted by data is based on the facts of the past.

Data can’t make magic. Computers can’t be a human. No amount of data analysis can tip the world on its head and say ‘to hell with yesterday, this feels right for tomorrow.’ Only people can do that. And they do it in moments of magic. That’s because computers can only tell you what they’ve been told to tell you. People add the catalyst of how they feel.

The best supercomputer in the world.

Supercomputers are getting better all the time. One day they’ll manage billions of bytes of data and somehow make sense from the noise. But every one of us has a supercomputer like that sitting on our shoulders right now.

That’s why things that seem counter-intuitive can sometimes feel right. It’s how we make connections between the otherwise unconnectable. And it’s precisely why gut-feel is one of the most powerful forces we can harness. Also, in the spirit of self-promotion, it’s why we called our business Hunch.

That’s what I reckon, what do you think?