In 2009, four rhesus monkeys sat down in front of a computer. Two coloured boxes soon flashed up on the screen. One gave the monkeys specific information about an upcoming reward – an image of water of differing volumes. The other displayed information about the prize, but did not detail its size. After a few days, the animals showed an overwhelming preference for the box that revealed the quantity, even when they didn’t receive a gift.
The supervising neuroscientists at the National Eye Institute, in Maryland, US, reasoned that not only did the monkeys prefer immediate information, but accessing it was a form of reward. Records of their brainwaves showed their levels of dopamine – a chemical neurotransmitter that controls the brain’s pleasure centres – shot up.
And what is true for rhesus monkeys is often true for humans. “Human beings are designed to be ‘infovores’. It’s a craving that begins with simple preference for certain types of stimuli, then proceeds to more sophisticated levels of perception and cognition,” wrote neuroscientists Irving Biederman and Edward Vessel in American Scientist magazine in 2006.
Today, we are living in the golden age of information, where people can access as much dopamine-stimulating data as they want – a condition fomented by the rise of a digital world. From the dawn of time until the turn of 2003, human beings created roughly five billion gigabytes of digital information. That’s five exabytes, to give it the proper binary prefix. By 2011, the same amount was created every two days. In the next few years, it will likely increase to every two minutes.
Up until the turn of the 21st century, the growth of data outpaced capabilities of assessing it. But by the 2000s, the digerati began talking about a way of collecting this magnitude of information and turning it into something useful and tangible. They called it ‘big data’.
At first, big data remained within the realms of academia. But by the beginning of the 2010s, it had gravitated towards the world of business, piquing the attention of retailers and marketers who had long harboured an interest in knowing as much as possible about customers. In 2011, a McKinsey Global Institute report suggested that if used properly, big data could increase company profits by as much as 60%.
“Because of big data, managers can measure, and hence know, radically more about their businesses, and directly translate that knowledge into improved decision making and performance,” read a 2012 article by the Harvard Business Review. It gave the example of booksellers. Traditional, physical stores could monitor which books sold well and which didn’t and, if customers had loyalty programmes, track their purchasing history.
Once booksellers moved online “they could track not only what customers bought, but also what else they looked at; how they navigated through the site; how much they were influenced by promotions, reviews, and page layouts; and similarities across individuals and groups. Before long, they developed algorithms to predict what books individual customers would like to read next – algorithms that performed better every time the customer responded to or ignored a recommendation.”
As the article concluded: “Traditional retailers simply couldn’t access this kind of information, let alone act on it in a timely manner. It’s no wonder that Amazon has put so many brick-and-mortar bookstores out of business.”
Other industries also found they could use big data in similar ways, and it wasn’t long before global brands began devoting huge sums of cash and manpower to finding out as much as possible about customers, competitors and their own operations. Marketing firms began compiling vast databases filled with the tiniest fragments of consumer’s lives and habits, which could then be sold to companies or used to target specific audiences in campaigns.
What’s more, an appendage of the big data surge gave birth to new professions to manage it. The Straits Times recently reported that the data analytics sector in Singapore was expected to be worth at least $1 billion in 2016. This pales in comparison to the $125 billion that the global analytics market was worth last year, according to the International Institute of Analytics.
Big data analysis
However, while the benefits of big data are unquestionable and few could imagine companies longing to return to the days before it was established, by late 2013 there appeared to be a growing backlash. In 2014, the Financial Times ran the headline “Big data: are we making a big mistake?”
Commentators began extolling the virtues of ‘right data’ or ‘wide data’ – concepts that would keep the virtues of big data while purging is imperfections. As the Economist put it, “criticisms are not intrinsic to big data per se, but endemic to data analysis”.
For example, in Small Data: The Tiny Clues that Uncover Huge Trends, the best-selling Danish author and branding expert Martin Lindstrom argued that if businesses want to glean real insights about their consumers, they should still pay attention to the small, personal and, at times, harder to comprehend elements of their lives.
After decades of first-hand investigation into consumer behaviour, he believes that the way people store their toothbrushes, for instance, says a lot about their personality. If they stand it in a cup, bristles up, they tend to be less sexually active. Bristles down means people are ignoring trouble, subconsciously burying their heads in the sand. Lindstrom concludes that big data and small data should be “partners in a dance”.
And while big data poses revolutionary possibilities, some suggest that companies should not forget traditional ways of doing business, nor overlook the importance of human interactions.
In February, Charles Duhigg, a Pulitzer Prize-winning reporter, described the history of Project Aristotle, an initiative established by Google in 2012, in the New York Times. A group of statisticians, psychologists, sociologists and engineers set out to discover the answer to a fundamental question: why do some teams work well together and why do some not? They began by amassing data from academic studies on teamwork over the past half-century. They also interviewed thousands of employees and gathered records, quarterly reviews and meeting minutes from more than 150 teams.
By late 2014, few conclusions had been drawn. However, after observing a team gathering away from Google’s headquarters, where a team leader initiated a group discussion, revealing that he had inoperable Stage IV cancer, they realised that what mattered most to a team’s development was human interaction. “Google’s intense data collection and number crunching… led it to the same conclusions that good managers have always known. In the best teams, members listen to one another and show sensitivity to feelings and needs,” Duhigg wrote.
Another consideration lies in information bias – a natural foible shared by all people where we look for more and more information to make a decision, even if that information is irrelevant.
In a famous 1998 study, university participants were divided into two groups and told that a course they were very interested in was on offer. The first group was informed that an excellent professor normally taught the course, but he would be on leave. Instead, they would be taught by a less popular teacher. 82% of people decided to register for the course.
The second group was told the same information, except they were given an option of waiting for confirmation of who would teach it. 56% chose to defer their decision. When they were later told that it would be the unpopular teacher – the same information that the first group had known all along – they were split roughly 50-50 between accepting and turning it down.
Put simply, it showed that decision-making capabilities can be significantly affected if we are provided with uncertain or contentious information. The danger, warn many critics of big data, is that the demand for greater and greater quantities of information will do just that. More doesn’t always mean better.
The age we now live in allows businesses to access as much data as they want. For many, this information will be used wisely, and be implemented in sound plans that allow strategies to be quantified and assessed. However, like a junkie with an unlimited supply of heroin, the temptation for more and more can be counterproductive. And while few people are hoping for the end of big data, it is just as important to remember its shortcomings as well as its virtues.
“Statisticians have spent the past 200 years figuring out what traps lie in wait when we try to understand the world through data,” read a 2014 Financial Times article. “The data are bigger, faster and cheaper these days – but we must not pretend that the traps have all been made safe.”
