Kenneth Cukier is an award-winning journalist at The Economist and co-author of the bestselling book “Big Data.” Here, he discusses the future of big data and cloud computing.
Can you describe the relationship between cloud computing and big data? How do they work/interact with each other?
When all the data is stored in the same place, running big data applications is easier – and cloud computing simplifies that. In the past, companies had many independent systems spread across different geographies. It was outrageously hard to manage. And it was virtually impossible for the data to be combined and used as a corporate asset and strategic lever.
But once the data sits in the cloud, it’s easier to integrate it into new business processes or combine it with other data for analytical insight. Cloud computing is a huge enabler of big data activities in business and scientific research. And big data is giving organizations a new justification to go through the process of shifting IT operations to the cloud.
For readers who are not familiar with your work, why is big data better data?
My work as a business-technology journalist for two decades on three continents has given me a front-row view to see how data is changing the business landscape and international competitiveness. Just a decade ago, companies made critical decisions on annual cycles based on a dollop of data and heaps of hope. Today, the smartest companies have changed their operations around data.
Moreover, the data is near real time, so decision cycles can be compressed: “weekly is the new quarterly.” And with more data, there are more things companies can learn about the market, and new opportunities – and threats. As a keynote speaker at corporate events, I’ve seen how senior managers need to change corporate cultures to get the most from cloud computing and big data. More data gives new insight, just as the microscope let doctors put aside studying the symptoms of disease and get closer to understanding the cause.
What questions should a company be asking itself before investing in big data solutions?
The first question to ask is not “What are we going to do with the technology?” This is because the initial ideas are usually very basic: smarter, more sophisticated uses only emerge after people start experimenting. There will be no shortage of ideas. Instead, the most important question is “Does big data have top management buy-in?” Without that support, big data is a dead duck.
Next, the questions are: What data do we have – and is it in the right form to be used? What other data do we need, and how can we get it? Then the question is: do we have the right people or do we need to fill gaps so we know what kit to buy? Finally, the question becomes: Do we have the right culture to listen to data and change our processes, or do we need to work on that as well?
What is an out-of-the-box example of how an individual, company, government, nonprofit, etc. utilized big data for a successful outcome?
A great example is the big German e-retailer Otto. It found that shoppers were more likely to return items if they arrived after more than three days. So to improve service for customers and protect its margins, it needed to ship goods faster. To do that, it built a predictive model using an advanced algorithm by Blue Yonder that was originally developed for analyzing experiments at CERN, a particle physics lab.
The algorithm analyzed around 200 variables to predict what shoppers would buy a month before the purchases actually happened. Then Otto would buy the goods so it could ship them right after the order. The big-data project was a triple win: it improved delivery times, it reduced returns (and saved Otto millions of dollars a year), and it helped the environment since fewer packages were sent back.
Is there any specific industry in which you’ve seen business analytics have the biggest impact?
The winner last year and this year has got to be marketing, as both an industry and corporate division. For years, marketing decisions have famously been made by HIPPOs, i.e., the “highest-paid person’s opinion.” Yet just as advertising has been transformed by the rise of programmatic online ads to the detriment of campaigns, now marketing is the next in line for a revolution. It’s now hard to find a marketing program that does not have data baked into its design and evaluation.
The good news is that marketing performance can be measured, and finally the industry is putting some science around its practices and less art. The drawback is that the practice of marketing is indeed black magic: a special bit of creativity and excellence will be lost in all decisions needed to pass through a big-data ROI filter. The key will be marrying the data to human ingenuity.