The IDC recently reported that data is doubling in size every two years, and by 2020, the data we create and copy annually will reach 44 zettabytes, or 44 trillion gigabytes. In the same report, only 5 percent of the information created in 2013 was actually used for analytics.

If all of this data exists, why is so much of it unused? Many organizations think about this challenge in context of how they need to store the data. And others think about how they need to analyze the data and the tools required for those analyses.

Prepping the data

While it’s critical to address those needs, the real pain is identifying the right data and making it worth analyzing. It takes a lot of work to get raw data, clean it, shape and organize it into contextual information the business can use quickly and confidently. Historically, data preparation has been the most challenging aspect of any analytic exercise, with teams struggling to address their needs with inefficient IT-enforced processes or time-consuming, manual work done by siloed business teams or costly outsourced consultants.

Today, information-driven organizations recognize the need for a platform approach that stops the pendulum from swinging between data lock-down and freedom-driven chaos. This platform, which I call the enterprise information fabric, needs to:

  1. Empower the business. Many organizations suffer delays in every analytic exercise due to the back-and-forth work that happens between the developers preparing the data and the analysts who have the business context. Putting the power of intelligent, self-service data preparation into the hands of the people asking the questions eliminates that pain and allows analysts to move through their work in a rapid, agile manner.

  2. Support all analytical use cases. In analytics, a one-size-fits-most approach won’t work because organizations have varied use cases that require different analytical tools. One department made have needs for ad-hoc data visualization and discovery, where another may require deep predictive analytics, and another just needs packaged analytic application, for example to do financial planning. The modern platform must support data preparation regardless of the organizations analytic use cases, tools or teams.

  3. Remove barriers to getting data you need. Today, most analysts “make do” with the data they are given or data they can access without a lot of friction. As long as they are entitled to it, the business should not feel hindered to get the data they need, regardless of whether it is from internal or external sources, regardless of size, whether it structured, semi-structured or unstructured data, or whether it is static or dynamic.

  4. Balance security and governance with freedom. Today’s platform must enable the IT team to carry out their responsibilities to manage data securely while accounting for the business needs for flexibility in how they exploit data for better, more confident decision-making.

  5. Enable teams to work smarter. In addition to governance, the modern platform must drive productivity through transparency. Rather than rigid, top-down business rules being imposed on data teams, the platform captures the work they perform, seamlessly and in real-time. That means both keeping track of how data was used and allowing others to see what work was performed, while giving the business the flexibility and the ability to undo it or re-use it, as needed.

This intelligent enterprise information fabric needs to be built from the ground up for the next generation of big data and BI requirements, with a set of enabling technologies: distributed computing, artificial intelligence, cloud capabilities and dynamic user experience.

Only with this fabric in place can organizations become truly information-driven—where over 80 percent, not 5 percent, of their data is used or contextual analytics that transform their productivity, ability to innovative and succeed in ways they never imagined.