There is a lot of confusion around careers in data; to be convinced, one only needs to look at how much of the mindshare is occupied by buzzwords such as “big data.” Add to that the fact the industry is ever-changing, with roles and tools being constantly redefined, and it can be difficult for outsiders considering entering the field to find their mark. A good example is the “data scientist” title, which has come to designate anything from a technical business analyst to machine learning researchers. Data science encompasses many different roles, each requiring different skill sets.

Bridging the gap

I anticipate the exact roles and qualification requirements to shift significantly over the next few years. A big driver of this evolution is the development of tools that abstract out some of the technical aspects of the work of data practitioners. This evolution produces two contradictory tendencies. On the one hand, better tools flatten technical barriers, and promotes interdisciplinarity. It is now easier for data scientists coming from academia to write production code; easy-to-use libraries make machine learning increasingly accessible; and BI software helps business analysts perform most of their analyses by themselves. On the other hand, as the field matures clear specializations emerge on the more technical end of the spectrum, deep learning being the latest example.

"What kind of problems are you solving? More than just knowing the tools and techniques, knowing how to apply them in a way that creates value is critical."

I think it’s necessary to take a step back and understand why data science is important in a business setting. I see three main ways data is bringing value to the industry, each requiring different skills, from common sense to machine learning Ph.D.s.

Impact of information

The first one is solving old problems with new tools. In the context of business analytics, what big data really means is that companies now have the ability to leverage previously unused information. They now have more data available—they also have better tools to process or interpret it. Analysts can use software like Tableau or GoodData (or, increasingly, run their own queries and analyses) to make sense of data at a glance and better inform business decisions; customer churn prediction is a great example of an old business need that has become more powerful in the data era.

The second is building products that solve new problems. Machine learning in particular enables companies to build products or provide services that were difficult to scale cheaply before by automating human decision processes. Amazon and Netflix come to mind as great examples of companies that pioneered providing personalized recommendations to millions of users.

And finally, the third is building the new tools and inventing new techniques that will be used for the above purposes.

Looking ahead

Depending on which category you fit into, you may work more closely with senior management (providing insights), customers (building products), or engineers (building tools). For those who plan on entering the field, my advice would be to answer this question: what are you bringing to the table? What kind of problems are you solving? More than just knowing the tools and techniques, knowing how to apply them in a way that creates value is critical.