The changing role of BI professionals: from analysts to scientists

The following is the first in a series of four blog posts on the evolving roles, skills and functions played by business intelligence and data professionals.

It’s been estimated that 65 percent of current primary school students will eventually hold jobs that currently don’t exist.

That’s a lot of new jobs. But also consider this: according to the Bureau of Labor Statistics, the U.S. is expected to create around 11.5M jobs during the 2016-2026 decade — more jobs than the previous decade, and something that contradicts the argument that the robots will take all our jobs.

Much of this job growth relates to new types of employment with foundations in data. The World Economic Forum says that cloud technology and the mobile internet will be the Number 1 technological driver of change across all industries, while Number 2 will be processing power and big data. New energy supplies and technologies are pegged at Number 3.

Much of the above is predicated on the immense and growing business value companies have placed on data, and by extension, business intelligence (BI) professionals who, in this new reality, have essentially become data professionals.

How data is dominating the job market

A glance at LinkedIn’s recent Emerging Jobs Report shows that among the top 20 emerging jobs of the past few years, data-related positions like machine learning engineer, data scientist, big data and full stack developer and engineer, unity developer and director of data science are at the forefront.

More tellingly, it also indicates that most data scientists are also business analysts.

Data scientist and other data-related roles in the US have grown by 650 percent since 2012, with some of the new, emerging roles including:

Data scientist: A hotly debated role that always seems poorly defined, data scientists are generally those who do statistical analysis, data mining and retrieval on gigantic datasets to solve business problems, find efficiencies and optimize performance.

Data engineer: The designers, builders and managers of big data infrastructure to ensure data is accessible and available for analysis. Data engineers gather, collect and store data, while also running batch or real-time processing to serve it up to data scientists.

Data wrangler: A data transformer who maps data from raw into appropriate formats for actionable purposes, such as analytics.

Data architect: Develops architecture to capture, organize and maintain data, including data warehouse/data platform solutions, ETL and data modeling.

Data analyst: Similar to data scientists, but typically run analyses on smaller amounts of data.

Machine learning engineer: Runs machine learning experiments and deploys machine learning solutions.

It has also been predicted that the rise of artificial intelligence will, while also disrupting more traditional tasks, also be a job creator of new types of positions that don’t yet exist such as trainers (such as empathy and personality trainers), explainers (such as algorithm forensics analysts and transparency analysts) and sustainers (such as AI safety engineers).

What skills are required in the new reality?

Traditional BI roles typically involved running after-the-fact reports for stakeholders using tools like SQL and SSRS. Good problem-solving skills and business acumen were always a prerequisite. But with the explosion of big data, new data types and rising user demands for real-time analytics, the role — and skills required — for BI professionals has become much more complex.

Statistician Michael Hochster says there are two types of data professionals: Type A, who focus on making sense of data through statistical analysis, and Type B, developers of predictive models and algorithms to power data products.

And as the roles have changed, the skill sets have changed along with them. CIO Magazine says the top skills required for data scientists in today’s job market are:

• Critical thinking
• Coding (Python, C++, R, Scala, Clojure, Java, Octave, etc.)
• Math/statistics
• Machine learning/deep learning/AI
• Communication
• Data architecture
• Risk analysis/process improvement/systems engineering
• Problem-solving and business intuition

For the BI professional in the data era, it’s a good bet your next job will require multiple skill sets from multiple disciplines. The most in-demand roles combine a variety of technical skills and data ingenuity with softer skills such as creativity, storytelling and communications. And since good data professionals are so tough to find, and most companies — technology-related or not — are looking to fill these types of roles, most jobs requiring data science or machine learning skills pay in the six figures-plus.