Companies encountering a data engineering talent vacuum

This post originally appears on our sister publication, CIO Dive. Our mission is to provide busy professionals like you with a bird’s-eye-view of the Information Technology industry in 60 seconds. To subscribe to our daily newsletter click here.

Contents

Dive Brief:Dive Insight:

Dive Brief:

Demand for data engineers is growing faster than demand for data scientists, according to research from Stitch.
From 2013 to 2015, the number of data engineers grew 122%, while the number of data scientists grew 47% in the same time frame, according to the report. But though the pipeline for data engineers is growing, demand is far outpacing the number of data engineers on the market. On LinkedIn there are only 6,500 people who call themselves data engineers. But, according to the report, there are at least 6,600 job listings for data engineers in San Francisco alone.
More than 40% of data engineers come from a software engineering background and the top five skills include SQL, Java, Python, Hadoop and Linux.

Dive Insight:

The labor market for talented tech personnel is growing increasingly competitive, especially in places with a large concentration of technology companies. In January, Robert Half Technology released its 2016 Salary Guide for Technology Professionals and found there is still more demand than supply for IT professionals, creating a competitive recruiting environment.

But organizations have increased demand for specialized roles, especially those involving data, which can help companies better understand their customer and how their products work on the market. To understand that data, companies have to have clean pipelines that can facilitate analysis. That’s where data engineers step in.

Data engineering skill sets can vary depending on company size, according to the report, but now data engineers are tasked with working on large data architecture problems rather than working with business intelligence systems.