Data science, machine learning (ML), and AI are no longer just cute buzzwords. Nearly all organizations, companies and governments now recognize the immense potential of AI-enabled products and services, and many of them have already made the very real investment of hiring employees with skills in these emerging fields.
However, as is true with most things in data science, one size does not fit all! Simply hiring a self-proclaimed data scientist or machine learning expert with advanced degrees isn’t likely to fit the bill. A successful machine learning application often requires looking at a business problem critically, crafting a creative technical solution to that problem, designing and executing (many) experiments to see if that solution is possible, and then (if so) developing a scalable way to integrate that solution into an application framework. Expecting a single person to have all of these skills (and also to want to apply them all at the same time) is a bit much.
Of course, the answer is to create a team full of highly competent people with creative, technical and product skills, who are all passionate about AI/ML, right? Easier said than done!
It’s becoming more and more clear that the true competitive advantage in AI/ML isn’t some killer algorithm, or even a cutting-edge technology platform (complete with robust experimentation capabilities and slick visualizations); it’s the ability to build, engage, and retain an awesome team.
Much has been written on the best ways to structure data science organizations (DJ Patil even wrote a book on the subject!) But what are the roles that make up these prolific groups, and more importantly, how should leaders best apply the rare skills that each of these roles bring to the table?
In an article on the subject earlier this summer, Forbes describes three critical roles: data scientists, data engineers, and machine learning engineers. These roles and team structure are great for smaller but agile organizations seeking to uncover AI/ML use cases and then rapidly address them with scrappy solutions.
Hackernoon takes a more detailed approach to describing AI/ML roles in an article describing the Top 10 Roles in AI and data science. Staffing out a team or team(s) with these skills and responsibilities might be more appealing (and realistic) for a more mature department, especially when enterprise software engineering, compliance and/or regulatory requirements, are a major consideration.
Ultimately, there are no cookie-cutter AI/ML team structures, and thus no perfect roles. But as the community of practice grows and enjoys more and more success, best practices have certainly emerged. For example, here at Cloudera Fast Forward Labs, we often work with our customers to help them discover the right AI/ML vision, team structure, core skills, and enabling technology necessary to reach their respective AI/ML goals–and beyond!
More from the Blog
Sep 28 2018
by — Machine learning (and more specifically, deep learning) techniques are recognized for their ability to obtain high accuracy for a variety of classification problems. Deep learning models are more complex than most traditional models and often have thousands of parameters and commensurately require more labeled training data. Obtaining this data can be an expensive and time-consuming process, ...
Oct 29 2018
by — As humans, we can quickly adapt our actions in new situations, be it recognizing objects from a few examples, or learning new skills and applying them in a matter of just a few minutes. But when it comes to deep learning techniques, an understandably large amount of time and data is required. So the challenge is to help our deep models do the same thing we can - to learn and quickly adapt fro...
Dec 6 2018
by — Our prototypes are designed to demonstrate the value of the technologies we research. For our most recent prototype, Turbofan Tycoon, we decided that the best way to demonstrate the value of federated learning was to place you in an interactive simulation where you’re in charge of maintaining four turbofan engines. In this post, I’m going to try and explain a bit about why we decided that, an...