I was interviewing a candidate recently for a Machine Learning Engineering role. I asked some questions about Machine Learning Deployment and Drift. In general, the questions were about Machine Learning System Design. “I am not into MLOps,” came the answer. It did not surprise me. Sometimes, when I talk to people about Machine Learning System Design, I get a blank stare, even though there are copious amounts of literature on the same.
Even on our website, we have touched upon various aspects of ML System Design. For instance, in our introductory article on ML System Design, we did touch upon why we need the same, but at a tactical level. In this article, we will expand a bit deeper into the importance of Machine Learning System Design.
Machine Learning System Design is a mindset along with a skillset
You can have a Data Scientist, with zero to minimum engineering knowledge. On the other hand, we can have Platform Engineers/MLOps Engineers who maintain ML infrastructure, with practically no knowledge of Data Science/ML.
Now, take a look at the below image:
We have seen this many times in the aforementioned google paper Hidden Technical Debt in Machine Learning Systems. The small section of ML Code can be written by a Data Scientist. Data Collection and Feature Extraction code can be written by a Data Engineer. Infrastructure or MLOps Engineers can manage the other systems like Serving Infrastructure, resource management, configuration etc.
While all of these could be developed in silos, it is important to connect all of them in a logical and cohesive system. That takes a mindset which can see Machine Learning and Engineering Systems in one place.
For instance, Data Collection and Feature Extraction could happen in Data Engineering Systems. However, Machine Learning workflows have additional needs like Dataset Versioning. Building good features and datasets is an art in itself.
Another aspect that’s different in Machine Learning Systems is logging and monitoring. Along with monitoring system health, it is imperative to monitor issues like data drift or model drift, since ML Systems are data-dependent. This takes a scientific mindset to anticipate and analyze the stochastic nature of data and models.
There are many more concerns like Orchestrated Experimentation, Responsible AI, Validation etc. that need a decent understanding of ML as well as Engineering Design. For example, it takes good engineering knowledge along with Data Science knowledge, to automate experimentation.
Last, but not least, all the above concerns need a mindset that can deal with the Probabilistic Nature of Machine Learning along with the robustness of Engineering systems. It needs the creativity of an artist, the inquisitiveness of a Scientist and the rigour of engineers to design a Machine Learning System effectively.
Conclusion
This is an opinion article, learnt from building and growing a Machine Learning team from scratch. Experiences may vary across organizations. But, this article aims to show the grander picture of Machine Learning System Design. This is only for information purposes. Hence, we do not claim any guarantees regarding its accuracy or completeness.
