MLOps maturity of an Organization

What is MLOps? MLOps, like DevOps, is a discipline to put ML models into production. It is the journey of an ML algorithm from notebooks to a production environment. However, unlike traditional DevOps, MLOps is more complicated because of the inherently probabilistic nature of Machine Learning. For instance, In traditional Software Engineering, Versioning includes Code and Dependencies. However, with Machine Learning, Versioning increases in complexity. Apart from the aforementioned artifacts, ML workflows need to track Experiments, Models, and Data. MLOps is the set of practices at the intersection of Machine Learning, DevOps and Data Engineering

Apparently, a common problem across teams is that they do not know how to design their Machine Learning/MLOps Journey. However, there is a deeper problem. In any endeavour, a team’s journey should start with a clear set of goals. It’s important to know the destination. Better if they know the pathway and landmarks.

And, teams are typically plagued with this problem as far as MLOps is concerned. They don’t have clarity regarding where they would like to reach. This brings us to MLOps Readiness/Maturity of an Organization.

ML Readiness

There could be many ways in which one can gauge the ML Readiness of a team. It could be on the lines of Business Decision Making maturity where business decision-making could vary from human-driven to data-driven and AI-driven. Read more about this in our article on Evolution of Business Decision Making

However, from a technical standpoint, ML/MLOps readiness could fall into three categories:

1. Tactical Phase
2. Strategic Phase
3. Transformational Phase

Tactical Phase: Manual ML Modeling

In the tactical phase, organizations are exploring the ML Landscape. Typically, it’s a small team with minimal/no potential use cases. Moreover, use cases may not align with business goals. Furthermore, no standard processes and tools are in place.

• Organizations exploring the potential for ML to deliver, with a focus on short-term projects.
• Use cases are more narrow, focusing more on proofs of concept or prototypes.
• A direct link to the business goals may not always be clear.
• Driven primarily by individual contributors or outsourced entirely to partners.
• No tools are in place to automate the various phases of the ML development cycle.
• The little distinction between testing and production environments.

Typical skills/assets:

• Data and Analytics Infrastructure.
• Data Engineering and Analytics
• An understanding of Machine Learning and its potential

Strategic phase: Utilizing pipelines

In the strategic phase, ML efforts align with business requirements. Hence, a dedicated team is in place for ML efforts. Moreover, scalable, secure, and reliable infrastructure is in place. Phases of ML Lifecycle are automated using ML Pipelines.

• ML efforts align with business objectives and priorities.
• ML is a pivotal accelerator for the business.
• A dedicated budget for ML projects that are executed by skilled teams and strategic partners.
• Infrastructure is in place for these teams to easily share assets and develop ML systems that leverage both ready-to-use and custom models.
• There is a clear distinction between development and production environments.
• The development of ML models occurs as an orchestrated experiment.
• The ML assets and source code for these pipelines is stored in a centralized source repository.
• Automation of ML workflows for training, evaluation, and batch prediction.

Typical skills/assets:

Apart from the above skills/assets in the tactical phase:

• Business Alignment
• Secure/Scalable Data Science Infrastructure
• Model Registry, Feature Stores, etc.
• ML System Design
• ML Engineering

Transformational phase: Fully automated processes

In the transformational phase, ML workflows are fully automated with CI/CD Pipelines. Besides, AI/ML-based decision-making is a norm amongst various products and for analytics teams.

• Actively using ML to stimulate innovation, support agility, and cultivate a culture where experimentation and learning are ongoing.
• Strategic partnerships to innovate, co-create, and augment technical resources within the company.
• Product-specific ML teams are embedded into the broader product teams and supported by the advanced analytics team.
• Incorporated CI/CD practices into each of the various stages of their ML workflow as well.
• These CI/CD best practices focus on reliability, reproducibility, and version control for the code to produce the ML models and the data and the data pipelines and their orchestration.

Typical skills/assets:

Apart from the above skills/assets in the tactical and strategic phase:

• Fully automated MLOps – CI/CD processes for different aspects of ML Workflows:
• Ability to host/manage multiple ML use cases.
• AI/ML-Driven Decision Making culture

Conclusion

These maturity levels aren’t cast in stone, but act as a reference for a team/organization to frame their MLOps journey. This is for information. We do not claim guarantees regarding its accuracy/completeness.

P.S.

Reference book: Machine Learning Design Patterns

Image Credit: Wikipedia