-BY CHANDRASHEKHAR DESHPANDE (MCT, SYNERGETICS INDIA)

We are in the era of cut-throat competition among business houses in making quick, timely decisions with high and reliable precision and offering services to the utmost satisfaction to customers with the help of technologies like Machine Learning, Artificial Intelligence, Chatbots, and so on. Data Science deals with a modern approach to processing a vast volume of data to find unseen patterns, and derive meaningful information to make business predictions and so, is becoming an essential part of ‘Data Driven Decisions’ for business houses today. It uses complex machine learning algorithms to build predictive models with the support of an ecosystem for voluminous storages capable of dealing with high-velocity data, distributed, high speed and scalable computing, deployment environment responding with minimal latency, data flow, and workflow management with tools to secure whole system. And above all, with the added feature of dealing with a variety of data sources/sinks with a variety of data formats. The life cycle of Data Science has been posing challenges for its complexity, demand for entirely different eco-system of tools and technologies, departments/teams working in silos, reusing and recycling of life cycle pipelines, and above all, keeping performance all-time high. Problems have always been a motivation for the corporate fraternity to find solutions and go ahead. The Model Ops/MLOps/AI Ops are innovations in this direction.

Need of ML Ops in the ecosystem–

Productionizing the Data Science life cycle has always been difficult because it consists of many complex steps such as data ingestion, data preparation, model training, model tuning, model deployment, monitoring, explaining ability, and so on. Naturally, it requires stringent operational efforts to synchronize all these processes to make to work them in tandem. The ML Ops approach facilitates Continuous Implementation and Deployment (CI/CD) practices smoothly weaved with monitoring, governance, and validations over the ML Model life cycle. Thereby, it offers great help to Data Scientists, ML Engineers can smoothen the collaboration among them to increase the pace of development, deployment, and production. MLOps encompasses the experimentation, iteration, and continuous improvement of the machine learning lifecycle.

Once in production, the performance of ML model degrades specifically because it received new data with the new change of market trends, business practices, etc on which it is yet to be trained. Soon business realizes a need of retraining the model on new data to keep its performance always up to expectations. It becomes a never-ending cycle of training a model on new data regularly after a specific interval of time.

The MLOps is a machine Learning Operation intended to automate the process of applying life cycle steps of data science, taking machine learning models to production, maintaining and monitoring them all the time, accessing their performance, and when realizing that performance has been downgraded below the threshold, re-cycle them to create a re-trained and improved highly performing model based on recent data changed with trends.

Features and Capabilities:

1. Creating a re-producible ML Pipeline: A pipeline object can stitch together different steps like Data Cleaning, Preparation, Feature Extractions, Hyper Parameter Tuning, Model Evaluation, etc. Once created, an object can be deep cloned, used, or re-used as per need. In Azure ML Designer, the version of a pipeline can be created by copying it.
2. Create Re-usable software environment: Environment describes ‘pip’ and ‘Conda’ dependencies of software requirements for Training, Testing, and Production environments. Tracks and builds software dependencies without manual intervention or hard-coded software configurations.
3. Register Models from Anywhere: A ML Model registration service stores and versions the model automatically on the cloud and provides access all over the globe. It’s a logical container of one or multiple files uniquely identified by a name and the version. A model is accessible for deployment, download, search on tags, and adding a new version model.
4. Package and Deploy Models: The model has packaged in Docker Image automatically while deploying on Kubernetes services. When deployed on Kubernetes services, multiple versions of endpoints can be created which allows Split/Bucket (A-B Testing) to compare the performance of multiple models by routing traffic to multiple endpoints.
5. Support of Open Neural Network Exchange (ONNE): On conversion of the model into ONNE format which offers around double, uniform, and optimal performance on different hardware like CPU/GPU/FPGA and platforms like OnPrem/Cloud/Edge.
6. Capturing Governance Data: Captures metadata to let us …
   1. Track end-to-end audit trail of all ML assets.
   2. Automatically maintains the versions and profile of data sets.
   3. When plugged with Git Hub, allows commit/branch the code snippets
   4. Interoperability allows model explanation and meets regulatory compliances.
   5. Maintains Run History with Snapshots of data, code, and compute used to train the model
   6. The Model Registry captures metadata of registered models.
7. Notify and alert on events in ML Life Cycle by linking Azure ML with Azure Event Grid.
8. Monitor ML Applications for operational and ML issues to understand the change in the performance of the model. Alerts can be raised on performance if goes below the threshold consistently for a significant duration. On alerts being raised, an automated pipeline can be triggered to retrain the model on new data, run evaluation and testing phases, and compare performances of the old and new models to decide whether to deploy a new model in the production environment.
9. Design the CI/CD using GitHub and Azure Pipeline.

Major Steps of Implementation–

1. Ingesting data from various data sources and accumulating in a Blob Storage: The Azure ML Data Store is an abstraction over the connection and secret details to connect to the Data Source/Sink. It identifies the data source/sink by a name and is attached to the Azure ML Workspace. The step exercises Extract Load and Transform to make data ready for consumption in Azure ML Service.
2. Creating Azure ML Environment. If it is Python, an environment comprises Conda Dependencies declared through the configuration file and the Environment Object. The Configuration file declares Python Core Packages, external packages, and libraries like Sci-Kit Learn.

3. Model Training and Registration to Azure ML Registration service. This step includes training one or more models using different algorithms and/or the same algorithm with different sets of hyperparameters. Identify the best-performing one and register its object to the ML Registration Service. The registration service automatically manages and increments versions for the models registered with the same name. All the steps here are to be executed on the compute well equipped with a well-needed environment defined and created in step 2. For pulling the data for training and testing purposes, integration with the Blob Storage is again a need. The Data Store created in Step 1 is handy here to connect to Data Source and ingest the data from.

4. Model Deployment. Once the best-performing model is identified, it’s time to deploy it in Production Environment. There are two types of endpoints created around the model the way interaction is needed i.e. the Web Service endpoint and batch processing endpoint. Also, there are two types of Production Environments Azure Kubernetes Services and Azure Container Instances on which the ML Model can be deployed. For the Production Environment, it’s obvious that the Environment declared in step 2 must be submitted to create a similar environment to Model Training and Testing.

5. Continuous Integration and Deployment. The ML Pipeline is developed by writing code by members of a team, it’s essential to enforce the Unit Tests and Evaluation of Coding Standards. Whenever the user makes a PULL request on Git, the system must be automatically enforcing calls to the tools for Static Code Analysis and running of Unit Tests. The test results and code coverage must be published back to Azure DevOps for users before choosing to merge the code.

Wrap-Up:

ML Ops is a one-stop solution for different and many problems related to Machine Learning Life Cycle. It offers a compelling proposition to Businesses to run their ML Models. ML Ops streamlines the collaboration among teams and members by automating functions, and activities and thus reducing human interventions, and errors.

References:

1. https://docs.microsoft.com/en-us/azure/machine-learning/concept-model-management-and-deployment
2. https://www.researchgate.net/publication/357552787_MLOps_–_Definitions_Tools_and_Challenges
3. https://benalexkeen.com/creating-end-to-end-mlops-pipelines-using-azure-ml-and-azure-pipelines-part-1 to part-7
4. https://neptune.ai/blog/modelops