ML team = Data Scientist, AI Engineer, AI Architect etc
Ops team = Admin, Network engg, security engg etc
MLops = ML + DevOps
is a set of practice that combines ML and DevOps to automate and streamline the deployment, monitoring and management of ML models in production env.
It bridges the gap between the data science experimentation and production env.
SDLC (software Development Life Cycle):
plan –> Define –> Design –> Build –> Test –> Deploy
ML project:
Understanding problem –> Data collection -> data preparation –> Model traning –> Model Evaluation –> Model Deployment –> Monitoring & logging –> restart.
1. Experiment Tracking:
is the process of systematically recording and organizing all the relevant information from ML experiments.
Popular tools: MLflow, Weights & Biases (W&B), Comet, Mepture.ai
Importance to track Experiment tracking: for Reproducibility, for organization, for collaboration, for debugging and insights, for efficiency
| Code and versioning | git commit | train.py |
| Data | DVC | dataset.csv |
| Hyperparameters | controls the training code | |
| Metrics | quantitative measure of model performance | |
| Model Artifacts | saved model along with serialized versions | model.pkl |
| env | Software & hardware | python version, GPU’s |
| logs |
MLFlow: opensource platform for managing the end-to-end ML lifecycle
components: 1.Tracking, 2.projects, 3.Models, 4.Model Registry.
DVC: Data Version Control (Git for data)
version control system for large files, datasets and models.
essential for reproducible ML and MLops
pip install dvc
pip install ‘dvc[s3]’
dvc –version
#create a new project directory
git init
dvc init
git commit -m “first commit”
dvc add /data/raw/dataset.csv
setting local remote
dvc remote add -d myremote /tmp/dvc-storate
for cloud:
dvc remote add -d myremote s3://my-bucket/dvc-storage
Weights & Biases:
MLflow platform for machine learning experiment tracking
run : single execution of training script. It is a fundamental building block of W&B. contains hyperparameter, metrics, output file, system info.
experiments: collection of runs
Projects: collection of experiments (or runs)
Dashboard: web interface for visualization. Realtime metrics tracking. Interactive charts and graphs.
Artifacts: track datasets, models and file etc.
Sweeps: automated parameter search. Multiple strategies: grid, random, bayesian. Finds optimal configuration.
| Kserve | standardization & scalability | |
| seldon | complex pipeline | |
| triton | raw performance | |
| TF serving | tensorflow integration | |
| MLFlow | model packaging |
Best practice for effective model serving:
treat model as microservice
standardize the model format (ONNX, PMML)
implement CI/CD for model
monitor everything, not just infrastructure
Plan for failure
BentoML:
framework for AI application development and deployment
used in model serving – production-ready serving for any ML framework
platform for building, shipping and scaling AI app.
Unified framework for packaging, running and deploying ML models
bento service : main abstraction for model serving
bento deploy
bento monitor
yatai
bento repository
bento registry
Prefect:
workflow orchestration tool used to build, schedule, run and monitor data pipeline and automation workflows.
workflow as flow: define workflow using pure python. Uses flows (containers for workflow logic) and tasks (individual units of work)
Dynamic: workflows can change based on runtime conditions.
Async-first: build on Async/await pattern
developer-friendly: simple python decorators,
Apache Airflow
Rishabh's Space 