Speeding Up Machine Learning Development with MLflow Hien Luu - - PowerPoint PPT Presentation

▶
speeding up machine learning development with mlflow
SMART_READER_LITE
LIVE PREVIEW

Speeding Up Machine Learning Development with MLflow Hien Luu - - PowerPoint PPT Presentation

May 18, 2023 353 likes •838 views

Speeding Up Machine Learning Development with MLflow Hien Luu Agenda Unique Challenges in ML Development Process Machine Learning Platform Tour Introduction to MLflow Demo ML Development Process Overview Hidden Technical Debt in

slide-1
SLIDE 1

Speeding Up Machine Learning Development with MLflow

Hien Luu

slide-2
SLIDE 2

Agenda

  • Unique Challenges in ML Development Process
  • Machine Learning Platform Tour
  • Introduction to MLflow
  • Demo
slide-3
SLIDE 3

ML Development Process Overview

Hidden Technical Debt in Machine Learning Systems (paper from Google)

The required surrounding infrastructure is vast & complex

slide-4
SLIDE 4

ML Development Process Overview

Software 2.0

Data Driven Model Driven

New dawn of a new age of software

slide-5
SLIDE 5

ML Development Process Overview

Software Model

Meet function requirements Optimize for business metric Depend on code Depend on data, code, algorithm, parameters Standardized stack Many tools and libraties Fairly deterministic Changing with data

Goal Quality Tool Outcome

Software vs Model Development

slide-6
SLIDE 6

ML Development Process Overview

Experimentation Environments Big Data

Machine Learning Development Dimensions

slide-7
SLIDE 7

ML Development Process Overview

Experimentation Dimension

Model Performance

Data Algorithm Parameters Machine Learning develooment is a scientific endeavor

slide-8
SLIDE 8

ML Development Process Overview

Multiple Environments Dimension

Data Features Training Evaluation Prediction Offline Online

slide-9
SLIDE 9

ML Development Process Overview

Data Volume Dimension

Machine Learning Pipeline & ML Training Infrastructure

slide-10
SLIDE 10

Machine Learning Platform Tour

People Process Technology

Other Challenges Moving To Model Driven

Scientific exploration and engineering rigorous and automation

slide-11
SLIDE 11

ML Development Process Overview

https://medium.com/netflix-techblog/open-sourcing-metaflow-a-human-centric-framework-for-data-science-fa72e04a5d9

ML Platform Data Scientist

Data Lake & Feature Store ML Pipeline & Workflow Management Compute Resources Model Training, Deployment & Management Feature Engineering Model Development & Experimentation Create Business Values

Separation of Concerns

slide-12
SLIDE 12

ML Development Process Overview ML development vs traditional software development

slide-13
SLIDE 13

Machine Learning Platform Tour

Michelangelo

Google TFX FBLearner

AI Backbone ML as-a-service Deploying production ML pipelines ML as-a-service

In-house Machine Learning Platforms

Automation, productivity and fast iteration

slide-14
SLIDE 14

Machine Learning Platform Tour

Feature Infrastructure Model Training Model Serving Model Management Model Monitoring

Minimize incidental complexity in Machine Learning to increase efficiency

Anatomy of Machine Learning Platform

slide-15
SLIDE 15

Machine Learning Platform Tour

Productivity, reusability, scalability, and ease-of-use

Facebook - FBLearner

slide-16
SLIDE 16

Machine Learning Platform Tour

Uber - Michelangelo

ML as software engineering Democratize & scale AI to make it as easy as requesting a ride

Iterative, tested, and methodical

slide-17
SLIDE 17

Machine Learning Platform Tour

LinkedIn - Pro-ML

Use Cases

  • Feeds
  • Search
  • Recommendation
  • Advertisement
  • Fraud

To double effectiveness

  • f ML engineers

https://towardsdatascience.com/introducing-pro-ml-68f37574e1f4

slide-18
SLIDE 18

Machine Learning Platform Tour

Major pain points associated with a machine learning project dramatically change as the scale of the project increases.

AWS SageMaker Azure ML Google Cloud Machine Learning Engine

Cloud Based Machine Learning Platforms

Machine Learning as a Service - MLaaS

slide-19
SLIDE 19

Machine Learning Platform Tour

Vision Speech Language

Pre-trained Machine Learning Models

Text within image, facial expressions Chat bots, disease predictions, fake news Translation, language detection

slide-20
SLIDE 20

Machine Learning Platform Tour

Cloud MLaaS

AI Services

(Computer vision, object recognition, NLP)

ML Services

(ML IDE, experimentation, model training, management & monitoring)

ML Frameworks & Compute Infrastructure

(Tensorflow, Pytorch, Caffe, GPUs, Kubernetes, prediction infrastructure)

slide-21
SLIDE 21

MLflow – Open ML Platform

Manage the ML lifecycle, including experimentation, reproducibility and deployment

  • Open
  • Ease of use
  • Extensible
  • Scalable

Principles

slide-22
SLIDE 22

MLflow – Open ML Platform

Tracking

Record and query experiments (code, configs, data,result)

Project

Packaging format for reproducible runs on various platforms

Model

Model format that standardize deployment

Model Registry

Model lifecycle management

slide-23
SLIDE 23

MLflow – Tracking

Tracking

Record and query experiments: code, configs, results, …etc

Track and analyze experiments

slide-24
SLIDE 24

MLflow – Tracking

Tracking Experiments

“ML experimentation is like the wild

  • west. Ad-hoc tools and processes

because of a lack of standardized

  • tooling. Forget reproducibility, it is

difficult to track experiments and results.”

https://towardsdatascience.com/tracking-ml-experiments-using-mlflow-7910197091bb

slide-25
SLIDE 25

MLflow – Tracking

UI API Tracking APIs (REST, Python, Java, R)

Tracking Server

  • K-V parameters
  • Metrics
  • Artifacts
  • Source code
  • Version (git)
  • Tags & notes

Notebooks Applications Cloud Jobs

Key Concepts Metadata Artifacts

slide-26
SLIDE 26

MLflow – Tracking

Python, R, Java, REST

mlflow.keras.autolog() Recently added:

slide-27
SLIDE 27

MLflow – Open ML Platform

  • UI Demo
  • Show MLFlow parameter and metric logging
  • Visualize the experiments
  • mlflow ui
  • http://127.0.0.1:7000/#/
slide-28
SLIDE 28

MLflow – Projects

Projects

Packaging format for reproducible runs on any platform

Reproducibility via self-contained ML project specification

slide-29
SLIDE 29

MLflow – Open ML Platform

Reproducibility, Sharing, Productionalization Project Spec

Code Data Config

Local Execution Remote Execution

Dependencies

slide-30
SLIDE 30

MLflow – Projects

conda_env: conda.yaml entry_points: main: parameters: training_data: path lambda: {type: float, default: 0.1} command: python main.py {training_data} {lambda} my_project/ ├── MLproject │ │ │ │ │ ├── conda.yaml ├── main.py └── model.py ...

$ mlflow run <directory> or git://<my_project> mlflow.run(“<directory> or git://<my_project>”)

slide-31
SLIDE 31

MLflow – Projects

  • Demo
  • mlflow run <local directory>
  • mlflow run <github>
slide-32
SLIDE 32

MLflow – Models

Models

General model format that supports diverse deployment tools

Simplify model deployment

slide-33
SLIDE 33

MLflow – Models

Model Format

Flavor 2 Flavor 1

ML Frameworks

Inference Code Batch & Stream Scoring

Serving Tools

Standard for ML models

slide-34
SLIDE 34

MLflow – Models

run_id: <uuid> time_created: 2019-06-20T08:11 flavors: flavors: tensorflow tensorflow: saved_model_dir: estimator signature_def_key: predict python_function python_function: loader_module: mlflow.tensorflow my_model/ ├── MLmodel │ │ │ │ │ └ estimator/ ├─ saved_model.pb └─ variables/ ...

mlflow mlflow.tensorflow.log_model .tensorflow.log_model(...) (...)

slide-35
SLIDE 35

MLflow – Models

  • Python Function
  • R,H2O
  • Keras
  • Mleap
  • PyTorch
  • Skiki-learn
  • Spark Mllib
  • TensorFlow
  • ONNX

Built-In Flavors

predict = mlflow.pyfunc.load_pyfunc(…) predict(input_dataframe)

Model Flavors

slide-36
SLIDE 36

MLflow – Model Registry

Model Registry

Model lifecycle management

Collaboratively manage the full lifecycle of a model

slide-37
SLIDE 37

MLflow – Model Registry

  • Model administration and review
  • Sharing, versioning, approval workflow
  • Integrate with MLflow tracking
  • Centralized activity logs and

comments

  • Integration with CI/CD

Managing Models Collaboratively

slide-38
SLIDE 38

MLflow

  • Demo
  • Deploy model to a REST endpoint
  • mlflow models serve
  • Simple application to call REST endpoint to perform prediction
  • Flask application
slide-39
SLIDE 39

Summary

  • Interesting & unique challenges with ML development process
  • In-house & cloud-based ML infrastructures
  • MLflow – open source ML platform
  • Tracking
  • Project
  • Model
  • Model registry
slide-40
SLIDE 40

Q&A

slide-41
SLIDE 41

Machine Learning Platform Tour

Google - TFX

slide-42
SLIDE 42

ML Development Process Overview

Software Engineer Data Engineer Data Scientist People Technologies Environments

slide-43
SLIDE 43

Machine Learning Platform Tour

AWS AI Platform

slide-44
SLIDE 44

Machine Learning Platform Tour

Google AI Platform

slide-45
SLIDE 45

Machine Learning Platform Tour

Azure AI Platform

slide-46
SLIDE 46

MLflow – Models

  • Packaging format for ML Model
  • Directory structure
  • Different flavors
  • (keras, pytorch, sklearn, spark, tensorflow)
  • Define dependencies
  • Easy reproducibility & deployment
  • Model creation utilities
  • To save model in MLFlow format
  • Deployment APIs
  • CLI – Python,R,Java
slide-47
SLIDE 47

MLflow – Projects

  • Packaging format for reproducible for ML runs
  • Training code folder or Github repository
  • Project configuration file in YMAL format
  • Define dependencies for reproducibility
  • Using Conda to specifying and managing dependencies
  • Execution API for running projects
  • Local or remote execution
  • CLI – Python,R,Java
  • mlflow run