ONE DOES NOT T SIMPLY DEPLOY ML INTO PRODUCTI TION Henrik Brink - - PowerPoint PPT Presentation

▶
one does not t simply deploy ml into producti tion
SMART_READER_LITE
LIVE PREVIEW

ONE DOES NOT T SIMPLY DEPLOY ML INTO PRODUCTI TION Henrik Brink - - PowerPoint PPT Presentation

Jan 16, 2024 199 likes •653 views

ONE DOES NOT T SIMPLY DEPLOY ML INTO PRODUCTI TION Henrik Brink Machine Learning Engineering @ Wise.io / GE Digital brinkar Agenda From space to industrial machine learning Challenges Optimization dimensions Infrastructure

slide-1
SLIDE 1

brinkar

ONE DOES NOT T SIMPLY DEPLOY ML INTO PRODUCTI TION

Henrik Brink

Machine Learning Engineering @ Wise.io / GE Digital

slide-2
SLIDE 2

brinkar

Agenda

  • From space to industrial machine learning
  • Challenges
  • Optimization dimensions
  • Infrastructure
  • Development and deployment
  • Solutions
  • The ML meta-algorithm
  • Containerization
  • Best engineering practices
  • Wrap-up and questions
slide-3
SLIDE 3

brinkar

From astronomy to industrial machine learning...

slide-4
SLIDE 4

brinkar

slide-5
SLIDE 5

brinkar

slide-6
SLIDE 6

brinkar

slide-7
SLIDE 7

brinkar

slide-8
SLIDE 8

brinkar

slide-9
SLIDE 9

brinkar

slide-10
SLIDE 10

brinkar

slide-11
SLIDE 11

brinkar

Everyone ready...?

slide-12
SLIDE 12

brinkar

ONE DOES NOT SIMPLY DEPLOY ML INTO PRODUCTION

slide-13
SLIDE 13

brinkar

Wh What t to opti timi mize for when bui buildi ding ng ML ML systems?

slide-14
SLIDE 14

brinkar

Accu Accuracy? cy?

slide-15
SLIDE 15

brinkar

Implemen Implementatio ion c n cost?

slide-16
SLIDE 16

brinkar

1920 CPUs 280 GPUs

Ru Runti time co cost?

slide-17
SLIDE 17

brinkar

In Inter erpr pretabilit ability?

slide-18
SLIDE 18

brinkar

arxiv.org/abs/1602.04938

LOCAL INTERPRETABLE MODEL- AGNOSTIC EXPLANATIONS (LIME

slide-19
SLIDE 19

brinkar

Au Autom

  • mati

tion

  • n

vs vs aug augmen mentatio ion? n?

slide-20
SLIDE 20

brinkar

Wh What t to opti timi mize for when bui buildi ding ng ML ML systems?

  • Accuracy?
  • Implementation cost?
  • Runtime cost?
  • Interpretability?
  • Automation vs augmentation?
slide-21
SLIDE 21

brinkar

1. Identify and define the problem 2. Collect and understand the data 3. Build and deploy a simple model that works end-to-end 4. Iterate to optimize an deploy improved models (inner loop) 5. Monitor and back-propagate changes to problem parameters (outer loop)

A A 5-st step ML meta-alg algorit ithm hm

slide-22
SLIDE 22

brinkar

  • 1. Identify and define the problem
  • 5. Monitor and back-propagate changes to problem parameters
slide-23
SLIDE 23

brinkar

  • 2. Collect and understand the data
slide-24
SLIDE 24

brinkar

  • 2. Collect and understand the data
slide-25
SLIDE 25

brinkar

  • 2. Collect and understand the data
slide-26
SLIDE 26

brinkar

  • 3. Deploy simple model end-to-end
  • 4. Iterate to optimize an deploy improved models
  • Simplest possible model that solves the

problem

  • End-to-end production deployment:

automated deployment, testing, logging, monitoring, feedback

slide-27
SLIDE 27

brinkar

Getting cl closer...

slide-28
SLIDE 28

brinkar

Mach chine learning infrastruct cture

slide-29
SLIDE 29

brinkar

Co Continuous integration

slide-30
SLIDE 30

brinkar

slide-31
SLIDE 31

brinkar

3 r 3 reaso eason f n for using using c container ainers in s in mac machine lear hine learning ning...

slide-32
SLIDE 32

brinkar

1 P 1 Pac ackag aging ing

slide-33
SLIDE 33

brinkar

2 Inher 2 Inherit itanc ance

FROM tensorflow/tensorflow:latest-gpu # Your specialized modeling pipeline FROM my-special-pipeline # Your even more specialized pipeline

tensorflow images segmentation classification text sentiment

slide-34
SLIDE 34

brinkar

Immut Immutabilit ability

slide-35
SLIDE 35

brinkar

“Data should go through the exact same pipeline when making predictions, as when the model was built.”

  • - Good ML practitioner
slide-36
SLIDE 36

brinkar

Us Use gr great t framewor

  • rks and service

ces... ...

slide-37
SLIDE 37

brinkar

Op Open sou

  • urce

ce ML framewor

  • rks
slide-38
SLIDE 38

brinkar

RISELab Clipper

  • Deploy models trained in your choice of framework to Clipper

with a few lines of code by using an existing model container or writing your own

  • Easily update or add models to running applications
  • Use adversarial bandit algorithms to dynamically select best

model for prediction at serving time

  • Set latency service level objectives for reliable query latencies
  • Run each model in a separate Docker container for simple

cluster management and resource allocation

  • Deploy models running on CPUs, GPUs, or both in the same

application

slide-39
SLIDE 39

brinkar

Ho Hosted M ed ML ser servic ices es

slide-40
SLIDE 40

brinkar

Mach chine learning infrastruct cture

  • Continuous integration
  • Containerization
  • Utilize ML platforms
slide-41
SLIDE 41

brinkar

Wr Wrap up...

slide-42
SLIDE 42

brinkar

Real-World Machine Learning

manning.com/brink

slide-43
SLIDE 43

brinkar

meetup.com/datacph nordic.ai

slide-44
SLIDE 44

brinkar

Qu Questi tions? s?