CS 744: RAY Shivaram Venkataraman Fall 2020 ADMINISTRIVIA late - - PowerPoint PPT Presentation

CS 744: RAY

Shivaram Venkataraman Fall 2020

a.1!

ADMINISTRIVIA

• Assignment Grades?
• Project proposal aka Introduction (10/16)

Introduction Related Work Timeline (with eval plan)

• Midterm: Oct 22

→

by

mall

late

next

week

→ early

next

week

MACHINE LEARNING: STACK

→

ypytorch

Data parallelism

train

→

,y→

Performance

Portability

TVM

D

\

Pipedream

Pipeline

parallelism

'm⇒

REINFORCEMENT LEARNING

Reward

Affeldt

wt

by

algo

"

n;÷:i

tea"

RL SETUP

:*

.in#ja:hna

→ perforation -7

¥

. -

• re
• do
• w
• -

rein

at

Imitation

.!qm

:*

RL REQUIREMENTS

Simulation Training Serving

¥

.

static exeat'm plan

→

fine

grained

computation

→ flexibility

• ↳

each

simulation

could

be

~

ms

• r

hours

stateless

processing

stated

,dm!nFu7e

, simeator state I

↳ data pre

• proving

→ very

low

latency

• Inane

execution

↳ future computation

very

High throughput

)

depends

• n outfit
• f

past

compute

IM

tasks

I see

RAY API

Tasks Actors

futures = f.remote(args) actor = Class.remote(args) futures = actor.method.remote(args)

• bjects = ray.get(futures)

ready = ray.wait(futures, k,timeout)

a' ←

qdfeadegr.am?t:aFIImtereqg-8osl9

" @yeifzI÷mg↳ :

• →

That

'kiEg¥eµ

tasks

Tfeu:EE

:b

variable I

¥

• - -

T Im

,I¥mItfauI

Fwd

• 2mg

name

• handle

method

't)

not 8.I

→ -

I -

arap will

(or

before

I

futures

arguments

to

tasks

args1#

↳

you

spawn ( or

wait) for

tasks

within

a task

RAY API

Tasks Actors

futures = f.remote(args) actor = Class.remote(args) futures = actor.method.remote(args)

• bjects = ray.get(futures)

ready = ray.wait(futures, k,timeout)

• o

a•img÷

• Nested

tasks

• \

""

"" /

def

f Largs)

• for

i

in

Ito lo :

fo . g.

remote ( 443)

Hof

• ray

• wait

EP

Frigg.

to

rmthprtati

"

COMPUTATION MODEL

Lineage !

g.

create - policy

Dotted

lines

↳

Control

edges

spawn

• an actor
• spam

@

get C)①

Sold

lines

✓

✓

↳

Data

edges

• y

⇒

stateful

edges /

X

action

• n

arts

happen

sequentially

ARCHITECTURE

Deterministic hash

key, had

can

Mtn

state.gr

tasks

I

to

1

Fagin

• •

← idea:

:

¥

"anger 01¥

Emir:&

Global control store

Object table Task table Function table

Sort of

a

Database

Externalizes

↳

list r

all

• bjets

state !

donations )

Namde

metadata

and

their

↳ shard

↳

Lineage

• f

tasks

Replicate

\

Scale more

↳

code

blocks corresponding

easily , simplify

ached

design

1

To tasks

fault

tolerance

RAY SCHEDULER

Global Scheduler Global Control Store

Can local scheduler

take locality ?

§

lags )

C

.

(

if busy

wait for timeout

g.*

/✓bcdi5

&

fund

:Fasching

• →

¥

←

length

to

locality

determine if

node is busy

FAULT TOLERANCE

Tasks Actors GCS Scheduler

→

lineage

replay

• k

• execution
• f

tasks

periodically

→

checkpoint

actors

tree

restore

cleft replay

messages

→ 775ha 'd

I

replication

drain

→

stateless ! Nothing ? Re -spawn

• r launch

new scheduler

SUMMARY

Ray: Unified system for ML training, serving, simulation Flexible API with support for Stateless tasks Stateful Actors Distributed scheduling, Global control store

DISCUSSION

https://forms.gle/PN5FSJB6vVkDjoih8

Consider you are implementing two apps: a deep learning model training and a sorting application. When will use tasks vs actors and why ?

stateless

bad

• balancing

state

Actors

Tasks

location

Sorting

Does

external Deterministic

• perations

unity

→ it

still

have

dependencies

!

* +

• a. d

into smallerparts?

Model

weights

are

can

do

dependencies Training

state ,

Multiple for

between

iterators ?

data parallel

fine

• grained

recovery

✓

YET

replica

new

node

→

has

better

if

;÷÷£

:*:*.

• to replica ?

• goes

doin

/

we# after

gift

v

)

recovery ?

godwin

?

ageing Minkowski

NEXT STEPS

Next class: Clipper Last lecture on ML!

Linear

scalability

sillier

¢

a

super

linear

l2hB/mc

hardware