AI and Predictive Analytics in Data-Center Environments Performance - - PowerPoint PPT Presentation

▶

Sep 24, 2023 35 likes •289 views

AI and Predictive Analytics in Data-Center Environments Performance & Executing Experiments Josep Ll. Berral @BSC Intel Academic Education Mindshare Initiative for AI Introduction We have to choose where/how to run AI algorithms &

SLIDE 1

AI and Predictive Analytics in Data-Center Environments

Performance & Executing Experiments

Josep Ll. Berral @BSC

Intel Academic Education Mindshare Initiative for AI

SLIDE 2

Introduction

“We have to choose where/how to run AI algorithms & experiments”

SLIDE 3

Introduction

Algorithms have computing/data requirements
Computation Resources
CPU, Memory, GPUs, accelerators,

storage, ...

Time to run
Train models, infer new data, …
Data
What we are modeling and imitating

Machines to run our algorithms Data to feed our algorithms

SLIDE 4

Algorithms have computing requirements

Resources

Mem Disk CPU

Algorithms

CPU CPU Mem Mem

Machine Resources

Time

SLIDE 5

Algorithms also have data requirements

Resources

In Disks

Disk

Data Algorithms

Mem

In Memory

Disk Disk

From Users From the Network

SLIDE 6

Environment

We need a COMPUTING ENVIRONMENT!

SLIDE 7

Environment

Local Machines
Own computer
Workstations at work
...

SLIDE 8

Environment

Cluster Machines
DataCenter at work
DataCener at labs
Scientific grids
...

Data-Center / Cluster Your Machine Execution Submission

SLIDE 9

Environment

The Cloud
Data-Centers from Resource Providers

Data-Centers Resource Provider Experiments

SLIDE 10

Environment

Choosing the environment

X 1 X 16GB

CPU Mem

X 2 X 128GB

CPU Mem

X 4 X 256 GB

CPU Mem

X 32 X 2GB

CPU Mem

X 16 X 4GB

CPU Mem

X 64 X 128 GB

CPU Mem

SLIDE 11

Environment

Choosing the environment

X 128 X 1TB

CPU Mem

X 1 X 4GB

CPU Mem

X 2 X 2GB

CPU Mem

X 4 X 4GB

CPU Mem

X 16 X 16GB

CPU Mem

X 16 X 16GB

CPU Mem

X 64 X 128GB

CPU Mem

X ∞ X ∞GB

CPU Mem

SLIDE 12

Performance

“Work x Time” “Capacity to Progress”

SLIDE 13

Performance

Performance is (usually) linked to Resources

Mem Disk CPU

Resources Performance

SLIDE 14

Performance

Performance is (usually) linked to Resources

Mem Disk CPU

Resources Performance

(Up to a point) (Not always) (Not necessarily linear) (etc...)

SLIDE 15

Performance

Computing Environment
Pool of Resources
Algorithms/Apps/Experiments
Require resources

Mem Disk CPU CPU CPU Mem Mem

X 1 X 32GB

CPU Mem

OK!

SLIDE 16

Some Theory

Little’s Law

Little’s Law: L = λW
“Arrival rate x Dedicated time per input = Average load”

SLIDE 17

Some Theory

Little’s Law

Relation between
Received load
Resources/Time required
Average load // Required resources

Mem Disk CPU CPU CPU CPU

SLIDE 18

Little’s Law: L = λW

Example
we submit λ = 100 experiments / hour
Exps. take an average of W = 0.5 hours
[with 1 CPU per exp.]
Average number of exps. on our system: L = 50 exps
[avg. 50 CPUs in use]

CPU

100 Exp / hour ½ hour 1 CPU

BUSY x 50

CPU

What we expect (what we need)

SLIDE 19

Little’s Law

Demo!

SLIDE 20

100 CPUs

100 Jobs 100 CPUs BUSY Running

Resource Limits

CPU

x 100 x 100

30 60

CPU

0 Jobs 0 CPUs BUSY Running

Arrival rate: λ = 100 experiments / hour

Exps. take:

W = 0.5 hours [1 CPU per exp.] Average exps. in: L = 50 exps

SLIDE 21

50 CPUs

(What Little’s Law indicated)

50 Jobs 50 CPUs BUSY Running

Resource Limits

CPU

x 100 x 50

30 60

CPU

50 Jobs 50 CPUs BUSY Running 50 Jobs Queue 0 Jobs Queue

CPU Arrival rate: λ = 100 experiments / hour

Exps. take:

W = 0.5 hours [1 CPU per exp.] Average exps. in: L = 50 exps

SLIDE 22

25 CPUs
Less than needed!

25 Jobs 25 CPUs BUSY Running

Resource Limits

CPU

x 100 x 25

30 60

CPU

25 Jobs 25 CPUs BUSY Running 75 Jobs Queue 50 Jobs Queue

CPU Arrival rate: λ = 100 experiments / hour

Exps. take:

W = 0.5 hours [1 CPU per exp.] Average exps. in: L = 50 exps

SLIDE 23

25 CPUs
Less than needed!

25 Jobs 25 CPUs BUSY Running

Resource Limits

CPU

x 100 x 25

30 60 90

CPU

25 Jobs 25 CPUs BUSY Running 75 Jobs Queue 50 Jobs Queue

CPU

25 Jobs 25 CPUs BUSY Running 125 Jobs Queue

CPU x 100 Arrival rate: λ = 100 experiments / hour

Exps. take:

W = 0.5 hours [1 CPU per exp.] Average exps. in: L = 50 exps

SLIDE 24

Throughput

Throughput: outcome per time unit
E.g. experiments finished per hour
E.g. data-sets processed per minute
E.g. data points trained per second

Load Throughput

Resources Limit

SLIDE 25

Resource Competition

Systems do not always have “queues”
Processes (applications) compete in the system
For using the CPU
For getting some memory
For accessing the disk and network (I/O)