CS184c: Computer Architecture [Parallel and Multithreaded] Day 14: - - PDF document

▶
cs184c computer architecture parallel and multithreaded
SMART_READER_LITE
LIVE PREVIEW

CS184c: Computer Architecture [Parallel and Multithreaded] Day 14: - - PDF document

Apr 28, 2023 853 likes •1.16k views

CS184c: Computer Architecture [Parallel and Multithreaded] Day 14: May 24, 2001 SCORE CALTECH cs184c Spring2001 -- DeHon Previously Interfacing Array logic with Processors Single thread, single-cycle operations Scaling

slide-1
SLIDE 1

1

CALTECH cs184c Spring2001 -- DeHon

CS184c: Computer Architecture [Parallel and Multithreaded]

Day 14: May 24, 2001 SCORE

CALTECH cs184c Spring2001 -- DeHon

Previously

  • Interfacing Array logic with Processors
  • Single thread, single-cycle operations
  • Scaling

– models weak on allowing more active hardware

  • Can imagine a more general,

heterogeneous, concurrent, multithreaded compute model….

slide-2
SLIDE 2

2

CALTECH cs184c Spring2001 -- DeHon

Today

  • SCORE

– scalable compute model – architecture to support – mapping and runtime issues

CALTECH cs184c Spring2001 -- DeHon

UCB BRASS RISC+HSRA

  • Integrate:

– processor – reconfig. Array – memory

  • Key Idea:

– best of both worlds temporal/spatial

slide-3
SLIDE 3

3

CALTECH cs184c Spring2001 -- DeHon

Bottom Up

  • GARP

– Interface – streaming

  • HSRA

– clocked array block – scalable network

  • Embedded DRAM

– high density/bw – array integration

Good handle on: raw building blocks tradeoffs

CALTECH cs184c Spring2001 -- DeHon

HSRA Architecture

CS184a: Day16

slide-4
SLIDE 4

4

CALTECH cs184c Spring2001 -- DeHon

Top Down

  • Question remained

– How do we control this? – Allow hardware to scale?

  • What is the higher

level model

– capture computation? – allows scaling?

CALTECH cs184c Spring2001 -- DeHon

SCORE

  • An attempt at defining a computational

model for reconfigurable systems

– abstract out

  • physical hardware details
  • especially size / # of resources
  • timing
  • Goal

– achieve device independence – approach density/efficiency of raw hardware – allow application performance to scale based

  • n system resources (w/out human intervention)
slide-5
SLIDE 5

5

CALTECH cs184c Spring2001 -- DeHon

SCORE Basics

  • Abstract computation is a dataflow graph

– stream links between operators – dynamic dataflow rates

  • Allow instantiation/modification /destruction of

dataflow during execution

– separate dataflow construction from usage

  • Break up computation into compute pages

– unit of scheduling and virtualization – stream links between pages

  • Runtime management of resources

CALTECH cs184c Spring2001 -- DeHon

Stream Links

  • Sequence of data flowing between
  • perators

– e.g. vector, list, image

  • Same

– source – destination – processing

slide-6
SLIDE 6

6

CALTECH cs184c Spring2001 -- DeHon

Virtual Hardware Model

  • Dataflow graph is arbitrarily large
  • Hardware has finite resources

– resources vary from implementation to implementation

  • Dataflow graph must be scheduled on

the hardware

  • Must happen automatically (software)

– physical resources are abstracted in compute model

CALTECH cs184c Spring2001 -- DeHon

Example

slide-7
SLIDE 7

7

CALTECH cs184c Spring2001 -- DeHon

Ex: Serial Implementation

CALTECH cs184c Spring2001 -- DeHon

Ex: Spatial Implementation

slide-8
SLIDE 8

8

CALTECH cs184c Spring2001 -- DeHon

Compute Model Primitives

  • SFSM

– FA with Stream Inputs – each state: required input set

  • STM

– may create any of these nodes

  • SFIFO

– unbounded – abstracts delay between operators

  • SMEM

– single owner (user)

CALTECH cs184c Spring2001 -- DeHon

SFSM

  • Model view for an operator or compute

page

– FIR, FFT, Huffman Encoder, DownSample

  • Less powerful than an arbitrary software

process

– bounded physical resources (no dynamic allocation) – only interface to state through streams

  • More powerful than an SDF operator

– dynamic input and output rates – dynamic flow rates

slide-9
SLIDE 9

9

CALTECH cs184c Spring2001 -- DeHon

SFSM

Operators are FSMs not just Dataflow graphs

  • Variable Rate Inputs

– FSM state indicates set of inputs require to fire

  • Lesson from hybrid dataflow

– control flow cheaper when succ. known

  • DF Graph of operators gives task-level

parallelism

– GARP and C models are all just one big TM

  • Gives programmer convenience of writing

familiar code for operator

– use well-known techniques in translation to extract ILP within an operator

CALTECH cs184c Spring2001 -- DeHon

STM

  • Abstraction of a process running on the

sequential processor

  • Interfaced to graph like SFSM
  • More restricted/stylized than threads

– cannot side-effect shared state arbitrarily – stream discipline for data transfer – single-owner memory discipline

slide-10
SLIDE 10

10

CALTECH cs184c Spring2001 -- DeHon

STM

  • Adds power to allocate memory

– can give to SFSM graphs

  • Adds power to create and modify

SCORE graph

– abstraction for allowing the logical computation to evolve and reconfigure – Note different from physical reconfiguration

  • f hardware
  • that happens below the model of computation
  • invisible to the programmer, since hardware

dependent

CALTECH cs184c Spring2001 -- DeHon

Model consistent across levels

  • Abstract computational model

– think about at high level

  • Programming Model

– what programmer thinks about – no visible size limits – concretized in language: e.g. TDF

  • Execution Model

– what the hardware runs – adds fixed-size hardware pages – primitive/kernel operations (e.g. ISA)

slide-11
SLIDE 11

11

CALTECH cs184c Spring2001 -- DeHon

Architecture

Lead: Randy Huang

CALTECH cs184c Spring2001 -- DeHon

Array

CP CP

CMB CMB Processor I Cache D Cache

SCORE Processor

Architecture for SCORE

Compute page interface Configurable memory block interface

instruction stream ID stream data

GPR Global Controller

SID PID location process ID

Memory & DMA Controller Memory & DMA Controller

data addr/cntl addr/cntl data data addr/cntl addr/cntl data

Processor to array interface

slide-12
SLIDE 12

12

CALTECH cs184c Spring2001 -- DeHon

Processor ISA Level Operation

  • User operations

– Stream write STRMWR Rstrm, Rdata – Stream read STRMRD Rstrm, Rdata

  • Kernel operation (not visible to users)

– {Start,stop} {CP,CMB,IPSB} – {Load,store} {CP,CMB,IPSB} {config,state,FIFO} – Transfer {to,from} main memory – Get {array processor, compute page} status

CALTECH cs184c Spring2001 -- DeHon

Communication Overhead

Note

  • single cycle to send/receive data
  • no packet/communication overhead

– once a connection is setup and resident

  • contrast with MP machines and NI we

saw earlier

slide-13
SLIDE 13

13

CALTECH cs184c Spring2001 -- DeHon

SCORE Graph on Hardware

  • One master

application graph

  • Operators run
  • n processor

and array

  • Communicate

directly amongst

CALTECH cs184c Spring2001 -- DeHon

SCORE OS: Reconfiguration

  • Array managed

by OS

  • Only OS can

manipulate array configuration

slide-14
SLIDE 14

14

CALTECH cs184c Spring2001 -- DeHon

  • Allocation

goes through OS

  • Similar to sbrk

in conventional API

SCORE OS: Allocation

CALTECH cs184c Spring2001 -- DeHon

Performance Scaling: JPEG Encoder

5 10 15 20 25 30 35 40 1 3 5 7 9 11 13 Array Size (# of CPs) Runtime (Mcycles)

SCORE Simulation Processor - Pentium III (500MHz/256MB)

slide-15
SLIDE 15

15

CALTECH cs184c Spring2001 -- DeHon

Performance Scaling: JPEG Encoder

2 4 6 8 10 12 14 1 3 5 7 9 11 13 Array Size (# of CPs) Runtime (Mcycles)

CALTECH cs184c Spring2001 -- DeHon

Page Generation (work in progress)

Eylon Caspi, Laura Pozzi

slide-16
SLIDE 16

16

CALTECH cs184c Spring2001 -- DeHon

SCORE Compilation in a Nutshell

Programming Model Execution Model

  • Graph of TDF FSMD operators
  • Graph of page

configs

  • unlimited size, # IOs
  • fixed size, # IOs
  • no timing constraints
  • timed, single-cycle firing

Compile

memory segment TDF

  • perator

stream memory segment compute page stream CALTECH cs184c Spring2001 -- DeHon

How Big is an Operator?

  • Wavelet Decode
  • Wavelet Encode
  • JPEG Encode
  • MPEG Encode

Area for 47 Operators

(Before Pipeline Extraction)

500 1000 1500 2000 2500 3000 3500

Operator (sorted by area) Area (4-LUTs)

FSM Area DF Area

  • JPEG Encode
  • JPEG Decode
  • MPEG (I)
  • MPEG (P)
  • Wavelet Encode
  • IIR
slide-17
SLIDE 17

17

CALTECH cs184c Spring2001 -- DeHon

Unique Synthesis / Partitioning Problem

  • Inter-page stream delay not known by

compiler:

– HW implementation – Page placement – Virtualization – Data-dependent token emission rates

  • Partitioning must retain stream abstraction

– also gives us freedom in timing

  • Synchronous array hardware

CALTECH cs184c Spring2001 -- DeHon

Clustering is Critical

  • Inter-page comm. latency may be long
  • Inter-page feedback loops are slow
  • Cluster to:

– Fit feedback loops within page – Fit feedback loops on device

slide-18
SLIDE 18

18

CALTECH cs184c Spring2001 -- DeHon

Pipeline Extraction

  • Hoist uncontrolled FF data-flow out of FSMD
  • Benefits:

– Shrink FSM cyclic core – Extracted pipeline has more freedom for scheduling and partitioning

Extract state foo(i): acc=acc+2*i state foo(two_i): acc=acc+two_i

i

state DF CF

*2

two_i i pipeline pipeline CALTECH cs184c Spring2001 -- DeHon

Pipeline Extraction – Extractable Area

Extractable Data-Path Area

for 47 Operators

500 1000 1500 2000 2500 3000 3500

Operator (sorted by data-path area) Area (4-LUTs)

Extracted DF Area Residual DF Area

  • JPEG Encode
  • JPEG Decode
  • MPEG (I)
  • MPEG (P)
  • Wavelet Encode
  • IIR
slide-19
SLIDE 19

19

CALTECH cs184c Spring2001 -- DeHon

Page Generation

  • Pipeline extraction

– removes dataflow can freely extract from FSMD control

  • Still have to partition potentially large

FSMs

– approach: turn into a clustering problem

CALTECH cs184c Spring2001 -- DeHon

State Clustering

  • Start: consider each state to be a unit
  • Cluster states into page-size sub-

FSMDs

– Inter-page transitions become streams

  • Possible clustering goals:

– Minimize delay (inter-page latency) – Minimize IO (inter-page BW) – Minimize area (fragmentation)

IA IB O A O B

slide-20
SLIDE 20

20

CALTECH cs184c Spring2001 -- DeHon

State Clustering to Minimize Inter-Page State Transfer

  • Inter-page state transfer is slow
  • Cluster to:

– Contain feedback loops – Minimize frequency of inter-page state transfer

  • Previously used in:

– VLIW trace scheduling [Fisher ‘81] – FSM decomposition for low power

[Benini/DeMicheli ISCAS ‘98]

– VM/cache code placement – GarpCC code selection[Callahan ‘00]

CALTECH cs184c Spring2001 -- DeHon

Scheduling (work in progress)

Lead: Yury Markovskiy

slide-21
SLIDE 21

21

CALTECH cs184c Spring2001 -- DeHon

Scheduling

  • Time-multiplex the operators onto the

hardware

  • To exploit scaling:

– page capacity is a late-bound

parameter –cannot do scheduling at compile time

  • To exploit dynamic data

–want to look at application, data characteristics

CALTECH cs184c Spring2001 -- DeHon

Scheduling: First Try Dynamic

  • Fully Dynamic
  • Time sliced
  • List-scheduling based
  • Very expensive:

– 100,000-200,000 cycles – scheduling 30 virtual pages – onto 10 physical

slide-22
SLIDE 22

22

CALTECH cs184c Spring2001 -- DeHon

Overhead Effects

Wavelet Encode Dynamic Scheduler Performance

0.5 1 1.5 2 2.5 3 3.5 4 4.5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Array Size (#CP) Makespan (Mcycl) No Overhead Reconfig Time Reconfig + Sched Time

CALTECH cs184c Spring2001 -- DeHon Wavelet Encode Dynamic Scheduler Overhead per Timeslice 50 100 150 200 250 300 350 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Array Size (#CP) Kcycles Scheduling Reconfiguration

Overhead Costs

slide-23
SLIDE 23

23

CALTECH cs184c Spring2001 -- DeHon

Scheduling: Why Different, Challenging

  • Distributed Memory vs. Uniform

Memory

– placement/shuffling matters

  • Multiple memory ports

– increase bandwidth – fixed limit on number of ports available

  • Schedule subgraphs

– reduce latency and memory

CALTECH cs184c Spring2001 -- DeHon

Scheduling: Taxonomy (How Dynamic?)

  • Static/Dynamic Boundary?

Placement Sequence Rate Timing

slide-24
SLIDE 24

24

CALTECH cs184c Spring2001 -- DeHon

Dynamic→Load Time Scheduling

Dynamic Scheduler Static Scheduler TDFC QueryArray Sequence Allocation Reconfigure

Compile Time Run Time

TDFC QueryArray Reconfigure Sequence Allocation Load Time

CALTECH cs184c Spring2001 -- DeHon

Static Scheduler Overhead

Wavelet Encode Static Scheduler Overhead per Timeslice

10 20 30 40 50 60 70 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Array Size (#CP) Kcycles Reconfiguration Scheduling

slide-25
SLIDE 25

25

CALTECH cs184c Spring2001 -- DeHon

Static Scheduler Performance

Overall Performance

0.5 1 1.5 2 2.5 3 3.5 4 4.5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Array Size Makespan Dynamic Static CALTECH cs184c Spring2001 -- DeHon

Anomalies and How Dynamic?

  • Anomalies on previous graph

– early stall on stream data – from assuming fixed timeslice model

  • Solve by

– dynamic epoch termination – detect when appropriate to advance schedule Placement Sequence Rate Timing

slide-26
SLIDE 26

26

CALTECH cs184c Spring2001 -- DeHon

Static Scheduler w/ Early Stall Detection

Wavelet Runtime

200000 400000 600000 800000 1000000 1200000 1400000 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Array Size Cycles Runtime Runtime+Reconfig Runtime+Reconfig+SchedBookKeep

CALTECH cs184c Spring2001 -- DeHon

More Heterogeneous Programmable SoC

slide-27
SLIDE 27

27

CALTECH cs184c Spring2001 -- DeHon

Broader Programmable SOC Applicability

  • Model potentially

valuable beyond homogenous array

  • Already

introduced idea

  • f different page

types

CALTECH cs184c Spring2001 -- DeHon

Heterogeneous Pages

Small conceptual step to generalize

  • Memory (CMB)
  • Processor
  • FPGA

– vary granularity – vary depth

  • IO
  • Custom (e.g. FPU)
slide-28
SLIDE 28

28

CALTECH cs184c Spring2001 -- DeHon

Summary

  • Advantage and value for programmable

spatial computing components

  • Need a compute model

– to permit device scaling – while preserving human effort

  • SCORE model captures parallelism and

freedom in these applications

  • Believe it can be efficient
  • Starting to get a handle on

hardware/compiler/runtime support

CALTECH cs184c Spring2001 -- DeHon

Additional Information

  • SCORE:

– http://brass.cs.berkeley.edu/SCORE – especially see “Introduction and Tutorial”

  • CALTECH:

– http://www.cs.caltech.edu/research/ic/

slide-29
SLIDE 29

29

CALTECH cs184c Spring2001 -- DeHon

Big Ideas

  • Model

– basis for virtualization – basis for scaling – allows common-case optimizations – supports kind of computations which exploit this architecture

  • spatial composition of computing blocks

CALTECH cs184c Spring2001 -- DeHon

Big Ideas

  • Expose parallelism

– hidden by sequential control flow in ISA- based models

  • Communication to operator

– not to resource (ala. GARP)

  • Support spatial composition

– contrast sequential composition in ISA

  • Data presence [self timed!]

– tolerant to timing and resource variations

slide-30
SLIDE 30

30

CALTECH cs184c Spring2001 -- DeHon

Big Ideas

  • Persistent Dataflow

– separate creation and use – use many times (amortize cost of creation)

  • Persistent Communication

– separate setup/allocation form use – amortize out cost of routing/negotiation/setup