Synchronous Elastic Systems Synchronous Elastic Systems Mike - - PowerPoint PPT Presentation
Synchronous Elastic Systems Synchronous Elastic Systems Mike Kishinevsky and Jordi Cortadella Mike Kishinevsky and Jordi Cortadella Intel Intel Universitat Politecnica Politecnica Universitat Strategic CAD Labs Strategic CAD Labs de
Mike Kishinevsky and Jordi Cortadella Mike Kishinevsky and Jordi Cortadella
Universitat Universitat Politecnica Politecnica de de Catalunya Catalunya Barcelona, Spain Barcelona, Spain Intel Intel Strategic CAD Labs Strategic CAD Labs Hillsboro, USA Hillsboro, USA
2
I. I.
II. II.
III. III.
IV. IV.
3
Token (of data)
1 4 7
Clock cycle 1 2
4
1 4 7
1 2
Clock cycle Token
4 1 7
1 2
3 4 5 Clock cycle
Bubble (no data) 5
1 4 7
3 4 8 2 1
6
1 4 7
3 4 8 2 1
3 4 8
1 4 7
2 1
7
A C D B A C D B
Changing latencies changes behavior
8
A C D B A C D B e e e e e e e e e
Changing latencies does NOT change behavior = time elasticity
9
Elasticity refers to elasticity of time, i.e. tolerance to chang Elasticity refers to elasticity of time, i.e. tolerance to changes in es in timing parameters, not properties of materials timing parameters, not properties of materials Luca Carloni et al. in the first systematic study of such system Luca Carloni et al. in the first systematic study of such systems s called them Latency Insensitive Systems called them Latency Insensitive Systems Other used names: Other used names: – – Latency tolerant systems Latency tolerant systems – – Synchronous emulation of asynchronous systems Synchronous emulation of asynchronous systems – – Synchronous handshake circuits Synchronous handshake circuits We use term We use term “ “synchronous elastic synchronous elastic” ” to link to asynchronous elastic to link to asynchronous elastic systems that have been developed before systems that have been developed before e.g., David Muller e.g., David Muller’ ’s pipelines of late 1950s s pipelines of late 1950s Ivan Sutherland Ivan Sutherland’ ’s micro s micro-
pipelines 1989 Tolerate the variability of input data arrival and computation d Tolerate the variability of input data arrival and computation delays elays Asynchronous elastic tolerate changes in continuous time Asynchronous elastic tolerate changes in continuous time Synchronous elastic Synchronous elastic -
in discrete time
10
Scalable Scalable Modular (Plug & Play) Modular (Plug & Play) Better energy Better energy-
delay trade-
(design for typical case instead of worst case) (design for typical case instead of worst case) New micro New micro-
architectural opportunities in digital design in digital design Not asynchronous: use existing design Not asynchronous: use existing design experience, CAD tools and flows... but have experience, CAD tools and flows... but have some advantages of asynchronous some advantages of asynchronous 11
12
sender receiver Data Data
What if the sender does not always send valid data?
13
sender receiver Data Data Valid Valid
What if the receiver is not always ready ? 14
sender Data Valid Stop receiver Data Valid Stop
15
1 1 1 1
sender Data Valid Stop receiver Data Valid Stop
16
1 1 1 1 1 1
sender Data Valid Stop receiver Data Valid Stop
17
1 1 1 1 1 1 1 1 1 1
sender Data Valid Stop receiver Data Valid Stop
18
1 1
sender Data Valid Stop receiver Data Valid Stop
Long combinational path 19
20
Combinational cycle
Data Valid Stop One can build circuits with combinational cycles (constructive cycles by Berry), but synthesis and timing tools do not like them
Example: pipelined linear communication chain Example: pipelined linear communication chain with transparent latches with transparent latches
sender receiver
H L H L ½ cycle ½ cycle
Master and slave latches with independent control
21
22
D Q clk
En En
sender receiver
V V V V S S S En En En En S 1 1 Data Valid Stop Data Valid Stop 1 1
23
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
1 1 24
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
1 1 25
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
1 1 26
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
1 1 27
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
1 1 28
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
29
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
30
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
31
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
32
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
33
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
1 1 1 1 34
sender receiver
V V V V S S S En En En En S Data Valid Stop
1 1 1 1
Data Valid Stop
35
sender receiver
V V V V S S S En En En En S Data Valid Stop
1 1 1 1
Data Valid Stop
36
sender receiver
V V V V S S S En En En En S Data Valid Stop
1 1 1 1
Data Valid Stop
37
sender receiver
V V V V S S S En En En En S Data Valid Stop
1 1 1 1
Data Valid Stop
38
sender receiver
V V V V S S S En En En En S Data Valid Stop
1 1 1 1
Data Valid Stop
39
sender receiver
V V V V S S S En En En En S Data Valid Stop
1 1 1 1
Data Valid Stop
40
sender receiver
V V V V S S S En En En En S Data Valid Stop
1 1 1 1
Data Valid Stop
41
sender receiver
V V V V S S S En En En En S Data Valid Stop
1 1 1 1
Data Valid Stop
42
sender receiver
V V V V S S S En En En En S Data Valid Stop
1 1
Data Valid Stop
43
sender receiver
V V V V S S S En En En En
1 1
S Data Valid Stop Data Valid Stop
44
sender receiver
V V V V S S S En En En En
1 1
S Data Valid Stop Data Valid Stop
45
sender receiver
V V V V S S S En En En En
1 1
S Data Valid Stop Data Valid Stop
46
sender receiver
V V V V S S S En En En En
1 1
S Data Valid Stop Data Valid Stop
47
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
1 1 48
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
1 1 49
sender receiver
V V V V S S S En En En En S Data Valid Stop Data Valid Stop
1 1 50
Idle Retry Transfer
Valid * not Stop not Valid Valid * Stop
Sender Sender Receiver Receiver
Data Valid Stop
51
Sender Sender Receiver Receiver Data Data
Valid Valid Stop Stop
Data Data
Valid Valid Stop Stop
* D D * C C C B * A * D D * C C C B * A 0 1 1 0 1 1 1 1 0 1 0 1 1 0 1 1 1 1 0 1 0 0 1 0 0 1 1 0 0 0 0 0 1 0 0 1 1 0 0 0
Transfer Transfer
Retry Retry Idle Idle 52
En Eni
i
V Vi
i
En Eni
i
V Vi
i-
53 V Vi
i-
V Vi
i 1 1
S Si
i-
S Si S Si
i-
S Si
1 i 1 i
VS block + data-path latch = elastic HALF-buffer (EHB) EHB + EHB = elastic buffer with capacity 2
54
55
Elastic buffer keeps data while stop is in flight Elastic buffer keeps data while stop is in flight
W1R1 W2R1 W1R2
56
W2R2 W1R1 Cannot be done with Single Edge Flops without double pumping Can use latches inside Master-Slave as shown before
EBs = FIFOs with two parameters:
Forward latency Capacity Backward latency for stop propagation assumed (but need not be) equal to fwd latency Typical case: (1,2) - 1 cycle forward latency with capacity of 2 Replaces “normal” registers Decoupling buffers
57 VS
V1 V2 S1 S2 V S VS VS
V V1 S1 V2 S S2 58
S1
^ ^ ^ ^
V1 V V2 S S2 59
VS VS VS VS VS
60
61
[0 - k] cycles [0 - k] cycles V/S V/S done go clear
62
Synchronous Elastic
63
CLK CLK
64
CLK CLK
PC IF/ID ID/EX MEM/WB EX/MEM J J O O I I N N J J O O I I N N F F O O R R K K FORK FORK
65
66
V S
CLK CLK
V S V S V S V S J O I N J O I N F O R K FORK
67
1
CLK CLK
1 1 1 1 J O I N J O I N F O R K FORK
68
1 1 1 1 1
CLK CLK
Synchronous: stream of data
SELF: elastic stream of data
Transfer sub-stream = original stream
Called: transfer equivalence, flow equivalence, or latency equivalence
69
70
data data-
token bubble bubble data data-
token bubble bubble
71
bubble bubble data data-
token 2 data 2 data-
tokens
72 Hiding internal transitions of elastic buffers
73
Forward Forward (Valid or Request) (Valid or Request) Backward Backward (Stop or Acknowledgement) (Stop or Acknowledgement)
74
75 d=250ps d=151ps Delays in time units 250 151
76 d=1 d=1 Latencies in clock cycles 1 1
77 d=2 d=1 2 1
d {1,2} d=1 {1,2} 1 e.g. discrete probabilistic distribution: average latency 0.8*1 + 0.2*2 = 1.2
∈
78
d=1 1 79
d=1 d=0 1 80
An Elastic Marked Graph (EMG) is a Timed MG such that for any arc a there exists a complementary arc a’ satisfying the following condition
Initial number of tokens on a and a’ (M0(a)+M0(a’)) = capacity of the corresponding elastic buffer Similar forms of “pipelined” Petri Nets and Marked Graphs have been previously used for modeling pipelining in HW and SW (e.g. Patil 1974; Tsirlin, Rosenblum 1982) 81
Reminder Reminder: : Performance Performance analysis analysis of
Marked graphs graphs
Th Th = operations / cycle = number of firings per time unit = operations / cycle = number of firings per time unit
The throughput is given by the minimum mean-weight cycle Th=min(Th(A), Th(B), Th(C))=2/5
A B C Th(A)=3/7 Th(B)=3/5 Th(C)=2/5
Efficient algorithms: (Karp 1978), (Dasdan,Gupta 1998)
82
Na Naï ïve solution: introduce choice places ve solution: introduce choice places – – issue tokens at choice node only into one (some) relevant path issue tokens at choice node only into one (some) relevant path – – problem: tokens can arrive to merge nodes out problem: tokens can arrive to merge nodes out-
later token can overpass the earlier one later token can overpass the earlier one Solution: change enabling rule Solution: change enabling rule – – early evaluation early evaluation – – issue negative tokens to input places without tokens, issue negative tokens to input places without tokens, i.e. keep the same firing rule i.e. keep the same firing rule – – Add symmetric sub Add symmetric sub-
channels with negative tokens – – Negative tokens kill positive tokens when meet Negative tokens kill positive tokens when meet Two related problems: Two related problems: Early evaluation and Exceptions (how to kill a data Early evaluation and Exceptions (how to kill a data-
token)
83
MULTIPLEXOR a b c s
if s = T then c := a -- don’t wait for b else c := b -- don’t wait for a
T F MULTIPLIER a b c
if a = 0 then c := 0 -- don’t wait for b
84
Petri nets Petri nets
– – Extensions to model OR causality Extensions to model OR causality Kishinevsky et al. Change Diagrams [e.g. book of 1994] Kishinevsky et al. Change Diagrams [e.g. book of 1994] Yakovlev et al. Causal Nets 1996 Yakovlev et al. Causal Nets 1996
Asynchronous systems Asynchronous systems
– – Reese et al 2002: Early evaluation Reese et al 2002: Early evaluation – – Brej Brej 2003: Early evaluation with anti 2003: Early evaluation with anti-
tokens – – Ampalan Ampalan & Singh 2006: & Singh 2006: preemption preemption using anti using anti-
tokens 85
Marking: Marking: Arcs (places) −> Z (allow negative markings) Some nodes are labeled as early-enabling Enabling rules for a node:
– Positive enabling: M(a) > 0 for every input arc – Early enabling (for early enabling nodes): M(a) > 0 for some input arcs – Negative enabling: M(a) < 0 for every output arc
Firing rule: the same as in regular MG
86
Early enabling can be associated with an external guard that depends on data variables (e.g., a select signal of a multiplexor) Actual enabling guards are abstracted away (unless needed) Anti-token generation: When an early enabled node fires, it generates anti-tokens in the predecessor arcs that had no tokens Anti-token propagation counterflow: When negative enabled node fires, it propagates the anti-tokens from the successor to the predecessor arcs
87
Enabled !
88
Passive DMG = version of DMG without negative enabling Passive DMG = version of DMG without negative enabling Negative tokens can only be generated due to early Negative tokens can only be generated due to early enabling, but cannot propagate enabling, but cannot propagate Let Let D D be a strongly connected DMG such that all cycles be a strongly connected DMG such that all cycles have positive cumulative marking have positive cumulative marking Let Let D Dp
p be a corresponding passive DMG
be a corresponding passive DMG. . If environment (consumers) never generate negative If environment (consumers) never generate negative tokens, then tokens, then throughput ( throughput (D D) = throughput ( ) = throughput (D Dp
p)
)
– – If capacity of input places for early enabling transitions is un If capacity of input places for early enabling transitions is unlimited, limited, then active anti then active anti-
tokens do not improve performance – – Active anti Active anti-
tokens reduce activity in the data-
path (good for power reduction) (good for power reduction)
89
Firing invariant: Let node n be simultaneously positive (early) and negative enabled in marking M. Let M1 be the result of firing n from M due to positive (early) enabling. Let M2 be the result of firing n from M due to negative enabling. Then, M1 = M2 Token preservation. Let c be a cycle of a strongly connected DMG with initial marking M0. For every reachable marking M : M(c) = M0(c)
M(c) > 0.
– For DMGs this is a sufficient condition of liveness – It is also a necessary condition for positive liveness
Repetitive behavior. In a SC DMG: a firing sequence s from M leads to the same marking iff every node fires in s the same number of times DMGs have properties similar to regular MGs
90
91
Positive tokens Negative tokens 92
Positive tokens Negative tokens 93
Valid Valid+
+
94
Valid Valid+
+
Valid Valid+
+
Stop Stop+
+
Valid Valid–
–
Stop Stop–
–
Stop Stop+
+
Valid Valid–
–
Stop Stop–
–
V S V S Data
H H L L L H
V S V S En En
95
S+ V+ V- S- S+ V+ V- S- En En
96
97
98
Early evaluation function makes decision based on presence
Formally: EE is positive unate with respect to data input Example: legal EE function for a data-path MUX (s – select input) 99
Bigger capacity can be achieved by “injecting” anti-token up-down counters on elastic channels 100
Invariants: mutually exclusive Kill (V -) and Stop (S +) Valid (V +) and retain of a kill (S -)
101
Fetch Bypass Decode Execute Memory Write-back α 1−α β 1−β
Th Th = operations / cycle = operations / cycle
Throughput:
102
Late evaluation
Th=0.5 Th = 0.7 (α=0.3; β=0.3)
Applying early evaluation on “Execution” and “Write-back”
Early evaluation can increase performance Early evaluation can increase performance beyond the min cycle ratio beyond the min cycle ratio The duality between positive and negative The duality between positive and negative tokens suggests a clean and effective tokens suggests a clean and effective implementation implementation Dual Marked Graphs is a formal model for Dual Marked Graphs is a formal model for analytical analysis and optimization methods analytical analysis and optimization methods
103
104
Revisit Revisit Performance Performance Analysis Analysis of
Marked Graphs Graphs
The throughput can also be computed by means of The throughput can also be computed by means of linear programming linear programming
Average marking
∞ →
=
t p t t p
)d ( m m
1
lim τ τ
p p m
th min =
Throughput
2 1 p p m
t1 t2 t3 p1 p2
[Campos, Chiola, Silva 1991]
105
Revisit Revisit Performance Performance Analysis Analysis of
Marked Graphs Graphs max th
106 a b c p1 p2 p3 p4 p5
mp1 = 1 + tb – ta mp2 = 0 + ta – tb mp3 = 1 + td – ta mp4 = 0 + ta – tc mp5 = 1 + tc – td
Th Th = 0.5 = 0.5
reachability
th ≤ mp2 // transition b th ≤ mp4 // transition c th ≤ mp5 // transition d th ≤ min(mp1, mp3) // transition a
th constraints
d
Refinement of passive Refinement of passive DMGs DMGs Every node has a set of guards Every node has a set of guards Every guard is a set of input places (arcs) Every guard is a set of input places (arcs) Example: Example:
t1 t3 t2 t4 p1 p2 p3
G(t4)={{p1,p3},{p2,p3}} 107
α 1-α β 1-β
108
α 1−α β 1−β α β
0.60 0.60 0.54 0.54 0.49 0.49 0.46 0.46 0.44 0.44 0.43 0.43 1.0 1.0 0.54 0.54 0.51 0.51 0.48 0.48 0.46 0.46 0.44 0.44 0.43 0.43 0.8 0.8 0.49 0.49 0.48 0.48 0.47 0.47 0.45 0.45 0.44 0.44 0.43 0.43 0.6 0.6 0.45 0.45 0.45 0.45 0.45 0.45 0.44 0.44 0.44 0.44 0.43 0.43 0.4 0.4 0.43 0.43 0.43 0.43 0.42 0.42 0.42 0.42 0.42 0.42 0.42 0.42 0.2 0.2 0.40 0.40 0.40 0.40 0.40 0.40 0.40 0.40 0.40 0.40 0.40 0.40 0.0 0.0 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 0.60 0.60 0.54 0.54 0.49 0.49 0.46 0.46 0.44 0.44 0.43 0.43 1.0 1.0 0.54 0.54 0.51 0.51 0.48 0.48 0.46 0.46 0.44 0.44 0.43 0.43 0.8 0.8 0.49 0.49 0.48 0.48 0.47 0.47 0.45 0.45 0.44 0.44 0.43 0.43 0.6 0.6 0.45 0.45 0.45 0.45 0.45 0.45 0.44 0.44 0.44 0.44 0.43 0.43 0.4 0.4 0.43 0.43 0.43 0.43 0.42 0.42 0.42 0.42 0.42 0.42 0.42 0.42 0.2 0.2 0.40 0.40 0.40 0.40 0.40 0.40 0.40 0.40 0.40 0.40 0.40 0.40 0.0 0.0 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0
109 (0.43) (0.43) (0.60) (0.60) (0.40) (0.40)
α 1-α
a b d c p1 p2 p3 p4 p5
max th
mp1 = 1 + tb – ta mp2 = 0 + ta – tb mp3 = 1 + td – ta mp4 = 0 + ta – tc mp5 = 1 + tc – td th ≤ mp2 th ≤ mp4 th ≤ mp5 th = α mp1 + (1-α) mp3
Th Th = (2 = (2 -
α) / (3 ) / (3 -
α) )
110
Th Th = (2 = (2 -
α) / (3 ) / (3 -
α) )
α 1-α
a b d c p1 p2 p3 p4 p5
1/2 1/2 2/3 2/3
Averaging throughput of individual cycles Th Th’ ’ = = α α 1/2 + (1 1/2 + (1-
α) 2/3 = (4 ) 2/3 = (4 -
α) / 6 ) / 6 1/Th 1/Th” ” = 2 = 2α α + (1 + (1-
α) 3/2 ) 3/2 = = (3 + (3 + α α) / 2 ) / 2 Th Th” ” = 2/(3+ = 2/(3+ α α) ) Averaging effective cycle times
111
112
ALU ALU L = 1 L = 3 L = 2 L = 1 start done
113
# adds
Benchmark “Patricia” from Media Bench
Statistics
sizes bits of adder used
1st operand 2
nd
e r a n d S i g n i f i c a n t b i t s # adds
12 bits of an adder do 95% of additions 114
1 1.25 1.5
Compare 64 bits VLA and prefix adder
relative delay
115
12-cycle miss L2-cache 2-way associative 32KB 2-cycle hit
1-cycle hit
suggested by Joel Emer for ASIM experiment suggested by Joel Emer for ASIM experiment
L1-cache
116
12-cycle miss L2-cache Pseudo-associative 32KB {1-2} cycle hit
1-cycle hit
Sequential access: if hit in first access L = 1, if not – L=2 Trade-off: faster, or larger, or less power cache
L1-cache
117
12-cycle miss L2-cache Pseudo-associative 64KB {2-3} cycle hit
1-cycle hit
Sequential access: if hit in first access L = 1, if not – L=2 Trade-off: faster, or larger, or less power cache
L1-cache
118
Transforms:
– bypass – retiming – elasticize – early enabling – insert buffers and negative tokens – size elastic buffer capacity
ID E1 E2 RF
E1 E2 RF
SPEC Correct-by-construction IMP
[Joint work with Timothy Kam and Marc Galceran] 119
R R R R R
AGENT AGENT AGENT AGENT AGENT AGENT AGENT
[In collaboration with Ken Stevens, Charles Dike, Bill Grundmann] 120
121
Relative order of tokens between agents is preserved
122
123
Developed theory of elastic machines Developed theory of elastic machines (for late evaluation) (for late evaluation) Verify correctness of any elastic implementation = check Verify correctness of any elastic implementation = check conformance with the definition of elastic machine conformance with the definition of elastic machine All SELF controllers are verified for conformance All SELF controllers are verified for conformance Elasticization Elasticization is correct is correct-
by-
construction Theory for early evaluation and negative delays is more Theory for early evaluation and negative delays is more challenging challenging
– – Sketch of a theory, but no fully satisfactory compositional Sketch of a theory, but no fully satisfactory compositional properties found yet properties found yet – – Verification done on concrete systems and controllers Verification done on concrete systems and controllers
124
Summary Summary SELF gives a low cost implementation of elastic machines SELF gives a low cost implementation of elastic machines Functionality is correct when latencies change Functionality is correct when latencies change New micro New micro-
architectural opportunities Compositional theory proving correctness Compositional theory proving correctness Early evaluation Early evaluation -
mechanism for performance and power
Optimization methods (that we did not discuss): Optimization methods (that we did not discuss): Retiming and recycling, buffer optimization and pipelining Retiming and recycling, buffer optimization and pipelining Applications to design of Applications to design of NoC NoC link layer link layer 125
126
Flow Flow graph graph
Parameterized Parameterized library of library of controllers controllers
Control generation Verilog Verilog SMV SMV blif blif
Backend synthesis NuSMV SIS & ABC Simulator
Verification Logic synthesis
Netlist Netlist of
distributed distributed controllers controllers
127
Performance
128
Evaluation Evaluation Throughput Throughput No early evaluation No early evaluation 0.277 0.277 Passive anti Passive anti-
tokens M2 → → W W 0.280 0.280 Passive anti Passive anti-
tokens F3 → → W W 0.387 0.387 Active anti Active anti-
tokens 0.400 0.400
129