Living beyond your means Extracting Response Time Densities and - - PowerPoint PPT Presentation

▶
living beyond your means
SMART_READER_LITE
LIVE PREVIEW

Living beyond your means Extracting Response Time Densities and - - PowerPoint PPT Presentation

Nov 14, 2023 145 likes •670 views

Living beyond your means Extracting Response Time Densities and Quantiles from Stochastic Models Susanna Au-Yeung, Jeremy Bradley, Nicholas Dingle, Tony Field, Peter Harrison, William Knottenbelt , Aleksandar Trifunovic, Helen Wilson email: {

slide-1
SLIDE 1

Living beyond your means

Extracting Response Time Densities and Quantiles from Stochastic Models

Susanna Au-Yeung, Jeremy Bradley, Nicholas Dingle, Tony Field, Peter Harrison, William Knottenbelt, Aleksandar Trifunovic, Helen Wilson email: {wjk,aesop}@doc.ic.ac.uk

slide-2
SLIDE 2

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

1

Summary (NOT!)

There are three surefire ways to live beyond your means:

  • “Invest” (a.k.a. lose) your money in the stockmarket.
  • Play Texas Hold ’em poker for money.
  • Go out frequently on an academic salary.
slide-3
SLIDE 3

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

2

Motivation

  • Fast response times are important for almost all computer-

communication and transaction processing systems.

  • It is important to assess the performance of these systems

ahead of implementation via formal models (e.g. Petri nets).

  • Traditional analysis methods compute steady state behaviour.
  • But this is inadequate to compute response-time densities

and quantiles (percentiles).

slide-4
SLIDE 4

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

3

Response Times and QoS

  • 90th/95th percentile response time is a good indicator of

client-perceived quality of service in SLAs/benchmarks, e.g.:

  • Ambulance should be on scene in under 13 mins 90% of time.
  • 95% of text messages should be delivered in under 3 seconds.
  • TPC-C OLTP benchmark requirements:

90th percentile

  • Min. Mean of

Transaction Minimum Minimum response time Think Time Type % of mix Keying Time constraint Distribution Payment 43.0 3 sec. 5 sec. 12 sec. Order-Status 4.0 2 sec. 5 sec. 10 sec. Stock-Level 4.0 2 sec. 20 sec. 5 sec.

slide-5
SLIDE 5

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

4

Aim

p1 p3 p4 t2 (3r) t1 (r) t3 t4 (2r) p2 p5 t5 (v)

⇒

slide-6
SLIDE 6

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

5

Petri nets

  • A simple, graphical formalism for modelling concurrent sys-

tems (including hardware, software, communication, manu- facturing, chemical and biological systems).

  • Natural expression of synchronisation.
  • Stochastic Petri net models support both:
  • correctness analysis (deadlock, liveness, boundedness)
  • performance analysis (via Markov/semi-Markov chains)

within a single modelling framework.

slide-7
SLIDE 7

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

6

Petri nets (example)

L L

photosynthesis C6H12O6 O2 metabolism CO2 H20 sunrise sunset dark light

slide-8
SLIDE 8

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

7

Petri nets (example)

L L

photosynthesis C6H12O6 O2 metabolism CO2 H20 sunrise sunset dark light

slide-9
SLIDE 9

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

8

Petri nets (example)

L L

photosynthesis C6H12O6 O2 metabolism CO2 H20 sunrise sunset dark light

slide-10
SLIDE 10

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

9

Petri nets (example)

L L

photosynthesis C6H12O6 O2 metabolism CO2 H20 sunrise sunset dark light

slide-11
SLIDE 11

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

10

Petri nets (example)

L L

photosynthesis C6H12O6 O2 metabolism CO2 H20 sunrise sunset dark light

slide-12
SLIDE 12

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

11

Petri nets (example)

L L

photosynthesis C6H12O6 O2 metabolism CO2 H20 sunrise sunset dark light

slide-13
SLIDE 13

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

12

Stochastic Petri nets

  • Attach exponentially distributed firing delays on transitions

f(t) = λe−λt F(t) = 1 − e−λt

  • Competing transitions t1 (rate λ1) and t2 (rate λ2) race:

P[t1 fires] =

∞ x

0 λ1e−λ1ydy

  • λ2e−λ2xdx =

λ1 λ1 + λ2

  • Sojourn time in marking ∼ Exp(λ1 + λ2):

P[min(χ1, χ2) ≤ t] = 1 − P[χ1 > t and χ2 > t] = 1 − e−(λ1+λ2)t

slide-14
SLIDE 14

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

13

Markov Processes

  • State holding times are exponentially distributed.
  • Transition probabilities depend only current state.
  • Hence underlying state-transition behaviour of an SPN is iso-

morphic to a Continuous Time Markov Chain.

  • Stochastic Process Algebras and closed Queueing networks

map onto Markov Chains in a similar way.

  • Response time analysis will proceed at Markov Chain level.
slide-15
SLIDE 15

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

14

CTMC (example)

L L

p1 p2 p3 (2,0,0) t2 (2) t3 (3) t1 (1)

slide-16
SLIDE 16

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

15

CTMC (example)

L L

p1 p2 p3 (2,0,0) t2 (2) t3 (3) t1 (1) (1,1,0) 1

slide-17
SLIDE 17

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

16

CTMC (example)

L L

p1 p2 p3 (2,0,0) t2 (2) t3 (3) t1 (1) (1,1,0) 1 1 (0,2,0) (1,0,1) 2

slide-18
SLIDE 18

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

17

CTMC (example)

L L

p1 p2 p3 (2,0,0) t2 (2) t3 (3) t1 (1) (1,1,0) 1 1 (0,2,0) (1,0,1) 2

slide-19
SLIDE 19

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

18

CTMC (example)

L L

p1 p2 p3 (2,0,0) t2 (2) t3 (3) t1 (1) (1,1,0) 1 1 (0,2,0) (1,0,1) 2 (0,1,1) 2

slide-20
SLIDE 20

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

19

CTMC (example)

L L

p1 p2 p3 (2,0,0) t2 (2) t3 (3) t1 (1) (1,1,0) 1 1 (0,2,0) (1,0,1) 2 (0,1,1) 2 1 3 (0,0,2) 2 3 3

slide-21
SLIDE 21

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

20

CTMC (example)

1 2 3 5 6 4 2 3 4 6 5 1 1 2 3 4 5 6

L L

p1 p2 p3 (2,0,0) t2 (2) t3 (3) t1 (1) (1,1,0) 1 1 (0,2,0) (1,0,1) 2 (0,1,1) 2 1 3 (0,0,2) 2 3 3 Q= 1 2 1 2 1 3 3 2 3 −1 −3 −2 −4 −5 −3

slide-22
SLIDE 22

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

21

CTMC (example)

2 3 5 4 2 3 4 5 2 3 4 5 1 6 6 6 1 1

L L

p1 p2 p3 (2,0,0) t2 (2) t3 (3) t1 (1) (1,1,0) 1 1 (0,2,0) (1,0,1) 2 (0,1,1) 2 1 3 (0,0,2) 2 3 3 Q= 1 2 1 2 1 3 3 2 3 −1 −3 −2 −4 −5 −3

slide-23
SLIDE 23

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

22

Laplace Transforms

  • We calculate the Laplace Transform (LT) of required re-

sponse time density and then invert it numerically.

  • Laplace Transform of f(t) is

L(s) =

∞

e−stf(t)dt

  • For f(t) = λe−λt

L(s) =

∞

λe−(λ+s)tdt = λ/(λ + s)

∞

0 (λ + s)e−(λ+s)tdt

= λ/(λ + s)

slide-24
SLIDE 24

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

23

Why the Laplace domain

λ

L L

L(s) =

  • λ

λ + s

  • ⇔ f(t) = λe−λt

Uniqueness

slide-25
SLIDE 25

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

24

Why the Laplace domain

λ

L L

µ

L(s) =

  • λ

λ + s µ µ + s

  • Convolution (easier than h(t) = f(t) ∗ g(t) =

t

0 f(α)g(t − α)dα!)

slide-26
SLIDE 26

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

25

Why the Laplace domain

λ µ

L L

(1−p) p

L(s) = p

  • λ

λ + s

  • + (1 − p)
  • µ

µ + s

  • Linearity
slide-27
SLIDE 27

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

26

Why the Laplace domain

L L

Det(d) Uniform(a,b) Immediate

L(s) =

  • e−as − e−bs

(b − a)s

  • (1)
  • e−ds

Extensible to non-exponential time delays (SMCs)

slide-28
SLIDE 28

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

27

Why the Laplace domain

  • Integration in t domain corresponds to division in Laplace

domain.

  • By inverting L(s)/s instead of L(s) we obtain cumulative

density function (cdf) cheaply. Lf(t)(s) =

∞

e−stf(t)dt = e−stF(t)

  • ∞

0 + s

∞

e−stF(t)dt = sLF(t)(s)

  • Quantiles (percentiles) can be easily calculated from cdf.
slide-29
SLIDE 29

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

28

Why the Laplace Domain

  • Calculating (raw) moments is straightforward:

L′(s) =

∞

−te−stf(t)dt L′(0) = −

∞

tf(t)dt = −E[T]

  • Higher moments can be found via L′′(0), L′′′(0) etc.

i.e. From the Laplace Transform, we can easily find the mean, variance, skewness, kurtosis etc. of f(t)

slide-30
SLIDE 30

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

29

Response Time LT

  • First step analysis (source state i, target states

j): Li

j(s)

=

  • k/

∈ j

  • qik

−qii −qii s − qii

  • Lk

j(s) +

  • k∈

j

  • qik

−qii −qii s − qii

  • =
  • k/

∈ j

  • qik

s − qii

  • Lk

j(s) +

  • k∈

j

  • qik

s − qii

  • With multiple source states:

L

  • i

j(s) =

  • k∈
  • i

αkLk

j(s)

where αk = ˜ πk

  • j∈
  • i ˜

πj

slide-31
SLIDE 31

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

30

Response time LT

  • As a system of n linear equations in Ax = b form (

j = 1):

       

s − q11 −q12 · · · −q1n s − q22 · · · −q2n −q32 · · · −q3n . . . ... . . . −qn2 · · · s − qnn

                 

L1

j(s)

L2

j(s)

L3

j(s)

. . . Ln

j(s)

         

=

       

q21 q31 . . . qn1

       

  • Symbolic solution infeasible for large n.
  • Instead solve for particular values of s based on evaluation

demands of numerical LT inversion algorithm.

slide-32
SLIDE 32

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

31

Numerical LT Inversion

  • General principle:
  • Input: values of t
  • Inverter demands: value of L(s) set for several s
  • Output: values of f(t)
  • Computational cost (values of L(s) required):
  • Euler: ≈ 30|t|, Laguerre (modified): 400
slide-33
SLIDE 33

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

32

Data Partitioning Techniques

  • Euler inversion on a 15 000 000 state model and 50 t points

requires solving 1 500 systems of 15 000 000 × 15 000 000 (complex) sparse linear equations!

  • We need to take advantage of the combined compute power

and memory capacity of a network of workstations.

  • Main opportunity for parallelisation within groups is in sparse

matrix-vector type operations used when solving linear equa- tions.

  • Need to minimise the communication between processors

while maintaining load balance.

slide-34
SLIDE 34

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

33

Hypergraph Partitioning

  • Hypergraphs are generalisations of graph data structures in

which hyperedges connect sets of vertices.

  • Hypergraphs model exact communication cost in distributed

sparse-matrix vector multiplication (unlike graphs).

  • Every system of linear equations has the same non-zero struc-

ture so hypergraph partition is reused thousands of times.

  • Parallel multilevel hypergraph partitioner for very large hy-

pergraphs now constructed! (PDSEC 2004?)

slide-35
SLIDE 35

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

34

Hypergraph Partitioning

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 13 7 16 10 15 9 1 3 14 8 11 4 2 12 5 6 P1 P2 P3 P4 13 7 16 10 15 9 1 3 14 8 11 4 2 12 5 6 x

slide-36
SLIDE 36

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

35

Semi-Markov Extensions

  • Semi-Markov processes are more expressive generalisations
  • f Markov processes
  • An n-state SMP is characterised by:
  • an n × n one-step probability matrix P
  • an n × n matrix H where hij(t) is the density function of

the sojourn time of the process in state i, given that it’s going to state j

slide-37
SLIDE 37

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

36

Semi-Markov Extensions

  • Now (SIGMETRICS 2002):

Li

j(s) =

  • k/

∈ j

pikh∗

ik(s)Lk j(s) +

  • k∈

j

pikh∗

ik(s)

  • We have developed a Semi-Markov Stochastic Petri net for-

malism whose underlying state space maps onto an SMP (PMEO-PDS 2003).

p1 p2 (t1, 2*m(p1), 1, erlang(3,2))

slide-38
SLIDE 38

DTMC Steady State Solver Distributed Laplace Transform Inverter Hypergraph Partitioner master processor LT inverter with no L(s) evaluation master disk cache filter disk cache master partitioned matrix files Enhanced DNAmaca high−level specification Generator Space State s 1 s 2 LT inverter L(s) evaluation with L(s ) 1 L(s ) n memory cache master L(s ) 2 L(s ) 2 L(s ) 1 L(s ) 1 L(s ) n memory cache master L(s ) 2 s−value work queue s−values groups slave processor

slide-39
SLIDE 39

n courier1 network delay sender application task sender session task sender transport task receiver application task receiver session task receiver transport task m p2 t2 p4 p3 p5 p6 p8 t5 p10 p9 p11 p13 p12 p16 p15 p14 p17 p20 p18 p19 t14 t13 p22 p21 t15 p23 p24 p25 p26 p27 p28 p29 t23 t24 p31 p30 p32 t22 p33 p34 t27 p35 p36 p37 t29 p38 p39 p40 p41 p42 t32 p44 p43 p45 p46 p1 courier3 courier2 courier4 network delay t1 (r7) t3 (r1) t4 (r2) t6 (r1) t7 (r8) t12 (r3) t8 (q1) t9 (q2) t11 (r5) t10 (r5) t18 (r4) t16 (r6) t17 (r6) t34 (r10) t33 (r1) t31 (r2) t30 (r1) t28 (r9) t25 (r5) t26 (r5) t19 (r3) t20 (r4) t21 (r4)

slide-40
SLIDE 40

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

39

Numerical Results I

0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.005 0.01 0.015 0.02 f(t) t numerical f(t) simulated f(t)

slide-41
SLIDE 41
slide-42
SLIDE 42

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

41

Numerical Results II

0.001 0.002 0.003 0.004 0.005 0.006 300 400 500 600 700 800 900 Probability density

  • Time

Error from 10 simulation runs of 1 billion transitions each

slide-43
SLIDE 43

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

42

Numerical Results II

0.001 0.002 0.003 0.004 0.005 0.006 300 400 500 600 700 800 900 Probability density

  • Time

Error from 10 simulation runs of 1 billion transitions each Passage time density: 15.4 million state web server model

slide-44
SLIDE 44

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

43

Numerical Results III

0.2 0.4 0.6 0.8 1 400 500 600 700 800 900 Cumulative probability

  • Time

Cumulative passage time distribution: 15.4 million state web server model

slide-45
SLIDE 45

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

44

Numerical Results III

0.2 0.4 0.6 0.8 1 400 500 600 700 800 900 Cumulative probability

  • Time

Cumulative passage time distribution: 15.4 million state web server model

slide-46
SLIDE 46

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

45

Scalability Results

p Time (s) Speedup Efficiency 1 3968.07 1.00 1.000 2 2199.98 1.80 0.902 4 1122.97 3.53 0.883 8 594.07 6.68 0.835 16 320.19 12.39 0.775 32 188.14 21.09 0.659

slide-47
SLIDE 47

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

46

Scalability Results

p Time (s) Speedup Efficiency 1 3968.07 1.00 1.000 2 2199.98 1.80 0.902 4 1122.97 3.53 0.883 8 594.07 6.68 0.835 16 320.19 12.39 0.775 32 188.14 21.09 0.659 p Time (s) Speedup Efficiency 32 × 1 150.13 26.43 0.830 16 × 2 159.55 24.87 0.777 8 × 4 162.13 24.47 0.765 4 × 8 165.24 24.01 0.750 16 × 2 173.76 22.84 0.714 1 × 32 188.14 21.09 0.659

slide-48
SLIDE 48

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

47

Selected Other Work

  • Extension to GSPNs (WOSP 2002)
  • Iterative Algorithm for Response Time Densities in Semi-

Markov Models (NSMC 2003/LAA 2004)

  • eCSL: A Stochastic Logic for Performance Queries (PNPM

2003)

  • Aggregation Strategies in Semi-Markov Models (SPECTS

2003)

slide-49
SLIDE 49

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

48

Selected Other Work

  • Response Times in Stochastic Process Algebra Models (MAS-

COTS 2003)

  • HYDRA: HYpergraph-based Distributed Response time-Analyser

(PDPTA 2003)

  • Parallel Uniformization with application to cycle times in QNs

(JPDC 2004?)

  • Efficient Approximation of Response Time Densities via mo-

ments (WOSP 2004)

slide-50
SLIDE 50

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

49

AESOP projects

  • PASTRAMI: Passage Times in Large Markov and Semi-Markov

Chains

  • EMU: Extracting performance models from UML
  • PROFORMA: Product Forms for Markovian Analysis
  • MEGAN: MM CPP/GE/C/L G-Queues and Networks
  • QUAINT: QUantitative Analysis of INternet Traffic
slide-51
SLIDE 51

Departmental Seminar wjk@doc.ic.ac.uk 10th December 2003

50

The End

  • AESOP Group web site:

http://aesop.doc.ic.ac.uk

  • These slides:

http://www.doc.ic.ac.uk/~wjk

  • Thanks to:

ICPC/LESC, EPSRC, British Council, MCASF