Tracking in a Spaghetti Bowl: Monitoring Transactions Using - - PowerPoint PPT Presentation

tracking in a spaghetti bowl monitoring transactions
SMART_READER_LITE
LIVE PREVIEW

Tracking in a Spaghetti Bowl: Monitoring Transactions Using - - PowerPoint PPT Presentation

Tracking in a Spaghetti Bowl: Monitoring Transactions Using Footprints Anima Anandkumar 1 , 2 Chatschik Bisdikian 2 Dakshi Agrawal 2 1 ECE Dept., Cornell University, Ithaca, NY 14853. 2 Networking Tech., IBM Watson Research, Hawthorne, NY 10532.


slide-1
SLIDE 1

Tracking in a Spaghetti Bowl: Monitoring Transactions Using Footprints

Anima Anandkumar1,2 Chatschik Bisdikian2 Dakshi Agrawal2

1ECE Dept., Cornell University, Ithaca, NY 14853. 2Networking Tech., IBM Watson Research, Hawthorne, NY 10532.

ACM SIGMETRICS 2008

3 June 2008, Annapolis, MD, USA Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 1 / 20

slide-2
SLIDE 2

Problem Motivation

End-to-End Service Level Transactions

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 2 / 20

slide-3
SLIDE 3

Problem Motivation

End-to-End Service Level Transactions

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 2 / 20

slide-4
SLIDE 4

Problem Motivation

End-to-End Service Level Transactions

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 2 / 20

slide-5
SLIDE 5

Problem Motivation

End-to-End Service Level Transactions

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 2 / 20

slide-6
SLIDE 6

Problem Motivation

End-to-End Service Level Transactions

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 2 / 20

slide-7
SLIDE 7

Problem Motivation

End-to-End Service Level Transactions Goals

Consider systems in absence of monitoring instrumentation Limited information available as footprints

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 2 / 20

slide-8
SLIDE 8

State-based Transaction Model

ATM transaction Application

Terminated

  • Started

Verifying information Incorrect login Offered services Performed services Completed Incorrect login again Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 3 / 20

slide-9
SLIDE 9

State-based Transaction Model

ATM transaction Application

  • Started

Verifying information Incorrect login Offered services Performed services Completed Incorrect login again Terminated

1:00:00 PM

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 3 / 20

slide-10
SLIDE 10

State-based Transaction Model

ATM transaction Application

  • Started

Verifying information Incorrect login Offered services Performed services Completed Incorrect login again Terminated

1:00:00 PM 1:00:01 PM

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 3 / 20

slide-11
SLIDE 11

State-based Transaction Model

ATM transaction Application

  • Started

Verifying information Incorrect login Offered services Performed services Completed Incorrect login again Terminated

1:00:00 PM 1:00:01 PM 1:00:02 PM

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 3 / 20

slide-12
SLIDE 12

State-based Transaction Model

ATM transaction Application

  • Started

Verifying information Incorrect login Offered services Performed services Completed Incorrect login again Terminated

1:00:00 PM 1:00:01 PM 1:00:02 PM 1:00:03 PM

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 3 / 20

slide-13
SLIDE 13

State-based Transaction Model

ATM transaction Application

  • Started

Verifying information Incorrect login Offered services Performed services Completed Incorrect login again Terminated

1:00:00 PM 1:00:01 PM 1:00:02 PM 1:00:03 PM 1:00:04 PM

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 3 / 20

slide-14
SLIDE 14

State-based Transaction Model

ATM transaction Application

Started Verifying information Incorrect login Offered services Performed services Completed Incorrect login again Terminated

1:00:00 PM 1:00:01 PM 1:00:02 PM 1:00:03 PM 1:00:04 PM 1:00:05 PM

  • Anandkumar, Bisdikian, Agrawal

Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 3 / 20

slide-15
SLIDE 15

State-based Transaction Model

ATM transaction Application

Started Verifying information Incorrect login Offered services Performed services Completed Incorrect login again Terminated

1:00:00 PM 1:00:01 PM 1:00:02 PM 1:00:03 PM 1:00:04 PM 1:00:05 PM

  • Monitoring

Footprints may not have identifiers or tokens

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 3 / 20

slide-16
SLIDE 16

State-based Transaction Model

ATM transaction Application

Started Verifying information Incorrect login Offered services Performed services Completed Incorrect login again Terminated

1:00:00 PM 1:00:01 PM 1:00:02 PM 1:00:03 PM 1:00:04 PM 1:00:05 PM

  • Monitoring

Footprints may not have identifiers or tokens Which transaction had the wrong login?

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 3 / 20

slide-17
SLIDE 17

State-based Transaction Model

ATM transaction Application

Started Verifying information Incorrect login Offered services Performed services Completed Incorrect login again Terminated

1:00:00 PM 1:00:01 PM 1:00:02 PM 1:00:03 PM 1:00:04 PM 1:00:05 PM

  • Monitoring

Footprints may not have identifiers or tokens Which transaction had the wrong login?

Probabilistic Monitoring Using Footprints Without Tokens

Maximum likelihood rule: best probabilistic guarantee

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 3 / 20

slide-18
SLIDE 18

State-based Transaction Model

ATM transaction Application

Started Verifying information Incorrect login Offered services Performed services Completed Incorrect login again Terminated

1:00:00 PM 1:00:01 PM 1:00:02 PM 1:00:03 PM 1:00:04 PM 1:00:05 PM

  • Monitoring

Footprints may not have identifiers or tokens Which transaction had the wrong login?

Probabilistic Monitoring Using Footprints Without Tokens

Maximum likelihood rule: best probabilistic guarantee

Maximum Likelihood Rule Maximize probability that each footprint is matched correctly to the unique transaction that generated it

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 3 / 20

slide-19
SLIDE 19

Problem Formulation

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Information in Footprint State & time stamp, optionally tokens (identifiers)

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 4 / 20

slide-20
SLIDE 20

Problem Formulation

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Information in Footprint State & time stamp, optionally tokens (identifiers) Real-time match: all footprints may not yet have been generated

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 4 / 20

slide-21
SLIDE 21

Problem Formulation

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Information in Footprint State & time stamp, optionally tokens (identifiers) Real-time match: all footprints may not yet have been generated

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 4 / 20

slide-22
SLIDE 22

Problem Formulation

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Information in Footprint State & time stamp, optionally tokens (identifiers) Real-time match: all footprints may not yet have been generated Problem Statement Given footprints and transaction model, find maximum likelihood match between footprints and transactions

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 4 / 20

slide-23
SLIDE 23

Problem Formulation

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Information in Footprint State & time stamp, optionally tokens (identifiers) Real-time match: all footprints may not yet have been generated Problem Statement Given footprints and transaction model, find maximum likelihood match between footprints and transactions Questions & Issues Complexity of maximum likelihood?

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 4 / 20

slide-24
SLIDE 24

Problem Formulation

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Information in Footprint State & time stamp, optionally tokens (identifiers) Real-time match: all footprints may not yet have been generated Problem Statement Given footprints and transaction model, find maximum likelihood match between footprints and transactions Questions & Issues Complexity of maximum likelihood? Decentralized implementation?

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 4 / 20

slide-25
SLIDE 25

Problem Formulation

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Information in Footprint State & time stamp, optionally tokens (identifiers) Real-time match: all footprints may not yet have been generated Problem Statement Given footprints and transaction model, find maximum likelihood match between footprints and transactions Questions & Issues Complexity of maximum likelihood? Decentralized implementation? Probabilistic bounds on accuracy?

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 4 / 20

slide-26
SLIDE 26

Problem Formulation

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Information in Footprint State & time stamp, optionally tokens (identifiers) Real-time match: all footprints may not yet have been generated Problem Statement Given footprints and transaction model, find maximum likelihood match between footprints and transactions Questions & Issues Complexity of maximum likelihood? Decentralized implementation? Probabilistic bounds on accuracy? Effect of transaction model ?

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 4 / 20

slide-27
SLIDE 27

Problem Formulation

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Transaction Model Information in Footprint State & time stamp, optionally tokens (identifiers) Real-time match: all footprints may not yet have been generated Problem Statement Given footprints and transaction model, find maximum likelihood match between footprints and transactions Questions & Issues Complexity of maximum likelihood? Decentralized implementation? Probabilistic bounds on accuracy? Effect of transaction model ?

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 4 / 20

slide-28
SLIDE 28

Problem Formulation

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Transaction Model IID transitions of transactions Information in Footprint State & time stamp, optionally tokens (identifiers) Real-time match: all footprints may not yet have been generated Problem Statement Given footprints and transaction model, find maximum likelihood match between footprints and transactions Questions & Issues Complexity of maximum likelihood? Decentralized implementation? Probabilistic bounds on accuracy? Effect of transaction model ?

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 4 / 20

slide-29
SLIDE 29

Problem Formulation

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Transaction Model IID transitions of transactions Acyclic semi-Markov process Information in Footprint State & time stamp, optionally tokens (identifiers) Real-time match: all footprints may not yet have been generated Problem Statement Given footprints and transaction model, find maximum likelihood match between footprints and transactions Questions & Issues Complexity of maximum likelihood? Decentralized implementation? Probabilistic bounds on accuracy? Effect of transaction model ?

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 4 / 20

slide-30
SLIDE 30

Summary of Results

Problem Statement

Maximum likelihood match between footprints & transactions

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 5 / 20

slide-31
SLIDE 31

Summary of Results

Problem Statement

Maximum likelihood match between footprints & transactions

Maximum Likelihood for Two-state Systems

S1 S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 5 / 20

slide-32
SLIDE 32

Summary of Results

Problem Statement

Maximum likelihood match between footprints & transactions

Maximum Likelihood for Two-state Systems

S1 S0 ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1

W (2, 2) . W (1, 2) . W (1, 1) . W (2, 1) .

Reduction to bipartite minimum weight perfect matching

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 5 / 20

slide-33
SLIDE 33

Summary of Results

Problem Statement

Maximum likelihood match between footprints & transactions

Maximum Likelihood for Two-state Systems

S1 S0 ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1

W (2, 2) . W (1, 2) . W (1, 1) . W (2, 1) .

Reduction to bipartite minimum weight perfect matching Reduction to FIFO for a class of transition-time pdfs

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 5 / 20

slide-34
SLIDE 34

Summary of Results

Problem Statement

Maximum likelihood match between footprints & transactions

Maximum Likelihood for Two-state Systems

S1 S0 ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1

W (2, 2) . W (1, 2) . W (1, 1) . W (2, 1) .

Reduction to bipartite minimum weight perfect matching Reduction to FIFO for a class of transition-time pdfs

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 5 / 20

slide-35
SLIDE 35

Summary of Results

Problem Statement

Maximum likelihood match between footprints & transactions

Maximum Likelihood for Two-state Systems

S1 S0 ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1

W (2, 2) . W (1, 2) . W (1, 1) . W (2, 1) .

Reduction to bipartite minimum weight perfect matching Reduction to FIFO for a class of transition-time pdfs

Worst Case Analysis of Maximum Likelihood Performance

Uniform and exponential transition times are worst-case distributions

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 5 / 20

slide-36
SLIDE 36

Summary of Results

Problem Statement

Maximum likelihood match between footprints & transactions

Maximum Likelihood for Two-state Systems

S1 S0 ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1

W (2, 2) . W (1, 2) . W (1, 1) . W (2, 1) .

Reduction to bipartite minimum weight perfect matching Reduction to FIFO for a class of transition-time pdfs

Worst Case Analysis of Maximum Likelihood Performance

Uniform and exponential transition times are worst-case distributions Equal to reciprocal of number of unique perfect matchings

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 5 / 20

slide-37
SLIDE 37

Summary of Results (cont.)

Maximum Likelihood for Multi-state Systems

Series of bipartite matchings in high-level two-state systems

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 6 / 20

slide-38
SLIDE 38

Summary of Results (cont.)

Maximum Likelihood for Multi-state Systems

Series of bipartite matchings in high-level two-state systems Constructive proof of optimality: Decentralized rules

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 6 / 20

slide-39
SLIDE 39

Summary of Results (cont.)

Maximum Likelihood for Multi-state Systems

Series of bipartite matchings in high-level two-state systems Constructive proof of optimality: Decentralized rules

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 6 / 20

slide-40
SLIDE 40

Summary of Results (cont.)

Maximum Likelihood for Multi-state Systems

Series of bipartite matchings in high-level two-state systems Constructive proof of optimality: Decentralized rules

1 2 3 4 5 2 1 3 5 4 ({1,3},{2,4,5}) 2 3 ({2},{3})

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 6 / 20

slide-41
SLIDE 41

Summary of Results (cont.)

Maximum Likelihood for Multi-state Systems

Series of bipartite matchings in high-level two-state systems Constructive proof of optimality: Decentralized rules

1 2 3 4 5 2 1 3 5 4 ({1,3},{2,4,5}) 2 3 ({2},{3})

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 6 / 20

slide-42
SLIDE 42

Summary of Results (cont.)

Maximum Likelihood for Multi-state Systems

Series of bipartite matchings in high-level two-state systems Constructive proof of optimality: Decentralized rules

1 2 3 4 5 2 1 3 5 4 ({1,3},{2,4,5}) 2 3 ({2},{3})

Presence of Tokens

For linear model, ML rule is still decentralized bipartite matching

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 6 / 20

slide-43
SLIDE 43

Outline

1

Introduction

2

Two-state System

3

Multi-state System

4

Conclusion

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 7 / 20

slide-44
SLIDE 44

Two-state System

S1 S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-45
SLIDE 45

Two-state System

S1 S0

2 1

S1 Time S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-46
SLIDE 46

Two-state System

S1 S0

?

1 2

S1 Time S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-47
SLIDE 47

Two-state System

S1 S0

?

1 2

S1 Time S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-48
SLIDE 48

Two-state System

S1 S0

3 1 2

S1 Time S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-49
SLIDE 49

Two-state System

S1 S0

3 1 2

S1 Time S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-50
SLIDE 50

Two-state System

S1 S0

3 1 2

S1 Time S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-51
SLIDE 51

Two-state System

S1 S0

3 1 2

S1 Time S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-52
SLIDE 52

Two-state System

S1 S0

3 1 2

S1 Time S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-53
SLIDE 53

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-54
SLIDE 54

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch Transaction 1 Y0 Y1

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-55
SLIDE 55

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch Transaction 1 Transaction 2 Y0 Y1

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-56
SLIDE 56

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch Footprint Node Transaction 1 Transaction 2 Y0 Y1

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-57
SLIDE 57

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch Footprint Node Transaction 1 Transaction 2 Y0 Y1

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-58
SLIDE 58

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch Footprint Node Transaction 1 Transaction 2 Y0 Y1 − log[fT (Y1(1) − Y0(1))] .

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-59
SLIDE 59

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch Footprint Node Transaction 1 Transaction 2 Y0 Y1 − log[fT (Y1(1) − Y0(2))] . − log[fT (Y1(1) − Y0(1))] .

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-60
SLIDE 60

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1 − log[fT (Y1(1) − Y0(2))] . − log[fT (Y1(1) − Y0(1))] .

Real Time Matching: All Footprints Not Yet Available Add virtual node: event that transaction is still resident at S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-61
SLIDE 61

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1 − log[fT (Y1(1) − Y0(2))] . − log[fT (Y1(1) − Y0(1))] .

Real Time Matching: All Footprints Not Yet Available Add virtual node: event that transaction is still resident at S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-62
SLIDE 62

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1 − log[ ¯ FT (Y1(1) − Y0(2))] − log[ ¯ FT (Y1(1) − Y0(1))] − log[fT (Y1(1) − Y0(2))] . − log[fT (Y1(1) − Y0(1))] .

Real Time Matching: All Footprints Not Yet Available Add virtual node: event that transaction is still resident at S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-63
SLIDE 63

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1 − log[ ¯ FT (Y1(1) − Y0(2))] − log[ ¯ FT (Y1(1) − Y0(1))] − log[fT (Y1(1) − Y0(2))] . − log[fT (Y1(1) − Y0(1))] .

Real Time Matching: All Footprints Not Yet Available Add virtual node: event that transaction is still resident at S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-64
SLIDE 64

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1 − log[ ¯ FT (Y1(1) − Y0(2))] − log[ ¯ FT (Y1(1) − Y0(1))] − log[fT (Y1(1) − Y0(2))] . − log[fT (Y1(1) − Y0(1))] .

Real Time Matching: All Footprints Not Yet Available Add virtual node: event that transaction is still resident at S0

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-65
SLIDE 65

Two-state System

S1 S0

3 1 2

S1 Time S0 Count Time Batch ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1 − log[ ¯ FT (Y1(1) − Y0(2))] − log[ ¯ FT (Y1(1) − Y0(1))] − log[fT (Y1(1) − Y0(2))] . − log[fT (Y1(1) − Y0(1))] .

Real Time Matching: All Footprints Not Yet Available Add virtual node: event that transaction is still resident at S0

Maximum Likelihood ≡ Minimum Weight Perfect Match

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 8 / 20

slide-66
SLIDE 66

Worst Case Maximum Likelihood Performance

Definition Minimum probability of correct match over transition time pdfs

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 9 / 20

slide-67
SLIDE 67

Worst Case Maximum Likelihood Performance

Definition Minimum probability of correct match over transition time pdfs

Worst Case ML Performance= 1

  • No. of Unique Matches

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 9 / 20

slide-68
SLIDE 68

Worst Case Maximum Likelihood Performance

Definition Minimum probability of correct match over transition time pdfs

Worst Case ML Performance= 1

  • No. of Unique Matches

ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1 Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 9 / 20

slide-69
SLIDE 69

Worst Case Maximum Likelihood Performance

Definition Minimum probability of correct match over transition time pdfs

Worst Case ML Performance= 1

  • No. of Unique Matches

ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1 Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 9 / 20

slide-70
SLIDE 70

Worst Case Maximum Likelihood Performance

Definition Minimum probability of correct match over transition time pdfs

Worst Case ML Performance= 1

  • No. of Unique Matches

ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1 Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 9 / 20

slide-71
SLIDE 71

Worst Case Maximum Likelihood Performance

Definition Minimum probability of correct match over transition time pdfs

Worst Case ML Performance= 1

  • No. of Unique Matches

ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1

  • No. of unique matches =2
  • Prob. of correct ML match = 1

2

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 9 / 20

slide-72
SLIDE 72

Worst Case Maximum Likelihood Performance

Definition Minimum probability of correct match over transition time pdfs

Worst Case ML Performance= 1

  • No. of Unique Matches

ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1

  • No. of unique matches =2
  • Prob. of correct ML match = 1

2

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 9 / 20

slide-73
SLIDE 73

Worst Case Maximum Likelihood Performance

Definition Minimum probability of correct match over transition time pdfs

Worst Case ML Performance= 1

  • No. of Unique Matches

ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1

  • No. of unique matches =2
  • Prob. of correct ML match = 1

2

Partial Batch Exponential = Worst Case

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 9 / 20

slide-74
SLIDE 74

Worst Case Maximum Likelihood Performance

Definition Minimum probability of correct match over transition time pdfs

Worst Case ML Performance= 1

  • No. of Unique Matches

ccdf Node Footprint Node Transaction 1 Transaction 2 Y0 Y1

  • No. of unique matches =2
  • Prob. of correct ML match = 1

2

Partial Batch Exponential = Worst Case Complete Batch Uniform, Exponential = Worst Case

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 9 / 20

slide-75
SLIDE 75

Comparison of Maximum Likelihood with FIFO

S0 S1 Time

FIFO = Match footprint with earliest unmatched transaction

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 10 / 20

slide-76
SLIDE 76

Comparison of Maximum Likelihood with FIFO

S0 S1 Time

FIFO = Match footprint with earliest unmatched transaction

Always valid, Distribution free, Simpler rule than maximum likelihood

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 10 / 20

slide-77
SLIDE 77

Comparison of Maximum Likelihood with FIFO

S0 S1 Time

FIFO = Match footprint with earliest unmatched transaction

Always valid, Distribution free, Simpler rule than maximum likelihood

Can FIFO and maximum likelihood coincide?

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 10 / 20

slide-78
SLIDE 78

Comparison of Maximum Likelihood with FIFO

S0 S1 Time

FIFO = Match footprint with earliest unmatched transaction

Always valid, Distribution free, Simpler rule than maximum likelihood

Can FIFO and maximum likelihood coincide?

Yes: Quadrangle Inequality

fT (t1)fT (t2) ≥ fT (t1 − τ)fT (t2 − τ)

1 2 3 4 5 0.2 0.4 0.6

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 10 / 20

slide-79
SLIDE 79

Outline

1

Introduction

2

Two-state System

3

Multi-state System

4

Conclusion

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 11 / 20

slide-80
SLIDE 80

Recap of Transaction Model

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Transaction Model Acyclic semi-Markov process Definition of Semi-Markov Process

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 12 / 20

slide-81
SLIDE 81

Recap of Transaction Model

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Transaction Model Acyclic semi-Markov process IID transitions of transactions Definition of Semi-Markov Process

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 12 / 20

slide-82
SLIDE 82

Recap of Transaction Model

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Transaction Model Acyclic semi-Markov process IID transitions of transactions Definition of Semi-Markov Process

1

Sequence of states visited is a Markov chain

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 12 / 20

slide-83
SLIDE 83

Recap of Transaction Model

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Transaction Model Acyclic semi-Markov process IID transitions of transactions Definition of Semi-Markov Process

1

Sequence of states visited is a Markov chain

2

Transition time depends only on states involved in transition

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 12 / 20

slide-84
SLIDE 84

Recap of Transaction Model

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Transaction Model Acyclic semi-Markov process IID transitions of transactions Definition of Semi-Markov Process

1

Sequence of states visited is a Markov chain

2

Transition time depends only on states involved in transition

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 12 / 20

slide-85
SLIDE 85

Recap of Transaction Model

  • S0

S1 S2 S3 S4 1 2 3 ? ? ? ? ? ? ?

Transaction Model Acyclic semi-Markov process IID transitions of transactions Definition of Semi-Markov Process

1

Sequence of states visited is a Markov chain

2

Transition time depends only on states involved in transition Maximum Likelihood Rule Maximize probability that each footprint is matched correctly to the unique transaction that generated it [ˆ πML

1 , . . . , ˆ

πML

Ns] := arg max

π1,...,πNs P(Yπ1

1 , . . . , YπNs Ns

|Y0).

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 12 / 20

slide-86
SLIDE 86

Recap of Transaction Model

  • S0

S1 S2 S3 S4 1 2 3 π4 π1 π2 π3

Transaction Model Acyclic semi-Markov process IID transitions of transactions Definition of Semi-Markov Process

1

Sequence of states visited is a Markov chain

2

Transition time depends only on states involved in transition Maximum Likelihood Rule Maximize probability that each footprint is matched correctly to the unique transaction that generated it [ˆ πML

1 , . . . , ˆ

πML

Ns] := arg max

π1,...,πNs P(Yπ1

1 , . . . , YπNs Ns

|Y0).

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 12 / 20

slide-87
SLIDE 87

Special Cases: Linear & Tree Models

Linear Model With Snapshot of Footprints

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 13 / 20

slide-88
SLIDE 88

Special Cases: Linear & Tree Models

Linear Model With Snapshot of Footprints

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Linear Model

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 13 / 20

slide-89
SLIDE 89

Special Cases: Linear & Tree Models

Linear Model With Snapshot of Footprints

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Linear Model

1 3 2 4 S0 S1 S2 S3 S4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 13 / 20

slide-90
SLIDE 90

Special Cases: Linear & Tree Models

Linear Model With Snapshot of Footprints

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Linear Model

1 3 2 4 S0 S1 S2 S3 S4 CCDF Node

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 13 / 20

slide-91
SLIDE 91

Special Cases: Linear & Tree Models

Linear Model With Snapshot of Footprints

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Linear Model

1 3 2 4 S0 S1 S2 S3 S4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 13 / 20

slide-92
SLIDE 92

Special Cases: Linear & Tree Models

Linear Model With Snapshot of Footprints

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Linear Model

1 3 2 4 S0 S1 S2 S3 S4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 13 / 20

slide-93
SLIDE 93

Special Cases: Linear & Tree Models

Linear Model With Snapshot of Footprints

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Linear Model

1 3 2 4 S0 S1 S2 S3 S4

Tree Model

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 13 / 20

slide-94
SLIDE 94

Special Cases: Linear & Tree Models

Linear Model With Snapshot of Footprints

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Linear Model

1 3 2 4 S0 S1 S2 S3 S4

Tree Model

1 3 2 5 6 7 4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 13 / 20

slide-95
SLIDE 95

Special Cases: Linear & Tree Models

Linear Model With Snapshot of Footprints

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Linear Model

1 3 2 4 S0 S1 S2 S3 S4

Tree Model

1 3 2 5 6 7 4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 13 / 20

slide-96
SLIDE 96

Special Cases: Linear & Tree Models

Linear Model With Snapshot of Footprints

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Linear Model

1 3 2 4 S0 S1 S2 S3 S4

Tree Model

1 3 2 5 6 7 4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 13 / 20

slide-97
SLIDE 97

Special Cases: Linear & Tree Models

Linear Model With Snapshot of Footprints

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Linear Model

1 3 2 4 S0 S1 S2 S3 S4

Tree Model

1 3 2 5 6 7 4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 13 / 20

slide-98
SLIDE 98

Special Cases: Linear & Tree Models

Linear Model With Snapshot of Footprints

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Linear Model

1 3 2 4 S0 S1 S2 S3 S4

Tree Model

1 3 2 5 6 7 4

ML ≡ Series of Bipartite Matchings

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 13 / 20

slide-99
SLIDE 99

Acyclic Semi-Markov Process

Acyclic Semi Markov Process: Two-Stage Systems No common imm. predecessor: States in end stage of different systems

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 14 / 20

slide-100
SLIDE 100

Acyclic Semi-Markov Process

Acyclic Semi Markov Process: Two-Stage Systems No common imm. predecessor: States in end stage of different systems

({2},{3}) 1 2 3 4 5 2 5 4 3

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 14 / 20

slide-101
SLIDE 101

Acyclic Semi-Markov Process

Acyclic Semi Markov Process: Two-Stage Systems No common imm. predecessor: States in end stage of different systems

({2},{3}) 1 2 3 4 5 2 1 5 4 3

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 14 / 20

slide-102
SLIDE 102

Acyclic Semi-Markov Process

Acyclic Semi Markov Process: Two-Stage Systems No common imm. predecessor: States in end stage of different systems

({2},{3}) 1 2 3 4 5 2 1 3 5 4 3

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 14 / 20

slide-103
SLIDE 103

Acyclic Semi-Markov Process

Acyclic Semi Markov Process: Two-Stage Systems No common imm. predecessor: States in end stage of different systems

({2},{3}) 1 2 3 4 5 2 1 3 5 4 ({1,3},{2,4,5}) 3

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 14 / 20

slide-104
SLIDE 104

Acyclic Semi-Markov Process

Acyclic Semi Markov Process: Two-Stage Systems No common imm. predecessor: States in end stage of different systems

({2},{3}) 1 2 3 4 5 2 1 3 5 4 ({1,3},{2,4,5}) 2 3

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 14 / 20

slide-105
SLIDE 105

Acyclic Semi-Markov Process

Acyclic Semi Markov Process: Two-Stage Systems No common imm. predecessor: States in end stage of different systems

1 2 3 4 5 2 1 3 5 4 ({1,3},{2,4,5}) 2 3 ({2},{3})

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 14 / 20

slide-106
SLIDE 106

Acyclic Semi-Markov Process

Acyclic Semi Markov Process: Two-Stage Systems No common imm. predecessor: States in end stage of different systems

1 2 3 4 5 2 1 3 5 4 ({1,3},{2,4,5}) 2 3 ({2},{3})

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 14 / 20

slide-107
SLIDE 107

Acyclic Semi-Markov Process

Acyclic Semi Markov Process: Two-Stage Systems No common imm. predecessor: States in end stage of different systems

1 2 3 4 5 2 1 3 5 4 ({1,3},{2,4,5}) 2 3 ({2},{3})

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 14 / 20

slide-108
SLIDE 108

Acyclic Semi-Markov Process

Acyclic Semi Markov Process: Two-Stage Systems No common imm. predecessor: States in end stage of different systems

1 2 3 4 5 2 1 3 5 4 ({1,3},{2,4,5}) 2 3 ({2},{3})

ML Rule ≡ Series of Bipartite Matchings in 2-State Systems

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 14 / 20

slide-109
SLIDE 109

Presence of Tokens

Linear Semi Markov Process

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Presence of Tokens

1 2 3 4 S0 S1 S2 S3 S4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 15 / 20

slide-110
SLIDE 110

Presence of Tokens

Linear Semi Markov Process

...

1 2 3 4

S0 S1 S3 S4 ? ? ? ?

Maximum Likelihood in Presence of Tokens

1 2 3 4 S0 S1 S2 S3 S4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 15 / 20

slide-111
SLIDE 111

Presence of Tokens

Linear Semi Markov Process

...

1 2 3 4 3

S0 S1 S3 S4 ? ? ?

Maximum Likelihood in Presence of Tokens

1 2 3 4 S0 S1 S2 S3 S4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 15 / 20

slide-112
SLIDE 112

Presence of Tokens

Linear Semi Markov Process

...

1 2 3 4 3

S0 S1 S3 S4 ? ? ?

Maximum Likelihood in Presence of Tokens

1 2 3

3

4 S0 S1 S2 S3 S4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 15 / 20

slide-113
SLIDE 113

Presence of Tokens

Linear Semi Markov Process

...

1 2 3 4 3

S0 S1 S3 S4 ? ? ?

Maximum Likelihood in Presence of Tokens

1 2 3

3

4 S0 S1 S2 S3 S4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 15 / 20

slide-114
SLIDE 114

Presence of Tokens

Linear Semi Markov Process

...

1 2 3 4 3

S0 S1 S3 S4 ? ? ?

Maximum Likelihood in Presence of Tokens

1 2 3

3

4 S0 S1 S2 S3 S4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 15 / 20

slide-115
SLIDE 115

Presence of Tokens

Linear Semi Markov Process

...

1 2 3 4 3

S0 S1 S3 S4 ? ? ?

Maximum Likelihood in Presence of Tokens

1 2 3

3

4 S0 S1 S2 S3 S4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 15 / 20

slide-116
SLIDE 116

Presence of Tokens

Linear Semi Markov Process

...

1 2 3 4 3

S0 S1 S3 S4 ? ? ?

Maximum Likelihood in Presence of Tokens

1 2 3

3

4 S0 S1 S2 S3 S4

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 15 / 20

slide-117
SLIDE 117

Presence of Tokens

Linear Semi Markov Process

...

1 2 3 4 3

S0 S1 S3 S4 ? ? ?

Maximum Likelihood in Presence of Tokens

1 2 3

3

4 S0 S1 S2 S3 S4

ML ≡ Series of Bipartite Matchings for Tokenized Linear Model

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 15 / 20

slide-118
SLIDE 118

Simulation Results

Effect of Tokens in Linear Model

1 2 3 4 5 0.2 0.4 0.6 0.8 1

Matching Accuracy Ns = No. of States - 1 No Token Token

ML vs. FIFO for 2-state System

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 16 / 20

slide-119
SLIDE 119

Simulation Results

Effect of Tokens in Linear Model

1 2 3 4 5 0.2 0.4 0.6 0.8 1

Matching Accuracy Ns = No. of States - 1 No Token Token

ML vs. FIFO for 2-state System

0.2 0.6 1 1.4 1.8 0.2 0.4 0.6 0.8 1

Matching Accuracy Load factor ML (actual pdf) FIFO (pdf-free) ML (est. pdf) Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 16 / 20

slide-120
SLIDE 120

Outline

1

Introduction

2

Two-state System

3

Multi-state System

4

Conclusion

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 17 / 20

slide-121
SLIDE 121

Conclusion

Summary

End-to-end monitoring of transactions using footprints Optimal maximum likelihood rule for matching footprints to transactions Reduction of ML rule to bipartite matching for two-state systems Reduction of ML rule to a series of bipartite matching for multi-state systems

Outlook

General transaction models e.g., higher order Markov, petri-nets Relaxation of assumptions

◮ Perfect knowledge of transaction model ◮ Missing footprints, lack of synchronization Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 18 / 20

slide-122
SLIDE 122

Related Work

Identification of Global System States

Classical work by Chandy & Lamport (85) Do not deal with monitoring individual transaction instances

Whitebox/Tokenized Methods (Chen et al. 04, Schmid et al. 07)

Require industry standards like Open Group ARM instrumentation Not applicable if such instrumentation is not available

Blackbox Methods (Aguilera et al. 03, Liu et al. 07)

Do not require token generating instrumentation Do not deal with monitoring individual transaction instances

Discovery of Transaction Model

Extensive work in this area (Agrawal, Gunopulos, Leymann 98)

Related Publication (Sengupta, Banerjee, Anandkumar, Bisdikian NOMS 08)

Implementation of monitoring system, experimental results on response times Bounds on aggregate number of transactions in any state of the model

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 19 / 20

slide-123
SLIDE 123

Thank You !

Anandkumar, Bisdikian, Agrawal Monitoring Transactions Using Footprints ACM SIGMETRICS ‘08 20 / 20