The RV System Tutorial Patrick Meredith and Grigore Rosu joint - - PowerPoint PPT Presentation

▶
the rv system tutorial
SMART_READER_LITE
LIVE PREVIEW

The RV System Tutorial Patrick Meredith and Grigore Rosu joint - - PowerPoint PPT Presentation

Oct 09, 2023 46 likes •1.29k views

The RV System Tutorial Patrick Meredith and Grigore Rosu joint work with Feng Chen, Dongyun Jin, Dennis Gri ffj th, Michael Ilseman Runtime Verification, Inc. University of Illinois Wednesday, November 10, 2010 The RV System

slide-1
SLIDE 1

The RV System Tutorial

Patrick Meredith and Grigore Rosu joint work with Feng Chen, Dongyun Jin, Dennis Griffjth, Michael Ilseman Runtime Verification, Inc. University of Illinois

Wednesday, November 10, 2010

slide-2
SLIDE 2

The RV System

  • Combines Runtime Monitoring and

Predictive Analysis

2

!"#$%&'(%) !"#*)+,'-(

.(%/'-'(01 "'%23('%&1 4+(+-('%& 5+&+)'-1*)%6+)(0 4+(+-('%& !3-+14+(+-('%& 7&8()9/+&(3('%& :398321;2'-'&< =%<<'&< !9&('/+ $%&'(%)'&<1 *)+,'-('%&1 =%<<'&<1.86+-( *)+,'-('%&1 $%&'(%)1='>)3)0 1*)%<)3/1 ?&,+)1@+8( *3)3/+()'- ;2'-'&<1

  • Fig. 1. System Overview

New Version of JavaMOP New Version of jPredictor

Wednesday, November 10, 2010

slide-3
SLIDE 3

Overview

  • Monitoring
  • RV-Monitor Demo
  • RV-Monitor Techniques and Implementation

– Monitor Synthesis – Parametric Monitoring – Optimizations

  • Prediction
  • RV-Predict Demo
  • RV-Predict Techniques and Implementation

– Sliced Causality – Pipeline – Race Prediction

Wednesday, November 10, 2010

slide-4
SLIDE 4

Why Monitoring

  • Monitoring is well-adopted in many

engineering disciplines

– Fuses, watchdogs, fire-alarms, etc.

  • Monitoring adds redundancy

– Increases reliability, robustness and confidence in correct behavior, reduces risk

  • Provably correct systems can fail, too

– Unexpected environment, wrong/strong assumptions, hardware or OS errors, etc.

Wednesday, November 10, 2010

slide-5
SLIDE 5

Applications of Monitoring

  • Debugging

– Development – Error messages

  • Testing

– Development – Error messages

  • Security/Reliability/Robustness/…

– Production – Recovery mechanisms

  • Programming Paradigm

– Production – General actions

Wednesday, November 10, 2010

slide-6
SLIDE 6

Runtime Monitoring Systems

(a few of them)

  • ≤ 2001

– MAC (UPenn), PAX (NASA), TimeRover (commercial)

  • 2002-2004

– HAWK/Eagle (NASA), MOP (UIUC), POTA (UTA)

  • ≥ 2005:

– PQL (Stanford) – Tracematches (Oxford) – PTQL (Berkeley/Stanford/Novell) – Pal (UPenn) – RuleR (Manchester) – … many others

Wednesday, November 10, 2010

slide-7
SLIDE 7

FailFast Iterator

Vector v = new Vector(); Iterator i = v.iterator(); v.add(new Integer(2)); while (i.hasNext()) …

  • Following code throws exception in Java (FailFast):
  • FailFast: if the underlying vector is changed when an iterator

is used for enumerating elements, the iterator fails.

  • However …

Wednesday, November 10, 2010

slide-8
SLIDE 8

MOP Example: Safe Enumeration

  • No exception raised if one uses

Enumeration instead of Iterator

– Java language decision, showing that

SafeEnum(Vector v, Enumeration+ e) { event create after(Vector v) returning(Enumeration e): ... event updatesource after(Vector v) : ... event next before(Enumeration e) : ... ere : create next* updatesource+ next @match { System.out.println(“Failed Enumeration!"); } }

> 250 AspectJ LOC generated …

Wednesday, November 10, 2010

slide-9
SLIDE 9

Complexity of SafeEnum

  • Tricky to check SafeEnum manually

– Two counters needed, one in the vector (the current timestamp) and the other in the enumeration (the vector’s timestamp when creating the enumeration). – Accesses to vector / enumerator can be scattered all over the code, both in program and in libraries – Accesses to vector’s counter must be synchronized – Every implementation of the Enumeration interface should repeat the above work

Wednesday, November 10, 2010

slide-10
SLIDE 10
  • Bug found in jHotDraw
  • The underlying vector should not be

changed when one of its enumerations is being used!

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

Example of Unsafe Enumeration

Wednesday, November 10, 2010

slide-11
SLIDE 11
  • Bug found in jHotDraw
  • The underlying vector should not be

changed when one of its enumerations is being used!

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

Example of Unsafe Enumeration

May cause unexpected behaviors, e.g., a NoSuchElement Exception

  • r some errors later.

Wednesday, November 10, 2010

slide-12
SLIDE 12

RV-Monitor (Based on Monitoring Oriented Programming - MOP)

http://fsl.cs.uiuc.edu/mop

A generic runtime verification framework

  • proposed in 2003 –

RV’03, ICFEM’04, RV’05, CAV’05, TACAS’05, CAV’06, CAV’07, OOPSLA’07, ASE’08, RTSS’08, AOSD’08,TACAS’09, …

Wednesday, November 10, 2010

slide-13
SLIDE 13

What RV-Monitor Supports

  • Observe a run of a system

– Requires instrumentation – Can be offmine or online

  • Check it against desired properties

– Specified using patterns or in a logical formalism

  • React/Report (if needed)

– Error messages – Recovery mechanisms – General code

Wednesday, November 10, 2010

slide-14
SLIDE 14

RV-Monitor Model

Program Execution

Wednesday, November 10, 2010

slide-15
SLIDE 15

RV-Monitor Model

Program Execution Abstract Trace

Observation/Abstraction

Wednesday, November 10, 2010

slide-16
SLIDE 16

RV-Monitor Model

Program Execution Abstract Trace

M1 M2 M3

…

Monitors

Observation/Abstraction Verification

Wednesday, November 10, 2010

slide-17
SLIDE 17

RV-Monitor Model

Program Execution Abstract Trace

M1 M2 M3

…

Monitors

Observation/Abstraction Verification Action Action

Wednesday, November 10, 2010

slide-18
SLIDE 18

RV-Monitor Model

Program Execution Abstract Trace

M1 M2 M3

…

Monitors

Observation/Abstraction Verification Action Action

Monitors verify abstract traces against desired properties; can be dynamically created or destroyed

Wednesday, November 10, 2010

slide-19
SLIDE 19

Overview

  • Monitoring
  • RV-Monitor Demo
  • RV-Monitor Techniques and Implementation

– Monitor Synthesis – Parametric Monitoring – Optimizations

  • Prediction
  • RV-Predict Demo
  • RV-Predict Techniques and Implementation

– Sliced Causality – Pipeline – Race Prediction

Wednesday, November 10, 2010

slide-20
SLIDE 20

Overview

  • Monitoring
  • RV-Monitor Demo
  • RV-Monitor Techniques and Implementation

– Monitor Synthesis – Parametric Monitoring – Optimizations

  • Prediction
  • RV-Predict Demo
  • RV-Predict Techniques and Implementation

– Sliced Causality – Pipeline – Race Prediction

Wednesday, November 10, 2010

slide-21
SLIDE 21

Monitor Synthesis

Program Execution Abstract Trace

M1 M2 M3

…

Monitors

Observation/Abstraction Verification Action Action

Wednesday, November 10, 2010

slide-22
SLIDE 22

Monitor Synthesis

Program Execution Abstract Trace

M1 M2 M3

…

Monitors

Observation/Abstraction Verification Action Action

How do we generate effjcient monitors?

Wednesday, November 10, 2010

slide-23
SLIDE 23

MOP: Extensible Logic Framework

  • Generic in specification formalisms
  • Logic plugin: plugable monitor synthesis

components for difgerent logics

  • Already provides plugins for many logics

– FSM (Finite State Machine), ERE (extended regular expressions), PTLTL (Past-time LTL), FTLTL (Future-time LTL), ATL (Allen temporal logic), JML (Java modeling language), PtCaRet (Past-time Call/Return), CFG (Context-free grammars), …

  • The desired property can be arbitrarily

complex: Raw specifications

Wednesday, November 10, 2010

slide-24
SLIDE 24

FSM Plugin Finite State Machine

  • Easy to use, yet powerful
  • Many approaches/users encode

important properties directly in finite state machines, e.g., Typestates

  • Monitoring FSM

– Direct translation from an FSM specification to a monitor

Wednesday, November 10, 2010

slide-25
SLIDE 25

ERE Plugin Extended Regular Expressions

  • Regular expressions

– Widely used in programming, easy to master for ordinary programmers – Existing monitor synthesis algorithm

  • Extended regular expressions

– Extend regular exps with complement (negation) – Specify properties non-elementarily more compactly – More complicated to monitor

Wednesday, November 10, 2010

slide-26
SLIDE 26

LTL Plugins Linear Temporal Logic

  • MOP includes both a past-time plugin

(PTLTL) and an over-all LTL plugin for LTL

  • PTLTL uses a dynamic programming

algorithm, low resources, suitable for hardware

  • LTL uses a translation through

alternating automata. Semantics of past is difgerent than PTLTL

Wednesday, November 10, 2010

slide-27
SLIDE 27
  • Most systems support finite state

monitors

– Regular languages – Linear temporal logics

  • These cannot monitor structured

properties:

21

CFG Plugins Context-Free Grammar

Wednesday, November 10, 2010

slide-28
SLIDE 28
  • Most systems support finite state

monitors

– Regular languages – Linear temporal logics

  • These cannot monitor structured

properties:

21

CFG Plugins Context-Free Grammar

Wednesday, November 10, 2010

slide-29
SLIDE 29

22

GLR Parsing Yields CFG Monitors

  • Reads input Left to right, produces Right-

most derivation; table driven

  • Bottom-up parsing

– keeps stack with the current, and previous states

  • Effjcient
  • Handles all context-free grammars, even

those with ambiguity

  • Makes it a good candidate for CFG monitor

synthesis!

Wednesday, November 10, 2010

slide-30
SLIDE 30

Overview

  • Monitoring
  • RV-Monitor Demo
  • RV-Monitor Techniques and Implementation

– Monitor Synthesis – Parametric Monitoring – Optimizations

  • Prediction
  • RV-Predict Demo
  • RV-Predict Techniques and Implementation

– Sliced Causality – Pipeline – Race Prediction

Wednesday, November 10, 2010

slide-31
SLIDE 31

MOP Monitoring Model

Program Execution Abstract Trace

M1 M2 M3

…

Monitors

Observation/Abstraction Verification Action Action

Monitors can be dynamically created or destroyed Parametric monitoring

Wednesday, November 10, 2010

slide-32
SLIDE 32

Parametric Properties

Needed, but hard to monitor effjciently

SafeEnum(Vector v, Enumeration+ e) { event create after(Vector v) returning(Enumeration e): ... event updatesource after(Vector v) : ... event next before(Enumeration e) : ... ere : create next* updatesource+ next @match { System.out.println(“Failed Enumeration!"); } }

Parameters

Wednesday, November 10, 2010

slide-33
SLIDE 33

Safe Enumeration as Parametric Property

Usage pattern (using regular expressions) of three events updatesource(v) : change vector v create(v,e) : create enumeration e from vector v next(e) : use enumeration e

1 2

create updatesource updatesource next next

3 Violation state Monitor

Wednesday, November 10, 2010

slide-34
SLIDE 34

Monitoring Safe Enum

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

…

1 2

create updatesource updatesource next next

3

Wednesday, November 10, 2010

slide-35
SLIDE 35

Monitoring Safe Enum

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

…

1 2

create updatesource updatesource next next

3

Wednesday, November 10, 2010

slide-36
SLIDE 36

Monitoring Safe Enum

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

create

…

1 2

create updatesource updatesource next next

3

Wednesday, November 10, 2010

slide-37
SLIDE 37

Monitoring Safe Enum

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

create

…

1 2

create updatesource updatesource next next

3

Wednesday, November 10, 2010

slide-38
SLIDE 38

Monitoring Safe Enum

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

create

…

1 2

create updatesource updatesource next next

3

Wednesday, November 10, 2010

slide-39
SLIDE 39

Monitoring Safe Enum

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

create updatesource

…

1 2

create updatesource updatesource next next

3

Wednesday, November 10, 2010

slide-40
SLIDE 40

Monitoring Safe Enum

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

create updatesource

…

1 2

create updatesource updatesource next next

3

Wednesday, November 10, 2010

slide-41
SLIDE 41

Monitoring Safe Enum

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

create updatesource next

…

1 2

create updatesource updatesource next next

3

Wednesday, November 10, 2010

slide-42
SLIDE 42

Monitoring Safe Enum

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

create updatesource next

…

1 2

create updatesource updatesource next next

3

Wednesday, November 10, 2010

slide-43
SLIDE 43

Lack of Parameters Leads to False Alarms

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

…

Wednesday, November 10, 2010

slide-44
SLIDE 44

Lack of Parameters Leads to False Alarms

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

…

v2.remove(0);

Wednesday, November 10, 2010

slide-45
SLIDE 45

Lack of Parameters Leads to False Alarms

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

create updatesource next

…

v2.remove(0);

Wednesday, November 10, 2010

slide-46
SLIDE 46

Lack of Parameters Leads to False Alarms

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v.remove(0);

…

create updatesource next

…

Appear to be a violation but it is not; false alarm!

v2.remove(0);

Wednesday, November 10, 2010

slide-47
SLIDE 47

Adding Parameters to Events

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v2.remove(0);

…

Wednesday, November 10, 2010

slide-48
SLIDE 48

Adding Parameters to Events

create(v, e) update(v2) next(e)

… …

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v2.remove(0);

…

Wednesday, November 10, 2010

slide-49
SLIDE 49

Adding Parameters to Events

create(v, e) update(v2) next(e)

… …

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v2.remove(0);

…

update(v) v.remove(0);

Wednesday, November 10, 2010

slide-50
SLIDE 50

Adding Parameters to Events

create(v, e) update(v2) next(e)

… …

Parametric traces: traces containing events with parameters; Abundant in practice, especially in object-oriented programs

Main Thread: Vector v = //initialization;

…

Enumeration e = v.elements();

…

Object obj = e.nextElement();

…

Task Thread:

…

v2.remove(0);

…

update(v) v.remove(0);

Wednesday, November 10, 2010

slide-51
SLIDE 51

Checking Parametric Traces

Wednesday, November 10, 2010

slide-52
SLIDE 52

Checking Parametric Traces

updatesource(v1) create (v1,e1) updatesource(v2) next(e1) create(v1,e2) updatesource(v1) next(e1)

parametric trace

Wednesday, November 10, 2010

slide-53
SLIDE 53

Checking Parametric Traces

updatesource(v1) create (v1,e1) updatesource(v2) next(e1) create(v1,e2) updatesource(v1) next(e1)

parametric trace 1 2

create updatesource updatesource next next

3 non-parametric monitor

Wednesday, November 10, 2010

slide-54
SLIDE 54

Checking Parametric Traces

updatesource(v1) create (v1,e1) updatesource(v2) next(e1) create(v1,e2) updatesource(v1) next(e1)

parametric trace 1 2

create updatesource updatesource next next

3 non-parametric monitor

Wednesday, November 10, 2010

slide-55
SLIDE 55

Checking Parametric Traces

updatesource(v1) create (v1,e1) updatesource(v2) next(e1) create(v1,e2) updatesource(v1) next(e1)

parametric trace 1 2

create updatesource updatesource next next

3

parametric monitor

Wednesday, November 10, 2010

slide-56
SLIDE 56

Parametric Monitors

  • Other approaches: Monolithic (centralized)

monitors

– Tracematches [Oxford], Program Query Language (PQL) [Stanford], Eagle [NASA], etc. – Bound to specific formalisms/checking mechanisms – Limited expressiveness, specific to application domains

  • Our solution: decentralized monitors

– Formalism-independent, works with any formalism

  • More expressive, adaptive to difgerent domains

– Facilitates optimization (separation of concerns)

  • Evaluation shows better performance

Wednesday, November 10, 2010

slide-57
SLIDE 57

next

Parametric Trace Slicing

updatesource updatesource(v1) create updatesource(v2) next create(v1,e2) updatesource next(e1) create (v1,e1)

next(e1)

For given parameters (v, e)

updatesource(v1)

Wednesday, November 10, 2010

slide-58
SLIDE 58

next

Parametric Trace Slicing

updatesource updatesource(v1) create updatesource(v2) next create(v1,e2) updatesource next(e1) v1, e1 v1, e2 v2, e1 v2, e2 create (v1,e1)

next(e1)

For given parameters (v, e)

updatesource(v1)

Wednesday, November 10, 2010

slide-59
SLIDE 59

next

Parametric Trace Slicing

updatesource updatesource(v1) create updatesource(v2) next create(v1,e2) updatesource next(e1) v1, e1 v1, e2 v2, e1 create (v1,e1)

next(e1)

For given parameters (v, e)

updatesource(v1)

Wednesday, November 10, 2010

slide-60
SLIDE 60

next

Parametric Trace Slicing

updatesource updatesource(v1) create updatesource(v2) next create(v1,e2) updatesource next(e1) v1, e1 v1, e2 v2, e1 create (v1,e1)

next(e1)

For given parameters (v, e)

updatesource(v1)

Wednesday, November 10, 2010

slide-61
SLIDE 61

next

Parametric Trace Slicing

updatesource updatesource(v1) create updatesource(v2) next create(v1,e2) updatesource next(e1) v1, e1 v1, e2 v2, e1 create (v1,e1)

next(e1)

For given parameters (v, e) trace slice

updatesource(v1)

Wednesday, November 10, 2010

slide-62
SLIDE 62

next

Parametric Trace Slicing

updatesource updatesource(v1) create updatesource(v2) next create(v1,e2) updatesource next(e1) v1, e1 v1, e2 v2, e1 create (v1,e1)

next(e1)

updatesource create updatesource next next

For given parameters (v, e) trace slice

updatesource(v1)

Wednesday, November 10, 2010

slide-63
SLIDE 63

Naive monitoring of Parametric Traces

  • Every parametric trace contains multiple non-

parametric trace slices, each corresponding to a particular parameter binding

1 2

create updatesource updatesource next next

3 1 2

create updatesource updatesource next next

3

Wednesday, November 10, 2010

slide-64
SLIDE 64

Naive monitoring of Parametric Traces

  • Every parametric trace contains multiple non-

parametric trace slices, each corresponding to a particular parameter binding

v1, e1 v1, e2

1 2

create updatesource updatesource next next

3 1 2

create updatesource updatesource next next

3

Wednesday, November 10, 2010

slide-65
SLIDE 65

Parametric Trace Slicing - Challenges

update(v1) createEnum(v1,e1) update(v2) useEnum(e1) createEnum(v1,e2) update(v1) useEnum(e1) v1, e1 v1, e2 v2, e1 update(v1) createEnum(v1,e1) useEnum(e1) update(v1) useEnum(e1) update createEnum useEnum update useEnum update createEnum update useEnum useEnum

For given parameters (v, e)

Wednesday, November 10, 2010

slide-66
SLIDE 66

Parametric Trace Slicing - Challenges

update(v1) createEnum(v1,e1) update(v2) useEnum(e1) createEnum(v1,e2) update(v1) useEnum(e1) v1, e1 v1, e2 v2, e1 update(v1) createEnum(v1,e1) useEnum(e1) update(v1) useEnum(e1) update createEnum useEnum update useEnum update createEnum update useEnum useEnum

How to do it efficiently?

For given parameters (v, e)

Wednesday, November 10, 2010

slide-67
SLIDE 67

Parametric Trace Slicing - Challenges

update(v1) createEnum(v1,e1) update(v2) useEnum(e1) createEnum(v1,e2) update(v1) useEnum(e1) v1, e1 v1, e2 v2, e1 update(v1) createEnum(v1,e1) useEnum(e1) update(v1) useEnum(e1) update createEnum useEnum update useEnum update createEnum update useEnum useEnum

How to do it efficiently?

For given parameters (v, e)

What if the trace is not complete?

Wednesday, November 10, 2010

slide-68
SLIDE 68
  • Online: process events as receiving

them and do not look back for the previous events

  • Effjcient

– Scan the trace once – Events discarded immediately after being processed

  • What information should be kept for

the unknown future?

Online Parametric Trace Slicing

Wednesday, November 10, 2010

slide-69
SLIDE 69

Overview

  • Monitoring
  • RV-Monitor Demo
  • RV-Monitor Techniques and Implementation

– Monitor Synthesis – Parametric Monitoring – Optimizations

  • Prediction
  • RV-Predict Demo
  • RV-Predict Techniques and Implementation

– Sliced Causality – Pipeline – Race Prediction

Wednesday, November 10, 2010

slide-70
SLIDE 70

Slicing Example

For given parameters (v, e)

Wednesday, November 10, 2010

slide-71
SLIDE 71

Slicing Example

update(v1) v1 update update

For given parameters (v, e)

Wednesday, November 10, 2010

slide-72
SLIDE 72

Slicing Example

update(v1) createEnum(v1,e1) v1 v1, e1 update update

For given parameters (v, e)

Wednesday, November 10, 2010

slide-73
SLIDE 73

Slicing Example

update(v1) createEnum(v1,e1) v1 v1, e1 update update

For given parameters (v, e)

Wednesday, November 10, 2010

slide-74
SLIDE 74

Slicing Example

update(v1) createEnum(v1,e1) v1 v1, e1 update update createEnum

For given parameters (v, e)

Wednesday, November 10, 2010

slide-75
SLIDE 75

Slicing Example

update(v1) createEnum(v1,e1) update(v2) v1 v1, e1 v2 update update createEnum update

For given parameters (v, e)

Wednesday, November 10, 2010

slide-76
SLIDE 76

Slicing Example

update(v1) createEnum(v1,e1) update(v2) useEnum(e1) v1 v1, e1 v2 update update createEnum update update e1 useEnum

For given parameters (v, e)

Wednesday, November 10, 2010

slide-77
SLIDE 77

Slicing Example

update(v1) createEnum(v1,e1) update(v2) useEnum(e1) v1 v1, e1 v2 update update createEnum update update useEnum e1 useEnum

For given parameters (v, e)

Wednesday, November 10, 2010

slide-78
SLIDE 78

Slicing Example

update(v1) createEnum(v1,e1) update(v2) useEnum(e1) v2, e1 v1 v1, e1 v2 update update createEnum update update useEnum e1 useEnum

For given parameters (v, e)

Wednesday, November 10, 2010

slide-79
SLIDE 79

Slicing Example

update(v1) createEnum(v1,e1) update(v2) useEnum(e1) v2, e1 v1 v1, e1 v2 update update createEnum update update useEnum useEnum e1 useEnum

For given parameters (v, e)

Wednesday, November 10, 2010

slide-80
SLIDE 80

Slicing Example

update(v1) createEnum(v1,e1) update(v2) useEnum(e1) createEnum(v1,e2) v2, e1 v1 v1, e1 v1, e2 v2 update update createEnum update update update createEnum useEnum useEnum e1 useEnum

For given parameters (v, e)

Wednesday, November 10, 2010

slide-81
SLIDE 81

Slicing Example

update(v1) createEnum(v1,e1) update(v2) useEnum(e1) createEnum(v1,e2) v2, e1 v1 v1, e1 v1, e2 v2 update update createEnum update update update createEnum useEnum useEnum e1 useEnum

For given parameters (v, e)

Wednesday, November 10, 2010

slide-82
SLIDE 82

Slicing Example

update(v1) createEnum(v1,e1) update(v2) useEnum(e1) createEnum(v1,e2) v2, e1 v1 v1, e1 v1, e2 v2 update update createEnum update update update createEnum useEnum useEnum e1 useEnum

For given parameters (v, e) Optimization: based on static property analysis, generate specialized slicing code for the given specification

Wednesday, November 10, 2010

slide-83
SLIDE 83

Slicing Example

update(v1) createEnum(v1,e1) update(v2) useEnum(e1) createEnum(v1,e2) v1 v1, e1 v1, e2 v2 update update createEnum update update createEnum useEnum

For given parameters (v, e) Optimization: based on static property analysis, generate specialized slicing code for the given specification

Wednesday, November 10, 2010

slide-84
SLIDE 84

RV-Monitor Performance

38

Comparison of Tracematches (TM), JavaMOP (MOP), and RV: Average percent runtime overhead HasNext UnsafeIter Unsafe- MapIter Unsafe- SyncColl Unsafe- SyncMap All Prop TM MOP RV TM MOP RV TM MOP RV TM MOP RV TM MOP RV RV antlr 1 4

  • 2

3

  • 2

3 3 1

  • 1
  • 1
  • 1
  • 2

bloat 2119 448 116 19194 569 251 OOM 1203 178 1359 746 212 1942 716 130 982 chart 1

  • 2

15 2

  • 1

1

  • 2
  • 2
  • 2
  • 1
  • 2
  • 2
  • 2

eclipse 1

  • 4
  • 2

1

  • 5
  • 4
  • 5
  • 3
  • 5
  • 4
  • 5
  • 5
  • 2
  • 3
  • 3

fop 2 4

  • 2

4 7

  • 1

9 7

  • 2

1

  • 2
  • 2
  • 1
  • 3
  • 1

1 hsqldb 15

  • 3

13

  • 1
  • 3

13 1

  • 3

9

  • 4
  • 2

7

  • 3
  • 3
  • 3

jython 13 11 1 150 18 3 11 1 1 10 4 luindex

  • 7

1

  • 1

4

  • 2
  • 1

3

  • 1
  • 1

2

  • 1

2 12 lusearch 3

  • 1
  • 2

22 1 2 7

  • 7

3

  • 6

5 4 3 pmd 70 26

  • 1

207 12 5 OOM 181 56 40 13 2 58 17

  • 1

69 xalan 5 1

  • 1

16 4 5 5 7

  • 1
  • 2

7

  • 1

1

  • Fig. 6. Comparison of Tracematches (TM), JavaMOP (MOP), and RV:

Wednesday, November 10, 2010

slide-85
SLIDE 85

Overview

  • Monitoring
  • RV-Monitor Demo
  • RV-Monitor Techniques and Implementation

– Monitor Synthesis – Parametric Monitoring – Optimizations

  • Prediction
  • RV-Predict Demo
  • RV-Predict Techniques and Implementation

– Sliced Causality – Pipeline – Race Prediction

Wednesday, November 10, 2010

slide-86
SLIDE 86

Why Prediction

  • Concurrent programs are hard to

analyze

– Model checking: number of interleavings is prohibitively large – Testing: interleavings depend on environment

  • Combine dynamic and static methods

to find bad behaviors near correct executions

40 Wednesday, November 10, 2010

slide-87
SLIDE 87

Our Solution

  • Sliced Causality

– General purpose technique to predict (bad) behaviors from correct runs – Sound: No false alarms

  • RV-Predict

– Tool implementing Sliced Causality – Allows for prediction of any property for which an algorithm exists – Better than tools specialized simply for data race or atomicity violations

41 Wednesday, November 10, 2010

slide-88
SLIDE 88

Prediction Example

42

Task Thread: … … s3: if (! flag.value) Thread.yield() ; s4: resource.access(); … Main Thread: s1: resource.authenticate(); s2: flag.value = true; … Observed execution: … s1 s2 s3 s4 …

Property: “authenticate before access”

while (! flag.value)

Wednesday, November 10, 2010

slide-89
SLIDE 89

Prediction Example

  • Buggy S4 can be executed before S1
  • Low possibility to hit error in testing

43

Task Thread: … … s3: if (! flag.value) Thread.yield() ; s4: resource.access(); … Main Thread: s1: resource.authenticate(); s2: flag.value = true; … Observed execution: … s1 s2 s3 s4 …

Property: “authenticate before access”

Wednesday, November 10, 2010

slide-90
SLIDE 90

Prediction Example

  • Buggy S4 can be executed before S1
  • Low possibility to hit error in testing

43

Task Thread: … … s3: if (! flag.value) Thread.yield() ; s4: resource.access(); … Main Thread: s1: resource.authenticate(); s2: flag.value = true; … Observed execution: … s1 s2 s3 s4 …

Property: “authenticate before access”

Can we predict the error even when the above execution is observed? Yes! But not in the traditional way

Wednesday, November 10, 2010

slide-91
SLIDE 91

Special Case: Data Races

  • Our techniques work for any

behavioral property

  • One of the simplest properties is

race detection

– Two accesses to a shared variable can take place concurrently – At least one of the accesses is a write

44 Wednesday, November 10, 2010

slide-92
SLIDE 92

Overview

  • Monitoring
  • RV-Monitor Demo
  • RV-Monitor Techniques and Implementation

– Monitor Synthesis – Parametric Monitoring – Optimizations

  • Prediction
  • RV-Predict Demo
  • RV-Predict Techniques and Implementation

– Sliced Causality – Pipeline – Race Prediction

Wednesday, November 10, 2010

slide-93
SLIDE 93

Overview

  • Monitoring
  • RV-Monitor Demo
  • RV-Monitor Techniques and Implementation

– Monitor Synthesis – Parametric Monitoring – Optimizations

  • Prediction
  • RV-Predict Demo
  • RV-Predict Techniques and Implementation

– Sliced Causality – Pipeline – Race Prediction

Wednesday, November 10, 2010

slide-94
SLIDE 94

Predictive Runtime Analysis

47

Search space

Wednesday, November 10, 2010

slide-95
SLIDE 95

Search space Observed execution

Predictive Runtime Analysis

48 Wednesday, November 10, 2010

slide-96
SLIDE 96

Search space Observed execution Causal model

Predictive Runtime Analysis

49 Wednesday, November 10, 2010

slide-97
SLIDE 97

Search space Observed execution Causal model Inferred executions Bug

Predictive Runtime Analysis

50 Wednesday, November 10, 2010

slide-98
SLIDE 98

Search space Observed execution Causal model Inferred executions Bug

Predictive Runtime Analysis

50

More relaxed causal model yields more inferred executions

Wednesday, November 10, 2010

slide-99
SLIDE 99
  • Originally for distributed systems

[Lamport-78]

– Applied to shared memory systems by several authors

  • Causal model = non-permutable pairs of

events

– a – Causal dependency: if two events access the same location and one writes it, then their execution order matters

  • Inferred executions = extending the

causal model

Traditional Predictive Runtime Analysis: Happens-Before

51

= {intra-thread total orders} U {causal dependencies}

Wednesday, November 10, 2010

slide-100
SLIDE 100

Happens-Before Works... If Lucky

52

Main Thread: s1: resource.authenticate() s2: flag.value = true; Task Thread: s3: if (! flag.value) Thread.yield() ; s4: resource.access(); Observed execution: s3 s1 s2 s4

Property: “authenticate before access”

Wednesday, November 10, 2010

slide-101
SLIDE 101

Main Thread: s1: resource.authenticate() s2: flag.value = true; Task Thread: s3: if (! flag.value) Thread.yield() ; s4: resource.access(); Causal dependency: s3 < s2 Observed execution: s3 s1 s2 s4

Property: “authenticate before access”

Bad execution inferred: s3 s4 s1 s2. Bug detected! Chances of observing this execution are very low

Happens-Before Works... If Lucky

53 Wednesday, November 10, 2010

slide-102
SLIDE 102

Happens-Before Limitations

54

Main Thread: s1: resource.authenticate() s2: flag.value = true; Task Thread: s3: if (! flag.value) Thread.yield() ; s4: resource.access(); Causal dependency: s2 < s3. No bug found … Observed execution: s1 s2 s3 s4

Property: “authenticate before access”

Too constrained: access will be performed regardless of the flag

Wednesday, November 10, 2010

slide-103
SLIDE 103

Sliced Causality

  • Relaxes the Happens-Before causal model

– Formally proved in [chen-rosu-07]

  • How? Focus on the property
  • Use static information about the program
  • Remove events and causalities irrelevant

to the property

– Smaller and more relaxed causal model – (Exponentially) more inferred executions – Better predictive capability

55 Wednesday, November 10, 2010

slide-104
SLIDE 104

Sliced Causality

  • Start with those events relevant to the

property

  • Add events on which they are control

dependent (transitively, intrathread)

  • Add events on which they are data

dependent (transitively, interthread)

56 Wednesday, November 10, 2010

slide-105
SLIDE 105

Static Information: Control Scope

  • S2 is in the control scope of S1 if its

execution depends on a choice at S1

  • Extends to other control statements

– break/continue, return, exceptions

57

s1: if (flag) { s2: ... } else { s3: ... } s4: ... s0: i=0; s1: while (i<3) { s2: ... s3: i++ } s4: ... s1: while (!flag) { s2: ... } s3: ...

Wednesday, November 10, 2010

slide-106
SLIDE 106

Static Information: Control Scope

  • S2 is in the control scope of S1 if its

execution depends on a choice at S1

  • Extends to other control statements

– break/continue, return, exceptions

57

s1: if (flag) { s2: ... } else { s3: ... } s4: ... s0: i=0; s1: while (i<3) { s2: ... s3: i++ } s4: ... s1: while (!flag) { s2: ... } s3: ...

s1: if (flag) { s2: ... } else { s3: ... } s4: ...

Wednesday, November 10, 2010

slide-107
SLIDE 107

Static Information: Control Scope

  • S2 is in the control scope of S1 if its

execution depends on a choice at S1

  • Extends to other control statements

– break/continue, return, exceptions

57

s1: if (flag) { s2: ... } else { s3: ... } s4: ... s0: i=0; s1: while (i<3) { s2: ... s3: i++ } s4: ... s1: while (!flag) { s2: ... } s3: ...

s1: if (flag) { s2: ... } else { s3: ... } s4: ...

s0: i=0; s1: while (i<3) { s2: ... s3: i++ } s4: ...

Wednesday, November 10, 2010

slide-108
SLIDE 108

Static Information: Control Scope

  • S2 is in the control scope of S1 if its

execution depends on a choice at S1

  • Extends to other control statements

– break/continue, return, exceptions

57

s1: if (flag) { s2: ... } else { s3: ... } s4: ... s0: i=0; s1: while (i<3) { s2: ... s3: i++ } s4: ... s1: while (!flag) { s2: ... } s3: ...

s1: if (flag) { s2: ... } else { s3: ... } s4: ...

s0: i=0; s1: while (i<3) { s2: ... s3: i++ } s4: ...

s1: while (!flag) { s2: ... } s3: ...

Wednesday, November 10, 2010

slide-109
SLIDE 109

Slice Causality Works!

58

Main Thread: s1: resource.authenticate() s2: flag.value = true; Task Thread: s3: if (! flag.value) Thread.yield() ; s4: resource.access(); Observed execution: s1 s2 s3 s4

Property: “authenticate before access”

Only s1 and s4 directly relevant to the property

Wednesday, November 10, 2010

slide-110
SLIDE 110

Slice Causality Works!

59

Main Thread: s1: resource.authenticate() s2: flag.value = true; Task Thread: s3: if (! flag.value) Thread.yield() ; s4: resource.access(); s4 s1 is a potential execution. Bug detected! Observed execution: s1 s2 s3 s4 Sliced causality: s1 <> s4; Execution of s4 not dependent of s3; ignore the causal dependency s2 < s3

Property: “authenticate before access”

Only s1 and s4 directly relevant to the property

Wednesday, November 10, 2010

slide-111
SLIDE 111
  • Main Thread:

s1: resource.authenticate() s2: flag.value = true; Task Thread: s3: while (! flag.value) Thread.yield(); s4: resource.access(); Observed execution: s1 s2 s3 s4

Property: “authenticate before access”

No False Alarms ☺

60 Wednesday, November 10, 2010

slide-112
SLIDE 112
  • Main Thread:

s1: resource.authenticate() s2: flag.value = true; Task Thread: s3: while (! flag.value) Thread.yield(); s4: resource.access(); Sliced causality: s1 <s2 < s3 < s4, no false alarm! Observed execution: s1 s2 s3 s4 Execution of s4 depends on flag.value being true at s3 causal dependency s2 < s3 matters

Property: “authenticate before access”

No False Alarms ☺

61 Wednesday, November 10, 2010

slide-113
SLIDE 113

Overview

  • Monitoring
  • RV-Monitor Demo
  • RV-Monitor Techniques and Implementation

– Monitor Synthesis – Parametric Monitoring – Optimizations

  • Prediction
  • RV-Predict Demo
  • RV-Predict Techniques and Implementation

– Sliced Causality – Pipeline – Race Prediction

Wednesday, November 10, 2010

slide-114
SLIDE 114

RV-Predict Pipeline

63

jPredictor: Filling the box

Original Program Property Predicted Violations:

Counter-Examples

Wednesday, November 10, 2010

slide-115
SLIDE 115

jPredictor: Filling the box

Static Analyzer Original Program Property Predicted Violations:

Counter-Examples

RV-Predict Pipeline

64 Wednesday, November 10, 2010

slide-116
SLIDE 116

jPredictor: Filling the box

Static Analyzer Original Program Instrumented Program Structural Information Property Predicted Violations:

Counter-Examples

RV-Predict Pipeline

65 Wednesday, November 10, 2010

slide-117
SLIDE 117

jPredictor: Filling the box

Static Analyzer JVM Preprocessor Original Program Instrumented Program Recorded Trace Structural Information Property Complete Trace Predicted Violations:

Counter-Examples

RV-Predict Pipeline

66 Wednesday, November 10, 2010

slide-118
SLIDE 118

jPredictor: Filling the box

Static Analyzer JVM Preprocessor Original Program Instrumented Program Recorded Trace Structural Information Property Trace Slicer Complete Trace Sliced Trace Predicted Violations:

Counter-Examples

RV-Predict Pipeline

67 Wednesday, November 10, 2010

slide-119
SLIDE 119

jPredictor: Filling the box

Static Analyzer JVM Preprocessor Original Program Instrumented Program Recorded Trace Structural Information Property Trace Slicer Vector Clock Calculator Complete Trace Sliced Trace Causal Model Predicted Violations:

Counter-Examples

RV-Predict Pipeline

68 Wednesday, November 10, 2010

slide-120
SLIDE 120

jPredictor: Filling the box

Static Analyzer JVM Preprocessor Property Checker Original Program Instrumented Program Recorded Trace Structural Information Property Trace Slicer Vector Clock Calculator Complete Trace Sliced Trace Causal Model Predicted Violations:

Counter-Examples

RV-Predict Pipeline

69 Wednesday, November 10, 2010

slide-121
SLIDE 121

Overview

  • Monitoring
  • RV-Monitor Demo
  • RV-Monitor Techniques and Implementation

– Monitor Synthesis – Parametric Monitoring – Optimizations

  • Prediction
  • RV-Predict Demo
  • RV-Predict Techniques and Implementation

– Sliced Causality – Pipeline – Race Prediction

Wednesday, November 10, 2010

slide-122
SLIDE 122

Data Race Prediction

  • Consider all pairs of accesses in the trace

– We actually do something smarter

  • Check if either access is a write

– We are not worried about read-read races

  • Check if they have incomparable VCs

– Incomparable VCs means accesses could be reordered

  • If they have difgerent lock sets then race

found

71 Wednesday, November 10, 2010

slide-123
SLIDE 123

RV-Predict Performance

72

jPredictor RV-Predict Name Input Real Time Disk Usage Real Time Disk Usage account

  • 0:02.57

236K 0:06.07 364K elevator

  • 5:55.29

63M 1:20.31 864K tsp map4 2 4:24.44 16M 1:45.22 744K tsp map5 2 8:12.31 17M 2:45.28 868K tsp map10 2 > 3 hours > 230M 33:45.32 2.8M huge

  • crash

crash 0:42.22 13M medium

  • crash

crash 0:06.12 840K small

  • crash

crash 0:05.99 292K mixedlockshuge

  • > 2 hours

> 250M 0:05.68 2.9M mixedlocksbig

  • 4:39.08

25M 0:05.68 496K mixedlocksmedium

  • 0:08.92

2.7M 0:07.25 308K mixedlockssmall

  • 0:05.46

1.5M 0:05.67 296K jPredictor vs. RV-Predict

Wednesday, November 10, 2010

slide-124
SLIDE 124

Conclusion

  • RV-Monitor is a generic yet effjcient
  • monitoring system

– Extensible logic framework: FSM, ERE, PTLTL, FTLTL, LTL, CFG, PTCaRet, …

  • RV-Predict provides very effjcient

causal predict of generic properties

– Race detection, atomicity violations, monitoring properties

Wednesday, November 10, 2010