ITLTracer: Runtime Verification of Properties expressed in ITL Dr. - - PowerPoint PPT Presentation

ITLTracer: Runtime Verification of Properties expressed in ITL

• Dr. Helge Janicke (heljanic@dmu.ac.uk)

De Montfort University

Version: 906913f9b356(September 30, 2010) heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

1 / 29

Introduction

Runtime Verification

Definition (Runtime Verification) Combines formal verification and Program Execution. Detects faults by passively observing system behaviour during normal

• peration.

Validates that the system under test (SuT) satisfies formally expressed requirements (e.g. in Interval Temporal Logic (ITL)). Pros Testing in the real world! No (little) overhead during development. Reactive Systems. Cons Failure Recovery. Performance overhead. Not a substitute for rigorous testing.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

2 / 29

Introduction

Runtime Verification in Context

When is Runtime Verification useful? Information only available at run-time. Dependent on environment (reactive system). (security) critical system.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

3 / 29

Introduction

Runtime Verification in Context

When is Runtime Verification useful? Information only available at run-time. Dependent on environment (reactive system). (security) critical system.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

3 / 29

Introduction

Problem Statement

Currently we use AnaTempura for Runtime Verification. AnaTempura uses information obtained from the system via assertion points as input to Tempura which computes a trace. Properties are then evaluated as state invariants1. Tempura does not allow properties to contain negation of formulae, non-deterministic choice or implication, as these are not executable. (David’s Functional Programming implementation may be an alternative!) Properties, such as SANTA Policies, use some of these, which made their encoding cumbersome and lead to restrictions in the language.

1This including references to next or previous states heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

4 / 29

Introduction

Contribution and Applications

Semantics based on partitions instead of states. This allows for a more efficient evaluation of formulae containing sequential compositions (Chop). Efficient encoding of partitions. Traces are stored in a compacted form saving space and access time. Prototype of a Runtime Monitor (pure Java). See www.tech.dmu.ac.uk/~heljanic Applications include: Runtime Verification, where temporal assertions are needed. SANTA Policy Enforcement. Formal Log-File/ Attack analysis. ...

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

5 / 29

Framework Overview

Motivation

Observation Most programs are sequential, or contain large parts that are executed sequentially. In a sequential program only few variables can change at a time (typically only one). Depending on the interpretation of states this means that only very few variables are changed from one state to the next. Storing all variable/value mappings for each state can be expensive and it is more sensible to store only the changes. This is what partitions do ...

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

6 / 29

Semantics

Definition (Partitioning) Let a partitioning partx(σ) for a variable x ∈ Var be a nonempty, finite sequence of tuples (l0, r0), (l1, r1), ..., (ln, rn) where: l0 = 0 start at 0 rn = |σ| end with length of sigma ∀0≤i≤n · li ≤ ri

• rder

∀0≤i<n · ri + 1 = li+1 progression ∀0≤i≤n · ∀li≤t≤ri · σl(x) = σt(x) constant mapping Example

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

7 / 29

Semantics

Definition (Partitioning) Let a partitioning partx(σ) for a variable x ∈ Var be a nonempty, finite sequence of tuples (l0, r0), (l1, r1), ..., (ln, rn) where: l0 = 0 start at 0 rn = |σ| end with length of sigma ∀0≤i≤n · li ≤ ri

• rder

∀0≤i<n · ri + 1 = li+1 progression ∀0≤i≤n · ∀li≤t≤ri · σl(x) = σt(x) constant mapping Example grouping of states

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

7 / 29

Semantics

Definition (Partitioning) Let a partitioning partx(σ) for a variable x ∈ Var be a nonempty, finite sequence of tuples (l0, r0), (l1, r1), ..., (ln, rn) where: l0 = 0 start at 0 rn = |σ| end with length of sigma ∀0≤i≤n · li ≤ ri

• rder

∀0≤i<n · ri + 1 = li+1 progression ∀0≤i≤n · ∀li≤t≤ri · σl(x) = σt(x) constant mapping Example partitioning

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

7 / 29

Semantics

Definition (Partitioning) Let a partitioning partx(σ) for a variable x ∈ Var be a nonempty, finite sequence of tuples (l0, r0), (l1, r1), ..., (ln, rn) where: l0 = 0 start at 0 rn = |σ| end with length of sigma ∀0≤i≤n · li ≤ ri

• rder

∀0≤i<n · ri + 1 = li+1 progression ∀0≤i≤n · ∀li≤t≤ri · σl(x) = σt(x) constant mapping Example alternative partitioning

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

7 / 29

Semantics

Definition (Partition) Let δx map from any given partition in the partition interval of partx(σ) and variable x ∈ Var to the set of values (Val). δx : partx(σ) → Val such that: ∀l≤t≤r · δx(l, r) = σt(x) Definition (Unrolling) We define unrol(δx) as a function: unrol(δx) = λi ·                if (l0 ≤ i ≤ r0 → δx(l0, r0) [] (l1 ≤ i ≤ r1 → δx(l1, r1) ... [] (lk ≤ i ≤ rk → δx(lk, rk) fi

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

8 / 29

Semantics

Definition (Equivalence) Two partitions are equivalent δx ≈ δ′

x iff unrol(δx) = unrol(δ′ x)

δx(0, 1) = 0 δx(2, 4) = 1 δx(5, 5) = 0 δ′

x(0, 1) = 0

δ′

x(2, 3) = 1

δ′

x(4, 4) = 1

δ′

x(5, 5) = 0

The partitions contain sufficient information to reconstruct the underlying σ via unrolling. Equivalence with respect to a variable x is established by checking if both underlying intervals σ and σ′ are identical w.r.t. the variable x.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

9 / 29

Semantics

Definition (Aligned) We say any two partitions partx(σ) and party(σ) are aligned iff: |partx(σ)| = |party(σ)| ∀0≤i≤|partx(σ)| · (li, ri) ∈ partx ≡ (li, ri) ∈ party(σ) Example Not aligned!

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

10 / 29

Semantics

Definition (Aligned) We say any two partitions partx(σ) and party(σ) are aligned iff: |partx(σ)| = |party(σ)| ∀0≤i≤|partx(σ)| · (li, ri) ∈ partx ≡ (li, ri) ∈ party(σ) Example Aligned.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

10 / 29

Semantics

Definition Let the partition interval δ be defined as a tuple of all partitions. δ = δx0, δx1, ..., δxn where xi ∈ Var. With the restriction that all partitions are of equal length: ∀xi∈Var · len(δx0) = len(δxi) where len(δx) = rx

n is the right index of the last tuple in the partitioning.

We also use len(δ) to denote the length of the partition interval. We refer to ∆x(σ) as the set of all possible partitionings for the variable x. Correspondingly to ∆ for the set of all possible partition intervals.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

11 / 29

Semantics

The syntax below: e ::= g(e1, ..., en) | fin v | v | v | µ f ::= p(e1, ..., en) | f1 ; f2 | f1∗ | skip | f1 ∧ f2 | ¬ f1

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

12 / 29

Semantics

Semantics of Expressions

The semantics of expressions is defined by induction over the syntax: E[ [v] ](δ) = δv(0, rv

0 )

E[ [g(e1, ..., en)] ](δ) = ˆ g(E[ [e1] ](δ), ..., E[ [en] ](δ)) E[ [ fin v] ](δ) = δv(lv

n , rv n )

E[ [ v] ](δ) =      χ(Val) if rv

n = 0

δv(0, rv

0 )

if rv

0 > 0

δv(1, rv

1 )

• therwise

E[ [µ] ](δ) = µ Where lv

i and rv i are the left element and the right element of the i’s tuple

in the partition partv(σ).

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

13 / 29

Semantics

Semantics of Formulae

The semantics of formulae is defined by induction over the syntax: δ f1 ; f2 iff ∃δ′ ∈ ∆ · δ′ ≈ δ and δ′ is aligned. And δ′[0, k] f1 and δ′[k, |δ′|] f2 for some k, 0 ≤ k ≤ |δ′| δ f1 ∧ f2 iff δ f1 and δ f2 δ ¬ f iff σ f δ skip iff len(δ) equal to 1 δ empty iff len(δ) equal to 0 δ p(e1, ..., en) iff ˆ p(E[ [e1] ](δ), ..., E[ [en] ](δ)) δ f ∗ iff δ empty or δ f ∗ ; f The complexity of the chop is now reduced from being proportional to the interval length len(δ) to the number of partitions |δ|.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

14 / 29

Tool-Support

Tool-Support

Pure Java implementation javacc for parsing and rewriting. Java interface as well as stdin option. Evaluation of properties on demand.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

15 / 29

Tool-Support

Keeping a Trace

Variables States ... ... 1 2 3 4 5 x y z 1 1 1 1 1 1 Figure: Trace Variables ... x y z 0@0 1@2 0@5 1@0 0@2 0@0 1@5 Figure: Partitioned Trace

Instead of recording the values of variables in each state only the changes are recorded. Implicitly a variable keeps its value if it is not changed. The implementation keeps record of rn the interval length an thus only needs to store the li values of the tuples together with the value as the all other ri’s can be computed.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

16 / 29

Tool-Support

UML Classdiagram: Trace Recording

DataStore<T>

• state:long = 0L
• variables:HashMap<String,VariableStore<T»

+ add(id:String, value:T) + advanceState(delta:int) + get(state:long, id:String) : T + getInv(state:long, id:String, inv:long[]):long[] + getState():long VariableStore<T>

• data:List<DataEntry<T»
• lastpivot:long

+ (final) varid:String + VariableStore(varid: String) + add(state:long, value:T) + get(state:Long):T + getDataEntries():List<DataEntry<T» + getEntry(state:Long):DataEntry<T> + getEntryIndex(state:long):int + getInv(state:long,inv:long[]):long[] DataEntry + (final) state : long + (final) value : Object + DataEntry(state: long, value: Object) uk.ac.dmu.heljanic.satchk 0..n contains ID -> Variable mappings contains state -> value mappings Checker

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

17 / 29

Tool-Support

Unrolling and Partitions

The unrol function is implemented by the method get(state:long):T which returns the value for a given state by identifying the corresponding partition using an adapted binary search. The function uses a cursor, so that repeated calls for close states result in constant algorithmic complexity. The function getInv(state:long, inv:long[]):long[] implements the partitioning. For any state it returns the corresponding partition element. Data is recorded using the add(state:long, value:T) function that records the new variable value for the state. The class DataStore collates all Variable stores and handles the identifier mappings. It also is responsible for keeping the overall interval length.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

18 / 29

Tool-Support

Alignment

When evaluating a “chop”, the partitions must be aligned. The alignments are computed “on the fly”, ie. no restructuring of the recorded trace takes place. This is achieved through the efficient implementation of the unrol function. Only those partitions of variables that are evaluated as part of the “chop” are aligned. The partitioning of other variables cannot affect the outcome. Example true ; (x = 1 ∧ y = 3)

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

19 / 29

Tool-Support

Tool Interface

ITLTracer can be used interactively, via the stdin/stdout interface. This allows ITLTracer to trace almost any program and is also useful for testing and verification of recorded traces (e.g. log analysis) Starting the tool

$ java uk.ac.dmu.heljanic.itltracer.ITLTracer This is ITLTracer $Revision: 906913f9b356 $ (c) 2010 H. Janicke itlt:ESCAPE <escapecode> ... set the escape code for assertions. default = itlt: escape [x=v] [@s] ... records value v for variable x in state s. Omitting x=v advance only the state to s. Omitting @s records for the last set state. itlt:EXIT ... quit. itlt:PRINTSIGMA ... prints the recorded trace (sigma). itlt:PRINTDELTA ... prints the recorded trace (delta). itlt:CHECK <property> ... checks the <property> on the trace.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

20 / 29

Examples

Recording a Trace

To record a trace, we set the escape sequence that is recognised by ITLTracer (default “itlt:”). Any string received on stdin that is prefixed by the escape sequence will be interpreted as an assertion point. The example on the right is a sequence that constructs the interval used in the previous examples. Illegal inputs Attempting to record a variable with a different value in the same state or attempts to reduce the current state will result in an exception. Example

itlt:ESCAPE x=0 y=1 @2 x=1 y=2 y=3 @4 x=0 @5 y=7

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

21 / 29

Examples

Displaying Trace

ITLTrace can also output the trace to stdout in two formats: itlt:PRINTDELTA This command will output the partition interval δ. itlt:PRINTSIGMA This command will reconstruct the underlying interval σ using the unrol function. Example (itlt:PRINTSIGMA)

SIGMA= y= 1 x= 1 y= 1 x= 2 y= 2 x= 1 3 y= 2 x= 1 4 y= 3 x= 1 5 y= 7 x=

Example (itlt:PRINTDELTA)

DELTA= DELTA(y) (0, 1) -> 1 (2, 3) -> 2 (4, 4) -> 3 (5, 5) -> 7 DELTA(x) (0, 1) -> 0 (2, 4) -> 1 (5, 5) -> 0

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

22 / 29

Examples

Checking a Property

ITLTracer can check a property using the check command: itlt:CHECK some suffix {x=1 and y=2} ITLTracer only processes the primitive syntax and derived operators such as “some suffix” (viz. ♦) are expanded during the parsing: Example (Rewriting ♦f with true ; f (prefix notation))

rewriting DIAMOND node IN = Diamond(And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2]))) OUT = Chop(BLit[true], And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2]))) parsed successfully

Once reduced to primitive operators the property can be evaluated against the trace.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

23 / 29

Examples

Checking a Property (step by step)

DELTA= DELTA(y) (0, 1) -> 1 (2, 3) -> 2 (4, 4) -> 3 (5, 5) -> 7 DELTA(x) (0, 1) -> 0 (2, 4) -> 1 (5, 5) -> 0 Processing: l=0 r=5 f=Chop(BLit[true], And(Eq(Id[x], ILit[1]), Eq(Id[y], chop: k=5 Processing: l=5 r=5 f=And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2])) Processing: l=5 r=5 f=Eq(Id[x], ILit[1]) Processing: l=5 r=5 f=Id[x] return= 0 : [5, 5, 5, 5] Processing: l=5 r=5 f=ILit[1] return= 1 : [0, 5, 5, 5] return= false : [5, 5, 5, 5] return= false : [5, 5, 5, 5] chop: k=4 Processing: l=4 r=5 f=And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2])) Processing: l=4 r=5 f=Eq(Id[x], ILit[1]) Processing: l=4 r=5 f=Id[x] return= 1 : [2, 4, 4, 5] Processing: l=4 r=5 f=ILit[1] return= 1 : [0, 5, 4, 5] return= true : [2, 4, 4, 5] Processing: l=4 r=5 f=Eq(Id[y], ILit[2])...

Evaluation is backwards and uses left and right indicies instead of partition indicies, to avoid repartitioning. In this case a k in the partition (2,3) evaluates to true.

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

24 / 29

Conclusion and Future Work

Conclusions

ITLTracer checks ITL properties against a (recorded or streamed) system trace. Overcomes (executable) restriction of AnaTempura, viz. a wider range

• f properties can be used.

Partitions can reduce algorithmic complexity, as evaluation of chop is O(|δ|) rather than O(|σ|). Checks can be initiated by the target program. The checking of properties can be reduced to specific assertion points within the target program. Prototype implementation is available at www.tech.dmu.ac.uk/~heljanic

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

25 / 29

Conclusion and Future Work

Applications

Runtime Verification, where temporal properties are important. Policy Based Management: History Based Access Control Policies can be enforced. (Without refinement to executable form!) Compliance Checking:

Lean Engineering, Standard Operations. Management Applications: Business Process Monitoring. Service Oriented Computing: Contract Monitoring.

Incident Response and Forensic Analysis: Checking for known malicious behaviours (properties) against log files (system trace) Intrusion Detection Systems: Checking for known behaviour patterns (properties) in network traffic or access patterns (trace). Trust in Collaborative Systems: Establishing trust based on benevolent and malicious behaviour patterns. Archaeology (Properties of Data Analysis Processes).

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

26 / 29

Conclusion and Future Work

Future Work (I)

Reuse results from previous checks: σ = σ0...σn σ′ = σ0...σnσn+1

Reuse known prefix results. σ f ; true = ⇒ σ′ f ; true σ f ; true and σ′ f = ⇒ σ′ f ; true σ f ; true and σ′ f = ⇒ σ′ f ; true Reuse suffix results (state formulae only) σ true ; w = ⇒ σ′ true ; w σ true ; w and σn+1 w = ⇒ σ′ true ; w σ true ; w and σn+1 w = ⇒ σ′ true ; w

By annotating the formulae with the last evaluation result and reference to δ (including the computed partitioning).

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

27 / 29

Conclusion and Future Work

Future Work (II)

Extend supported Data Types (All immutable Java Objects can be supported). Allow query calls to stored objects in the trace (query means that calling the function is free of side-effects) Locking mechanism: Properties require only certain variables and based on previous result evaluations part of the trace becomes irrelevant (ie. σ f ; true = ⇒ σ′ f ; true) therefore any future query will return true, thus the record of σ can be released (if no

• ther property requires this part of the trace).

Temporal Formulae compute partitions based on invariants (currently expressed on σ). This will have to be converted to the new partition semantics (δ). testing, publications...

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

28 / 29

Conclusion and Future Work

Questions? Any Comments or Questions?

• Dr. Helge Janicke

Software Technology Research Laboratory Faculty of Technology E: heljanic@dmu.ac.uk De Montfort University P: ++44 (0) 116 257 7617 Bede Island 1.16 W: www.tech.dmu.ac.uk/~heljanic LE1 9BH Leicester, UK

heljanic@dmu.ac.uk (DMU) ITLTracer

906913f9b356

29 / 29