A practical introduction to active automata learning Bernhard - - PowerPoint PPT Presentation

Maik Merten, learning technology 1

A practical introduction to active automata learning

SFM2011

Bernhard Steffen, Falk Howar, Maik Merten TU Dortmund

Overview

• Motivation
• Introduction to active automata learning
• Practical aspects in active automata

learning

• Conclusions

learner connector try to use

X

inform about new service and device interrogate interrogate learn look for known models some service CONNECT environment

Connect Scenario

Learning in CONNECT

Develop techniques for learning … models of … behavior of networked peers and middleware through exploratory interaction…

WP4 Learning Enabler

Rich & abstract models:

• Data parameters & state

variables

• Pre- and post-conditions
• Non-functional properties

Interface descriptions Semantic information on

• Data domains
• Data dependencies
• Effects

Metrics of interest Counterexamples through monitoring

Overview

• Motivation
• Introduction to active automata learning
• Practical aspects in active automata

learning

• Conclusions

Mealy machines

 Mealy machine M=(S,, ,,)

 S finite set of states   finite input-alphabet   finite output-alphabet  : (S x ) → S transition-function  : (S x ) →  output-function

• Words * for (sS, a, w*)

 :(S x *) → S, (s,)=s, (s,aw) =((s,a),w)  : (S x *) → *, (s,)=, (s,aw) = (s,a).((s,a),w)

Passive learning or learning with traces

 tape-record communication  Create observation tree  Construct automaton without contradiction

a/0 a/1 b/1 a/0 b/0 a/0 b/1 a/0 b/0 b/1 b/0 a/0 a/0 b/1 a/1 b/0 b/1 a/0 a/0 b/0 b/0 a/0

Passive learning or learning with traces

a/0 a/1 b/1 a/0 b/0 a/0 b/1 a/0 b/0 b/1 b/0 a/0 "equivalent" states (no contradiction) a/0 b/1 a/1 b/0 b/1 a/0 a/0 b/0 b/0 a/0

Passive learning or learning with traces

a/0 a/1 b/1 a/0 b/0 a/0 b/1 a/0 b/0 b/1 b/0 a/0

Passive learning or learning with traces

a/0 a/1 b/1 a/0 b/0 a/0 b/1 a/0 b/0 b/1 b/0 a/0 a/0 a/1 b/1 b/0 b/0 a/0

Passive learning or learning with traces

Observations

 The relation "not in conflict" is very weak:

 Reflexive, symmetric, but not transitive!  `Not in conflict´ clusters typically overlap  The relation contain various eqivalence relations  Computing the best choice of equivalence is:  Expensive for criteria like state minimality  Impossible in terms of adequacy for the problem.

Active automata learning

Idea 1: Ask where infomation is incomplete!

• This requires an active testing mechanism:

 Membership Queries: Check the reaction of the

system to input sequences.

• Checking all inputs at all positions makes `not in

conflict´ an equivalence relation.

Angluin's algorithm

• Consequence:
• The underlying tree is homogeneous in the sense

that all nodes treat the same set of inputs.

• As the not in conflict relation is now an equivalence

relation, the corresponding clustering is unique

• Problem:
• The clustered graph may be non-deterministic in

general

Angluin's algorithm

Idea 2: Enforce consistency!

• Refine the ‚not in conflict‘ relation
• Also consider whether the target of the transitions
• f each cluster are unique for each input
• I.e.: consider the largest congruence wrt. The Transition

relation inside the not in conflict relation).

This yields determinism!

Angluin's algorithm

Consequence:

• Clustering yields an (input) deterministic graph /model)
• The projective quotient model of a consistent and

homogeneous abstraction) This simplifies the situation a lot: Termination Lemma 1 Given some execution tree, realizing the two ideas via Membership Queries provides a closed, consistent , and deterministic Hypothesis Model (Quality? Termination?)

Angluin's algorithm

Idea 3: Introduce qualitative termination!

 Equivalence Queries: Check for equivalence with

the target system, and produce a distinguishing test in case of failure.

• Conceptually a nice idea that leads to a very

elegant correctness proof.

• Practically typically not implementable.

Active automata learning

Learner MQ-Oracle EQ-Oracle Σ={a,b} a  L? no ? no, bb  L!

a a a a b b b b a a,b b

(queries) should word w be included in L(A)? (conjectures) here is an A – is L(A) = U? yes / no yes! no: word w should (not) be in L(A) the oracle L* learner

Angluin's alg. for Mealy machines

a b ε S a b

 Initialize

Distinguishing Set D with alphabet of inputs

state cover set

• ne transition

extensions SA distingushing set

D

a b ε a b Unknown system:

1 1 1 1

Angluin's alg. for Mealy machines

a b ε a 1 1 b 1 1

 Unclosure:

Rows in lower part that are not in upper part

Angluin's alg. for Mealy machines

a b ε a 1 1 b 1 1 aa 1 1 ab 1 1

 Unclosure:

Rows in lower part that are not in upper part

Angluin's alg. for Mealy machines

a b ε a 1 1 b 1 1 aa 1 1 ab 1 1

 Conjecture:  Unique rows in S become

states

 Rows in S and SA

become transitions

Angluin's alg. for Mealy machines

 Counterexample:

bbb / 010

a b ε a 1 1 b 1 1 aa 1 1 ab 1 1

Angluin's alg. for Mealy machines

a b ε a 1 1 b 1 1 bb aa 1 1 ab 1 1 ba … … … bbb

Angluin's alg. for Mealy machines

 Counterexample:

bbb / 010

 Insert all prefixes of the

counterexample to upper part

 Extend SA accordingly

a b ε a 1 1 b 1 1 bb aa 1 1 ab 1 1 ba … … … bbb

 Inconsistency:  Equal rows in upper

part have ‚different extensions‘

00 00 00

Angluin's alg. for Mealy machines

a b ε a 1 1 b 1 1 bb aa 1 1 ab 1 1 ba … … … bbb

 Inconsistency:  Equal rows in upper

part have ‚different extensions‘

 b and bbb differ, e.g.,

for suffix b => ε and bb will differ for suffix bb

Angluin's alg. for Mealy machines

 Inconsistencies lead

to new columns

Angluin's alg. for Mealy machines

a b bb ε 1 a 1 1 1 b 1 1 bb aa 1 1 1 ab 1 1 1 ba 1 … … … 1 bbb …

New Conjecture

Target System Learned System

Angluin's alg. for Mealy machines

Summarized Observations (1)

 Systematic completition of the observation table  New states arise as targets of transitions or from

counter examples of the equivalence queries. Technically: prefixes are added to S

 Closure procedure extends SA  Consistency is enforced by enlarging the

Distinguishing Set D

Angluin's algorithm

Hypothesis models or conjectures:

• Closed and consistent models (projective

quotients) of the so far expanded

• homogeously extended, and
• consistent

execution tree.

Summarized Observations (2)

Invariance Lemma:

 All hypothesis models are

 totally defined: each input is considered at each state,  input deterministic: there is only one transition per input at

each state,

 transition covered: each transition lies on a path of the original

system,

 state minimal: two different states in a hypothesis model

always have a separating future – á la Nerode).

Myhill–Nerode

Nerode relation:



For language L define relation RL (for u,u‘  Σ*)

u RL u‘ ↔ for all v  Σ*: (uv  L ↔ uv  L)

Myhill-Nerode Theorem:



Minimal number of states of an accepting deterministic automaton equals the number of equivalence classes of RL

Summarized Observations (4)

This (Nerode‘s theorem) directly yields:

 Corollary: Hypothesis automata have at most as

many states as the smallest deterministic equivalent automaton.

• We will denote the number of states by n.

• Lemma: The number of states of the hypothesis

model increases in response to a counterexample.

• Theorem: Angluin´s algorithm terminates after at

most n equivalence queries with the smallest deterministic system representing the behaviour

• f the system to be learned.

Summarized Observations (4)

Equivalence Queries At most |Q| Membership Queries At most O(m |Q| |ΣA| ) per EQ (m = length of max. counter example)

• Max. size of table = O(m |Q|2 |ΣA|).

Theorem (Complexity for const. Time MQs and EQs). O( m |Q|2 |ΣA| ). For m in O(|Q|) the complexity result reads: O(|Q|3 |ΣA|)

Complexity of Angluin

Remaining Problems

 High Computational Complexity  Even worse: equivalence queries in general

undecidable. In essence:

 Active automata learning always remains at the level

• f hypotheses:

 neither correct nor complete

39

Further Developments

Conceptual Improvements 1

a b bb ε a 1 1 b 1 1 bb aa 1 1 ab 1 1 ba … … … bbb

All prefixes of counterexample …

• ne erssential suffix

Reduced observation table



Rivest and Shapire: Analyze counterexample separately (not in the table)

• Only add one ‚essential‘ suffix (i.e., witness), as

column label to the table

Consequence: Guaranteed Consistency! BUT: Hypothesis Automata are no longer guaranteed to be minimal!

(cf. Pnueli / Mahler‘s criticism)

Reduced observation table (contd.)



Saves membership queries! (by saving rows in the

• bservation table)

Essential suffix

Equivalence Queries At most |Q| Membership Queries for guaranteed progress after Eqs At most O( log2(m) + |ΣA| |Q|) per EQ (m = length of max. counter example)

• Max. size of table = O(|Q|2 |ΣA|).

Theorem (Complexity for const. Time MQs and EQs). O( |Q|2 |ΣA| + |Q| log2(m) ). For m in O(|Q|) the complexity result reads: O(|Q|2 |ΣA|)

Complexity (reduced observation table)

Conceptual Improvements 2

a b bb ε a 1 1 b 1 1 bb aa 1 1 ab 1 1 ba … … … bbb

All rows are filled completely, even if unnecessary

Discrimination tree

Angluin: Add suffix globally to all rows

• leads to unclosedness
• resolved by new elements

in S Kearns & Vazirani: Add suffix only locally

• Suffix only added to one ‚essential‘

sub-table.

• Prefix known from counterexample

‘Sink’ words into table through discrimination tree

non uniform suffix classifying output

Discimination tree (contd.)



Saves membership queries! (by saving entries in the

• bservation table)



More equivalence queries! (using suffixes globally may be a good heuristic sometimes)



Worst case complexity unchanged

Kearns & Vazirani + discrimination tree

• Lemma. Each counterexample leads to at least one new

state.

• Lemma. The hypothesis automata are guaranteed to have

fewer states than the minimal deterministic finite automaton for the considered language.

Theorem (for perfect equivalence oracle)

The algorithm terminates with the smallest determinsitc automaton for the considered language / set of traces.

Correctness pattern (maintained)

Overview

• Motivation
• Introduction to active automata learning
• Practical aspects in active automata

learning

• Conclusions

49

Practical results II

The ZULU competition

The ZULU challenge

• Competition in active learning (2010)
• No equivalence queries allowed, limited

amount of membership queries

• Randomly generated automata
• Test-based evaluation
• http://labh-curien.univ-st-etienne.fr/zulu/

Evolving hypothesis

Continuous equivalence queries

ZULU competition results

Kearns & Vazirani: High impact even here!

Detailed results

Asymtotic costs per state

ZULU Problem 49763507

56

Practical results I

More Applications

Practical challenges

Behavioral models

Test-driver

<presence type=… /> <iq type= “result“ /> Available OK

reset interfacing real systems:

• alphabet generation
• abstraction
• data

equivalence queries membership queries

Interface description etc.

58

Practical results I

Learning assumptions

query c A true M2 Ai

• racle for WA in assume-guarantee

reasoning

L*

query: string s

s M1 P

conjecture: Ai

Ai M1 P

false+crex c c A c A (simulate s on M1 || Perr) (model check) (model check) false+crex c true / false true P satisfied P violated true false

1. A M1 P

• 2. true M2 A

true M1 || M2 P

60

Practical results I

Learning the OCS

User model: paper workflow

Hierarchy

Event Condition Action

Submit Report

Semantics of "phase expires"- edges (1)

Semantics of "phase expires"- edges (2)

Many participants

DB

Putting it all together

Jboss

• cs

Tomcat JEE JMS ECA Work- flows

Regular extrapolation

Optimized learning setup

Learning algorithm

Bernhard Steffen | EternalS' 2011 @ Budapest, HU

• Observation Table
• Mealy machine inference
• Regular extrapolation

First Hypothesis

Reusing system states

Bernhard Steffen | EternalS' 2011 @ Budapest, HU

Reuse tree on our example

Bernhard Steffen | EternalS' 2011 @ Budapest, HU

• 52 Membership Queries
• Saved 12 Resets

Exploitation: failure invariance

Bernhard Steffen | EternalS' 2011 @ Budapest, HU

• Domainknowledge
• Failing actions due to missing

permissions

• OCS is transaction secure

(roll back in case of error)

• Partition output alphabet into

successful and failed execution

• Reflexive edges indicate failure
• utput

Pumping: Unfolding edges

Bernhard Steffen | EternalS' 2011 @ Budapest, HU

• For 52 Membership Queries only 10

Resets necessary

• 50 Symbols executed (of 148)

Queries 9 to 20 will be ‘pumped’, e.g.

• UD UD or
• DP SP

already known

Exploitation: Invariant symbols

…

Bernhard Steffen | EternalS' 2011 @ Budapest, HU

• Downloading (reading) a document

(DD) does not change a system state

• The state can be kept for reuse
• Only 6 Resets and 35 Symbols need

to be executed Failure invariance + invariant input symbol DD. indicates allowed outputs.

Statistics: Learning the OCS

Bernhard Steffen | EternalS' 2011 @ Budapest, HU

• a) No reuse
• b) Only direct re-usage
• c) Exploit input knowledge
• d) Exploit output knowledge
• e) Exploit input and output

knowledge

Statistics: Learning the OCS

Bernhard Steffen | EternalS' 2011 @ Budapest, HU

MQs = Resets + Reuses + Pumped

• a) Only direct reusage
• b) Exploit of failure outputs
• c) Like b) but one input marked as invariant
• d)-f) failure outputs and invariant for growing learn alphabets

Statistics: Learning the OCS

Bernhard Steffen | EternalS' 2011 @ Budapest, HU

• Reset times are growing
• Observed runtime included execution
• f input symbols

Accumulated reset time is highly optimistic!

Simple User Process

SP: Submit Paper UD: Upload Document IS: Interrupt Submission IU: Interrupt Upload RS: Restart Submission RU: Restart Upload

Learned automaton

Verification

Overview

• Motivation
• Introduction to active automata learning
• Practical aspects in active automata

learning

• Conclusions

Conclusions

Active Automata Learning:

• its practice has many facets:
• Abstraction
• Instrumentation
• Reuse/Optimization
• It establishes a new system perspective

Systems as evolving ‚beasts‘:

• to be observed continuously
• difficult to control:

Forget the ‚Why/How‘ focus on the ‚What‘ !

Bern hard Steff