Active automata learning Based on: Bernhard Steffen, Falk Howar und - - PowerPoint PPT Presentation
Active automata learning Based on: Bernhard Steffen, Falk Howar und Maik Merten: Introduction to Active Automata Learning from a Practical Perspective. SFM 2011: 256-296. Motivation We want to apply model-based techniques. But: often
Based on: Bernhard Steffen, Falk Howar und Maik Merten: Introduction to Active Automata Learning from a Practical Perspective. SFM 2011: 256-296.
◮ We want to apply model-based techniques. ◮ But: often there are no models. ◮ Or: existing models are unrelated to implementations.
◮ We want to apply model-based techniques. ◮ But: often there are no models. ◮ Or: existing models are unrelated to implementations. ◮ Produce models from implementations.
Example (River crossing game)
Mealy machines
◮ Σ, Ω: sets of symbols, ranged over by αi (oi resp.) ◮ w, w′, . . .: words, sequences of symbols, e.g., w = α1α2α1.
◮ Σ, Ω: sets of symbols, ranged over by αi (oi resp.) ◮ w, w′, . . .: words, sequences of symbols, e.g., w = α1α2α1. ◮ u, u′, . . .: prefixes; v, v′, . . .: suffixes.
◮ Σ, Ω: sets of symbols, ranged over by αi (oi resp.) ◮ w, w′, . . .: words, sequences of symbols, e.g., w = α1α2α1. ◮ u, u′, . . .: prefixes; v, v′, . . .: suffixes.
Concatenation:
◮ Let u = αu,1 . . . αu,n and v = αv,1 . . . αv,m. ◮ Then: w = uv = αu,1 . . . αu,nαv,1 . . . αv,m
A Mealy machine is defined as a tuple M = S, s0, Σ, Ω, δ, λ where
A Mealy machine is defined as a tuple M = S, s0, Σ, Ω, δ, λ where
◮ S is a finite nonempty set of states (be n = |S| the size of the Mealy
machine),
◮ s0 ∈ S is the initial state,
A Mealy machine is defined as a tuple M = S, s0, Σ, Ω, δ, λ where
◮ S is a finite nonempty set of states (be n = |S| the size of the Mealy
machine),
◮ s0 ∈ S is the initial state, ◮ Σ is a finite input alphabet, ◮ Ω is a finite output alphabet,
A Mealy machine is defined as a tuple M = S, s0, Σ, Ω, δ, λ where
◮ S is a finite nonempty set of states (be n = |S| the size of the Mealy
machine),
◮ s0 ∈ S is the initial state, ◮ Σ is a finite input alphabet, ◮ Ω is a finite output alphabet, ◮ δ : S × Σ → S is the transition function, and ◮ λ : S × Σ → Ω is the output function.
(a) empty (b) with pod (c) with water (d) with pod and water (e) success (f) error
Mcm = S, s0, Σ, Ω, δ, λ, where
◮ S = {a, b, c, d, d′, e, f} ◮ s0 = a ◮ Σ = {water, pod,
button, clean}
◮ Ω = {, K, ✷}
a b c d d′ e f pod/ water/ clean/ button/✷ water/ pod/ button/✷ pod/ water/ button/✷ button/K {water, pod}/ button/K {water, pod}/ clean/ Σ \ {clean}/✷ Σ/✷
δ∗ : S × Σ∗ → S:
◮ δ∗(s, ǫ) = s ◮ δ∗(s, αw) = δ∗(δ(s, α), w)
δ∗ : S × Σ∗ → S:
◮ δ∗(s, ǫ) = s ◮ δ∗(s, αw) = δ∗(δ(s, α), w)
λ∗ : S × Σ∗ → Ω:
◮ λ∗(s, ǫ) = ∅ ◮ λ∗(s, wα) = λ(δ∗(s, w), α)
◮ Mealy machine processing α1α2 . . . αn ◮ will produce sequence of outputs o1o2 . . . on
◮ Mealy machine processing α1α2 . . . αn ◮ will produce sequence of outputs o1o2 . . . on
We will abstract from the complete traces!
◮ Mealy machine processing α1α2 . . . αn ◮ will produce sequence of outputs o1o2 . . . on
We will abstract from the complete traces!
Let M : Σ∗ → Ω defined by M(w) = λ∗(s0, w).
The run water pod button clean button, ✷ is in Mcm, while the run water button clean, is not.
a b c d d′ e f pod/ water/ clean/ button/✷ water/ pod/ button/✷ pod/ water/ button/✷ button/K {water, pod}/ button/K {water, pod}/ clean/ Σ \ {clean}/✷ Σ/✷
Let M ≡ M′, iff M = M′.
Let M ≡ M′, iff M = M′.
◮ But: we do not have M (and its domain, Σ∗, is infinite) ◮ Need other means of comparing models
Let M ≡ M′, iff M = M′.
◮ But: we do not have M (and its domain, Σ∗, is infinite) ◮ Need other means of comparing models ◮ Construct unique canonical Mealy machine for M and M′ ◮ Test for isomorphism (e.g., by synchronized BFS)
Regularity
Let P : Σ∗ → Ω
Let P : Σ∗ → Ω
Two words u, u′ ∈ Σ∗ are equivalent wrt. ≡P , iff for all continuations v ∈ Σ∗, the concatenated words uv and u′v are mapped to the same
u ≡P u′ ⇔ (∀v ∈ Σ∗. P(uv) = P(u′v)).
Let P : Σ∗ → Ω
Two words u, u′ ∈ Σ∗ are equivalent wrt. ≡P , iff for all continuations v ∈ Σ∗, the concatenated words uv and u′v are mapped to the same
u ≡P u′ ⇔ (∀v ∈ Σ∗. P(uv) = P(u′v)). We write [u] to denote the equivalence class of u wrt. ≡P .
water pod ≡Mcm water water pod ≡Mcm pod pod water
a b c d d′ e f pod/ water/ clean/ button/✷ water/ pod/ button/✷ pod/ water/ button/✷ button/K {water, pod}/ button/K {water, pod}/ clean/ Σ \ {clean}/✷ Σ/✷
A mapping P : Σ∗ → Ω is a semantic functional for some Mealy machine iff ≡P has only finitely many equivalence classes (finite index).
A mapping P : Σ∗ → Ω is a semantic functional for some Mealy machine iff ≡P has only finitely many equivalence classes (finite index). Proof (⇒):
◮ We show: M is an arbitrary Mealy machine
⇒ ≡M has finite index
A mapping P : Σ∗ → Ω is a semantic functional for some Mealy machine iff ≡P has only finitely many equivalence classes (finite index). Proof (⇒):
◮ We show: M is an arbitrary Mealy machine
⇒ ≡M has finite index
◮ All words leading to the same state in M are equivalent wrt. ≡M.
Proof (⇐): We will construct a Mealy machine for P. Let MP = S, s0, Σ, Ω, δ, λ:
◮ S is given by the classes of ≡P .
Proof (⇐): We will construct a Mealy machine for P. Let MP = S, s0, Σ, Ω, δ, λ:
◮ S is given by the classes of ≡P . ◮ s0 is given by [ǫ].
Proof (⇐): We will construct a Mealy machine for P. Let MP = S, s0, Σ, Ω, δ, λ:
◮ S is given by the classes of ≡P . ◮ s0 is given by [ǫ]. ◮ the transition function is defined by δ([w], α) = [wα].
Proof (⇐): We will construct a Mealy machine for P. Let MP = S, s0, Σ, Ω, δ, λ:
◮ S is given by the classes of ≡P . ◮ s0 is given by [ǫ]. ◮ the transition function is defined by δ([w], α) = [wα]. ◮ the output function can be defined as λ([w], α) = o, where
P(wα) = o.
Proof (⇐): We will construct a Mealy machine for P. Let MP = S, s0, Σ, Ω, δ, λ:
◮ S is given by the classes of ≡P . ◮ s0 is given by [ǫ]. ◮ the transition function is defined by δ([w], α) = [wα]. ◮ the output function can be defined as λ([w], α) = o, where
P(wα) = o. MP is well-defined and has semantic functional P.
a b c d d′ e f pod/ water/ clean/ button/✷ water/ pod/ button/✷ pod/ water/ button/✷ button/K {water, pod}/ button/K {water, pod}/ clean/ Σ \ {clean}/✷ Σ/✷
[ǫ] clean/
[ǫ] clean/ [button] button/✷ Σ/✷
[ǫ] clean/ [button] button/✷ Σ/✷ [pod] [water] pod/ water/ pod/ water/
[ǫ] clean/ [button] button/✷ Σ/✷ [pod] [water] pod/ water/ pod/ water/ [podwater] water/ {water, pod}/ pod/
[ǫ] clean/ [button] button/✷ Σ/✷ [pod] [water] pod/ water/ pod/ water/ [podwater] water/ {water, pod}/ pod/ [podwaterbutton] button/K
[ǫ] clean/ [button] button/✷ Σ/✷ [pod] [water] pod/ water/ pod/ water/ [podwater] water/ {water, pod}/ pod/ [podwaterbutton] button/K button/✷ button/✷ Σ \ {clean}/✷
[ǫ] clean/ [button] button/✷ Σ/✷ [pod] [water] pod/ water/ pod/ water/ [podwater] water/ {water, pod}/ pod/ [podwaterbutton] button/K button/✷ button/✷ Σ \ {clean}/✷ clean/
We will call P regular if ≡P has finite index, i.e., if there is a corresponding Mealy machine MP .
We will call P regular if ≡P has finite index, i.e., if there is a corresponding Mealy machine MP .
For regular P, MP is minimal.
We will call P regular if ≡P has finite index, i.e., if there is a corresponding Mealy machine MP .
For regular P, MP is minimal. Proof:
◮ Every state of a Mealy machine M belongs to one class of ≡M.
We will call P regular if ≡P has finite index, i.e., if there is a corresponding Mealy machine MP .
For regular P, MP is minimal. Proof:
◮ Every state of a Mealy machine M belongs to one class of ≡M. ◮ MM has exactly one state per class of ≡M.
For regular P, every transition of MP can be covered by a sequence of length at most n from the initial state. (n: number of states of MP )
For regular P, every transition of MP can be covered by a sequence of length at most n from the initial state. (n: number of states of MP ) Especially: For every Mealy machine with N states, all transitions of its mininmal equivalent Mealy machine can be covered by sequences of length at most N.
Canonical Mealy machines
Question: How can we minimize non-minimal automata / proof minimality?
Question: How can we minimize non-minimal automata / proof minimality? First idea: merge states based on equivalence of words: s ≡ s′ ⇔ ∀v ∈ Σ+ . λ∗(s, v) = λ∗(s′, v)
Question: How can we minimize non-minimal automata / proof minimality? First idea: merge states based on equivalence of words: s ≡ s′ ⇔ ∀v ∈ Σ+ . λ∗(s, v) = λ∗(s′, v) But: it may be hard to argue about all future behaviors. ⇒ exchange quantifier!
Two states s, s′ ∈ S of some Mealy machine M are k-distinguishable, iff there is a word w ∈ Σ∗ of length k or shorter, for which λ∗(s, w) = λ∗(s′, w).
Two states s, s′ ∈ S of some Mealy machine M are k-distinguishable, iff there is a word w ∈ Σ∗ of length k or shorter, for which λ∗(s, w) = λ∗(s′, w). How big can k get?
Two states s, s′ ∈ S of some Mealy machine M are k-distinguishable, iff there is a word w ∈ Σ∗ of length k or shorter, for which λ∗(s, w) = λ∗(s′, w). How big can k get?
In a Mealy machine of size n, states s ≡ s′ ∈ S are n-distinguishable.
access sequence
access sequence
Ts(v) =def M(uv) where u is access sequence to s and v ∈ Σ≤n
access sequence
Ts(v) =def M(uv) where u is access sequence to s and v ∈ Σ≤n
access sequence
Ts(v) =def M(uv) where u is access sequence to s and v ∈ Σ≤n
Complexity for comparing states: O(n2 · |Σ|n)
Partition refinement
For a Mealy machine M and some state s in M, the λ valuation of s is a mapping λs : Σ → Ω, with λs(α) =def λ(s, α)
Input: A Mealy machine M = S, s0, Σ, Ω, δ, λ Output: A partition P on S, the set of states of the Mealy machine 1: i := 1 2: put all s ∈ S with the same λ valuation into the same class p of partition Pi 3: loop 4: for all p ∈ Pi do 5: for all s ∈ p do 6: construct mapping sig : Σ → Pi: 7: sig(α) = p′ such that δ(s, α) ∈ p′ 8: Si
sig := Si sig ∪ s
9: end for 10: Pi+1 :=
sig Si sig
11: end for 12: if Pi = Pi+1 then 13: return Pi 14: end if 15: i := i + 1 16: end loop
Input: A Mealy machine M = S, s0, Σ, Ω, δ, λ Output: A partition P on S, the set of states of the Mealy machine 1: i := 1 2: put all s ∈ S with the same λ valuation into the same class p of partition Pi 3: loop 4: for all p ∈ Pi do 5: for all s ∈ p do 6: construct mapping sig : Σ → Pi: 7: sig(α) = p′ such that δ(s, α) ∈ p′ 8: Si
sig := Si sig ∪ s
9: end for 10: Pi+1 :=
sig Si sig
11: end for 12: if Pi = Pi+1 then 13: return Pi 14: end if 15: i := i + 1 16: end loop
Input: A Mealy machine M = S, s0, Σ, Ω, δ, λ Output: A partition P on S, the set of states of the Mealy machine 1: i := 1 2: put all s ∈ S with the same λ valuation into the same class p of partition Pi 3: loop 4: for all p ∈ Pi do 5: for all s ∈ p do 6: construct mapping sig : Σ → Pi: 7: sig(α) = p′ such that δ(s, α) ∈ p′ 8: Si
sig := Si sig ∪ s
9: end for 10: Pi+1 :=
sig Si sig
11: end for 12: if Pi = Pi+1 then 13: return Pi 14: end if 15: i := i + 1 16: end loop
Input: A Mealy machine M = S, s0, Σ, Ω, δ, λ Output: A partition P on S, the set of states of the Mealy machine 1: i := 1 2: put all s ∈ S with the same λ valuation into the same class p of partition Pi 3: loop 4: for all p ∈ Pi do 5: for all s ∈ p do 6: construct mapping sig : Σ → Pi: 7: sig(α) = p′ such that δ(s, α) ∈ p′ 8: Si
sig := Si sig ∪ s
9: end for 10: Pi+1 :=
sig Si sig
11: end for 12: if Pi = Pi+1 then 13: return Pi 14: end if 15: i := i + 1 16: end loop
Input: A Mealy machine M = S, s0, Σ, Ω, δ, λ Output: A partition P on S, the set of states of the Mealy machine 1: i := 1 2: put all s ∈ S with the same λ valuation into the same class p of partition Pi 3: loop 4: for all p ∈ Pi do 5: for all s ∈ p do 6: construct mapping sig : Σ → Pi: 7: sig(α) = p′ such that δ(s, α) ∈ p′ 8: Si
sig := Si sig ∪ s
9: end for 10: Pi+1 :=
sig Si sig
11: end for 12: if Pi = Pi+1 then 13: return Pi 14: end if 15: i := i + 1 16: end loop
◮ Initial computation of λ valuations: (n · |Σ|) ◮ Computation of sig objects per rounds: (n · |Σ|) ◮ Maximum number of rounds n.
Overall complexity: O(n2 · |Σ|)
a b c d d′ e f clean/ button/✷ pod/ button/✷ water/ button/✷ clean/ {water, pod}/ {water, pod}/ water/ pod/ pod/ water/ button/K button/K Σ \ {clean}/✷ Σ/✷
clean/ button/✷ pod/ button/✷ water/ button/✷ clean/ {water, pod}/ {water, pod}/ water/ pod/ pod/ water/ a b c d d′ e f button/K button/K Σ \ {clean}(esp. water)/✷ Σ(esp. clean)/✷
P1 = {a, b, c}, {d, d′}, {e}, {f}
clean/ button/✷ pod/ button/✷ water/ button/✷ clean/ {water, pod}/ {water, pod}/ button/K button/K Σ \ {clean}/✷ Σ/✷ a b c d d′ e f water/ pod/ pod/ water/
P1 = {a, b, c}, {d, d′}, {e}, {f} P2 = {a}, {b}, {c}, {d, d′}, {e}, {f}
clean/ button/✷ pod/ button/✷ water/ button/✷ clean/ {water, pod}/ {water, pod}/ button/K button/K Σ \ {clean}/✷ Σ/✷ a b c d d′ e f water/ pod/ pod/ water/
P1 = {a, b, c}, {d, d′}, {e}, {f} P2 = {a}, {b}, {c}, {d, d′}, {e}, {f} P3 = {a}, {b}, {c}, {d, d′}, {e}, {f}
a b c d e f
pod/ water/ clean/ button/✷ water/ pod/ button/✷ pod/ water/ button/✷ button/K {water, pod}/ clean/ Σ \ {clean}/✷ Σ/✷
Extension to black-boxes
◮ System under learning (SUL) is a black-box ◮ No knowledge about (number of) internal
states
◮ Input alphabet given ◮ Tests can be executed on SUL, output can
be recorded
◮ System under learning (SUL) is a black-box ◮ No knowledge about (number of) internal
states
◮ Input alphabet given ◮ Tests can be executed on SUL, output can
be recorded Try to produce complete and correct behavioral model of (interface-)behavior
Bad news: in general impossible! The possibility that one has not tested enough will always remain.
Bad news: in general impossible! The possibility that one has not tested enough will always remain.
◮ Assume maximum number of states to be N. ◮ Modify naive minimization
Ts(v) =def M(uv) where u is access sequence to s and v ∈ Σ≤N
Ts(v) =def M(uv) where u is access sequence to s and v ∈ Σ≤N
Ts(v) =def M(uv) where u is access sequence to s and v ∈ Σ≤N
Complexity for comparing states: O(|Σ|N · |Σ|N) = O(|Σ|2N)
◮ States can be represented by access sequences ◮ States can be distinguished by suffixes
◮ States can be represented by access sequences ◮ States can be distinguished by suffixes
Open questions:
◮ Can we extend the partition refinement pattern to produce suffixes? ◮ Can the partition refinement pattern be applied in the black box
scenario?
The ID learning algorithm
For a SUL and some word u in Σ∗, the output-signature of u in SUL is a mapping SULD
u : D → Ω with D ⊂ Σ∗, and
SULu(v) =def SUL(uv)
Input: A SUL with inputs Σ, a set U ⊂ Σ∗ of access sequences to all states of SUL Output: A Mealy machine M = S, s0, Σ, Ω, δ, λ for SUL 1: i := 1, D := Σ, S := U ∪ U × Σ 2: start: 3: compute SULD
u for all u ∈ S
4: put u, u′ ∈ S into the same class p of partition Pi if SULD
u = SULD u′
5: for all p ∈ Pi do 6: for all α ∈ Σ do 7: for all u, u′ ∈ p ∩ U do 8: if uα ∈ p′ ∧ u′α / ∈ p′ then 9: Let v ∈ D be s.t. SULD
uα(v) = SULD u′α(v)
10: D := D ∪ {αv}, i := i + 1 11: goto start 12: end if 13: end for 14: end for 15: end for 16: construct MSUL from Pi
Input: A SUL with inputs Σ, a set U ⊂ Σ∗ of access sequences to all states of SUL Output: A Mealy machine M = S, s0, Σ, Ω, δ, λ for SUL 1: i := 1, D := Σ, S := U ∪ U × Σ 2: start: 3: compute SULD
u for all u ∈ S
4: put u, u′ ∈ S into the same class p of partition Pi if SULD
u = SULD u′
5: for all p ∈ Pi do 6: for all α ∈ Σ do 7: for all u, u′ ∈ p ∩ U do 8: if uα ∈ p′ ∧ u′α / ∈ p′ then 9: Let v ∈ D be s.t. SULD
uα(v) = SULD u′α(v)
10: D := D ∪ {αv}, i := i + 1 11: goto start 12: end if 13: end for 14: end for 15: end for 16: construct MSUL from Pi
Input: A SUL with inputs Σ, a set U ⊂ Σ∗ of access sequences to all states of SUL Output: A Mealy machine M = S, s0, Σ, Ω, δ, λ for SUL 1: i := 1, D := Σ, S := U ∪ U × Σ 2: start: 3: compute SULD
u for all u ∈ S
4: put u, u′ ∈ S into the same class p of partition Pi if SULD
u = SULD u′
5: for all p ∈ Pi do 6: for all α ∈ Σ do 7: for all u, u′ ∈ p ∩ U do 8: if uα ∈ p′ ∧ u′α / ∈ p′ then 9: Let v ∈ D be s.t. SULD
uα(v) = SULD u′α(v)
10: D := D ∪ {αv}, i := i + 1 11: goto start 12: end if 13: end for 14: end for 15: end for 16: construct MSUL from Pi
Input: A SUL with inputs Σ, a set U ⊂ Σ∗ of access sequences to all states of SUL Output: A Mealy machine M = S, s0, Σ, Ω, δ, λ for SUL 1: i := 1, D := Σ, S := U ∪ U × Σ 2: start: 3: compute SULD
u for all u ∈ S
4: put u, u′ ∈ S into the same class p of partition Pi if SULD
u = SULD u′
5: for all p ∈ Pi do 6: for all α ∈ Σ do 7: for all u, u′ ∈ p ∩ U do 8: if uα ∈ p′ ∧ u′α / ∈ p′ then 9: Let v ∈ D be s.t. SULD
uα(v) = SULD u′α(v)
10: D := D ∪ {αv}, i := i + 1 11: goto start 12: end if 13: end for 14: end for 15: end for 16: construct MSUL from Pi
Input: A SUL with inputs Σ, a set U ⊂ Σ∗ of access sequences to all states of SUL Output: A Mealy machine M = S, s0, Σ, Ω, δ, λ for SUL 1: i := 1, D := Σ, S := U ∪ U × Σ 2: start: 3: compute SULD
u for all u ∈ S
4: put u, u′ ∈ S into the same class p of partition Pi if SULD
u = SULD u′
5: for all p ∈ Pi do 6: for all α ∈ Σ do 7: for all u, u′ ∈ p ∩ U do 8: if uα ∈ p′ ∧ u′α / ∈ p′ then 9: Let v ∈ D be s.t. SULD
uα(v) = SULD u′α(v)
10: D := D ∪ {αv}, i := i + 1 11: goto start 12: end if 13: end for 14: end for 15: end for 16: construct MSUL from Pi
We will construct a Mealy machine from final Pi by: Let MSUL = S, s0, Σ, Ω, δ, λ:
◮ S is given by the classes of Pi.
We will construct a Mealy machine from final Pi by: Let MSUL = S, s0, Σ, Ω, δ, λ:
◮ S is given by the classes of Pi. ◮ s0 is given by p that contains ǫ.
We will construct a Mealy machine from final Pi by: Let MSUL = S, s0, Σ, Ω, δ, λ:
◮ S is given by the classes of Pi. ◮ s0 is given by p that contains ǫ. ◮ the transition function is defined by δ(p, α) = p′ if exists
u ∈ p ∩ U with uα ∈ p′
We will construct a Mealy machine from final Pi by: Let MSUL = S, s0, Σ, Ω, δ, λ:
◮ S is given by the classes of Pi. ◮ s0 is given by p that contains ǫ. ◮ the transition function is defined by δ(p, α) = p′ if exists
u ∈ p ∩ U with uα ∈ p′
◮ the output function can be defined as λ(p, α) = o if exists
u ∈ p ∩ U with SULD
u (α) = o
We will construct a Mealy machine from final Pi by: Let MSUL = S, s0, Σ, Ω, δ, λ:
◮ S is given by the classes of Pi. ◮ s0 is given by p that contains ǫ. ◮ the transition function is defined by δ(p, α) = p′ if exists
u ∈ p ∩ U with uα ∈ p′
◮ the output function can be defined as λ(p, α) = o if exists
u ∈ p ∩ U with SULD
u (α) = o
MSUL is well-defined and has semantic functional SUL if U contains access sequences to all classes of SUL.
a b c d d′ e f pod/ water/ clean/ button/✷ water/ pod/ button/✷ pod/ water/ button/✷ button/K {water, pod}/ button/K {water, pod}/ clean/ Σ \ {clean}/✷ Σ/✷
pod water button clean λ
✷ ✷ ✷ ✷ pod water
✷ ✷ ✷
. . . . . . . . . . . .
pod water button clean λ
✷ ✷ ✷ ✷ pod water
✷ ✷ ✷
. . . . . . . . . . . .
pod water button clean λ
✷ ✷ ✷ ✷ pod water
✷ ✷ ✷
. . . . . . . . . . . .
pod water button clean pod button λ
pod
water
button ✷ ✷ ✷ ✷ ✷ pod water
water pod
pod water button ✷ ✷ ✷
. . . . . . . . . . . . . . . . . .
pod water button clean pod button λ
pod
water
button ✷ ✷ ✷ ✷ ✷ pod water
water pod
pod water button ✷ ✷ ✷
. . . . . . . . . . . . . . . . . .
pod water button clean pod button λ
pod
water
button ✷ ✷ ✷ ✷ ✷ pod water
water pod
pod water button ✷ ✷ ✷
. . . . . . . . . . . . . . . . . .
pod water button clean pod button water button λ
✷ pod
K water
✷ button ✷ ✷ ✷ ✷ ✷ ✷ pod water
K water pod
K pod water button ✷ ✷ ✷
✷ . . . . . . . . . . . . . . . . . . . . .
pod water button clean pod button water button λ
✷ pod
K water
✷ button ✷ ✷ ✷ ✷ ✷ ✷ pod water
K water pod
K pod water button ✷ ✷ ✷
✷ . . . . . . . . . . . . . . . . . . . . .
pod water button clean pod button water button λ
✷ pod
K water
✷ button ✷ ✷ ✷ ✷ ✷ ✷ pod water
K water pod
K pod water button ✷ ✷ ✷
✷ water pod button ✷ ✷ ✷
✷ pod water water
K water pod water
K pod water pod
K water pod pod
K pod water clean
✷ water pod clean
✷ . . . . . . . . . . . . . . . . . . . . .
a clean/
a clean/ f button/✷ Σ/✷
a clean/ f button/✷ Σ/✷ b c pod/ water/ pod/ water/
a clean/ f button/✷ Σ/✷ b c pod/ water/ pod/ water/ d water/ {water, pod}/ pod/
a clean/ f button/✷ Σ/✷ b c pod/ water/ pod/ water/ d water/ {water, pod}/ pod/ e button/K
a clean/ f button/✷ Σ/✷ b c pod/ water/ pod/ water/ d water/ {water, pod}/ pod/ e button/K button/✷ button/✷ Σ \ {clean}/✷ clean/
Measure: number of tests to be conducted
◮ Number of prefixes is in O(U|Σ|) ◮ Number of suffixes is in O(|Σ| + n)
Number of tests is in O(U|Σ|2 + U|Σ|n) In case U in O(n), we get O(n|Σ|2 + n2|Σ|)
Learning in general incomplete! The possibility that one has not tested enough will always remain. Solutions for restricted problems so far:
O( (|Σ|N)|Σ|2 + (|Σ|N)|Σ|n) = O(|Σ|Nn)
Is it possible to construct prefixes incrementally?
Learning in general incomplete! The possibility that one has not tested enough will always remain. Solutions for restricted problems so far:
O( (|Σ|N)|Σ|2 + (|Σ|N)|Σ|n) = O(|Σ|Nn)
Is it possible to construct prefixes incrementally?
Learning in general incomplete! The possibility that one has not tested enough will always remain. Solutions for restricted problems so far:
O( (|Σ|N)|Σ|2 + (|Σ|N)|Σ|n) = O(|Σ|Nn)
Is it possible to construct prefixes incrementally?
Learning in general incomplete! The possibility that one has not tested enough will always remain. Solutions for restricted problems so far:
O( (|Σ|N)|Σ|2 + (|Σ|N)|Σ|n) = O(|Σ|Nn)
Is it possible to construct prefixes incrementally?
Learning in general incomplete! The possibility that one has not tested enough will always remain. Solutions for restricted problems so far:
O( (|Σ|N)|Σ|2 + (|Σ|N)|Σ|n) = O(|Σ|Nn)
Is it possible to construct prefixes incrementally?
Learning in general incomplete! The possibility that one has not tested enough will always remain. Solutions for restricted problems so far:
O( (|Σ|N)|Σ|2 + (|Σ|N)|Σ|n) = O(|Σ|Nn)
Is it possible to construct prefixes incrementally?
The L∗
M-algorithm
Handling counterexamples
Let [u]H denote the access sequence of u in hypothesis H.
For every counterexample ¯ c there exists a decomposition ¯ c = uαv into a prefix u, an action α, and a suffix v such that mq([u]Hαv) = mq([uα]Hv). [u]Hα and [uα]H lead to same state in H ⇒ use suffix v to produce un-closedness in observation table
Let [u]H denote the access sequence of u in hypothesis H.
For every counterexample ¯ c there exists a decomposition ¯ c = uαv into a prefix u, an action α, and a suffix v such that mq([u]Hαv) = mq([uα]Hv). [u]Hα and [uα]H lead to same state in H ⇒ use suffix v to produce un-closedness in observation table
a b c d d′ e f pod/ water/ clean/ button/✷ water/ pod/ button/✷ pod/ water/ button/✷ button/K {water, pod}/ button/K {water, pod}/ clean/ Σ \ {clean}/✷ Σ/✷
a f Σ \ {button}/ button/ Σ/✷
a f Σ \ {button}/ button/ Σ/✷
Counterexample: ¯ c = pod water pod water button, leads to K in UCM but to ✷ in Hyp.
a f Σ \ {button}/ button/ Σ/✷
Counterexample: ¯ c = pod water pod water button, leads to K in UCM but to ✷ in Hyp.
i u [u]H α v Mcm() 1 ǫ ǫ pod water pod water button K 2 pod ǫ water pod water button K
a f Σ \ {button}/ button/ Σ/✷
Counterexample: ¯ c = pod water pod water button, leads to K in UCM but to ✷ in Hyp.
i u [u]H α v Mcm() 1 ǫ ǫ pod water pod water button K 2 pod ǫ water pod water button K 3 pod water ǫ pod water button K
a f Σ \ {button}/ button/ Σ/✷
Counterexample: ¯ c = pod water pod water button, leads to K in UCM but to ✷ in Hyp.
i u [u]H α v Mcm() 1 ǫ ǫ pod water pod water button K 2 pod ǫ water pod water button K 3 pod water ǫ pod water button K 4 pod water pod ǫ water button ✷
a f Σ \ {button}/ button/ Σ/✷
Counterexample: ¯ c = pod water pod water button, leads to K in UCM but to ✷ in Hyp.
i u [u]H α v Mcm() 1 ǫ ǫ pod water pod water button K 2 pod ǫ water pod water button K 3 pod water ǫ pod water button K 4 pod water pod ǫ water button ✷ 5 pod water pod water ǫ button ǫ ✷
Both strategies (i.e., all prefixes to Sp, all suffixes to D) are expensive. Is there a more efficient strategy?
Input: counterexample ¯ c = ¯ c1 . . . ¯ cm, hypothesis H. Output: new suffix for D 1: o¯
c := mq(¯
c) 2: lower := 2, upper := m − 1 3: loop 4: mid := ⌊(lower + upper) / 2⌋ 5: s := ¯ c1 . . . ¯ cmid−1, s′ := [s]H, d := ¯ cmid . . . ¯ cm 6:
7: if omid = o¯
c then
8: lower := mid + 1 // same as reference output: move right 9: if upper < lower then 10: return ¯ cmid+1 . . . ¯ cm 11: end if 12: else 13: upper := mid − 1 // not same as reference output: move left 14: if upper < lower then 15: return ¯ cmid . . . ¯ cm 16: end if 17: end if 18: end loop
Input: counterexample ¯ c = ¯ c1 . . . ¯ cm, hypothesis H. Output: new suffix for D 1: o¯
c := mq(¯
c) 2: lower := 2, upper := m − 1 3: loop 4: mid := ⌊(lower + upper) / 2⌋ 5: s := ¯ c1 . . . ¯ cmid−1, s′ := [s]H, d := ¯ cmid . . . ¯ cm 6:
7: if omid = o¯
c then
8: lower := mid + 1 // same as reference output: move right 9: if upper < lower then 10: return ¯ cmid+1 . . . ¯ cm 11: end if 12: else 13: upper := mid − 1 // not same as reference output: move left 14: if upper < lower then 15: return ¯ cmid . . . ¯ cm 16: end if 17: end if 18: end loop
Input: counterexample ¯ c = ¯ c1 . . . ¯ cm, hypothesis H. Output: new suffix for D 1: o¯
c := mq(¯
c) 2: lower := 2, upper := m − 1 3: loop 4: mid := ⌊(lower + upper) / 2⌋ 5: s := ¯ c1 . . . ¯ cmid−1, s′ := [s]H, d := ¯ cmid . . . ¯ cm 6:
7: if omid = o¯
c then
8: lower := mid + 1 // same as reference output: move right 9: if upper < lower then 10: return ¯ cmid+1 . . . ¯ cm 11: end if 12: else 13: upper := mid − 1 // not same as reference output: move left 14: if upper < lower then 15: return ¯ cmid . . . ¯ cm 16: end if 17: end if 18: end loop
Input: counterexample ¯ c = ¯ c1 . . . ¯ cm, hypothesis H. Output: new suffix for D 1: o¯
c := mq(¯
c) 2: lower := 2, upper := m − 1 3: loop 4: mid := ⌊(lower + upper) / 2⌋ 5: s := ¯ c1 . . . ¯ cmid−1, s′ := [s]H, d := ¯ cmid . . . ¯ cm 6:
7: if omid = o¯
c then
8: lower := mid + 1 // same as reference output: move right 9: if upper < lower then 10: return ¯ cmid+1 . . . ¯ cm 11: end if 12: else 13: upper := mid − 1 // not same as reference output: move left 14: if upper < lower then 15: return ¯ cmid . . . ¯ cm 16: end if 17: end if 18: end loop
Input: counterexample ¯ c = ¯ c1 . . . ¯ cm, hypothesis H. Output: new suffix for D 1: o¯
c := mq(¯
c) 2: lower := 2, upper := m − 1 3: loop 4: mid := ⌊(lower + upper) / 2⌋ 5: s := ¯ c1 . . . ¯ cmid−1, s′ := [s]H, d := ¯ cmid . . . ¯ cm 6:
7: if omid = o¯
c then
8: lower := mid + 1 // same as reference output: move right 9: if upper < lower then 10: return ¯ cmid+1 . . . ¯ cm 11: end if 12: else 13: upper := mid − 1 // not same as reference output: move left 14: if upper < lower then 15: return ¯ cmid . . . ¯ cm 16: end if 17: end if 18: end loop
Input: counterexample ¯ c = ¯ c1 . . . ¯ cm, hypothesis H. Output: new suffix for D 1: o¯
c := mq(¯
c) 2: lower := 2, upper := m − 1 3: loop 4: mid := ⌊(lower + upper) / 2⌋ 5: s := ¯ c1 . . . ¯ cmid−1, s′ := [s]H, d := ¯ cmid . . . ¯ cm 6:
7: if omid = o¯
c then
8: lower := mid + 1 // same as reference output: move right 9: if upper < lower then 10: return ¯ cmid+1 . . . ¯ cm 11: end if 12: else 13: upper := mid − 1 // not same as reference output: move left 14: if upper < lower then 15: return ¯ cmid . . . ¯ cm 16: end if 17: end if 18: end loop
Input: counterexample ¯ c = ¯ c1 . . . ¯ cm, hypothesis H. Output: new suffix for D 1: o¯
c := mq(¯
c) 2: lower := 2, upper := m − 1 3: loop 4: mid := ⌊(lower + upper) / 2⌋ 5: s := ¯ c1 . . . ¯ cmid−1, s′ := [s]H, d := ¯ cmid . . . ¯ cm 6:
7: if omid = o¯
c then
8: lower := mid + 1 // same as reference output: move right 9: if upper < lower then 10: return ¯ cmid+1 . . . ¯ cm 11: end if 12: else 13: upper := mid − 1 // not same as reference output: move left 14: if upper < lower then 15: return ¯ cmid . . . ¯ cm 16: end if 17: end if 18: end loop
Input: counterexample ¯ c = ¯ c1 . . . ¯ cm, hypothesis H. Output: new suffix for D 1: o¯
c := mq(¯
c) 2: lower := 2, upper := m − 1 3: loop 4: mid := ⌊(lower + upper) / 2⌋ 5: s := ¯ c1 . . . ¯ cmid−1, s′ := [s]H, d := ¯ cmid . . . ¯ cm 6:
7: if omid = o¯
c then
8: lower := mid + 1 // same as reference output: move right 9: if upper < lower then 10: return ¯ cmid+1 . . . ¯ cm 11: end if 12: else 13: upper := mid − 1 // not same as reference output: move left 14: if upper < lower then 15: return ¯ cmid . . . ¯ cm 16: end if 17: end if 18: end loop
a f Σ \ {button}/ button/ Σ/✷
Counterexample: ¯ c = pod water pod water button, leads to K in UCM but to ✷ in Hyp.
a f Σ \ {button}/ button/ Σ/✷
Counterexample: ¯ c = pod water pod water button, leads to K in UCM but to ✷ in Hyp.
u [u]H lower mid upper 1 2 3 4 5 ǫ ǫ K pod water pod water ǫ ✷
a f Σ \ {button}/ button/ Σ/✷
Counterexample: ¯ c = pod water pod water button, leads to K in UCM but to ✷ in Hyp.
u [u]H lower mid upper 1 2 3 4 5 ǫ ǫ K pod water pod water ǫ ✷ pod water ǫ 2 3 4 K
a f Σ \ {button}/ button/ Σ/✷
Counterexample: ¯ c = pod water pod water button, leads to K in UCM but to ✷ in Hyp.
u [u]H lower mid upper 1 2 3 4 5 ǫ ǫ K pod water pod water ǫ ✷ pod water ǫ 2 3 4 K pod water pod ǫ 3+1 4 4 ✷
a f Σ \ {button}/ button/ Σ/✷
Counterexample: ¯ c = pod water pod water button, leads to K in UCM but to ✷ in Hyp.
u [u]H lower mid upper 1 2 3 4 5 ǫ ǫ K pod water pod water ǫ ✷ pod water ǫ 2 3 4 K pod water pod ǫ 3+1 4 4 ✷
Summary
◮ System under learning (SUL) is a
black-box
◮ Input alphabet given ◮ Tests can be executed on SUL, output can
be recorded
◮ Membership queries: SUL : Σ∗ → Ω
Bad news: in general impossible! The possibility that one has not tested enough will always remain.
◮ States can be represented by access sequences ◮ States can be distinguished by suffixes ◮ Partition refinement pattern yields suffixes! ◮ In general, we cannot construct prefixes incrementally!
Good news: We can introduce some magic that resolves prefix problem. Equivalence queries test wether H = SUL, or not.
Bad news: in general impossible! The possibility that one has not tested enough will always remain.
◮ States can be represented by access sequences ◮ States can be distinguished by suffixes ◮ Partition refinement pattern yields suffixes! ◮ In general, we cannot construct prefixes incrementally!
Good news: We can introduce some magic that resolves prefix problem. Equivalence queries test wether H = SUL, or not.
Bad news: in general impossible! The possibility that one has not tested enough will always remain.
◮ States can be represented by access sequences ◮ States can be distinguished by suffixes ◮ Partition refinement pattern yields suffixes! ◮ In general, we cannot construct prefixes incrementally!
Good news: We can introduce some magic that resolves prefix problem. Equivalence queries test wether H = SUL, or not.
Bad news: in general impossible! The possibility that one has not tested enough will always remain.
◮ States can be represented by access sequences ◮ States can be distinguished by suffixes ◮ Partition refinement pattern yields suffixes! ◮ In general, we cannot construct prefixes incrementally!
Good news: We can introduce some magic that resolves prefix problem. Equivalence queries test wether H = SUL, or not.
depth N trees and trees Ts. MQ complexity: O(|Σ|2N).
find suffixes. MQ complexity: O(U|Σ|2 + U|Σ|n)
O( (|Σ|N)|Σ|2 + (|Σ|N)|Σ|n) = O(|Σ|Nn)
depth N trees and trees Ts. MQ complexity: O(|Σ|2N).
find suffixes. MQ complexity: O(U|Σ|2 + U|Σ|n)
O( (|Σ|N)|Σ|2 + (|Σ|N)|Σ|n) = O(|Σ|Nn)
depth N trees and trees Ts. MQ complexity: O(|Σ|2N).
find suffixes. MQ complexity: O(U|Σ|2 + U|Σ|n)
O( (|Σ|N)|Σ|2 + (|Σ|N)|Σ|n) = O(|Σ|Nn)
Angluins algorithm:
◮ Use observation tables to organize results from tests ◮ Partition states according to (growing partial) λs : Σ∗ → Ω. ◮ Iteratively refine partition:
◮ Closedness: Move“new”row from lower to upper part of table.
(Allows to drop the assumption from ID algo. that all necessary prefixes are given as input).
◮ Consistency: Partition refinement to generate suffixes.
Angluins algorithm:
◮ Use observation tables to organize results from tests ◮ Partition states according to (growing partial) λs : Σ∗ → Ω. ◮ Iteratively refine partition:
◮ Closedness: Move“new”row from lower to upper part of table.
(Allows to drop the assumption from ID algo. that all necessary prefixes are given as input).
◮ Consistency: Partition refinement to generate suffixes.
Angluins algorithm (contd.):
◮ Use equivalence queries to get counterexamples:
◮ Counterexample passes additional state (cf. proof in lecture & in
paper)
◮
⇒ put all prefixes of ce. to upper part of table. (i.e., “incremental”ID algorithm)
◮ Complexity:
◮ MQ: O(nm|Σ| · (n + |Σ|)) = O(n2|Σ|m + n|Σ|2m) ◮ EQ: O(n)
Angluins algorithm (contd.):
◮ Use equivalence queries to get counterexamples:
◮ Counterexample passes additional state (cf. proof in lecture & in
paper)
◮
⇒ put all prefixes of ce. to upper part of table. (i.e., “incremental”ID algorithm)
◮ Complexity:
◮ MQ: O(nm|Σ| · (n + |Σ|)) = O(n2|Σ|m + n|Σ|2m) ◮ EQ: O(n)
Angluins algorithm (contd.):
◮ Use equivalence queries to get counterexamples:
◮ Counterexample passes additional state (cf. proof in lecture & in
paper)
◮
⇒ put all prefixes of ce. to upper part of table. (i.e., “incremental”ID algorithm)
◮ Complexity:
◮ MQ: O(nm|Σ| · (n + |Σ|)) = O(n2|Σ|m + n|Σ|2m) ◮ EQ: O(n)
Other strategies to handle counterexamples (due to decomposition theorem):
◮ Add all suffixes of ce. to suffixes:
◮ No inconsistencies! ◮ Complexity (MQ): O(n|Σ| · (nm + |Σ|)) = O(n2|Σ|m + n|Σ|2)
◮ Find one suffix by binary search:
◮ Complexity (MQ):
O(n|Σ| · (n + |Σ|) + n · log2(m)) = O(n2|Σ| + n|Σ|2 + n · log2(m))
◮ Complexity (EQ): O(n)
Other strategies to handle counterexamples (due to decomposition theorem):
◮ Add all suffixes of ce. to suffixes:
◮ No inconsistencies! ◮ Complexity (MQ): O(n|Σ| · (nm + |Σ|)) = O(n2|Σ|m + n|Σ|2)
◮ Find one suffix by binary search:
◮ Complexity (MQ):
O(n|Σ| · (n + |Σ|) + n · log2(m)) = O(n2|Σ| + n|Σ|2 + n · log2(m))
◮ Complexity (EQ): O(n)
Other strategies to handle counterexamples (due to decomposition theorem):
◮ Add all suffixes of ce. to suffixes:
◮ No inconsistencies! ◮ Complexity (MQ): O(n|Σ| · (nm + |Σ|)) = O(n2|Σ|m + n|Σ|2)
◮ Find one suffix by binary search:
◮ Complexity (MQ):
O(n|Σ| · (n + |Σ|) + n · log2(m)) = O(n2|Σ| + n|Σ|2 + n · log2(m))
◮ Complexity (EQ): O(n)
Other strategies to handle counterexamples (due to decomposition theorem):
◮ Add all suffixes of ce. to suffixes:
◮ No inconsistencies! ◮ Complexity (MQ): O(n|Σ| · (nm + |Σ|)) = O(n2|Σ|m + n|Σ|2)
◮ Find one suffix by binary search:
◮ Complexity (MQ):
O(n|Σ| · (n + |Σ|) + n · log2(m)) = O(n2|Σ| + n|Σ|2 + n · log2(m))
◮ Complexity (EQ): O(n)