On transducers determinization Pierre-Alain Reynier Modelization - - PowerPoint PPT Presentation
On transducers determinization Pierre-Alain Reynier Modelization and Verification team LIF, Aix-Marseille University & CNRS Formal methods to improve software Software systems are complex and ubiquitous critical systems reliability
On transducers determinization
Pierre-Alain Reynier Modelization and Verification team LIF, Aix-Marseille University & CNRS
Formal methods to improve software
Software systems are complex and ubiquitous critical systems ‹ reliability widespread ‹ efficiency, scalability ‹ need for formal methods
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 2 / 25
Formal methods to improve software
Software systems are complex and ubiquitous critical systems ‹ reliability widespread ‹ efficiency, scalability ‹ need for formal methods Automata-based approaches: model checking controller synthesis performance evaluation model optimization Objective: Improve our theoretical understanding of automata models
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 2 / 25
From Languages to Transductions
Languages Transductions Input ! {0, 1} Input ! Outputs automata transducers accept inputs transform inputs
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 3 / 25
From Languages to Transductions
Languages Transductions Input ! {0, 1} Input ! Outputs automata transducers accept inputs transform inputs Applications: Word-to-word transducers:
I language and speech processing I model-checking infinite state-space systems I reactive systems I verification of web sanitizers
Nested-word-to-word transducers:
I XML transformations I model for recursive programs Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 3 / 25
Simplification of models
General Problem
Given a (complex) model of a transformation, does there exist an equivalent simpler model? ‹ Natural question: minimization of automata determinism reduce number of registers 2way: reduce number of passes . . .
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 4 / 25
Overview
1
Introduction
2
Determinization of transducers
3
Register minimization
4
Multi-sequentiality
5
Conclusion
Overview
1
Introduction
2
Determinization of transducers
3
Register minimization
4
Multi-sequentiality
5
Conclusion
Finite-state Transducers
= associate output words with transitions of a finite state automaton
Example (A transducer T)
`|✏ a|a b|✏ a|✏ Semantics JTK: f : `wa 7! a#a(w), with w 2 {a, b}⇤ Non-determinism: semantics is a relation
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 5 / 25
Finite-state Transducers
= associate output words with transitions of a finite state automaton
Example (A transducer T)
`|✏ a|a b|✏ a|✏ Semantics JTK: f : `wa 7! a#a(w), with w 2 {a, b}⇤ Non-determinism: semantics is a relation A transducer is: functional if it realizes a function deterministic if the underlying automaton is deterministic Classes: DFT, fNFT, NFT
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 5 / 25
Comparison of classes
DFT= Deterministic Finite-state Transducers fNFT= Functional Finite-state Transducers NFT= Non-deterministic Finite-state Transducers
Theorem
DFT ( fNFT ( NFT Examples: See blackboard
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 6 / 25
Determinization of Finite-state Transducers
Determinization Problem
Input: a fNFTT Question: does there exist an equivalent DFT? Standard technique: subset construction starting from the set of initial states.
store the unproduced outputs in the state States of the form {(p, a), (q, "), (s, bb)}
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 7 / 25
An example
dom(f ) = Σ3 f (u) = last(u)|u|
i p1 p2 p3 q1 q2 q3
|a |a a|a |b |b b|b
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 8 / 25
An example
dom(f ) = Σ3 f (u) = last(u)|u|
i p1 p2 p3 q1 q2 q3
|a |a a|a |b |b b|b
{(i, ε)} ⇢ (p1, a) (q1, b)
(q2, bb)
{(q3, ε)}
|" |" a|aaa b|bbb
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 8 / 25
An example
dom(f ) = Σ3 f (u) = last(u)|u|
i p1 p2 p3 q1 q2 q3
|a |a a|a |b |b b|b
{(i, ε)} ⇢ (p1, a) (q1, b)
(q2, bb)
{(q3, ε)}
|" |" a|aaa b|bbb Goal: characterize termination of subset construction
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 8 / 25
Delay between words
Definition (Longest common prefix)
Given two words u, v 2 Σ⇤, lcp(u, v) denotes the longest common prefix of u and v. Example: lcp(aaa, aab) = aa
Definition (Delay)
Given two words u, v 2 Σ⇤, we define: delay(u, v) = lcp(u, v)1.(u, v) Example: delay(aaa, aab) = (a, b)
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 9 / 25
Twinning Property [Choffrut77]
Consider some NFTT.
Definition (Twinning Property)
We say that T satisfies the twinning property iff for all situations as depicted on the right, we have: delay(v1, w1) = delay(v1v2, w1w2)
u1|v1 u1|w1 u2|v2 u2|w2
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 10 / 25
Characterization of sequentiality
Lemma
If a fNFT satisfies the Twinning Property, then the delays computed by the subset construction are bounded.
Corollary
Twinning Property ( ) Termination of subset construction.
Theorem ([WK95])
Twinning Property can be decided in PTime.
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 11 / 25
An example violating the Twinning Property
dom(f ) = Σ+ f (u) = last(u)|u|
i1 i2 p1 p2
a|a b|b |a |b After reading an input word u: longest common prefix of outputs = " subset construction ◆ {(i1, a|u|), (i2, b|u|)} ‹ The subset construction does not terminate. The TP is violated: consider synchronised loops around i1 and i2.
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 12 / 25
Overview
1
Introduction
2
Determinization of transducers
3
Register minimization
4
Multi-sequentiality
5
Conclusion
Streaming String Transducers [Alur and Cerny, 2010]
Definition
Streaming String Transducers (SST for short) are defined as deterministic Finite-state automata extended with registers. Register updates allowed have the following form: X := u · Y · v X := YZ where X, Y , Z denote registers and u, v are words in Σ⇤.
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 13 / 25
Streaming String Transducers [Alur and Cerny, 2010]
Definition
Streaming String Transducers (SST for short) are defined as deterministic Finite-state automata extended with registers. Register updates allowed have the following form: X := u · Y · v X := YZ where X, Y , Z denote registers and u, v are words in Σ⇤. Examples: X ` a, X := X.a b, X := X a `wa 7! a#a(w)
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 13 / 25
Streaming String Transducers [Alur and Cerny, 2010]
Definition
Streaming String Transducers (SST for short) are defined as deterministic Finite-state automata extended with registers. Register updates allowed have the following form: X := u · Y · v X := YZ where X, Y , Z denote registers and u, v are words in Σ⇤. Examples: XaXb ` a, ⇢ Xa := Xa.a Xb := Xb b, ⇢ Xa := Xa Xb := Xb.b a `wa 7! a#a(w)b#b(w)
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 13 / 25
Another example of SST
Consider the following SST:
pa
Xa
pb
Xb b, upd a, upd a, upd b, upd where upd: ⇢ Xa := Xa.a Xb := Xb.b Which function does it realize?
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 14 / 25
Another example of SST
Consider the following SST:
pa
Xa
pb
Xb b, upd a, upd a, upd b, upd where upd: ⇢ Xa := Xa.a Xb := Xb.b Which function does it realize? Solution: dom(f ) = Σ⇤ f (u) = last(u)|u|
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 14 / 25
Examples of SST
How to implement these transformations? mirror u = a1 . . . an 7! ˜ u = an . . . a1
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 15 / 25
Examples of SST
How to implement these transformations? mirror u = a1 . . . an 7! ˜ u = an . . . a1 copy u 7! uu
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 15 / 25
Examples of SST
How to implement these transformations? mirror u = a1 . . . an 7! ˜ u = an . . . a1 copy u 7! uu mirror and copy u 7! ˜ uu
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 15 / 25
Examples of SST
How to implement these transformations? mirror u = a1 . . . an 7! ˜ u = an . . . a1 copy u 7! uu mirror and copy u 7! ˜ uu replace u#v 7! v[a u]
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 15 / 25
Examples of SST
How to implement these transformations? mirror u = a1 . . . an 7! ˜ u = an . . . a1 copy u 7! uu mirror and copy u 7! ˜ uu replace u#v 7! v[a u] replace2 (k is fixed) u1# . . . #uk#v 7! v[ai ui] How many registers did you use? Is this number minimal?
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 15 / 25
Expressiveness results
Comparison of SST and Finite-state Transducers New class: Two-way Deterministic Finite-state Transducers (2DFT)
expressiveness
DFTs fNFTs 2DFTs ( (
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 16 / 25
Expressiveness results
Comparison of SST and Finite-state Transducers New class: Two-way Deterministic Finite-state Transducers (2DFT)
expressiveness
DFTs fNFTs 2DFTs ( ( ⌘ 1-register appending SST
X := Xu
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 16 / 25
Expressiveness results
Comparison of SST and Finite-state Transducers New class: Two-way Deterministic Finite-state Transducers (2DFT)
expressiveness
DFTs fNFTs 2DFTs ( ( ⌘ 1-register appending SST
X := Xu ⌘ appending SST
X := Yu
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 16 / 25
Expressiveness results
Comparison of SST and Finite-state Transducers New class: Two-way Deterministic Finite-state Transducers (2DFT)
expressiveness
DFTs fNFTs 2DFTs ( ( ⌘ 1-register appending SST
X := Xu ⌘ appending SST
X := Yu ⌘ copyless SST forbid (X, Y ) := (X, X)
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 16 / 25
Expressiveness results
Comparison of SST and Finite-state Transducers New class: Two-way Deterministic Finite-state Transducers (2DFT)
expressiveness
DFTs fNFTs 2DFTs ( ( ⌘ 1-register appending SST
X := Xu ⌘ appending SST
X := Yu ⌘ copyless SST forbid (X, Y ) := (X, X) ( SST
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 16 / 25
Expressiveness results
Comparison of SST and Finite-state Transducers New class: Two-way Deterministic Finite-state Transducers (2DFT)
expressiveness
DFTs fNFTs 2DFTs ( ( ⌘ 1-register appending SST
X := Xu ⌘ appending SST
X := Yu ⌘ copyless SST forbid (X, Y ) := (X, X) ( SST ‹ Long-term objective: Register Minimization for SST
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 16 / 25
Register minimization for appending SST
dom(f ) = Σ⇤ f (u) = last(u)|u| upd: ⇢ Xa := Xa.a Xb := Xb.b
pa
Xa
pb
Xb b, upd a, upd a, upd b, upd ‹ can be realized with 2 registers Can we do better?
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 17 / 25
Register minimization for appending SST
dom(f ) = Σ⇤ f (u) = last(u)|u| upd: ⇢ Xa := Xa.a Xb := Xb.b
pa
Xa
pb
Xb b, upd a, upd a, upd b, upd ‹ can be realized with 2 registers Can we do better? No! 1 register is DFT
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 17 / 25
Register minimization for appending SST
dom(f ) = Σ⇤ f (u) = last(u)|u| upd: ⇢ Xa := Xa.a Xb := Xb.b
pa
Xa
pb
Xb b, upd a, upd a, upd b, upd ‹ can be realized with 2 registers Can we do better? No! 1 register is DFT
Register Complexity Problem
Input: An appending SST T and an integer k Question: Does there exist a k-appending SST T 0 with T ⌘ T 0?
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 17 / 25
Register complexity using Twinning Property [LICS’16]
Intuition: 2 registers needed if there are 2 runs generating arbitrarily large delays k registers needed if there are k runs generating pairwise arb. large delays
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 18 / 25
Register complexity using Twinning Property [LICS’16]
Intuition: 2 registers needed if there are 2 runs generating arbitrarily large delays k registers needed if there are k runs generating pairwise arb. large delays Contraposition:
T satisfies the Twinning Property of order k if:
k synchronised loops
for every situation like:
k + 1 runs
‹ there are two runs that remain ”close”
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 18 / 25
Register complexity using Twinning Property [LICS’16]
Lemma
If T satisfies the TP of order k, then from any set of runs on the same input word, one can extract k runs such that every run is ”close” from one
Theorem
A fNFT is definable by a k-appending SST iff it satisfies the TP of order k.
Theorem
Given a fNFT T and k (in unary), deciding whether T satisfies the TP of
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 19 / 25
Register complexity using Twinning Property [LICS’16]
An example: how many registers for the following function? f : u1#u2 7! last(u1)|u1|#last(u2)|u2|
a|a a|a b|b b|b #|# #|# #|# #|# a|a b|b a|a b|b |a |a |b |b |a |b |a |b
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 20 / 25
Register complexity using Twinning Property [LICS’16]
An example: how many registers for the following function? f : u1#u2 7! last(u1)|u1|#last(u2)|u2|
a|a a|a b|b b|b #|# #|# #|# #|# a|a b|b a|a b|b |a |a |b |b |a |b |a |b
Only 2 registers!
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 20 / 25
Register complexity using Twinning Property [LICS’16]
An example: how many registers for the following function? f : u1#u2 7! last(u1)|u1|#last(u2)|u2|
pa pb
b, upd a, upd a, upd b, upd Xa Xb b, upd a, upd a, upd b, upd #, Xa := Xb.# Xb := Xb.# #, Xa := Xa.# Xb := Xa.#
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 20 / 25
Overview
1
Introduction
2
Determinization of transducers
3
Register minimization
4
Multi-sequentiality
5
Conclusion
Multi-sequential functions [CS86]
Definition
A function f is multi-sequential if there exists a finite number of DFTs T1, . . . , Tn such that f = S
iJTiK
Example: f (u) = last(u)|u|
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 21 / 25
Multi-sequential functions [CS86]
Definition
A function f is multi-sequential if there exists a finite number of DFTs T1, . . . , Tn such that f = S
iJTiK
Example: f (u) = last(u)|u|
Theorem
DFT ( multi-seq ( fNFT Examples: see blackboard
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 21 / 25
Multi-sequential functions [CS86]
Definition
A function f is multi-sequential if there exists a finite number of DFTs T1, . . . , Tn such that f = S
iJTiK
Example: f (u) = last(u)|u|
Theorem
DFT ( multi-seq ( fNFT Examples: see blackboard
k-sequentiality Problem
Input: A fNFTT and an integer k Question: Does there exist k DFTT1, . . . , Tk such that JTK = S
iJTiK?
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 21 / 25
Multi-sequential functions as SST [FoSSaCS’17]
Theorem
A function can be realized as a union of k DFT iff it can be realized by a SST with k registers and updates of the form X := Xu. ‹ Solving the k-sequentiality problem amounts to solve register minimization in this class.
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 22 / 25
Multi-sequential functions as SST [FoSSaCS’17]
Theorem
A function can be realized as a union of k DFT iff it can be realized by a SST with k registers and updates of the form X := Xu. ‹ Solving the k-sequentiality problem amounts to solve register minimization in this class. ‹ introduction of a “Branching” Twinning Property of order k (input words can be different between runs)
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 22 / 25
Multi-sequential functions as SST [FoSSaCS’17]
Theorem
A function can be realized as a union of k DFT iff it can be realized by a SST with k registers and updates of the form X := Xu. ‹ Solving the k-sequentiality problem amounts to solve register minimization in this class. ‹ introduction of a “Branching” Twinning Property of order k (input words can be different between runs)
Theorem
A fNFT is definable by a union of k DFT iff it satisfies the Branching TP
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 22 / 25
Multi-sequential functions as SST [FoSSaCS’17]
Theorem
A function can be realized as a union of k DFT iff it can be realized by a SST with k registers and updates of the form X := Xu. ‹ Solving the k-sequentiality problem amounts to solve register minimization in this class. ‹ introduction of a “Branching” Twinning Property of order k (input words can be different between runs)
Theorem
A fNFT is definable by a union of k DFT iff it satisfies the Branching TP
Theorem
Given a fNFT T and k (in unary), deciding whether T satisfies the Branching TP of order k is PSpace-complete.
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 22 / 25
Example
Consider this function: f : u1#u2 7! last(u1)|u1|#last(u2)|u2| Is it 2-sequential? Is it 3-sequential? Is it 4-sequential?
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 23 / 25
Example
Consider this function: f : u1#u2 7! last(u1)|u1|#last(u2)|u2| Is it 2-sequential? Is it 3-sequential? Is it 4-sequential? Minimum=4 a | a b | a b | a a | a a | a b | a b | a a | a # | # a | a b | a b | a a | a b | b a | b a | b b | b # | # b | b a | b a | b b | b a | a b | a b | a a | a # | # b | b a | b a | b b | b b | b a | b a | b b | b # | #
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 23 / 25
Overview
1
Introduction
2
Determinization of transducers
3
Register minimization
4
Multi-sequentiality
5
Conclusion
Summary
Some “old” results: Finite-state Transducers Determinization not always possible for transducers Twinning Property to characterize determinizability Recent applications to: Streaming String Transducer Register minimization Multi-sequential functions
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 24 / 25
Summary
Some “old” results: Finite-state Transducers Determinization not always possible for transducers Twinning Property to characterize determinizability Recent applications to: Streaming String Transducer Register minimization Multi-sequential functions I did not present...
logical characterization algebraic characterization
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 24 / 25
Some perspectives
Extend register minimization to larger classes of SST
algebraic presentation, canonical object Specification languages for transformations Alternative semantics to break undecidability/high complexity
Pierre-Alain Reynier (LIF, team MoVe) On transducers determinization Nov 23, 2017 25 / 25
Research topics:
transducers, higher-order languages, weighted automata
infinite state systems such as Petri nets, timed systems
workflows, access control, quantitative information flow