Equivalence of Deterministic Tree-to-String Transducers Is - - PowerPoint PPT Presentation

Equivalence of Deterministic Tree-to-String Transducers Is Decidable

Helmut Seidl, Sebastian Maneth, Gregor Kemper TU M¨ unchen, U. of Edinburgh Dagstuhl, April 4, 2017

Equivalence of Deterministic Tree-to-String Transducers and More Is Decidable

Helmut Seidl, Sebastian Maneth, Gregor Kemper TU M¨ unchen, U. of Edinburgh Dagstuhl, April 4, 2017

Overview Part 1: The General Setting Part 2: Tree-to-Matrix Transducers Part 3: Polynomial Invariants

Overview Part 1: The General Setting Part 2: Tree-to-Matrix Transducers Part 3: Polynomial Invariants

Tree-to-String Translation

Input

content content frame button Do not press! defs defs end height 20 width 50

Tree-to-String Translation

Output

<frame height=20 width=50> <button>Do not press!</button> ... </frame>

Tree-to-String Translation

Output

<frame height=20 width=50> <button>Do not press!</button> ... </frame>

Realized by:

q(frame(x1, x2)) → <frame q1(x1)q(x2)</frame> q1(end) → > q1(defs(x1, x2)) → q(x1)q1(x2) q(height(x1)) → height = q(x1) . . .

Tree-to-String Translation

Output

<frame height=20 width=50> <button>Do not press!</button> ... </frame>

Or realized by:

q′(frame(x1, x2)) → <frame q′

1(x1)> q′(x2)</frame>

q′

1(end)

→ ǫ q′

1(defs(x1, x2))

→ q′(x1)q′

1(x2)

q′(height(x1)) → height = q′(x1) . . .

Tree-to-String Translation

These two translations are equivalent.

Tree-to-String Translation

These two translations are equivalent. Unstructured output can be generated in surprisingly different ways ... q(f(x1, x2, x3)) → q(x3) a q1(x2) b q(x2) q1(f(x1, x2, x3)) → q1(x3)q1(x2)q1(x2) ba q1(e) → ba q(e) → ab

Tree-to-String Translation

These two translations are equivalent. Unstructured output can be generated in surprisingly different ways ... q(f(x1, x2, x3)) → q(x3) a q1(x2) b q(x2) q1(f(x1, x2, x3)) → q1(x3)q1(x2)q1(x2) ba q1(e) → ba q(e) → ab versus q′(f(x1, x2, x3)) → ab q′(x2)q′(x2)q′(x3) q′(e) → ab

Related Work

problem statement Engelfriet, 1980

Related Work (cont.)

with monadic input Culik II, Karhum¨ aki, 1986 Ruohonen, 1986 Honkala, 2000

Related Work (cont.)

with monadic input Culik II, Karhum¨ aki, 1986 Ruohonen, 1986 Honkala, 2000 MSO-definable Engelfriet, Maneth, 2006 sequential Staworko et al., 2009 linear Boiret, Palenta, 2016 polynomial program invariants Letichevsky, Lvov, 1993 M¨ uller-Olm, S., 2004

Related Work (cont.)

with monadic input Culik II, Karhum¨ aki, 1986 Ruohonen, 1986 Honkala, 2000 MSO-definable Engelfriet, Maneth, 2006 sequential Staworko et al., 2009 linear Boiret, Palenta, 2016 polynomial program invariants Letichevsky, Lvov, 1993 M¨ uller-Olm, S., 2004

General Idea

Obvious: In-equivalence can be verified by counter example

General Idea

Obvious: In-equivalence can be verified by counter example Required: Complete proof system for equivalence

General Idea

Obvious: In-equivalence can be verified by counter example Required: Complete proof system for equivalence Issues:

• Practical method for proof search
• Extension to outputs in the free group

Overview Part 1: The General Setting Part 2: Tree-to-Matrix Transducers Part 3: Polynomial Invariants

Simplification

• A single transducer with states Q = {1, . . . , n}.
• The transducer is total.

Simplification

• A single transducer with states Q = {1, . . . , n}.
• The transducer is total.
• There is a topdown-deterministic automaton B with

states p ∈ P dom(p) is the set of trees accepted at state p

Simplification

• A single transducer with states Q = {1, . . . , n}.
• The transducer is total.
• There is a topdown-deterministic automaton B with

states p ∈ P // generalization of algebraic data type dom(p) is the set of trees accepted at state p // trees of type p

Topdown Automaton

content content frame button Do not press! defs defs end height 20 width 50 p0

Topdown Automaton

content content frame button Do not press! defs defs end height 20 width 50 p0 p2 p1

Simplified Question

For states q, q′ of the transducer, p0 ∈ P, does it hold that [ [q] ](t) = [ [q′] ](t) (t ∈ dom(p0))

From Arbitrary Output to Matrices

Unary Output q(f(x1, x2)) → aa q1(x1) a q1(x1) q2(x2)

From Arbitrary Output to Matrices

Unary Output q(f(x1, x2)) → aa q1(x1) a q1(x1) q2(x2) Succinct representation: Tree-to-int transducer q(f(x1, x2)) → 3 + 2 · q1(x1) + q2(x2)

From Arbitrary Output to Matrices

Unary Output q(f(x1, x2)) → aa q1(x1) a q1(x1) q2(x2) Succinct representation: Tree-to-int transducer q(f(x1, x2)) → 3 + 2 · q1(x1) + q2(x2) Arbitary Output letters a, b ˆ = matrices

3 1 1

• ,

3 2 1

From Arbitrary Output to Matrices

Unary Output q(f(x1, x2)) → aa q1(x1) a q1(x1) q2(x2) Succinct representation: Tree-to-int transducer q(f(x1, x2)) → 3 + 2 · q1(x1) + q2(x2) Arbitary Output letters a, b ˆ = matrices

3 1 1

• ,

3 2 1

• string aab

ˆ =

3 1 1

• ·

3 1 1

• ·

3 2 1

From Arbitrary Output to Matrices

Unary Output q(f(x1, x2)) → aa q1(x1) a q1(x1) q2(x2) Succinct representation: Tree-to-int transducer q(f(x1, x2)) → 3 + 2 · q1(x1) + q2(x2) Arbitary Output letters a, b ˆ = matrices

3 1 1

• ,

3 2 1

• string aab

ˆ =

3 1 1

• ·

3 1 1

• ·

3 2 1

• =

27 22 1

From Arbitrary Output to Matrices

Unary Output q(f(x1, x2)) → aa q1(x1) a q1(x1) q2(x2) Succinct representation: Tree-to-int transducer q(f(x1, x2)) → 3 + 2 · q1(x1) + q2(x2) Arbitary Output letters a, b ˆ = matrices

1 1 1

• ,

1 1 1

• string aab

ˆ =

1 1 1

• ·

1 1 1

• ·

1 1 1

From Arbitrary Output to Matrices

Unary Output q(f(x1, x2)) → aa q1(x1) a q1(x1) q2(x2) Succinct representation: Tree-to-int transducer q(f(x1, x2)) → 3 + 2 · q1(x1) + q2(x2) Arbitary Output letters a, b ˆ = matrices

1 1 1

• ,

1 1 1

• string aab

ˆ =

1 1 1

• ·

1 1 1

• ·

1 1 1

• =

1 1 2 3

From Arbitrary Output to Matrices

Unary Output q(f(x1, x2)) → aa q1(x1) a q1(x1) q2(x2) Succinct representation: Tree-to-int transducer q(f(x1, x2)) → 3 + 2 · q1(x1) + q2(x2) Arbitary Output letters a, b ˆ = matrices

1 2 1

• ,

1 2 1

• string aab

ˆ =

1 2 1

• ·

1 2 1

• ·

1 2 1

• =

1 2 4 9

Fact

Let Ma = 1 2 1

• Mb =

1 2 1

Fact

Let Ma = 1 2 1

• Mb =

1 2 1

• The matrix monoid generated by Ma, Mb is free.

Fact

Let Ma = 1 2 1

• Mb =

1 2 1

• The matrix monoid generated by Ma, Mb is free.

The matrix group generated by Ma, Mb is free

Fact

Let Ma = 1 2 1

• Mb =

1 2 1

• The matrix monoid generated by Ma, Mb is free.

The matrix group generated by Ma, Mb is free 1 2 1

• ·
• 1

−2 1

• =

1 1

Fact

Let Ma = 1 2 1

• Mb =

1 2 1

• The matrix monoid generated by Ma, Mb is free.

The matrix group generated by Ma, Mb is free 1 2 1

• ·
• 1

−2 1

• =

1 1

• =

⇒

Sanov group

Transformation

Wanted Transformation of tree-to-string into tree-to-matrix ...

Transformation

Wanted Transformation of tree-to-string into tree-to-matrix ... q(f(x1, x2)) → a q1(x1) b q2(x2)

Transformation

Wanted Transformation of tree-to-string into tree-to-matrix ... q(f(x1, x2)) → a q1(x1) b q2(x2) is simulated by: q(f(x1, x2)) → Ma · q1(x1) · Mb · q2(x2)

Overview Part 1: The General Setting Part 2: Tree-to-Matrix Transducers Part 3: Polynomial Invariants

Tree-to-Matrix Transducers

q1(f(x1, x2)) → b q2(x1)q2(x2) q2(f(x1, x2)) → q2(x1)q2(x1) q3(f(x1, x2)) → q4(x2)q4(x1) b q4(f(x1, x2)) → q4(x1)q4(x1) q1(c) → b q2(c) → ab q3(c) → b q4(c) → ba translates to

Tree-to-Matrix Transducers

q1(f(x1, x2)) →

• 1

2 1

• · q2(x1) · q2(x2)

q2(f(x1, x2)) → q2(x1) · q2(x1) q3(f(x1, x2)) → q4(x2) · q4(x1) ·

• 1

2 1

• q4(f(x1, x2))

→ q4(x1) · q4(x1) q1(c) →

• 1

2 1

• q2(c)

→

• 1

2 2 5

• q3(c)

→

• 1

2 1

• q4(c)

→

• 5

2 2 1

Semantics

The semantics of a tree t is a vector of matrices [ [t] ] : (Ql×l)n

Semantics

The semantics of a tree t is a vector of matrices [ [t] ] : (Ql×l)n = = ⇒ can be represented by a matrix ([ [t] ]qij) ∈ Qn×l×l.

Semantics

The semantics of a tree t is a vector of matrices [ [t] ] : (Ql×l)n = = ⇒ can be represented by a matrix ([ [t] ]qij) ∈ Qn×l×l. The Semantics of Constructors [ [f] ] : (Qn×l×l × . . . × Qn×l×l) → Qn×l×l

Semantics

The semantics of a tree t is a vector of matrices [ [t] ] : (Ql×l)n = = ⇒ can be represented by a matrix ([ [t] ]qij) ∈ Qn×l×l. The Semantics of Constructors [ [f] ] : (Qn×l×l × . . . × Qn×l×l) → Qn×l×l is of the form ([ [f] ](x1, . . . , xk))qij = polynomial in the xκq′i′j′

In the Example

[ [f] ]1(x1, x2) =

• 1

2 1

• · x12 · x22

[ [f] ]3(x1, x2) = x24 · x14 ·

• 1

2 1

In the Example

[ [f] ]1(x1, x2) =

• 1

2 1

• · x12 · x22

[ [f] ]2(x1, x2) = x12 · x12 [ [f] ]3(x1, x2) = x24 · x14 ·

• 1

2 1

• [

[f] ]4(x1, x2) = x14 · x14

Inductive Invariant

An assignment p → Ip(z) is inductive if Ip(z) ⇐ = (z . = [ [f] ](x1, . . . , xk)) ∧ Ip1(x1) ∧ . . . ∧ Ipk(xk) whenever p → f(p1, . . . , pk) holds.

Inductive Invariant

An assignment p → Ip(z) is inductive if Ip(z) ⇐ = (z . = [ [f] ](x1, . . . , xk)) ∧ Ip1(x1) ∧ . . . ∧ Ipk(xk) whenever p → f(p1, . . . , pk) holds.

Proposition

If p → Ip is inductive, then Ip([ [t] ]) = true (t ∈ dom(p))

Questions

• Can everything which holds be proven by an

inductive invariant?

Questions

• Can everything which holds be proven by an

inductive invariant?

• Is there a suitably expressive logic for the relevant

assertions?

Questions

• Can everything which holds be proven by an

inductive invariant?

• Is there a suitably expressive logic for the relevant

assertions?

• Is it decidable?

Polynomials

polynomial equality: zq1 · zq′1 · zq′0 − 2 · zq0 + 3 . = 0

Polynomials

polynomial equality: zq1 · zq′1 · zq′0 − 2 · zq0 + 3 . = 0 r . = 0 poynomial invariant at p iff r([ [t] ]) = 0 (t ∈ dom(p))

Polynomials

polynomial equality: zq1 · zq′1 · zq′0 − 2 · zq0 + 3 . = 0 r . = 0 poynomial invariant at p iff r([ [t] ]) = 0 (t ∈ dom(p)) can be described by polynomial ideals ...

Polynomial Ideals: A Primer

R ring. I ⊆ R ideal, if

• a + b ∈ I whenever a, b ∈ I;
• r · a ∈ I whenever a ∈ I and r ∈ R.

Polynomial Ideals: A Primer

R ring. I ⊆ R ideal, if

• a + b ∈ I whenever a, b ∈ I;
• r · a ∈ I whenever a ∈ I and r ∈ R.

I is finitely generated, if I = a1, . . . , asR = {s

i=1ri · ai | ri ∈ R}

Polynomial Ideals: A Primer

R ring. I ⊆ R ideal, if

• a + b ∈ I whenever a, b ∈ I;
• r · a ∈ I whenever a ∈ I and r ∈ R.

I is finitely generated, if I = a1, . . . , asR = {s

i=1ri · ai | ri ∈ R}

R = Q[z] polynomial ring

Polynomial Ideals — Basis Theorem

David Hilbert (1890) Every ideal of Q[z] is finitely generated.

Consequence

• Polynomial Invariants can be represented by

polynomial ideals.

• Finite conjunctions suffice!

Consequence

• Polynomial Invariants can be represented by

polynomial ideals.

• Finite conjunctions suffice!
• There are effective algorithms for

◮ membership ◮ inclusion ◮ equality

Completeness Theorem

Collect all polynomal invariants: ¯ Ip = {r ∈ Q[z] | r([ [t] ]) = 0 (t ∈ dom(p))} (p ∈ P)

Completeness Theorem

Collect all polynomal invariants: ¯ Ip = {r ∈ Q[z] | r([ [t] ]) = 0 (t ∈ dom(p))} (p ∈ P) Then p → ¯ Ip is inductive.

Completeness Theorem

Collect all polynomal invariants: ¯ Ip = {r ∈ Q[z] | r([ [t] ]) = 0 (t ∈ dom(p))} (p ∈ P) Then p → ¯ Ip is inductive.

Corollary

Let Hij(z) = zqij − zq′ij. Then q, q′ are equivalent (at p0) iff Hij ∈ Ip0 (1 ≤ i, j ≤ l) for some inductive polynomial invariant p → Ip.

Key Lemma

¯ I(V) = {r | ∀ v ∈ V. r(v) = 0}

vanishing ideal

Key Lemma

¯ I(V) = {r | ∀ v ∈ V. r(v) = 0}

vanishing ideal

V1 ⊆ Qm, V2 ⊆ Qn

Key Lemma

¯ I(V) = {r | ∀ v ∈ V. r(v) = 0}

vanishing ideal

V1 ⊆ Qm, V2 ⊆ Qn Then ¯ I(V1 × V2) = ¯ I(V1 × Qn) ⊕ ¯ I(Qm × V2)

Discussion

• The best inductive invariant p → ¯

Ip is a greatest fixpoint. Greatest fixpoint iteration over polynomial ideals may not terminate.

Discussion

• The best inductive invariant p → ¯

Ip is a greatest fixpoint. Greatest fixpoint iteration over polynomial ideals may not terminate.

• All inductive invariants, though, can be recursively

enumerated!

Discussion

• The best inductive invariant p → ¯

Ip is a greatest fixpoint. Greatest fixpoint iteration over polynomial ideals may not terminate.

• All inductive invariants, though, can be recursively

enumerated!

• All potential counter examples can be enumerated ...

Discussion (cont.)

The ith iterate of GFP iteration ¯ I

(i) p = {r | ∀ t ∈ dom(p), depth(t) < i.

r([ [t] ]) = 0}

Discussion (cont.)

The ith iterate of GFP iteration ¯ I

(i) p = {r | ∀ t ∈ dom(p), depth(t) < i.

r([ [t] ]) = 0} If inequivalence holds, then H ∈ ¯ I

(i) p0 for some i.

Discussion (cont.)

The ith iterate of GFP iteration ¯ I

(i) p = {r | ∀ t ∈ dom(p), depth(t) < i.

r([ [t] ]) = 0} If inequivalence holds, then H ∈ ¯ I

(i) p0 for some i.

= = ⇒ search for counter example

Discussion (cont.)

Let p → Id,p denote the inductive invariant of vector spaces of polynomials of degree at most d = = ⇒ Abstraction

Discussion (cont.)

Let p → Id,p denote the inductive invariant of vector spaces of polynomials of degree at most d = = ⇒ Abstraction For every d, greatest fixpoint computation terminates

Discussion (cont.)

Let p → Id,p denote the inductive invariant of vector spaces of polynomials of degree at most d = = ⇒ Abstraction For every d, greatest fixpoint computation terminates = = ⇒ p → Id,p is computable.

Abstraction Refinement

Sequence of abstractions satisfies:

• I0,p ⊆ I1,p ⊆ . . . ⊆ ¯

Ip

Abstraction Refinement

Sequence of abstractions satisfies:

• I0,p ⊆ I1,p ⊆ . . . ⊆ ¯

Ip

• Id,p = ¯

Ip for some d.

Abstraction Refinement

Sequence of abstractions satisfies:

• I0,p ⊆ I1,p ⊆ . . . ⊆ ¯

Ip

• Id,p = ¯

Ip for some d. = = ⇒ search for inductive invariant

Wrap-up

Theorem

• Equivalence of deterministic tree-to-matrix

transducers is decidable.

Wrap-up

Theorem

• Equivalence of deterministic tree-to-matrix

transducers is decidable.

• Equvalence of general deterministic tree-to-string

transducers is decidable.

Wrap-up

Theorem

• Equivalence of deterministic tree-to-matrix

transducers is decidable.

• Equvalence of general deterministic tree-to-string

transducers is decidable.

• Equvalence of general deterministic tree-to-free

transducers is decidable.

Summary

Matrices allow to encode general output alphabets

Summary

Matrices allow to encode general output alphabets

Summary

Matrices allow to encode general output alphabets — w/o inverses. Equivalence for tree-to-matrix transducers can be handled by means of techniques from precise program analysis, i.e., program proving, and abstraction refinement.

Thank you!