Compositions of Extended Top-down Tree Transducers Andreas Maletti - PowerPoint PPT Presentation

Compositions of Extended Top-down Tree Transducers Andreas Maletti March 30, 2007

Short Introduction Motivation ◮ Extended tree transducers are used in machine translation [Knight & Graehl 05, Shieber 04] ◮ Compositions occur naturally 1. transducers for specific (small) tasks are easier to train 2. small transducers are simpler to understand 3. “component” tree transducers can be reused

Short Introduction Motivation ◮ Extended tree transducers are used in machine translation [Knight & Graehl 05, Shieber 04] ◮ Compositions occur naturally 1. transducers for specific (small) tasks are easier to train 2. small transducers are simpler to understand 3. “component” tree transducers can be reused ◮ Extended tree transducers are (essentially) as powerful as tree substitution grammars [Knight & Graehl & Hopkins 07] ◮ Closure under composition of synchronous tree substitution grammar transformations open (since introduction in 80’s)

Outline Extended Top-down Tree Transducer Bimorphism Multi Bottom-up Tree Transducer Composition

Principal Problem of Top-down Tree Transducers S S PRO VP PR NP = ⇒ ∗ There VB NP Hay CD NN are CD NN dos hombres men two

Principal Problem of Top-down Tree Transducers S S PRO VP PR NP = ⇒ ∗ There VB NP Hay CD NN are CD NN dos hombres men two Notes: ◮ difficult to implement without regular look-ahead ◮ solution: use copying

Principal Problem of Top-down Tree Transducers S S PRO VP PR NP = ⇒ ∗ There VB NP Hay CD NN are CD NN dos hombres men two Notes: ◮ difficult to implement without regular look-ahead ◮ solution: use copying — No! — closure under composition

The new device Why do we not have multi-level rules? [Knight, Graehl: Training Tree Transducers. HLT-NAACL 2004]

The new device Why do we not have multi-level rules? [Knight, Graehl: Training Tree Transducers. HLT-NAACL 2004] Then we could have rules like trans S S = ⇒ PR trans PRO VP x Hay x There VB are

Formal Syntax Definition (cf. Knight & Graehl 04) An extended top-down tree transducer is a tuple M = ( Q , Σ , ∆ , S , R ) ◮ Q a finite set of states ◮ Σ and ∆ input and output ranked alphabet, respectively; ◮ S ⊆ Q a set of initial states

Formal Syntax Definition (cf. Knight & Graehl 04) An extended top-down tree transducer is a tuple M = ( Q , Σ , ∆ , S , R ) ◮ Q a finite set of states ◮ Σ and ∆ input and output ranked alphabet, respectively; ◮ S ⊆ Q a set of initial states ◮ R ⊆ Q ( T Σ ( X )) × T ∆ ( Q ( X )) a finite set of rules such that var ( r ) ⊆ var ( l ) and l is linear for every rule ( l , r ) ∈ R .

An extended top-down tree transducer Example ◮ Q = S = { ⋆ } ; ◮ Σ = ∆ = { σ ( 2 ) , α ( 0 ) } ; ◮ R contains the rules ⋆ ( σ ( σ ( x 1 , x 2 ) , x 3 )) → σ ( ⋆ ( x 1 ) , σ ( ⋆ ( x 2 ) , ⋆ ( x 3 ))) ⋆ ( α ) → α σ ⋆ σ ⋆ σ ⋆ → → α x 3 x 1 σ ⋆ ⋆ α x 1 x 2 x 2 x 3

... in action Example Rules: ⋆ ( σ ( σ ( x 1 , x 2 ) , x 3 )) → σ ( ⋆ ( x 1 ) , σ ( ⋆ ( x 2 ) , ⋆ ( x 3 ))) ⋆ ( α ) → α Derivation: σ σ σ α σ σ α σ α α α α α

... in action Example Rules: ⋆ ( σ ( σ ( x 1 , x 2 ) , x 3 )) → σ ( ⋆ ( x 1 ) , σ ( ⋆ ( x 2 ) , ⋆ ( x 3 ))) ⋆ ( α ) → α Derivation: ⋆ σ σ σ α σ σ α σ α α α α α

... in action Example Rules: ⋆ ( σ ( σ ( x 1 , x 2 ) , x 3 )) → σ ( ⋆ ( x 1 ) , σ ( ⋆ ( x 2 ) , ⋆ ( x 3 ))) ⋆ ( α ) → α Derivation: σ ⋆ σ σ ⋆ σ σ α ⋆ ⋆ ⇒ α σ σ α σ σ σ α α α σ α σ α α α α α α α

... in action Example Rules: ⋆ ( σ ( σ ( x 1 , x 2 ) , x 3 )) → σ ( ⋆ ( x 1 ) , σ ( ⋆ ( x 2 ) , ⋆ ( x 3 ))) ⋆ ( α ) → α Derivation: σ σ ⋆ σ ⋆ σ α ⋆ ⋆ α σ σ ⇒ 2 σ σ ⋆ σ ⋆ σ σ α σ α α ⋆ ⋆ α ⋆ ⋆ α α α α α α α α

... in action Example Rules: ⋆ ( σ ( σ ( x 1 , x 2 ) , x 3 )) → σ ( ⋆ ( x 1 ) , σ ( ⋆ ( x 2 ) , ⋆ ( x 3 ))) ⋆ ( α ) → α Derivation: σ σ ⋆ σ α σ α σ σ σ σ ⇒ 7 ⋆ σ ⋆ σ α σ α σ α ⋆ ⋆ α ⋆ ⋆ α α α α α α α α

... in action Example Rules: ⋆ ( σ ( σ ( x 1 , x 2 ) , x 3 )) → σ ( ⋆ ( x 1 ) , σ ( ⋆ ( x 2 ) , ⋆ ( x 3 ))) ⋆ ( α ) → α Derivation: σ σ σ σ α σ α σ σ α σ σ ⇒ ∗ σ α α α α σ α σ α α α α α α

Semantics Definition The tree transformation computed by M is τ M ⊆ T Σ × T ∆ τ M = { ( t , u ) | q ( t ) ⇒ ∗ u for some initial state q } Notation XTOP = class of transf. computed by extended tree transducers

Syntactic Restrictions Let M = ( Q , Σ , ∆ , S , R ) be an extended tree transducer. Definition M is called linear and nondeleting if for every rule l → r var ( l ) = var ( r ) and no variable appears more than once in r . Example Our example transducer with rules ⋆ ( σ ( σ ( x 1 , x 2 ) , x 3 )) → σ ( ⋆ ( x 1 ) , σ ( ⋆ ( x 2 ) , ⋆ ( x 3 ))) ⋆ ( α ) → α is linear and nondeleting.

Quest Log Question Is the class of transformations computed by linear and nondeleting extended tree transducers closed under composition? Answer [Knight & Graehl & Hopkins 07]

Quest Log Question Is the class of transformations computed by linear and nondeleting extended tree transducers closed under composition? Answer [Knight & Graehl & Hopkins 07] No! σ γ m γ i δ σ α γ j γ m Transform into γ k γ j γ k α α α α α Two linear and nondeleting extended tree transducers can do that; but a single one cannot.

Quest Log Open Problems ◮ Understand linear and nondeleting extended tree transducers better! ◮ Find subclasses that are closed under composition! ◮ Identify a suitable superclass that is closed under composition!

Quest Log Open Problems ◮ Understand linear and nondeleting extended tree transducers better! (bimorphism) ◮ Find subclasses that are closed under composition! (unsolved) ◮ Identify a suitable superclass that is closed under composition! (transformations induced by certain bottom-up devices)

Extended Top-down Tree Transducer Bimorphism Multi Bottom-up Tree Transducer Composition

Bimorphism Let Σ , ∆ , Γ be ranked alphabets. Definition A bimorphism is a triple ( ϕ, L , ψ ) with ◮ ϕ : T Γ → T Σ the input homomorphism; ◮ L ⊆ T Γ the recognizable center; ◮ ψ : T Γ → T ∆ the output homomorphism. Definition Let B = ( ϕ, L , ψ ) be a bimorphism. The tree transformation computed by B is τ B ⊆ T Σ × T ∆ τ B = { ( ϕ ( s ) , ψ ( s )) | s ∈ L } Equivalently: τ B = ϕ − 1 ◦ id L ◦ ψ (composition of relations)

Illustration Example ( ϕ, L , ψ ) bimorphism with ◮ Σ = ∆ = { σ ( 2 ) , α ( 0 ) } and Γ = { γ ( 3 ) , α ( 0 ) } ; ◮ L = T Γ ; ◮ ϕ and ψ be the homomorphisms such that ϕ ( γ ) = σ ( σ ( x 1 , x 2 ) , x 3 ) ψ ( γ ) = σ ( x 1 , σ ( x 2 , x 3 )) ϕ ( α ) = α ψ ( α ) = α

Semantics γ γ γ α α α α α α α

Semantics γ γ γ α α α α α α α ϕ ψ ϕ ψ γ γ γ γ α γ γ α α α α α α α α α α α α α

Semantics γ γ γ α α α α α α α ϕ ψ σ σ ϕ ψ σ σ ϕ ϕ γ α ψ ψ γ α α α α γ γ α α α α α α α α α

Semantics γ γ γ α α α α α α α ϕ ψ σ σ α σ σ σ σ σ ϕ α σ σ σ σ ψ ψ ϕ ϕ ϕ σ α ϕ ϕ α α α α α ψ ψ ψ ψ α α α α α α

Semantics γ γ γ α α α α α α α ϕ ψ σ σ σ σ α σ α σ σ α σ σ σ α α α α σ α σ α α α α α α

A Relation Definition Homomorphism h : T Γ → T Σ is linear and complete if h ( γ ) is linear and nondeleting in X k for every k ≥ 0 and γ ∈ Γ ( k ) . Theorem (Knight & Graehl & Hopkins 07, M. 07) Bimorphisms with linear and complete homomorphisms are as powerful as linear and nondeleting extended tree transducers. BM ( LC , LC ) = ln- XTOP

A Relation Definition Homomorphism h : T Γ → T Σ is linear and complete if h ( γ ) is linear and nondeleting in X k for every k ≥ 0 and γ ∈ Γ ( k ) . Theorem (Knight & Graehl & Hopkins 07, M. 07) Bimorphisms with linear and complete homomorphisms are as powerful as linear and nondeleting extended tree transducers. BM ( LC , LC ) = ln- XTOP Theorem (Arnold & Dauchet 82) Bimorphisms with linear and complete ε -free homomorphisms are not closed under composition. BM ( LCE , LCE ) ⊂ BM ( LCE , LCE ) 2 = BM ( LCE , LCE ) 3

Quest Log Achievement We showed that extended tree transducers consist of three (simple) phases: ◮ an inverse homomorphism (pattern matcher) ◮ a recognizable restriction (finite control) ◮ an output homomorphism (interpretation) Question ◮ Which device can implement all phases? ◮ Is the class of transformations computed by the device closed under composition?

Extended Top-down Tree Transducer Bimorphism Multi Bottom-up Tree Transducer Composition