Fast(er) Exact Decoding and Global Training for Transition-Based Dependency Parsing via a Minimal Feature Set Tianze Shi* Liang Huangβ Lillian Lee* β Oregon State University * Cornell University π π 3 Minimal π π 3 π π 6 Feature Set Theoretical Practical
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Short Version β’ Transition-based dependency parsing has an exponentially-large search space β’ π π 3 exact solutions exist π β’ In practice, however, we needed rich features βΉ π π 6 π β’ (This work) with bi-LSTMs, now we can do π(π 3 ) ! π β’ And we get state-of-the-art results 2
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Short Version β’ Transition-based dependency parsing has an exponentially-large search space β’ π π 3 exact solutions exist π β’ In practice, however, we needed rich features βΉ π π 6 π β’ (This work) with bi-LSTMs, now we can do π(π 3 ) ! π β’ And we get state-of-the-art results 3
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Short Version β’ Transition-based dependency parsing has an exponentially-large search space β’ π π 3 exact solutions exist π β’ In practice, however, we needed rich features βΉ π π 6 π β’ (This work) with bi-LSTMs, now we can do π(π 3 ) ! π β’ And we get state-of-the-art results 4
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Short Version β’ Transition-based dependency parsing has an exponentially-large search space β’ π π 3 exact solutions exist π β’ In practice, however, we needed rich features βΉ π π 6 π β’ (This work) with bi-LSTMs, now we can do π(π 3 ) ! π β’ And we get state-of-the-art results 5
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Short Version β’ Transition-based dependency parsing has an exponentially-large search space β’ π π 3 exact solutions exist π β’ In practice, however, we needed rich features βΉ π π 6 π β’ (This work) with bi-LSTMs, now we can do π(π 3 ) ! π β’ And we get state-of-the-art results 6
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Dependency Parsing root obj xcomp nsubj mark det OUTPUT She wanted to eat an apple INPUT 7
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Transition-based Dependency Parsing β¦ Terminal β¦ states Initial state β¦ β¦ β¦ 8
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Transition-based Dependency Parsing Goal: max score( ) β¦ β¦ = max β score( ) Terminal β¦ states Initial state β¦ β¦ β¦ 9
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Exact Decoding with Dynamic Programming Goal: max score( ) β¦ Exponential to polynomial = max β score( ) β¦ Terminal β¦ states Initial state β¦ β¦ β¦ (Huang and Sagae, 2010; Kuhlmann, 10 GΓ³mez-RodrΓguez and Satta, 2011)
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Transition Systems DP Complexity # Action Types Arc-standard 3 π π 4 In our Arc-eager 4 π· π π paper Arc-hybrid 3 π· π π Presentational convenience 11
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System State β¦ β¦ π‘ 2 π‘ 1 π‘ 0 π 0 π 1 Stack Buffer Initial State ROOT She wanted β¦ Terminal State (Yamada and Matsumoto, 2003) ROOT (GΓ³mez-RodrΓguez et al., 2008) (Kuhlmann et al., 2011) 12
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Transitions β¦ β¦ β¦ β¦ β¦ β¦ π‘ 1 π‘ 0 π‘ 0 π 0 π 0 shift reduce β· reduce βΆ β¦ β¦ β¦ β¦ π‘ 1 β¦ β¦ π 0 π 0 π‘ 0 π‘ 0 Same as arc-standard 13
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Transitions β¦ β¦ β¦ β¦ β¦ β¦ π‘ 1 π‘ 0 π‘ 0 π 0 π 0 shift reduce β· reduce βΆ β¦ β¦ β¦ β¦ π‘ 1 β¦ β¦ π 0 π 0 π‘ 0 π‘ 0 Same as arc-standard 14
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Transitions β¦ β¦ β¦ β¦ β¦ β¦ π‘ 1 π‘ 0 π‘ 0 π 0 π 0 shift reduce β· reduce βΆ β¦ β¦ β¦ β¦ π‘ 1 β¦ β¦ π 0 π 0 π‘ 0 π‘ 0 Same as arc-standard 15
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Stack Buffer initial ROOT She wanted to eat an apple shift She wanted to eat an apple ROOT shift wanted to eat an apple ROOT She reduce βΆ wanted to eat an apple ROOT shift She to eat an apple ROOT wanted shift eat an apple ROOT wanted to 16
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Stack Buffer initial ROOT She wanted to eat an apple shift She wanted to eat an apple ROOT shift wanted to eat an apple ROOT She reduce βΆ wanted to eat an apple ROOT shift She to eat an apple ROOT wanted shift eat an apple ROOT wanted to 17
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Stack Buffer initial ROOT She wanted to eat an apple shift She wanted to eat an apple ROOT shift wanted to eat an apple ROOT She reduce βΆ wanted to eat an apple ROOT shift She to eat an apple ROOT wanted shift eat an apple ROOT wanted to 18
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Stack Buffer initial ROOT She wanted to eat an apple shift She wanted to eat an apple ROOT shift wanted to eat an apple ROOT She reduce βΆ wanted to eat an apple ROOT shift She to eat an apple ROOT wanted shift eat an apple ROOT wanted to 19
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Stack Buffer initial ROOT She wanted to eat an apple shift She wanted to eat an apple ROOT shift wanted to eat an apple ROOT She reduce βΆ wanted to eat an apple ROOT shift She to eat an apple ROOT wanted shift eat an apple ROOT wanted to 20
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Stack Buffer initial ROOT She wanted to eat an apple shift She wanted to eat an apple ROOT shift wanted to eat an apple ROOT She reduce βΆ wanted to eat an apple ROOT shift She to eat an apple ROOT wanted shift eat an apple ROOT wanted to 21
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Stack Buffer eat an apple ROOT wanted to reduce βΆ eat an apple ROOT wanted shift to an apple ROOT wanted eat shift apple ROOT wanted eat an reduce βΆ apple ROOT wanted eat shift an ROOT wanted eat apple 22
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Stack Buffer eat an apple ROOT wanted to reduce βΆ eat an apple ROOT wanted shift to an apple ROOT wanted eat shift apple ROOT wanted eat an reduce βΆ apple ROOT wanted eat shift an ROOT wanted eat apple 23
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Stack Buffer eat an apple ROOT wanted to reduce βΆ eat an apple ROOT wanted shift to an apple ROOT wanted eat shift apple ROOT wanted eat an reduce βΆ apple ROOT wanted eat shift an ROOT wanted eat apple 24
Background π(π 3 ) in theory π(π 6 ) in practice Back to π(π 3 ) Results Arc-hybrid Transition System Stack Buffer eat an apple ROOT wanted to reduce βΆ eat an apple ROOT wanted shift to an apple ROOT wanted eat shift apple ROOT wanted eat an reduce βΆ apple ROOT wanted eat shift an ROOT wanted eat apple 25
Recommend
More recommend
![In Transition: Gender [Identity], Law & Global Health USC LAW & GLOBAL HEALTH](https://c.sambuz.com/685997/in-transition-gender-identity-law-global-health-s.jpg)
