Recursive Subtree Composition in LSTM-Based Dependency Parsing - - PowerPoint PPT Presentation

▶
recursive subtree composition in lstm based dependency
SMART_READER_LITE
LIVE PREVIEW

Recursive Subtree Composition in LSTM-Based Dependency Parsing - - PowerPoint PPT Presentation

Jan 10, 2024 321 likes •1.05k views

Recursive Subtree Composition in LSTM-Based Dependency Parsing Miryam de Lhoneux , Miguel Ballesteros and Joakim Nivre @mdlhx 4 June 2019 Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 1/22

slide-1
SLIDE 1

Recursive Subtree Composition in LSTM-Based Dependency Parsing

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre

@mdlhx 4 June 2019

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 1/22

slide-2
SLIDE 2

Overview

1

Tree vs. sequential LSTMs for parsing

2

BiLSTM parsing

3

Results

4

Conclusion

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 2/22

slide-3
SLIDE 3

Outline for section 1

1

Tree vs. sequential LSTMs for parsing

2

BiLSTM parsing

3

Results

4

Conclusion

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 3/22

slide-4
SLIDE 4

Recursive vs recurrent NNs

The largest city in Minnesota Recurrent

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 4/22

slide-5
SLIDE 5

Recursive vs recurrent NNs

The largest city in Minnesota Recursive The largest city in Minnesota Recurrent

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 4/22

slide-6
SLIDE 6

Recursive NN for Transition-Based Parsing

the largest city left-arc left-arc

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 5/22

slide-7
SLIDE 7

Recursive NN for Transition-Based Parsing

the largest city left-arc left-arc

nmod Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 5/22

slide-8
SLIDE 8

Recursive NN for Transition-Based Parsing

the largest city left-arc left-arc

nmod det Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 5/22

slide-9
SLIDE 9

Recursive NN for Transition-Based Parsing

the largest city left-arc left-arc

nmod det

Recursive composition function in the stack-LSTM parser (Dyer et al., 2015):

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 5/22

slide-10
SLIDE 10

Recursive NN for Transition-Based Parsing

the largest city left-arc left-arc

nmod det

Recursive composition function in the stack-LSTM parser (Dyer et al., 2015): c(h, d, r) = tanh(W [h; d; r] + b)W [city, largest, left − nmod]

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 5/22

slide-11
SLIDE 11

Recursive NN for Transition-Based Parsing

the largest city left-arc left-arc

nmod det

Recursive composition function in the stack-LSTM parser (Dyer et al., 2015): c(h, d, r) = tanh(W [h; d; r] + b)W [city, largest, left − nmod] hi = c(hi−1, d, r)W [city, largest, left − nmod]

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 5/22

slide-12
SLIDE 12

Recursive NN for Transition-Based Parsing

the largest city left-arc left-arc

nmod det

Recursive composition function in the stack-LSTM parser (Dyer et al., 2015): c(h, d, r) = tanh(W [h; d; r] + b)W [city, largest, left − nmod] city1 = c(city0, largest, left − nmod)

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 5/22

slide-13
SLIDE 13

Recursive NN for Transition-Based Parsing

the largest city left-arc left-arc

nmod det

Recursive composition function in the stack-LSTM parser (Dyer et al., 2015): c(h, d, r) = tanh(W [h; d; r] + b)W [city, largest, left − nmod] city1 = c(city0, largest, left − nmod) city2 = c(city1, the, left − det)

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 5/22

slide-14
SLIDE 14

Recursive vs recurrent NNs

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 6/22

slide-15
SLIDE 15

Recursive vs recurrent NNs

English PTB Chinese CTB

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 6/22

slide-16
SLIDE 16

Recursive vs recurrent NNs

English PTB Chinese CTB S-LSTM without composition 89.6 83.6

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 6/22

slide-17
SLIDE 17

Recursive vs recurrent NNs

English PTB Chinese CTB S-LSTM without composition 89.6 83.6 S-LSTM with composition 90.9 85.7

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 6/22

slide-18
SLIDE 18

Recursive vs recurrent NNs

English PTB Chinese CTB S-LSTM without composition 89.6 83.6 S-LSTM with composition 90.9 85.7 BiLSTM 91.2 85.0

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 6/22

slide-19
SLIDE 19

Recursive vs recurrent NNs

English PTB Chinese CTB S-LSTM without composition 89.6 83.6 S-LSTM with composition 90.9 85.7 BiLSTM 91.2 85.0 Goals

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 6/22

slide-20
SLIDE 20

Recursive vs recurrent NNs

English PTB Chinese CTB S-LSTM without composition 89.6 83.6 S-LSTM with composition 90.9 85.7 BiLSTM 91.2 85.0 Goals BiLSTM + composition?

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 6/22

slide-21
SLIDE 21

Recursive vs recurrent NNs

English PTB Chinese CTB S-LSTM without composition 89.6 83.6 S-LSTM with composition 90.9 85.7 BiLSTM 91.2 85.0 Goals BiLSTM + composition? Examine composition in simple architecture

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 6/22

slide-22
SLIDE 22

Recursive vs recurrent NNs

English PTB Chinese CTB S-LSTM without composition 89.6 83.6 S-LSTM with composition 90.9 85.7 BiLSTM 91.2 85.0 Goals BiLSTM + composition? Examine composition in simple architecture Typologically varied languages

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 6/22

slide-23
SLIDE 23

Outline for section 2

1

Tree vs. sequential LSTMs for parsing

2

BiLSTM parsing

3

Results

4

Conclusion

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 7/22

slide-24
SLIDE 24

Transition-Based Parsing using BiLSTMs

LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b concat concat concat concat LSTM f LSTM b MLP concat LSTM f LSTM b

STACK jumped root BUFFER Configuration: the brown fox

X X

b r

  • n

f

  • x

j u e d w m p t h e Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root e(brown) pe(brown) e(fox) pe(fox) e(jumped) pe(jumped) pe(the) e(the) (score(LEFT−ARC),score(RIGHT−ARC),score(SHIFT),score(SWAP)) Scoring:

Kiperwasser and Goldberg (2016); de Lhoneux et al. (2017) Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 8/22

slide-25
SLIDE 25

Transition-Based Parsing using BiLSTMs

Xthe

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 9/22

slide-26
SLIDE 26

Transition-Based Parsing using BiLSTMs

e(the)

Xthe

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 9/22

slide-27
SLIDE 27

Transition-Based Parsing using BiLSTMs

pe(the) e(the)

Xthe

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 9/22

slide-28
SLIDE 28

Transition-Based Parsing using BiLSTMs

concat

Cf Cf Cf Cb Cb Cb h e t pe(the) e(the)

Xthe

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 9/22

slide-29
SLIDE 29

Transition-Based Parsing using BiLSTMs

X X X X the X brown

fox jumped root Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 10/22

slide-30
SLIDE 30

Transition-Based Parsing using BiLSTMs

LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b concat concat concat concat LSTM f LSTM b concat

Vthe Vfox Vjumped Vroot Vbrown

X X X X the X brown

fox jumped root Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 10/22

slide-31
SLIDE 31

Transition-Based Parsing using BiLSTMs

LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b concat concat concat concat LSTM f LSTM b concat

Vthe Vfox Vjumped Vroot Vbrown

X X X X the X brown

fox jumped root Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 10/22

slide-32
SLIDE 32

Transition-Based Parsing using BiLSTMs

LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b concat concat concat concat LSTM f LSTM b concat

Vthe Vfox Vjumped Vroot Vbrown

X X X X the X brown

fox jumped root Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 10/22

slide-33
SLIDE 33

Transition-Based Parsing using BiLSTMs

LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b concat concat concat concat LSTM f LSTM b MLP concat LSTM f LSTM b

X X

Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root (score(LEFT−ARC),score(RIGHT−ARC),score(SHIFT),score(SWAP)) Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 10/22

slide-34
SLIDE 34

Recursive Composition in the BiLSTM parser

LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b concat concat concat concat LSTM f LSTM b concat LSTM f LSTM b

X X

Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 11/22

slide-35
SLIDE 35

Recursive Composition in the BiLSTM parser

Cthe Cbrown Cfox Cjumped Croot

LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b concat concat concat concat LSTM f LSTM b concat LSTM f LSTM b

X X

Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 11/22

slide-36
SLIDE 36

Recursive Composition in the BiLSTM parser

nmod Cthe Cbrown Cfox Cjumped Croot

LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b concat concat concat concat LSTM f LSTM b concat LSTM f LSTM b

X X

Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 11/22

slide-37
SLIDE 37

Recursive Composition in the BiLSTM parser

Cfox nmod Cthe Cbrown Cfox Cjumped Croot

LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b concat concat concat concat LSTM f LSTM b concat LSTM f LSTM b

X X

Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 11/22

slide-38
SLIDE 38

Recursive Composition in the BiLSTM parser

Cfox = tanh(W[Cfox,Cbrown,left−nmod]+b)

Cfox nmod Cthe Cbrown Cfox Cjumped Croot

LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b concat concat concat concat LSTM f LSTM b concat LSTM f LSTM b

X X

Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 11/22

slide-39
SLIDE 39

Recursive Composition in the BiLSTM parser

det Cfox Cthe Cbrown Cfox Cjumped Croot

LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b concat concat concat concat LSTM f LSTM b concat LSTM f LSTM b

X X

Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 11/22

slide-40
SLIDE 40

Recursive Composition in the BiLSTM parser

Cfox det Cfox Cthe Cbrown Cfox Cjumped Croot

LSTM f LSTM b LSTM f LSTM b LSTM f LSTM b concat concat concat concat LSTM f LSTM b concat LSTM f LSTM b

X X

Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 11/22

slide-41
SLIDE 41

Recursive Composition in the BiLSTM parser

chead = tanh(W [h; d; r] + b) + rc

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 12/22

slide-42
SLIDE 42

Recursive Composition in the BiLSTM parser

chead = tanh(W [h; d; r] + b) + rc

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 12/22

slide-43
SLIDE 43

Recursive Composition in the BiLSTM parser

chead = tanh(W [h; d; r] + b) + rc chead = Lstm([h; d; r]) + lc

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 12/22

slide-44
SLIDE 44

Recursive Composition in the BiLSTM parser

chead = tanh(W [h; d; r] + b) + rc chead = Lstm([h; d; r]) + lc

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 12/22

slide-45
SLIDE 45

Outline for section 3

1

Tree vs. sequential LSTMs for parsing

2

BiLSTM parsing

3

Results

4

Conclusion

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 13/22

slide-46
SLIDE 46

Results: BiLSTM + composition

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 14/22

slide-47
SLIDE 47

Results: BiLSTM + composition

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 14/22

slide-48
SLIDE 48

Results: BiLSTM + composition

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 14/22

slide-49
SLIDE 49

LSTM Feature Extractors

concat concat concat concat concat

b

LSTM

b

LSTM

b

LSTM

b

LSTM

b

LSTM

f

LSTM

f

LSTM

f

LSTM

f

LSTM

f

LSTM

X X

Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root

bw

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 15/22

slide-50
SLIDE 50

LSTM Feature Extractors

b

LSTM

b

LSTM

b

LSTM

b

LSTM

b

LSTM

f

LSTM

f

LSTM

f

LSTM

f

LSTM

f

LSTM

X X

Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root

bw

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 15/22

slide-51
SLIDE 51

LSTM Feature Extractors

b

LSTM

b

LSTM

b

LSTM

b

LSTM

b

LSTM

X X

Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root

bw

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 15/22

slide-52
SLIDE 52

LSTM Feature Extractors

f

LSTM

f

LSTM

f

LSTM

f

LSTM

f

LSTM

X X

Vthe Vfox Vjumped Vroot

X

Vbrown

X the X brown

fox jumped root

fw

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 15/22

slide-53
SLIDE 53

Results: BiLSTM ablations

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 16/22

slide-54
SLIDE 54

Results: BiLSTM ablations

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 16/22

slide-55
SLIDE 55

Results: BiLSTM ablations

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 16/22

slide-56
SLIDE 56

Results: BiLSTM ablations + composition

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 17/22

slide-57
SLIDE 57

Results: BiLSTM ablations + composition

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 17/22

slide-58
SLIDE 58

Results: BiLSTM ablations + composition

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 17/22

slide-59
SLIDE 59

Word representation

concat

Cf Cf Cf Cb Cb Cb h e t pe(the) e(the)

Xthe

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 18/22

slide-60
SLIDE 60

Word representation

+pos

concat

Cf Cf Cf Cb Cb Cb h e t pe(the) e(the)

Xthe

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 18/22

slide-61
SLIDE 61

Word representation

+char +pos

concat

Cf Cf Cf Cb Cb Cb h e t pe(the) e(the)

Xthe

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 18/22

slide-62
SLIDE 62

Composition gap recovery

[bw+lc]-bw bi-bw %rec. [fw+lc]-fw bi-fw %rec. pos+char+ 1.4 1.6 87.5 0.6 6.3 9.5 pos+char- 1.3 1.8 72.2 0.6 6.6 9.1 pos-char+ 1.6 1.9 84.2 0.7 7.3 9.6 pos-char- 2 3.1 64.5 1 8.7 11.5

av.

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 19/22

slide-63
SLIDE 63

Composition gap recovery

[bw+lc]-bw bi-bw %rec. [fw+lc]-fw bi-fw %rec. pos+char+ 1.4 1.6 87.5 0.6 6.3 9.5 pos+char- 1.3 1.8 72.2 0.6 6.6 9.1 pos-char+ 1.6 1.9 84.2 0.7 7.3 9.6 pos-char- 2 3.1 64.5 1 8.7 11.5

av.

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 19/22

slide-64
SLIDE 64

Outline for section 4

1

Tree vs. sequential LSTMs for parsing

2

BiLSTM parsing

3

Results

4

Conclusion

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 20/22

slide-65
SLIDE 65

Conclusion Subtree composition does not reliably help a BiLSTM transition-based parser

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 21/22

slide-66
SLIDE 66

Conclusion Subtree composition does not reliably help a BiLSTM transition-based parser The backward part of the BiLSTM is crucial, especially for right-headed languages

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 21/22

slide-67
SLIDE 67

Conclusion Subtree composition does not reliably help a BiLSTM transition-based parser The backward part of the BiLSTM is crucial, especially for right-headed languages The forward part of the BiLSTM is less crucial

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 21/22

slide-68
SLIDE 68

Conclusion Subtree composition does not reliably help a BiLSTM transition-based parser The backward part of the BiLSTM is crucial, especially for right-headed languages The forward part of the BiLSTM is less crucial A backward LSTM + subtree composition performs close to a BiLSTM

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 21/22

slide-69
SLIDE 69

Conclusion Subtree composition does not reliably help a BiLSTM transition-based parser The backward part of the BiLSTM is crucial, especially for right-headed languages The forward part of the BiLSTM is less crucial A backward LSTM + subtree composition performs close to a BiLSTM POS information and subtree composition are two partially redundant ways of constructing contextual information

Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 21/22

slide-70
SLIDE 70

References

Miryam de Lhoneux, Yan Shao, Ali Basirat, Eliyahu Kiperwasser, Sara Stymne, Yoav Goldberg, and Joakim Nivre.

  • 2017. From raw text to universal dependencies - look, no tags! In Proceedings of the CoNLL 2017 Shared

Task: Multilingual Parsing from Raw Text to Universal Dependencies. Association for Computational Linguistics, Vancouver, Canada, pages 207–217. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers. pages 334–343. http://aclweb.org/anthology/P/P15/P15-1033.pdf. Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and accurate dependency parsing using bidirectional LSTM feature representations. Transactions of the Association for Computational Linguistics 4:313–327. Miryam de Lhoneux, Miguel Ballesteros and Joakim Nivre Recursive Subtree Composition in Parsing 22/22