On Different Approaches to Syntactic Analysis into Bi-Lexical - - PowerPoint PPT Presentation

Introduction DT Setup Results Conclusions

On Different Approaches to Syntactic Analysis into Bi-Lexical Dependencies

An Empirical comparison of Direct, PCFG-Based, and HPSG-Based Parsers Angelina Ivanova♠, Stephan Oepen♠♥, Rebecca Dridan♠, Dan Flickinger♣, Lilja Øvrelid♠

♠ University of Oslo ♥ Potsdam University ♣ Stanford University

The 13th International Conference on Parsing Technologies Nara, Japan, 2013

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 1/26

Introduction DT Setup Results Conclusions

Research Question

How does HPSG grammar-based parsing relate to PCFG and direct dependency approaches in terms of accuracy, efficiency and domain resilience for recovering bilexical dependencies?

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 2/26

Introduction DT Setup Results Conclusions

Motivation

◮ Comparison of parsers of different frameworks is challenging; ◮ Heuristic conversion introduces fuzziness in parsing results

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 3/26

Introduction DT Setup Results Conclusions

Related Work

◮ Grammar-based parser is not necessarily more accurate than

PCFG-based parser (Fowler and Penn, 2010)

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 4/26

Introduction DT Setup Results Conclusions

Related Work

◮ Grammar-based parser is not necessarily more accurate than

PCFG-based parser (Fowler and Penn, 2010)

◮ Grammar-based parser for Dutch is more accurate and

domain-resilient than direct dependency parsers (Plank and van Noord, 2010)

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 4/26

Introduction DT Setup Results Conclusions

Related Work

◮ Grammar-based parser is not necessarily more accurate than

PCFG-based parser (Fowler and Penn, 2010)

◮ Grammar-based parser for Dutch is more accurate and

domain-resilient than direct dependency parsers (Plank and van Noord, 2010)

◮ Grammatical Relations and Stanford Bilexical Dependencies as

a framework-independent parser comparison (Miyao et al., 2007)

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 4/26

Introduction DT Setup Results Conclusions

Related Work

◮ Grammar-based parser is not necessarily more accurate than

PCFG-based parser (Fowler and Penn, 2010)

◮ Grammar-based parser for Dutch is more accurate and

domain-resilient than direct dependency parsers (Plank and van Noord, 2010)

◮ Grammatical Relations and Stanford Bilexical Dependencies as

a framework-independent parser comparison (Miyao et al., 2007)

◮ CFG parsers are more accurate than direct dependency parsers

• n recovering bilexical Stanford Dependencies

(Cer et al., 2010)

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 4/26

Introduction DT Setup Results Conclusions

Experiment Setup

Cross-framework parser evaluation on bilexical syntactic dependencies

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 5/26

Introduction DT Setup Results Conclusions

DT Dependencies

Sun- filled Mountain View didn’t impress me. n - pn le v np noger le n - pn le v vp did-n le v np* le n - pr-me le

root sb-hd aj-hdn n-nh hd-cmp hd-cmp

⋆ DT: Derivation Tree–Derived Bi-Lexical Syntactic Dependencies (Ivanova et al., 2012) ⋆ Derived from English Resource Grammar (ERG; Flickinger, 2000) derivations ⋆ 48 broad HPSG constructions as dependency labels ⋆ About 1000 ERG lexical types as parts-of-speech ⋆ Deviate from PTB tokenization assumptions ⋆ Correspond to CoNLL in terms of Jaccard similarity over unlabeled dependencies

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 6/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

Reduction by Ivanova et al. (2012)

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Sun- filled Mountain View didn’t impress me. n - pn le v np noger le n - pn le v vp did-n le v np* le n - pr-me le

root sb-hd aj-hdn n-nh hd-cmp hd-cmp

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

Reduction by Ivanova et al. (2012)

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Sun- filled Mountain View didn’t impress me. n - pn le v np noger le n - pn le v vp did-n le v np* le n - pr-me le

root sb-hd aj-hdn n-nh hd-cmp hd-cmp

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

Reduction by Ivanova et al. (2012)

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Sun- filled Mountain View didn’t impress me. n - pn le v np noger le n - pn le v vp did-n le v np* le n - pr-me le

root sb-hd aj-hdn n-nh hd-cmp hd-cmp

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

ERG derivation & DT

Reduction by Ivanova et al. (2012)

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me. Sun- filled Mountain View didn’t impress me. n - pn le v np noger le n - pn le v vp did-n le v np* le n - pr-me le

root sb-hd aj-hdn n-nh hd-cmp hd-cmp

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions

DT Dependencies. Tokenization

PTB convention ERG convention Punctuation bark . bark. Multiword expressions such as such as Hyphenated words end-state end- state

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 8/26

Introduction DT Setup Results Conclusions

Motivation to use DT

Replication of results of (Ivanova et al., 2013) Gold PTB tags Predicted PTB tags Malt MST Bohnet and Nivre (2012) Stanford Basic 89.58 88.94 90.43 CoNLL 88.70 89.13 90.53 DT 87.19 88.16 90.48

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 9/26

Introduction DT Setup Results Conclusions

Motivation to use DT

Replication of results of (Ivanova et al., 2013) Gold PTB tags Predicted PTB tags Malt MST Bohnet and Nivre (2012) Stanford Basic 89.58 88.94 90.43 CoNLL 88.70 89.13 90.53 DT 87.19 88.16 90.48

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 9/26

Introduction DT Setup Results Conclusions

Data

Deepbank

◮ Flickinger et al. (2012) ◮ Sections 0-21 of WSJ annotated with the English Resource

Grammar (ERG; Flickinger, 2000).

◮ 15% of WSJ text is not covered by the gold standard

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 10/26

Introduction DT Setup Results Conclusions

Data

Training set sections 0-19 of DeepBank 33,783 sent. Tune set section 20 of DeepBank 1,721 sent. Test sets WSJ Section 21 of DeepBank 1,414 sent. SC Part of the SemCore Corpus 864 sent. WS Part of the WeScience corpus 520 sent.

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 11/26

Introduction DT Setup Results Conclusions

Parsers

⋆ PET (Callmeier, 2002) Two versions:

◮ “accurate” ERGa ◮ “efficient” ERGe (Dridan, 2013)

⋆ Bohnet and Nivre (2012) ⋆ Berkeley (Petrov et al., 2006)

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 12/26

Introduction DT Setup Results Conclusions

ERG derivation

sb-hd mc c hdn bnp-pn c aj-hdn norm c n-nh v-cpd c w hyphen plr n - pn le Sun- v pas odlr v np noger le filled n sg ilr n - pn le Mountain View hd-cmp u c v vp did-n le didn’t hd-cmp u c v n3s-bse ilr v np* le impress hdn bnp-qnt c w period plr n - pr-me le me.

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 13/26

Introduction DT Setup Results Conclusions

PCFG Parsing of HPSG Derivations

Unary rules preserved Unary rules removed Labels Long Short Gaps 2 Tagging Acc. 90.96 89.16 F1 76.39 75.15 LAS 86.26 80.49 UAS 89.34 87.95

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 14/26

Introduction DT Setup Results Conclusions

Evaluation

eval.pl scorer

• I. Problem: parse failure

Solution: dummy dependencies 1 Odds 2 and 3 Ends

• II. Problem: tokenization mismatch

Solution: character ranges instead of tokens (Dridan and Oepen, 2013)

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 15/26

Introduction DT Setup Results Conclusions

Evaluation

• II. Problem: tokenization mismatch

Solution: character ranges instead of tokens (Dridan and Oepen, 2013) RAW in the ’70s

0 1 2 3 4 5 6 7 8 9 10

GOLD PARSED in the ’ 70s in the ’70s < 0, 2, in > < 0, 2, in > < 3, 6, the > < 3, 6, the > < 7, 8, ’ > < 7, 11, ’70s > < 8, 11, 70s >

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 16/26

Introduction DT Setup Results Conclusions

In-Domain Results

Gaps Time TA LAS Berkeley 1+0 1.0 92.9 86.65 B&N 0+0 1.7 92.9 86.76 ERGa 0+0 10 97.8 92.87 ERGe 13+44 1.8 96.4 91.60

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 17/26

Introduction DT Setup Results Conclusions

In-Domain Results

Gaps Time TA LAS Berkeley 1+0 1.0 92.9 86.65 B&N 0+0 1.7 92.9 86.76 ERGa 0+0 10 97.8 92.87 ERGe 13+44 1.8 96.4 91.60

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 17/26

Introduction DT Setup Results Conclusions

In-Domain Results

Gaps Time TA LAS Berkeley 1+0 1.0 92.9 86.65 B&N 0+0 1.7 92.9 86.76 ERGa 0+0 10 97.8 92.87 ERGe 13+44 1.8 96.4 91.60

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 17/26

Introduction DT Setup Results Conclusions

In-Domain Results

Gaps Time TA LAS Berkeley 1+0 1.0 92.9 86.65 B&N 0+0 1.7 92.9 86.76 ERGa 0+0 10 97.8 92.87 ERGe 13+44 1.8 96.4 91.60

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 17/26

Introduction DT Setup Results Conclusions

In-Domain Results

Gaps Time TA LAS Berkeley 1+0 1.0 92.9 86.65 B&N 0+0 1.7 92.9 86.76 ERGa 0+0 10 97.8 92.87 ERGe 13+44 1.8 96.4 91.60

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 17/26

Introduction DT Setup Results Conclusions

In-Domain Results

Gaps Time TA LAS Berkeley 1+0 1.0 92.9 86.65 B&N 0+0 1.7 92.9 86.76 ERGa 0+0 10 97.8 92.87 ERGe 13+44 1.8 96.4 91.60

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 17/26

Introduction DT Setup Results Conclusions

Error Analysis

∗ Accuracy of dependency types Among the hardest types for all three parsers is an adjunct, especially hdn-aj: post-adjunction to a nominal head ∗ Long-distant dependencies Parsers perform comparably with ERGa having the highest accuracy

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 18/26

Introduction DT Setup Results Conclusions

Error Analysis

∗ LAS over lexical categories The hardest for all parsers: conjunctions (c) and prepositions (p and pp)

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 19/26

Introduction DT Setup Results Conclusions

Error Analysis

Examples where ERGa performs well p np ptcl-of le history of Linux p np ptcl le look for piece p np i le meet on Monday

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 20/26

Introduction DT Setup Results Conclusions

Cross-Domain Results

Gaps TA LAS WSJ Berkeley 1+0 92.9 86.65 B&N 0+0 92.9 86.76 ERGe 13 + 44 96.4 91.60 SC Berkeley 1+0 87.2 79.81 B&N 0+0 85.9 78.08 ERGe 11+7 94.9 89.94 WS Berkeley 7+0 87.7 80.31 B&N 0+0 88.4 80.63 ERGe 4+12 96.9 90.64

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 21/26

Introduction DT Setup Results Conclusions

Cross-Domain Results

Gaps TA LAS WSJ Berkeley 1+0 92.9 86.65 B&N 0+0 92.9 86.76 ERGe 13 + 44 96.4 91.60 SC Berkeley 1+0 87.2 79.81 B&N 0+0 85.9 78.08 ERGe 11+7 94.9 89.94 WS Berkeley 7+0 87.7 80.31 B&N 0+0 88.4 80.63 ERGe 4+12 96.9 90.64

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 21/26

Introduction DT Setup Results Conclusions

Cross-Domain Results

Gaps TA LAS WSJ Berkeley 1+0 92.9 86.65 B&N 0+0 92.9 86.76 ERGe 13 + 44 96.4 91.60 SC Berkeley 1+0 87.2 79.81 B&N 0+0 85.9 78.08 ERGe 11+7 94.9 89.94 WS Berkeley 7+0 87.7 80.31 B&N 0+0 88.4 80.63 ERGe 4+12 96.9 90.64

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 21/26

Introduction DT Setup Results Conclusions

Cross-Domain Results

Gaps TA LAS WSJ Berkeley 1+0 92.9 86.65 B&N 0+0 92.9 86.76 ERGe 13 + 44 96.4 91.60 SC Berkeley 1+0 87.2 79.81 B&N 0+0 85.9 78.08 ERGe 11+7 94.9 89.94 WS Berkeley 7+0 87.7 80.31 B&N 0+0 88.4 80.63 ERGe 4+12 96.9 90.64

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 21/26

Introduction DT Setup Results Conclusions

PTB Tokenization

Lexical Types PTB PoS Tags Gaps LAS LAS WSJ B&N 0+0 88.78 91.56 ERGe 13+9 92.38 92.38 SC B&N 0+0 81.69 85.17 ERGe 11+0 90.13 90.13 WS B&N 0+0 82.09 84.59 ERGe 4+0 91.61 91.61

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 22/26

Introduction DT Setup Results Conclusions

PTB Tokenization

Lexical Types PTB PoS Tags Gaps LAS LAS WSJ B&N 0+0 88.78 91.56 ERGe 13+9 92.38 92.38 SC B&N 0+0 81.69 85.17 ERGe 11+0 90.13 90.13 WS B&N 0+0 82.09 84.59 ERGe 4+0 91.61 91.61

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 22/26

Introduction DT Setup Results Conclusions

PTB Tokenization

Lexical Types PTB PoS Tags Gaps LAS LAS WSJ B&N 0+0 88.78 91.56 ERGe 13+9 92.38 92.38 SC B&N 0+0 81.69 85.17 ERGe 11+0 90.13 90.13 WS B&N 0+0 82.09 84.59 ERGe 4+0 91.61 91.61

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 22/26

Introduction DT Setup Results Conclusions

PTB Tokenization

Lexical Types PTB PoS Tags Gaps LAS LAS WSJ B&N 0+0 88.78 91.56 ERGe 13+9 92.38 92.38 SC B&N 0+0 81.69 85.17 ERGe 11+0 90.13 90.13 WS B&N 0+0 82.09 84.59 ERGe 4+0 91.61 91.61

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 22/26

Introduction DT Setup Results Conclusions

Conclusions

English Resource Grammar-driven parser vs. PCFG Berkeley parser and Bohnet and Nivre dependency parser: – superior in-domain and cross-domain accuracy – comparable efficiency – incomplete coverage Berkeley vs. Bohnet and Nivre (2012): – equal accuracy on in-domain data Berkeley performs better on SC and almost equally on WS – Berkeley has superior efficiency – Berkeley is challenged in coverage

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 23/26

Introduction DT Setup Results Conclusions

Conclusions

Not very likely but possible: 15% of missing WSJ sentences are difficult for PET but easy for statistical parsers

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 24/26

Introduction DT Setup Results Conclusions

Conclusions

Tentative generalization: linguistically richer representations enable more accurate parsing

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 25/26

Introduction DT Setup Results Conclusions

Thank You!

Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 26/26