dependency parsing as head selection
play

Dependency Parsing as Head Selection Xingxing Zhang , Jianpeng Cheng, - PowerPoint PPT Presentation

Feb 14, 2023 •272 likes •611 views

Dependency Parsing as Head Selection Xingxing Zhang , Jianpeng Cheng, Mirella Lapata Institute for Language, Cognition and Computation University of Edinburgh x.zhang@ed.ac.uk April 6, 2017 Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency

  1. Dependency Parsing as Head Selection Xingxing Zhang , Jianpeng Cheng, Mirella Lapata Institute for Language, Cognition and Computation University of Edinburgh x.zhang@ed.ac.uk April 6, 2017 Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 1 / 18

  2. Dependency Parsing Dependency Parsing is the task of transforming a sentence S = ( root , w 1 , w 2 , . . . , w N ) into a directed tree originating out of root . Parsing Algorithms Transition-based Parsing Graph-based Parsing Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 2 / 18

  3. Dependency Parsing Dependency Parsing is the task of transforming a sentence S = ( root , w 1 , w 2 , . . . , w N ) into a directed tree originating out of root . Parsing Algorithms Transition-based Parsing Graph-based Parsing Our parser is neither Transition-based nor Graph-based (during training) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 2 / 18

  4. Transition-based Parsing Data Structure Buffer, Stack, Arc Set Parsing: Choose an action from SHIFT REDUCE-Left REDUCE-Right Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 3 / 18

  5. Graph-based Parsing A Sentence → A Directed Complete Graph (Graphs from Kubler et al., 2009) Parsing: Finding Maximum Spanning Tree Chu-Liu-Edmond algorithm (Chu and Liu, 1965) Eisner algorithm (Eisner 1996) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 4 / 18

  6. Recent Advances Mostly replacing discrete features with Neural Network features. Transition-based Parsers Feed-Forward NN features (Chen and Manning, 2014) Bi-LSTM features (Kiperwasser and Goldberg, 2016) Stack LSTM: Buffer, Stack and Action Sequences modeled by Stack-LSTMs (Dyer et al., 2015) Graph-based Parsers Tensor Decomposition features (Lei et al., 2014) Feed-Forward NN features (Pei et al., 2015) Bi-LSTM features (Kiperwasser and Goldberg, 2016) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 5 / 18

  7. Do we need a transition system or graph algorithm? root kids love candy Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 6 / 18

  8. Do we need a transition system or graph algorithm? root kids love candy An important fact: Every word has only one head! Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 6 / 18

  9. Do we need a transition system or graph algorithm? root kids love candy An important fact: Every word has only one head! Why not just learn to select the head? Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 6 / 18

  10. Dependency Parsing as Head Selection DeNSe : De pendency N eural Se lection Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 7 / 18

  11. Dependency Parsing as Head Selection DeNSe : De pendency N eural Se lection Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 7 / 18

  12. Dependency Parsing as Head Selection DeNSe : De pendency N eural Se lection Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 7 / 18

  13. Dependency Parsing as Head Selection DeNSe : De pendency N eural Se lection exp( MLP ( a root , a love )) P head ( root | love , S ) = � 3 k =0 exp( MLP ( a k , a love )) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 7 / 18

  14. Dependency Parsing as Head Selection DeNSe : De pendency N eural Se lection exp( MLP ( a root , a love )) P head ( root | love , S ) = � 3 k =0 exp( MLP ( a k , a love )) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 7 / 18

  15. Decoding Greedy Decoding: The output may not be a (projective) tree! Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 8 / 18

  16. Decoding Greedy Decoding: The output may not be a (projective) tree! Greedy Decoding Dataset #Sent (Dev) Tree Proj PTB (English) 1,700 95.1 86.6 CTB (Chinese) 803 87.0 73.1 Czech 374 87.7 65.5 German 367 96.7 67.3 Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 8 / 18

  17. Decoding Greedy Decoding: The output may not be a (projective) tree! Greedy Decoding Dataset #Sent (Dev) Tree Proj PTB (English) 1,700 95.1 86.6 CTB (Chinese) 803 87.0 73.1 Czech 374 87.7 65.5 German 367 96.7 67.3 Decoding with a Maximum Spanning Tree Algorithm (relatively rare) Projective Parsing: Eisner Algorithm Non-projective Parsing: Chu-Liu-Edmond Algorithm Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 8 / 18

  18. Labelled Parser A two-layer Rectifier Network (Glorot et al., 2011) Dependent Word: Bi-LSTM Feature Word Embedding PoS Embedding Head Word: Bi-LSTM Feature Word Embedding PoS Embedding Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 9 / 18

  19. Experiments Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 10 / 18

  20. Projective Parsing Results (PTB; English) NN (Chen & Manning, 2014); S-LSTM (Dyer et al., 2015); Bi-LSTM (Kiperwasser & Goldberg, 2016); SynNet (Andor et al. 2016) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 11 / 18

  21. Projective Parsing Results (PTB; English) NN (Chen & Manning, 2014); S-LSTM (Dyer et al., 2015); Bi-LSTM (Kiperwasser & Goldberg, 2016); SynNet (Andor et al. 2016) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 11 / 18

  22. Projective Parsing Results (PTB; English) NN (Chen & Manning, 2014); S-LSTM (Dyer et al., 2015); Bi-LSTM (Kiperwasser & Goldberg, 2016); SynNet (Andor et al. 2016) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 11 / 18

  23. Projective Parsing Results (PTB; Chinese) NN (Chen & Manning, 2014); S-LSTM (Dyer et al., 2015); Bi-LSTM (Kiperwasser & Goldberg, 2016); 3rd-cubic (Zhang & McDonald 2014) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 12 / 18

  24. Non-projective Parsing Results (German) MST-1st, MST-2nd (McDonald et al., 2005) Turbo-1st, Turbo-3rd (Martins et al., 2013) RBG-1st RBG-3rd (Martins et al. 2013) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 13 / 18

  25. Non-projective Parsing Results (German) MST-1st, MST-2nd (McDonald et al., 2005) Turbo-1st, Turbo-3rd (Martins et al., 2013) RBG-1st RBG-3rd (Martins et al. 2013) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 13 / 18

  26. Non-projective Parsing Results (Czech) MST-1st, MST-2nd ((McDonald et al., 2005) Turbo-1st, Turbo-3rd (Martins et al., 2013) RBG-1st RBG-3rd (Martins et al. 2013) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 14 / 18

  27. Non-projective Parsing Results (Czech) MST-1st, MST-2nd ((McDonald et al., 2005) Turbo-1st, Turbo-3rd (Martins et al., 2013) RBG-1st RBG-3rd (Martins et al. 2013) Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 14 / 18

  28. Unlabeled Exact Match PTB CTB Parser Dev Test Dev Test C&M14 43.35 40.93 32.75 32.20 Dyer15 51.94 50.70 39.72 37.23 DeNSe 51.24 49.34 34.74 33.66 DeNSe +E 36.49 35.13 52.47 50.79 Table: UEM results on PTB and CTB. Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 15 / 18

  29. UAS v.s. Length 96 95 94 UAS (%) 93 92 91 C&M14 DeNSe+E 90 Dyer15 89 11 14 17 20 23 26 28 32 38 118 PTB sentence length Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 16 / 18

  30. UAS v.s. Length 93 92 91 90 89 88 UAS (%) 87 86 85 84 83 C&M14 DeNSe+E 82 Dyer15 81 80 5 9 14 18 22 26 30 37 49 116 PTB sentence length CTB CTB Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 16 / 18

  31. Conclusions We propose a dependency parser as greedily selecting the head of each word in sentence. Combine the greedy model with a MST algorithm can further increase the performance Code available: https://github.com/XingxingZhang/dense parser Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 17 / 18

  32. Thanks Q & A Zhang et al. (Univ. of Edinburgh) DeNSe : Dependency Neural Selection April 6, 2017 18 / 18

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Last class Dependency parsing and logistic regression Dependency parsing: a fully lexicalized

Last class Dependency parsing and logistic regression Dependency parsing: a fully lexicalized

Last class Dependency parsing and logistic regression Dependency parsing: a fully lexicalized formalism; tree edges connect words in the Shay Cohen sentence based on head-dependent relationships. (based on slides by Sharon Goldwater) a

359 views • 11 slides
Dynamic Feature Selection for Dependency Parsing He He, Hal Daum III and Jason Eisner EMNLP

Dynamic Feature Selection for Dependency Parsing He He, Hal Daum III and Jason Eisner EMNLP

Dynamic Feature Selection for Dependency Parsing He He, Hal Daum III and Jason Eisner EMNLP 2013, Seattle Structured Prediction in NLP Part-of-Speech Tagging Parsing N N V Det N

1.59k views • 102 slides
Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr

Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr

Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr Sprachwissenschaft November 2016 Recap/background Dependency grammar Dependency parsing Evaluation Summary Ingredients of a parser

1.44k views • 89 slides
Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP

Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP

Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP February 2, 2015 Roadmap Dependency parsing Graph-based dependency parsing Maximum spanning tree CLE Algorithm Learning

899 views • 87 slides
Dependency Parsing 2 CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre,

Dependency Parsing 2 CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre,

Dependency Parsing 2 CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan Jurafsky & James Martin Dependency Parsing Formalizing dependency trees Transition-based dependency parsing Shift-reduce parsing

1.02k views • 52 slides
Dependency Parsing CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan

Dependency Parsing CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan

Dependency Parsing CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan Jurafsky & James Martin Dependency Parsing Formalizing dependency trees Transition-based dependency parsing Shift-reduce parsing

676 views • 24 slides
Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency

Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency

Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency Parsing Dependency Parsing Pure dependency parsing is only cubic [Eisner 99] Lexicalized parsers can be seen as producing dependency trees

314 views • 5 slides
Dependency Parsing as Sequence Labeling with Head-Based Encoding and Multi-Task Learning

Dependency Parsing as Sequence Labeling with Head-Based Encoding and Multi-Task Learning

Dependency Parsing as Sequence Labeling with Head-Based Encoding and Multi-Task Learning Ophlie Lacroix Siteimprove, Copenhagen, Denmark ola@siteimprove.com August 27, 2019 1 / 15 Dependency Parsing as Sequence Labeling 1. Encoding the

349 views • 17 slides
Dependency parsing and logistic regression Shay Cohen (based on slides by Sharon Goldwater) 21

Dependency parsing and logistic regression Shay Cohen (based on slides by Sharon Goldwater) 21

Dependency parsing and logistic regression Shay Cohen (based on slides by Sharon Goldwater) 21 October 2019 Last class Dependency parsing: a fully lexicalized formalism; tree edges connect words in the sentence based on head-dependent

764 views • 43 slides
Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs

Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs

Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs Semantic Parsing Dependency parsing models the syntactic structure between words in a sentence. Dependency Parsing vs Semantic Parsing

637 views • 31 slides
Dependency Parsing II CMSC 470 Marine Carpuat Graph-based Dependency Parsing Slides credit:

Dependency Parsing II CMSC 470 Marine Carpuat Graph-based Dependency Parsing Slides credit:

Dependency Parsing II CMSC 470 Marine Carpuat Graph-based Dependency Parsing Slides credit: Joakim Nivre Directed Spanning Trees Dependency Parsing as Finding the Maximum Spanning Tree Views parsing as finding the best directed spanning

981 views • 61 slides
Dependency Grammars and Parsing CMSC 473/673 UMBC Outline Review: PCFGs and CKY Dependency

Dependency Grammars and Parsing CMSC 473/673 UMBC Outline Review: PCFGs and CKY Dependency

Dependency Grammars and Parsing CMSC 473/673 UMBC Outline Review: PCFGs and CKY Dependency Grammars Dependency Parsing (Shift-reduce) From syntax to semantics Probabilistic Context Free Grammar 1.0 S NP VP 1.0 PP P NP .4 NP Det

862 views • 75 slides
Algorithms for NLP CS 11-711 Fall 2020 Lecture 14: Graph-based dependency parsing Emma

Algorithms for NLP CS 11-711 Fall 2020 Lecture 14: Graph-based dependency parsing Emma

Algorithms for NLP CS 11-711 Fall 2020 Lecture 14: Graph-based dependency parsing Emma Strubell Announcements No recitation on Friday (Tartan Community Day). 2 Dependency parsing 3 Dependency parsing Input buffer Transition-based

1.15k views • 77 slides
Dependency Parsing II CMSC 470 Marine Carpuat Arc Standard Transition System defines 3

Dependency Parsing II CMSC 470 Marine Carpuat Arc Standard Transition System defines 3

Dependency Parsing II CMSC 470 Marine Carpuat Arc Standard Transition System defines 3 transition operators [Covington, 2001; Nivre 2003] SHIFT Remove word at head of input buffer Push it on the stack LEFT-ARC create head-dependent

605 views • 36 slides
Thoughts on Learner Data and Motivation Learner Language Dependency Parsing and Dependency

Thoughts on Learner Data and Motivation Learner Language Dependency Parsing and Dependency

Thoughts on Learner Data and Dependency Parsing Niels Ott, Ramon Ziai, Julia Krivanek, Detmar Meurers Introduction and Thoughts on Learner Data and Motivation Learner Language Dependency Parsing and Dependency Annotation Approximated

812 views • 53 slides
Marina Valeeva Outline 2 1. Introduction What is Dependency Parsing? What is a

Marina Valeeva Outline 2 1. Introduction What is Dependency Parsing? What is a

GRAPH-BASED DEPENDENCY PARSING Marina Valeeva Outline 2 1. Introduction What is Dependency Parsing? What is a Dependency Tree? Projectivity vs. Non-Projectivity 2. Graph-Based Dependency Parsing 3. Models Edge Based Factorization

572 views • 40 slides
Logic Programming: Search Strategies Alan Smaill Nov 5, 2012 Alan Smaill Logic Programming:

Logic Programming: Search Strategies Alan Smaill Nov 5, 2012 Alan Smaill Logic Programming:

N I V E U R S E I H T T Y O H F G R E U D I B N Logic Programming: Search Strategies Alan Smaill Nov 5, 2012 Alan Smaill Logic Programming: Search Strategies Nov 5, 2012 1/1 Today N I V E U R S E I H T T

284 views • 26 slides
Is a Graph Connected? Algorithm 1: Breadth-first search - From a starting node, find closest

Is a Graph Connected? Algorithm 1: Breadth-first search - From a starting node, find closest

1-1 Is a Graph Connected? Algorithm 1: Breadth-first search - From a starting node, find closest nodes first. a c Start at a: e b d 1-2 Is a Graph Connected? Algorithm 1: Breadth-first search - From a starting node, find closest nodes

544 views • 5 slides
Level-Balanced B-Trees Gerth Stlting Brodal BRICS University of Aarhus Pankaj K. Agarwal

Level-Balanced B-Trees Gerth Stlting Brodal BRICS University of Aarhus Pankaj K. Agarwal

Level-Balanced B-Trees Gerth Stlting Brodal BRICS University of Aarhus Pankaj K. Agarwal Lars Arge Jeffrey S. Vitter Center for Geometric Computing Duke University January 1999 1 Level-Balanced B-Trees B-Trees Bayer, McCreight 1972

427 views • 17 slides
30: General and Binary Trees Chris Wyatt Electrical and Computer Engineering Trees are

30: General and Binary Trees Chris Wyatt Electrical and Computer Engineering Trees are

ECE 2574 Introduction to Data Structures and Algorithms 30: General and Binary Trees Chris Wyatt Electrical and Computer Engineering Trees are non-linear, value oriented structures List Holding 7 Items A B C D E F G A B C Example of

775 views • 21 slides
Design of Multi-tier Wireless Mesh Issues in Wireless Network Design Networks Multi-tier Wless

Design of Multi-tier Wireless Mesh Issues in Wireless Network Design Networks Multi-tier Wless

Design of Wireless Mesh Networks Outline Introduction Design of Multi-tier Wireless Mesh Issues in Wireless Network Design Networks Multi-tier Wless Design Solution Approach Stage 1: AP-assignment Capacity of WLANs Raghuraman Rangarajan

1.44k views • 116 slides
Decompiling Boolean Expressions from Java TM Bytecode Mangala Gowri Nanda (IBM-IRL) and S.

Decompiling Boolean Expressions from Java TM Bytecode Mangala Gowri Nanda (IBM-IRL) and S.

Decompiling Boolean Expressions from JavaTM Bytecode Decompiling Boolean Expressions from Java TM Bytecode Mangala Gowri Nanda (IBM-IRL) and S. Arun-Kumar (IIT Delhi) Decompiling Boolean Expressions from JavaTM Bytecode Introduction The

570 views • 37 slides
Experimental SiPM parameter characterization from avalanche triggering probabilities G. Gallina ,

Experimental SiPM parameter characterization from avalanche triggering probabilities G. Gallina ,

Experimental SiPM parameter characterization from avalanche triggering probabilities G. Gallina , J.Kroeger , P. Giampa, F. Retire ,M. Ward, G. Zhang, L. Doria Electron vs hole triggered avalanches D. Orme, PD09 Following up from Oide PD07

398 views • 22 slides
Gates and Logic: From Transistors to Logic Gates and Logic Circuits Prof. Anne Bracy CS 3410

Gates and Logic: From Transistors to Logic Gates and Logic Circuits Prof. Anne Bracy CS 3410

Gates and Logic: From Transistors to Logic Gates and Logic Circuits Prof. Anne Bracy CS 3410 Computer Science Cornell University The slides are the product of many rounds of teaching CS 3410 by Professors Weatherspoon, Bala, Bracy, and

726 views • 34 slides

More recommend