dynamic programming for linear time incremental parsing
play

Dynamic Programming for Linear Time Incremental Parsing Liang Huang - PowerPoint PPT Presentation

Mar 15, 2024 •446 likes •1.1k views

Dynamic Programming for Linear Time Incremental Parsing Liang Huang Information Sciences Institute University of Southern California Kenji Sagae Institute for Creative Technologies University of Southern California ACL 2010, Uppsala, Sweden,

  1. Dynamic Programming for Linear Time Incremental Parsing Liang Huang Information Sciences Institute University of Southern California Kenji Sagae Institute for Creative Technologies University of Southern California ACL 2010, Uppsala, Sweden, July 2010 (slightly expanded)

  2. Ambiguities in Parsing I feed cats nearby in the garden ... • let’s focus on dependency structures for simplicity • ambiguous attachments of nearby and in • ambiguity explodes exponentially with sentence length • must design efficient (polynomial) search algorithm • typically using dynamic programming (DP); e.g. CKY DP for Incremental Parsing (Huang and Sagae) 2

  3. Ambiguities in Parsing I feed cats nearby in the garden ... • let’s focus on dependency structures for simplicity • ambiguous attachments of nearby and in • ambiguity explodes exponentially with sentence length • must design efficient (polynomial) search algorithm • typically using dynamic programming (DP); e.g. CKY DP for Incremental Parsing (Huang and Sagae) 2

  4. Ambiguities in Parsing I feed cats nearby in the garden ... • let’s focus on dependency structures for simplicity • ambiguous attachments of nearby and in • ambiguity explodes exponentially with sentence length • must design efficient (polynomial) search algorithm • typically using dynamic programming (DP); e.g. CKY DP for Incremental Parsing (Huang and Sagae) 2

  5. Ambiguities in Parsing I feed cats nearby in the garden ... • let’s focus on dependency structures for simplicity • ambiguous attachments of nearby and in • ambiguity explodes exponentially with sentence length • must design efficient (polynomial) search algorithm • typically using dynamic programming (DP); e.g. CKY DP for Incremental Parsing (Huang and Sagae) 2

  6. But full DP is too slow... I feed cats nearby in the garden ... • full DP (like CKY) is too slow (cubic-time) • while human parsing is fast & incremental (linear-time) DP for Incremental Parsing (Huang and Sagae) 3

  7. But full DP is too slow... I feed cats nearby in the garden ... • full DP (like CKY) is too slow (cubic-time) • while human parsing is fast & incremental (linear-time) • how about incremental parsing then? • yes, but only with greedy search (accuracy suffers) • explores tiny fraction of trees (even w/ beam search) DP for Incremental Parsing (Huang and Sagae) 3

  8. But full DP is too slow... I feed cats nearby in the garden ... • full DP (like CKY) is too slow (cubic-time) • while human parsing is fast & incremental (linear-time) • how about incremental parsing then? • yes, but only with greedy search (accuracy suffers) • explores tiny fraction of trees (even w/ beam search) • can we combine the merits of both approaches? • a fast, incremental parser with dynamic programming? • explores exponentially many trees in linear-time? DP for Incremental Parsing (Huang and Sagae) 3

  9. Linear-Time Incremental DP incremental parsing greedy (e.g. shift-reduce) search ☹ (Nivre 04; Collins/Roark 04; ...) this work: full DP principled fast shift-reduce parsing (e.g. CKY) search with dynamic programming ☺ (Eisner 96; Collins 99; ...) ☹ ☺ fast slow (linear-time) (cubic-time) DP for Incremental Parsing (Huang and Sagae) 4

  10. Preview of the Results • very fast linear-time dynamic programming parser • best reported dependency accuracy on PTB/CTB • explores exponentially many trees (and outputs forest) 1.4 parsing time (secs) 1.2 1 0.8 0.6 0.4 0.2 0 0 10 20 30 40 50 60 70 sentence length DP for Incremental Parsing (Huang and Sagae) 5

  11. Preview of the Results • very fast linear-time dynamic programming parser • best reported dependency accuracy on PTB/CTB • explores exponentially many trees (and outputs forest) 1.4 y k e parsing time (secs) a l i e 1.2 n k r r a 1 e h B C MST 0.8 0.6 0.4 this work 0.2 0 0 10 20 30 40 50 60 70 sentence length DP for Incremental Parsing (Huang and Sagae) 5

  12. Preview of the Results • very fast linear-time dynamic programming parser • best reported dependency accuracy on PTB/CTB • explores exponentially many trees (and outputs forest) 1.4 y 10 10 DP: exponential k e parsing time (secs) a l i e 1.2 n k 10 8 r r a 1 e h B 10 6 C MST 0.8 0.6 10 4 0.4 10 2 this work 0.2 non-DP beam search 10 0 0 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 sentence length sentence length DP for Incremental Parsing (Huang and Sagae) 5

  13. Outline • Motivation • Incremental (Shift-Reduce) Parsing • Dynamic Programming for Incremental Parsing • Experiments DP for Incremental Parsing (Huang and Sagae) 6

  14. Shift-Reduce Parsing I feed cats nearby in the garden. action stack queue I feed cats ... 0 - feed cats nearby ... cats nearby in ... cats nearby in ... nearby in the ... nearby in the ... in the garden ... DP for Incremental Parsing (Huang and Sagae) 7

  15. Shift-Reduce Parsing I feed cats nearby in the garden. action stack queue I feed cats ... 0 - I feed cats nearby ... 1 shift cats nearby in ... cats nearby in ... nearby in the ... nearby in the ... in the garden ... DP for Incremental Parsing (Huang and Sagae) 8

  16. Shift-Reduce Parsing I feed cats nearby in the garden. action stack queue I feed cats ... 0 - I feed cats nearby ... 1 shift I feed 2 shift cats nearby in ... cats nearby in ... nearby in the ... nearby in the ... in the garden ... DP for Incremental Parsing (Huang and Sagae) 9

  17. Shift-Reduce Parsing I feed cats nearby in the garden. action stack queue I feed cats ... 0 - I feed cats nearby ... 1 shift I feed 2 shift cats nearby in ... feed 3 l-reduce cats nearby in ... I nearby in the ... nearby in the ... in the garden ... DP for Incremental Parsing (Huang and Sagae) 10

  18. Shift-Reduce Parsing I feed cats nearby in the garden. action stack queue I feed cats ... 0 - I feed cats nearby ... 1 shift I feed 2 shift cats nearby in ... feed 3 l-reduce cats nearby in ... I 4 shift feed cats nearby in the ... I nearby in the ... in the garden ... DP for Incremental Parsing (Huang and Sagae) 11

  19. Shift-Reduce Parsing I feed cats nearby in the garden. action stack queue I feed cats ... 0 - I feed cats nearby ... 1 shift I feed 2 shift cats nearby in ... feed 3 l-reduce cats nearby in ... I 4 shift feed cats nearby in the ... I 5a r-reduce feed nearby in the ... cats I in the garden ... DP for Incremental Parsing (Huang and Sagae) 12

  20. Shift-Reduce Parsing I feed cats nearby in the garden. action stack queue I feed cats ... 0 - I feed cats nearby ... 1 shift I feed 2 shift cats nearby in ... feed 3 l-reduce cats nearby in ... I 4 shift feed cats nearby in the ... I 5a r-reduce feed nearby in the ... cats I 5b shift in the garden ... feed cats nearby I DP for Incremental Parsing (Huang and Sagae) 13

  21. Shift-Reduce Parsing I feed cats nearby in the garden. action stack queue I feed cats ... 0 - I feed cats nearby ... 1 shift I feed 2 shift cats nearby in ... feed 3 l-reduce cats nearby in ... I 4 shift feed cats nearby in the ... shift-reduce I 5a r-reduce feed conflict nearby in the ... cats I 5b shift in the garden ... feed cats nearby I DP for Incremental Parsing (Huang and Sagae) 14

  22. Choosing Parser Actions ← stack queue → ← stack queue → in the garden ... ... feed cats ... s 2 s 1 s 0 q 0 q 1 ... nearby I features: (s 0 . w , s 0 . rc , q 0 , ...) = (cats, nearby, in, ...) • score each action using features f and weights w • features are drawn from a local window • abstraction (or signature) of a state -- this inspires DP! • weights trained by structured perceptron (Collins 02) DP for Incremental Parsing (Huang and Sagae) 15

  23. Greedy Search • each state => three new states (shift, l-reduce, r-reduce) • search space should be exponential • greedy search: always pick the best next state DP for Incremental Parsing (Huang and Sagae) 16

  24. Greedy Search • each state => three new states (shift, l-reduce, r-reduce) • search space should be exponential • greedy search: always pick the best next state DP for Incremental Parsing (Huang and Sagae) 17

  25. Beam Search • each state => three new states (shift, l-reduce, r-reduce) • search space should be exponential • beam search: always keep top- b states DP for Incremental Parsing (Huang and Sagae) 18

  26. Dynamic Programming • each state => three new states (shift, l-reduce, r-reduce) • key idea of DP: share common subproblems • merge equivalent states => polynomial space DP for Incremental Parsing (Huang and Sagae) 19

  27. Dynamic Programming • each state => three new states (shift, l-reduce, r-reduce) • key idea of DP: share common subproblems • merge equivalent states => polynomial space “graph-structured stack” (Tomita, 1988) DP for Incremental Parsing (Huang and Sagae) 20

  28. Dynamic Programming • each state => three new states (shift, l-reduce, r-reduce) • key idea of DP: share common subproblems • merge equivalent states => polynomial space “graph-structured stack” (Tomita, 1988) DP for Incremental Parsing (Huang and Sagae) 21

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Dynamic Programming for Linear-Time Incremental Parsing Liang Huang Information Sciences

Dynamic Programming for Linear-Time Incremental Parsing Liang Huang Information Sciences

Dynamic Programming for Linear-Time Incremental Parsing Liang Huang Information Sciences Institute University of Southern California (Joint work with Kenji Sagae, USC/ICT) JHU CLSP Seminar September 14, 2010 Remembering Fred Jelinek

1.21k views • 88 slides
Linear Time Constituency Parsing with RNNs and Dynamic Programming Juneki Hong 1 Liang Huang 1,2 1

Linear Time Constituency Parsing with RNNs and Dynamic Programming Juneki Hong 1 Liang Huang 1,2 1

Linear Time Constituency Parsing with RNNs and Dynamic Programming Juneki Hong 1 Liang Huang 1,2 1 Oregon State University 2 Baidu Research Silicon Valley AI Lab Span Parsing is SOTA in Constituency Parsing Cross+Huang 2016 introduced Span

691 views • 53 slides
TAG, Dynamic Programming, and the Perceptron for Efficent, Feature-Rich Parsing Xavier Carreras,

TAG, Dynamic Programming, and the Perceptron for Efficent, Feature-Rich Parsing Xavier Carreras,

TAG, Dynamic Programming, and the Perceptron for Efficent, Feature-Rich Parsing Xavier Carreras, Michael Collins and Terry Koo MIT CSAIL Discriminative Models for Parsing Structured Prediction methods like CRF or Perceptron train linear models

403 views • 27 slides
Computational Geometry Linear programming Exercise session #8: Incremental construction &

Computational Geometry Linear programming Exercise session #8: Incremental construction &

Computational Geometry Linear programming Exercise session #8: Incremental construction & Problem : maximize c T x s.t. Ax b Arrangements c = ( c 1 ,, c d ) T Linear Programming x = ( x 1 ,, x d ) T

521 views • 4 slides
Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to

Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to

Lecture 11 Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O ( n 2 ) or O ( n 3 ) for which a naive approach would take exponential time. In this

259 views • 7 slides
Linear Programming Linear Programming In a linear programming problem, there is a set of

Linear Programming Linear Programming In a linear programming problem, there is a set of

Linear Programming Linear Programming In a linear programming problem, there is a set of variables, and we want to assign real values to CISC5835, Algorithms for Big Data them so as to satisfy a set of linear equations and/or linear

414 views • 6 slides
Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs

Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs

Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs Semantic Parsing Dependency parsing models the syntactic structure between words in a sentence. Dependency Parsing vs Semantic Parsing

637 views • 31 slides
Parsing with Dynamic Programming Graham Neubig Site https://phontron.com/class/nn4nlp2020/ Two

Parsing with Dynamic Programming Graham Neubig Site https://phontron.com/class/nn4nlp2020/ Two

CS11-747 Neural Networks for NLP Parsing with Dynamic Programming Graham Neubig Site https://phontron.com/class/nn4nlp2020/ Two Types of Linguistic Structure Dependency: focus on relations between words ROOT I saw a girl with a

730 views • 38 slides
Parsing with Dynamic Programming Graham Neubig Site https://phontron.com/class/nn4nlp2017/ Two

Parsing with Dynamic Programming Graham Neubig Site https://phontron.com/class/nn4nlp2017/ Two

CS11-747 Neural Networks for NLP Parsing with Dynamic Programming Graham Neubig Site https://phontron.com/class/nn4nlp2017/ Two Types of Linguistic Structure Dependency: focus on relations between words ROOT I saw a girl with a

396 views • 37 slides
A Dynamic Near-Optimal Algorithm for Online Linear Programming Yinyu Ye Department of Management

A Dynamic Near-Optimal Algorithm for Online Linear Programming Yinyu Ye Department of Management

Online Linear Programming Main Results and Key Ideas Related and More Recent Work A Dynamic Near-Optimal Algorithm for Online Linear Programming Yinyu Ye Department of Management Science and Engineering and Institute of Computational and

662 views • 52 slides
Incremental Parsing in Bounded Memory William Schuler Department of Linguistics The Ohio State

Incremental Parsing in Bounded Memory William Schuler Department of Linguistics The Ohio State

Incremental Parsing in Bounded Memory William Schuler Department of Linguistics The Ohio State University September 16, 2010 William Schuler Incremental Parsing in Bounded Memory Motivation Goal: simple processing model, matches observations

1.27k views • 80 slides
Type-driven Incremental Semantic Parsing with Polymorphism Kai Zhao and Liang Huang City

Type-driven Incremental Semantic Parsing with Polymorphism Kai Zhao and Liang Huang City

Type-driven Incremental Semantic Parsing with Polymorphism Kai Zhao and Liang Huang City University of New York The Problem weakly supervised semantic fully supervised semantic parsing parsing aka. parsing from Q/A pairs

547 views • 44 slides
Space/time tradeoffs; dynamic programming; y g g transform and conquer 1. Space/time

Space/time tradeoffs; dynamic programming; y g g transform and conquer 1. Space/time

6. Space-time tradeoffs and dynamic programming D. Keil Analysis of Algorithms 1/11 Topic 6: Space/time tradeoffs; dynamic programming; y g g transform and conquer 1. Space/time tradeoffs 2. Dynamic programming 3 Example: sequence

498 views • 18 slides
Batch-Incremental vs. Instance-Incremental Learning in Dynamic and Evolving Data Jesse Read 1 ,

Batch-Incremental vs. Instance-Incremental Learning in Dynamic and Evolving Data Jesse Read 1 ,

Batch-Incremental vs. Instance-Incremental Learning in Dynamic and Evolving Data Jesse Read 1 , Albert Bifet 2 , Bernhard Pfahringer 2 , Geoff Holmes 2 1 Department of Signal Theory and Communications Universidad Carlos III Madrid, Spain 2

568 views • 19 slides
CKY Parsing Ling 571 Deep Processing Techniques for NLP January 12, 2011 Roadmap

CKY Parsing Ling 571 Deep Processing Techniques for NLP January 12, 2011 Roadmap

CKY Parsing Ling 571 Deep Processing Techniques for NLP January 12, 2011 Roadmap Motivation: Parsing (In) efficiency Dynamic Programming Cocke-Kasami-Younger Parsing Algorithm Chomsky Normal Form Conversion

689 views • 42 slides
Programming Languages: Parsing Onur Tolga S ehito glu Computer Engineering,METU 27 May

Programming Languages: Parsing Onur Tolga S ehito glu Computer Engineering,METU 27 May

Programming Languages:Parsing Programming Languages: Parsing Onur Tolga S ehito glu Computer Engineering,METU 27 May 2009 Programming Languages:Parsing Outline 1 Parsing Top-down Parsing Recursive Descent Parser LL Parsers

215 views • 8 slides
TripleWave Andrea Mauri andrea.mauri@polimi.it @janez87 What is TripleWave? TripleWave an

TripleWave Andrea Mauri andrea.mauri@polimi.it @janez87 What is TripleWave? TripleWave an

Tutorial on RDF Stream Processing 2016 M.I. Ali, J-P Calbimonte, D. Dell'Aglio, E. Della Valle, and A. Mauri http://streamreasoning.org/events/rsp2016 How to publish RDF Stream with TripleWave Andrea Mauri andrea.mauri@polimi.it @janez87

697 views • 28 slides
CS314 Software Engineering Continuous Integration Dave Matthews Continuous Integration

CS314 Software Engineering Continuous Integration Dave Matthews Continuous Integration

2/13/18 CS314 Software Engineering Continuous Integration Dave Matthews Continuous Integration Improving Software Quality and Reducing Risk Paul M. Duvall 1 2/13/18 CI Value Reduce risks Reduce repetitive manual processes

549 views • 8 slides
Basic Support for Farm Event Tracking and GAP Compliance Alejandro Fernandez, Mariana del Pino,

Basic Support for Farm Event Tracking and GAP Compliance Alejandro Fernandez, Mariana del Pino,

RUCAPS 19th May 2020 Basic Support for Farm Event Tracking and GAP Compliance Alejandro Fernandez, Mariana del Pino, Francis Baumont de Oliveira Background Raise crop quality for higher value More exportable product Improved

460 views • 22 slides
Funding Sm Small Food & & Farm En Enter erprises ses Access to

Funding Sm Small Food & & Farm En Enter erprises ses Access to

` Funding Sm Small Food & & Farm En Enter erprises ses Access to Capital Workshop: Food + Beverage Enterprises Herkimer County Community College

445 views • 7 slides
Phil Baines Where are we now? Ampersand 2012 Phil Baines 1 Advertisment Public lettering

Phil Baines Where are we now? Ampersand 2012 Phil Baines 1 Advertisment Public lettering

Phil Baines Where are we now? Ampersand 2012 Phil Baines 1 Advertisment Public lettering walk Brighton 2 10:30 Saturday 16 June Meet outside the Dome (Photo: Jean Fraincoise Porchez, Dublin 2010) Ampersand 2012 Phil Baines 2

565 views • 52 slides
LLVM on the Web Using Portable Native Client to run Clang/LLVM in the Browser Brad Nelson

LLVM on the Web Using Portable Native Client to run Clang/LLVM in the Browser Brad Nelson

LLVM on the Web Using Portable Native Client to run Clang/LLVM in the Browser Brad Nelson February 1, 2015 @flagxor Motivation Author Ron Reuter (aka MainByte) I created the photo/file and it is released to the public domain. Motivation

692 views • 44 slides
Rendering Quality in WebKit's Open Source Graphics Stack Dominik Rttsches

Rendering Quality in WebKit's Open Source Graphics Stack Dominik Rttsches

Rendering Quality in WebKit's Open Source Graphics Stack Dominik Rttsches dominik.rottsches@intel.com 01.org/webkit About WebKit Software Engineer @ Intel Finland, OTC WebKit Committer Specialized in Tizen WebKit's Graphics Backend In

436 views • 26 slides
the internet must not be a place without civil rights! legitimacy and new forms of protest on

the internet must not be a place without civil rights! legitimacy and new forms of protest on

the internet must not be a place without civil rights! legitimacy and new forms of protest on the internet balccon 2013 bernd fix <bernd@wauland.de> Reason / Goal Reason / Goal Free speech Reason / Goal Petition

720 views • 49 slides

More recommend