a cyk variant for scfg decoding without a dot chart
play

A CYK+ Variant for SCFG Decoding Without a Dot Chart Rico Sennrich - PowerPoint PPT Presentation

Feb 21, 2023 •28 likes •408 views

A CYK+ Variant for SCFG Decoding Without a Dot Chart Rico Sennrich Institute for Language, Cognition and Computation University of Edinburgh October 25 2014 R. Sennrich Recursive CYK+ 1 / 15 Outline CYK+ and the role of the dot chart

  1. A CYK+ Variant for SCFG Decoding Without a Dot Chart Rico Sennrich Institute for Language, Cognition and Computation University of Edinburgh October 25 2014 R. Sennrich Recursive CYK+ 1 / 15

  2. Outline CYK+ and the role of the dot chart Recursive variant Evaluation R. Sennrich Recursive CYK+ 2 / 15

  3. Problem CYK+ parsing CYK+ and Earley-style variants are popular parsers for decoding with SCFGs (Moses, cdec, SAMT, Jane, ...). alternative: binarization and decoding with plain CYK. problem CYK+ parsing [with syntactic models] takes a lot of memory. n = 20 n = 40 n = 80 0.32 GB 2.63 GB 51.64 GB most of the memory is consumed by the dot chart. R. Sennrich Recursive CYK+ 3 / 15

  4. Problem CYK+ parsing CYK+ and Earley-style variants are popular parsers for decoding with SCFGs (Moses, cdec, SAMT, Jane, ...). alternative: binarization and decoding with plain CYK. problem CYK+ parsing [with syntactic models] takes a lot of memory. n = 20 n = 40 n = 80 0.32 GB 2.63 GB 51.64 GB most of the memory is consumed by the dot chart. solution in this talk, we present a variant of CYK+ without a dot chart. our variant requires less memory and is faster, with same result. R. Sennrich Recursive CYK+ 3 / 15

  5. CYK+ The CYK+ algorithm bottom-up chart parser generalization of CYK to n -ary rules two data structures: main chart: non-terminal symbols dot chart: rule prefix applications (dotted items) difference to Earley: dotted item represents all rules with same prefix dot chart allows dynamic binarization: rules that match span (i,j) are found by combining dotted item in (i,k) and (non-)terminal symbol in span (k,j). R. Sennrich Recursive CYK+ 4 / 15

  6. CYK+ it is a trap it is a trap 1 1 → S NP V NP 2 2 NP → ART NN 3 3 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  7. CYK+ it is a trap it is a trap 1 1 NP → S NP V NP 2 2 NP → ART NN 3 3 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  8. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 NP → ART NN 3 3 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  9. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  10. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 ART 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  11. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 ART • ART 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  12. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 ART • ART 4 4 NN → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  13. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 ART • ART 4 4 NN → NP it 5 NP V • 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  14. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 ART • ART 4 4 NN → NP it 5 NP V • 5 V → is 6 6 → ART a 7 7 NP 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  15. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 ART • ART 4 4 NN → NP it 5 NP V • 5 V → is 6 6 → ART a 7 7 NP 8 8 NN → trap 9 9 10 10 S dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  16. CYK+: complexity considerations monolingual 1-best parser main chart: O ( n 2 ) dot chart: O ( n 2 ) parsing steps: O ( n 3 ) SCFG decoding Non-locality of LM scores restricts recombination of dotted items [Hopkins and Langmead, 2010] main chart: O ( n 2 ) (with beam search) on the fast jet ski of mr smith the JJ NPB of NNP dot chart: O ( n scope ( G ) ) the JJ NPB of NNP the JJ NPB of NNP parsing steps: O ( n scope ( G ) ) the JJ NPB of NNP choice point choice point rule scope: number of choice points in rule R. Sennrich Recursive CYK+ 6 / 15

  17. The dot chart in SCFG decoding purpose of dot chart allows recombination of different dotted items → does not apply to SCFG decoding allows re-use of same dotted item for different spans it is a trap it is a trap 1 NP • 1 NP 2 2 V 3 3 ART • ART 4 4 NN 5 NP V • 5 6 6 7 7 NP 8 8 9 9 10 10 S dot chart main chart R. Sennrich Recursive CYK+ 7 / 15

  18. The dot chart in SCFG decoding purpose of dot chart allows recombination of different dotted items → does not apply to SCFG decoding allows re-use of same dotted item for different spans it is a trap it is a trap 1 NP • 1 NP 2 2 V 3 3 ART • ART 4 4 NN 5 NP V • 5 6 6 7 7 NP 8 8 9 9 10 10 S dot chart main chart R. Sennrich Recursive CYK+ 7 / 15

  19. The dot chart in SCFG decoding purpose of dot chart allows recombination of different dotted items → does not apply to SCFG decoding allows re-use of same dotted item for different spans it is a trap it is a trap 1 NP • 1 NP 2 2 V 3 3 ART • ART 4 4 NN 5 NP V • 5 6 6 7 7 NP 8 8 9 9 10 10 S dot chart main chart R. Sennrich Recursive CYK+ 7 / 15

  20. The dot chart in SCFG decoding purpose of dot chart allows recombination of different dotted items → does not apply to SCFG decoding allows re-use of same dotted item for different spans it is a trap it is a trap 1 NP • 1 NP 2 2 V 3 3 ART • ART 4 4 NN 5 NP V • 5 6 6 7 7 NP 8 8 9 9 10 10 S dot chart main chart R. Sennrich Recursive CYK+ 7 / 15

  21. A recursive variant Core idea we do not initially know if rule prefix application can be extended. → dotted items are re-visited throughout time. we can change chart traversal order to guarantee that when span (i,k) is visited, all spans (k,j) have been visited before. this eliminates need to store dotted items; instead, they are extended recursively, then discarded. it is a trap it is a trap 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 traditional proposed R. Sennrich Recursive CYK+ 8 / 15

  22. Recursive CYK+ it is a trap it is a trap 1 1 → S NP V NP 2 2 NP → ART NN 3 3 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar recursive CYK+ steps search for terminal rule of size 1. 1 initial call to rule consume function. 2 recursive call to rule consume function. 3 R. Sennrich Recursive CYK+ 9 / 15

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Chris Hopkins, PhD, MBA and CSO patient variant drug p.R292H pathogenicity Is this variant in

Chris Hopkins, PhD, MBA and CSO patient variant drug p.R292H pathogenicity Is this variant in

Chris Hopkins, PhD, MBA and CSO patient variant drug p.R292H pathogenicity Is this variant in my patient pathogenic or benign? "The Doctor" by Sir Luke Fildes (1891) Drop in genome price drives increase in patient variant

500 views • 22 slides
<? <?php php $ret = $ret = foo foo($a ($a); ); // C++ // C++ Variant v_ret Variant

<? <?php php $ret = $ret = foo foo($a ($a); ); // C++ // C++ Variant v_ret Variant

<? <?php php $ret = $ret = foo foo($a ($a); ); // C++ // C++ Variant v_ret Variant v_ret; ; Variant Variant v_a v_a; ; v_ret v_ret = = f_foo f_foo(v_a (v_a); ); Facebook 2010 (confidential) <?php <?

287 views • 12 slides
IDN Variant TLDs Program Update IDN Variant TLDs Program Origins No variants of gTLDs will be

IDN Variant TLDs Program Update IDN Variant TLDs Program Origins No variants of gTLDs will be

IDN Variant TLDs Program Update IDN Variant TLDs Program Origins No variants of gTLDs will be delegated until appropriate variant management solutions are developed http://www.icann.org/en/groups/board/do

533 views • 31 slides
Variant Effect Predictor Demo: The Variant Effect Predictor (VEP)

Variant Effect Predictor Demo: The Variant Effect Predictor (VEP)

Variant Effect Predictor Demo: The Variant Effect Predictor (VEP) We have identified five variants on human chromosome nine, an A deletion at 128328461, C->A

301 views • 7 slides
Working with Charts Objectives Understand and plan a chart Create a chart Move and

Working with Charts Objectives Understand and plan a chart Create a chart Move and

Working with Charts Objectives Understand and plan a chart Create a chart Move and resize charts and chart objects Apply chart layouts and styles Microsoft Office 2013-Illustrated Fundamentals 2 Objectives Customize chart

621 views • 26 slides
An Extended GHKM Algorithm for Inducing -SCFG Peng Li, Yang Liu and Maosong Sun THUNLP&CSS

An Extended GHKM Algorithm for Inducing -SCFG Peng Li, Yang Liu and Maosong Sun THUNLP&CSS

An Extended GHKM Algorithm for Inducing -SCFG Peng Li, Yang Liu and Maosong Sun THUNLP&CSS Tsinghua University, China Outline l Background l Rule extraction algorithm l Modeling l Experiments l Conclusion 2 Outline l

664 views • 40 slides
A General-Purpose Rule Extractor for SCFG-Based Machine Translation Greg Hanneman , Michelle

A General-Purpose Rule Extractor for SCFG-Based Machine Translation Greg Hanneman , Michelle

A General-Purpose Rule Extractor for SCFG-Based Machine Translation Greg Hanneman , Michelle Burroughs , and Alon Lavie Language Technologies Institute Carnegie Mellon University Fifth Workshop on Syntax and Structure in Statistical Translation

312 views • 28 slides
Observation Decoding with Sensor Models: Recognition Tasks via Classical Planning Diego Aineto,

Observation Decoding with Sensor Models: Recognition Tasks via Classical Planning Diego Aineto,

Observation Decoding with Sensor Models: Recognition Tasks via Classical Planning Diego Aineto, Sergio Jimenez, Eva Onaindia October 16, 2020 Universitat Polit` ecnica de Val` encia 1 What is decoding? What is decoding? Decoding : finding

848 views • 26 slides
Variant-Related Issues Variant TLD Issues Project Goal

Variant-Related Issues Variant TLD Issues Project Goal

Variant-Related Issues Variant TLD Issues Project Goal Iden0fy and agree on issues related to variants that should be included in the report 2

218 views • 10 slides
Chapter 6 Decoding Statistical Machine Translation Decoding We have a mathematical model for

Chapter 6 Decoding Statistical Machine Translation Decoding We have a mathematical model for

Chapter 6 Decoding Statistical Machine Translation Decoding We have a mathematical model for translation p ( e | f ) Task of decoding: find the translation e best with highest probability e best = argmax e p ( e | f ) Two types of

571 views • 34 slides
Jorge Jimnez Variant calling jjimeneza@cipf.es Index 1. Variant calling pipeline 2.

Jorge Jimnez Variant calling jjimeneza@cipf.es Index 1. Variant calling pipeline 2.

Jorge Jimnez Variant calling jjimeneza@cipf.es Index 1. Variant calling pipeline 2. Alignment processing 3. SNP calling 4. Short indel calling 5. VCF format 6. Structural Variation Jorge Jimnez Variant calling jjimeneza@cipf.es

1.12k views • 40 slides
Why decoding? Understanding the neural code. Neural Decoding Given spikes, what was the

Why decoding? Understanding the neural code. Neural Decoding Given spikes, what was the

Why decoding? Understanding the neural code. Neural Decoding Given spikes, what was the stimulus? What aspects of the stimulus does the system encode? (capacity is limited) Mark van Rossum What information can be extracted from spike trains:

779 views • 16 slides
By et al Siegfried Engelmann Decoding Strategies: Decoding B1- Teacher's Presentation Book

By et al Siegfried Engelmann Decoding Strategies: Decoding B1- Teacher's Presentation Book

By et al Siegfried Engelmann Decoding Strategies: Decoding B1- Teacher's Presentation Book (Corrective Reading) (3rd Revised edition) From McGraw-Hill Inc.,US By et al Siegfried Engelmann Decoding Strategies: Decoding B1- Teacher's Presentation

233 views • 5 slides
1 Variant II: Command Consensus (BG) Variant III: Interactive Consistency (IC) Variant II:

1 Variant II: Command Consensus (BG) Variant III: Interactive Consistency (IC) Variant II:

Coordination Coordination If the solution to availability and scalability is to decentralize and replicate functions and data, how do we coordinate the nodes? data consistency Failures and Consensus Failures and Consensus update

308 views • 6 slides
Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling

Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling

Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling Kyunghyun Cho New York University & Facebook AI Research Joint work with Jason Lee and Elman Mansimov Neural sequence modeling An arbitrary

461 views • 25 slides
IDN Variant TLD Implementation Status, Recommendations and Next Steps GNSO Council 18 April 2019

IDN Variant TLD Implementation Status, Recommendations and Next Steps GNSO Council 18 April 2019

IDN Variant TLD Implementation Status, Recommendations and Next Steps GNSO Council 18 April 2019 | 1 Objectives of This Session What: Understand 1. 1. IDN variant top-level domains (TLDs) 2. Status of IDN variant TLDs 3. Next

474 views • 24 slides
Predicting Energy Consumption by Extending the Palladio Component Model Symposium on Software

Predicting Energy Consumption by Extending the Palladio Component Model Symposium on Software

Stuttgart, Germany, 2014-11-27 Predicting Energy Consumption by Extending the Palladio Component Model Symposium on Software Performance (SOSP) 2014 Felix Willnecker 1 , Andreas Brunnert 1 , Helmut Krcmar 2 1 fortiss GmbH, 2 Technische

693 views • 22 slides
Economics 2 Professor Christina Romer Spring 2017 Professor David Romer LECTURE 20 PLANNED

Economics 2 Professor Christina Romer Spring 2017 Professor David Romer LECTURE 20 PLANNED

Economics 2 Professor Christina Romer Spring 2017 Professor David Romer LECTURE 20 PLANNED AGGREGATE EXPENDITURE AND OUTPUT April 6, 2017 I. O VERVIEW OF S HORT -R UN F LUCTUATIONS II. T HE K EY R OLE OF D EMAND A. Evidence B. Source III. P

653 views • 43 slides
Econometric Evaluation of Social Programs Part I: Causal Models, Structural Models, and

Econometric Evaluation of Social Programs Part I: Causal Models, Structural Models, and

Introduction Policy Econometric Evaluation of Social Programs Part I: Causal Models, Structural Models, and Econometric Policy Evaluation James J. Heckman and Edward J. Vytlacil Econ 312, Spring 2019 Heckman and Vytlacil Econometric

842 views • 73 slides
Efficiency with Endogenous Population Growth Mike Golosov Larry E. Jones Mich` ele Tertilt

Efficiency with Endogenous Population Growth Mike Golosov Larry E. Jones Mich` ele Tertilt

Efficiency with Endogenous Population Growth Mike Golosov Larry E. Jones Mich` ele Tertilt Minnesota MIT Stanford & UPenn October 2006 Efficiency p. 1/34 A Quote from The Economist Some people fret that if more women work

451 views • 34 slides
II.V. Consumption text, Ch 17 20 Objectives : Understand factors affecting consumption

II.V. Consumption text, Ch 17 20 Objectives : Understand factors affecting consumption

II.V. Consumption text, Ch 17 20 Objectives : Understand factors affecting consumption Understand the role of consumption in the Great Recession and its aftermath I. Keynesian consumption function Keynes (1936): consumers spend a

257 views • 7 slides
0

0

269 views • 4 slides
ANALYTICS OF CONTAGION ON INHOMOGENEOUS RANDOM SOCIAL NETWORKS Tom Hurd, with Hassan Chehaitli,

ANALYTICS OF CONTAGION ON INHOMOGENEOUS RANDOM SOCIAL NETWORKS Tom Hurd, with Hassan Chehaitli,

ANALYTICS OF CONTAGION ON INHOMOGENEOUS RANDOM SOCIAL NETWORKS Tom Hurd, with Hassan Chehaitli, Weijie Pang and Vladimir Nosov April 2020 Tom Hurd (McMaster) Contagion Analytics 1 / 33 Fields/CQAM Systemic Risk Analytics Lab Tom Hurd

492 views • 33 slides
CREDIT CLUSTER AND CONTAGION RISK RELATED TO DISTRESSED MUNICIPALITIES BY RICHARD A. CICCARONE

CREDIT CLUSTER AND CONTAGION RISK RELATED TO DISTRESSED MUNICIPALITIES BY RICHARD A. CICCARONE

CREDIT CLUSTER AND CONTAGION RISK RELATED TO DISTRESSED MUNICIPALITIES BY RICHARD A. CICCARONE MERRITT RESEARCH SERVICES, LLC DISCUSSANT -- SHARON N. KIOKO UNIVERSITY OF WASHINGTON Chicago Public Other Public Public Schools

183 views • 5 slides

More recommend