parsing
play

Parsing CSCI 3130 Formal Languages and Automata Theory Siu On CHAN - PowerPoint PPT Presentation

Oct 27, 2023 •468 likes •865 views

1/18 Parsing CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Chinese University of Hong Kong Fall 2015 2/18 Parsing input: 0011 If so, how to build a parse tree with a program? S 0 S 1 | 1 S 0 S | T T S | Is 0011 L

  1. 1/18 Parsing CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Chinese University of Hong Kong Fall 2015

  2. 2/18 Parsing input: 0011 If so, how to build a parse tree with a program? S → 0 S 1 | 1 S 0 S | T T → S | ε Is 0011 ∈ L ?

  3. 3/18 Parsing … 00 S 11 00 T 11 00 S 11 … 01 S 0 S 1 … 0 T 1 0 S 1 … 10 S 10 S … 1 S 0 S … S T S Try all derivations? input: 0011 This is (part of) the tree of all derivations, not the parse tree S → 0 S 1 | 1 S 0 S | T T → S | ε 0011 ✓ ε

  4. 3/18 Parsing … 00 S 11 00 T 11 00 S 11 … 01 S 0 S 1 … 0 T 1 0 S 1 … 10 S 10 S … 1 S 0 S … S T S Try all derivations? input: 0011 This is (part of) the tree of all derivations, not the parse tree S → 0 S 1 | 1 S 0 S | T T → S | ε 0011 ✓ ε

  5. 3/18 Parsing … 00 S 11 00 T 11 00 S 11 … 01 S 0 S 1 … 0 T 1 0 S 1 … 10 S 10 S … 1 S 0 S … S T S Try all derivations? input: 0011 This is (part of) the tree of all derivations, not the parse tree S → 0 S 1 | 1 S 0 S | T T → S | ε 0011 ✓ ε

  6. 3/18 Parsing … 00 S 11 00 T 11 00 S 11 … 01 S 0 S 1 … 0 T 1 0 S 1 … 10 S 10 S … 1 S 0 S … S T S Try all derivations? input: 0011 This is (part of) the tree of all derivations, not the parse tree S → 0 S 1 | 1 S 0 S | T T → S | ε 0011 ✓ ε

  7. 4/18 Problems 2. If input is not in the language, parsing will never stop Let’s tackle the 2nd problem 1. Trying all derivations may take too long

  8. Derviation may loop because 5/18 Derived string may shrink Remove of “unit productions” T S T S because of “ -productions” 01 When to stop 0 T 1 0 S 1 S Problems: Idea: Stop when and unit productions S → 0 S 1 | 1 S 0 S | T | derived string | > | input | T → S | ε

  9. Derviation may loop because 5/18 S Remove of “unit productions” T S T and unit productions When to stop Derived string may shrink Problems: Idea: Stop when S → 0 S 1 | 1 S 0 S | T | derived string | > | input | T → S | ε S ⇒ 0 S 1 ⇒ 0 T 1 ⇒ 01 because of “ ε -productions”

  10. 5/18 When to stop Idea: Stop when Problems: Derived string may shrink Derviation may loop because of “unit productions” S → 0 S 1 | 1 S 0 S | T | derived string | > | input | T → S | ε S ⇒ 0 S 1 ⇒ 0 T 1 ⇒ 01 S ⇒ T ⇒ S ⇒ T ⇒ . . . because of “ ε -productions” Remove ε and unit productions

  11. 6/18 D A S E C D AD S C Removing Add a new start variable T If S is the start variable and the not the (new) start variable Removing ε -productions Goal: remove all A → ε rules for every non-start variable A For every rule A → ε where A is rule S → ε exists 1. Remove the rule A → ε 2. If you see B → α A β Add the rule T → S Add a new rule B → αβ S → ACD A → a B → ε C → ED | ε D → BC | b E → b

  12. 6/18 AD D C E S A S Add a new start variable T not the (new) start variable If S is the start variable and the Removing ε -productions Goal: remove all A → ε rules for every non-start variable A For every rule A → ε where A is rule S → ε exists 1. Remove the rule A → ε 2. If you see B → α A β Add the rule T → S Add a new rule B → αβ S → ACD D → C A → a ✘✘✘ ✘ B → ε C → ED | ε D → BC | b E → b Removing B → ε

  13. 6/18 If S is the start variable and the Add a new start variable T D not the (new) start variable C E S A Removing ε -productions Goal: remove all A → ε rules for every non-start variable A For every rule A → ε where A is rule S → ε exists 1. Remove the rule A → ε 2. If you see B → α A β Add the rule T → S Add a new rule B → αβ S → ACD D → C A → a S → AD ✘✘✘ ✘ B → ε C → ED | ✁ ε D → BC | b E → b Removing C → ε

  14. 6/18 If S is the start variable and the Add a new start variable T C not the (new) start variable E S A Removing ε -productions Goal: remove all A → ε rules for every non-start variable A For every rule A → ε where A is rule S → ε exists 1. Remove the rule A → ε 2. If you see B → α A β Add the rule T → S Add a new rule B → αβ S → ACD D → C A → a S → AD ✘✘✘ ✘ B → ε D → ε C → ED | ✁ ε D → BC | b E → b Removing C → ε

  15. 6/18 If S is the start variable and the Add a new start variable T S not the (new) start variable A Removing ε -productions Goal: remove all A → ε rules for every non-start variable A For every rule A → ε where A is rule S → ε exists 1. Remove the rule A → ε 2. If you see B → α A β Add the rule T → S Add a new rule B → αβ S → ACD D → C A → a S → AD ✘✘✘ ✘ ✘ ✘✘✘ B → ε D → ε C → ED | ✁ ε C → E D → BC | b E → b Removing D → ε

  16. 6/18 If S is the start variable and the Add a new start variable T not the (new) start variable Removing ε -productions Goal: remove all A → ε rules for every non-start variable A For every rule A → ε where A is rule S → ε exists 1. Remove the rule A → ε 2. If you see B → α A β Add the rule T → S Add a new rule B → αβ S → ACD D → C A → a S → AD ✘✘✘ ✘ ✘ ✘✘✘ B → ε D → ε C → ED | ✁ ε C → E D → BC | b S → A E → b Removing D → ε

  17. 7/18 B A becomes B If B was removed earlier, don’t add it back Eliminating ε -productions For every A → ε rule where A is not the start variable 1. Remove the rule A → ε 2. If you see B → α A β Add a new rule B → αβ Do 2. every time A appears B → α A β A γ yields B → αβ A γ B → α A βγ B → αβγ

  18. 7/18 don’t add it back Eliminating ε -productions For every A → ε rule where A is not the start variable 1. Remove the rule A → ε 2. If you see B → α A β Add a new rule B → αβ B → A becomes B → ε Do 2. every time A appears B → α A β A γ yields If B → ε was removed earlier, B → αβ A γ B → α A βγ B → αβγ

  19. 8/18 Eliminating unit productions A unit production is a production of the form Grammar: Unit production graph: S T R A → B S → 0 S 1 | 1 S 0 S | T T → S | R | ε R → 0 SR

  20. 9/18 R 0 SR R R S 1 S 0 S 0 S 1 S T Removing unit productions S delete it and replace everything with A 1 If there is a cycle of unit productions Replace T by S A → B → · · · → C → A S → 0 S 1 | 1 S 0 S | T T → S | R | ε R → 0 SR

  21. 9/18 T R T S Removing unit productions Replace T by S 1 If there is a cycle of unit productions delete it and replace everything with A A → B → · · · → C → A S → 0 S 1 | 1 S 0 S | � � S → 0 S 1 | 1 S 0 S T → ✓ � � S | R | ε S → R | ε R → 0 SR R → 0 SR

  22. 10/18 0 S 1 R 0 SR S by 0 SR R S Replace 0 SR R 0 SR 1 S 0 S S Removal of unit productions R S by 2 replace any chain 0 SR A → B → · · · → C → α A → α, B → α, C → α . . . , S → 0 S 1 | 1 S 0 S | R | ε R → 0 SR

  23. 10/18 Replace R S Removal of unit productions by by 2 replace any chain A → B → · · · → C → α A → α, B → α, C → α . . . , S → 0 S 1 | 1 S 0 S S → 0 S 1 | 1 S 0 S | R | ε | 0 SR | ε R → 0 SR R → 0 SR S → R → 0 SR S → 0 SR , R → 0 SR

  24. 11/18 Recap Problems: 1. Trying all derivations may take too long 2. If input is not in the language, parsing will never stop Solution to problem 2: 2. Eliminate unit productions Try all possible derivations but stop parsing when ✓ 1. Eliminate ε productions | derived string | > | input |

  25. 12/18 Example Conclusion: 0011 00 S 11 too long 00 S 0 S 1 too long 0 S 1 0000 S 1 too long 0000 S 0 S too long 000 S 00 S 10 S too long 00 S 0 S 0 S too long 0 S 0 S S input: 0011 = L S → 0 S 1 | 0 S 0 S | T ⇒ S → 0 S 1 | 0 S 0 S | 0 T → S | 0 0 ✗ 001 ✗ 0000 ✗

  26. 12/18 Example Conclusion: 0011 00 S 11 too long 00 S 0 S 1 too long 0 S 1 0000 S 1 too long 0000 S 0 S too long 000 S 00 S 10 S too long 00 S 0 S 0 S too long 0 S 0 S S input: 0011 = L S → 0 S 1 | 0 S 0 S | T ⇒ S → 0 S 1 | 0 S 0 S | 0 T → S | 0 0 ✗ 001 ✗ 0000 ✗

  27. 12/18 00 S 0 S 0 S too long 00 S 11 too long 00 S 0 S 1 too long 0 S 1 0000 S 1 too long 0000 S 0 S too long 000 S Example 00 S 10 S too long 0 S 0 S S input: 0011 = S → 0 S 1 | 0 S 0 S | T ⇒ S → 0 S 1 | 0 S 0 S | 0 T → S | 0 0 ✗ 001 ✗ 0000 ✗ Conclusion: 0011 / ∈ L

  28. 13/18 Problems 2. If input is not in the language, parsing will never stop 1. Trying all derivations may take too long

  29. 14/18 Preparations A faster way to parse: Cocke–Younger–Kasami algorithm To use it we must perprocess the CFG: Eliminate unit productions Convert CFG to Chomsky Normal Form Eliminate ε productions

  30. 15/18 = variables with new sequences break up = variables with new terminals Chomsky Normal Form replace Convert to Chomsky Normal Form: or Noam Chomsky A CFG is in Chomsky Normal Form if for start variable S every production has the form but we also allow where neither B nor C is the start variable A → BC A → a S → ε ⇒ ⇒ A → B c DE A → BCDE A → BX C → c X → CY Y → DE C → c

  31. 16/18 a B A C S A B Cocke–Younger–Kasami algorithm B a b i a b A C A C let S C S A ℓ S → AB | BC 5 A → BA | a 4 B → CC | b 3 C → AB | a 2 1 Input: x = baaba 1 2 3 4 5 x [ i , ℓ ] = x i x i +1 . . . x i + ℓ − 1 For every substring x [ i , ℓ ] , remember all variables R that derive x [ i , ℓ ] Store in a table T [ i , ℓ ]

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

CSC 4181 Compiler Construction Parsing 1 1 Outline Top-down v.s. Bottom-up Top-down parsing

CSC 4181 Compiler Construction Parsing 1 1 Outline Top-down v.s. Bottom-up Top-down parsing

CSC 4181 Compiler Construction Parsing 1 1 Outline Top-down v.s. Bottom-up Top-down parsing Bottom-up parsing Recursive-descent Shift-reduce parsers parsing LR(0) parsing LL(1) parsing LR(0) items LL(1) parsing

429 views • 29 slides
Introduction to Bottom-Up Parsing Shift-reduce parsing The LR parsing algorithm

Introduction to Bottom-Up Parsing Shift-reduce parsing The LR parsing algorithm

Outline Review LL parsing Introduction to Bottom-Up Parsing Shift-reduce parsing The LR parsing algorithm Constructing LR parsing tables 2 Top-Down Parsing: Review Top-Down Parsing: Review Top-down parsing expands a

470 views • 13 slides
Top-Down Parsing Slides modified from Louden Book and Dr. Scherger Top Down Parsing A

Top-Down Parsing Slides modified from Louden Book and Dr. Scherger Top Down Parsing A

Compiler Design and Construction Top-Down Parsing Slides modified from Louden Book and Dr. Scherger Top Down Parsing A top-down parsing algorithm parses an input string of tokens by tracing out the steps in a leftmost derivation. Such an

1.12k views • 96 slides
Statistical Dependency Parsing in Korean: From Corpus Generation To Automatic Parsing Workshop on

Statistical Dependency Parsing in Korean: From Corpus Generation To Automatic Parsing Workshop on

Statistical Dependency Parsing in Korean: From Corpus Generation To Automatic Parsing Workshop on Statistical Parsing of Morphologically-Rich Languages 12th International Conference on Parsing Technologies Jinho D. Choi & Martha Palmer

199 views • 19 slides
Basic Parsing Algorithms Chart Parsing Seminar Recent Advances in Parsing Technology WS

Basic Parsing Algorithms Chart Parsing Seminar Recent Advances in Parsing Technology WS

Basic Parsing Algorithms Chart Parsing Seminar Recent Advances in Parsing Technology WS 2011/2012 Anna Schmidt Talk Outline Chart Parsing Basics Chart Parsing Algorithms Earley Algorithm CKY Algorithm Basics

1.12k views • 73 slides
Overview Introduction Lexicalized TAG, Advantages of parsing with LTAG Parsing LTAGs

Overview Introduction Lexicalized TAG, Advantages of parsing with LTAG Parsing LTAGs

Parsing with Lexicalized TAG (1) Extracting and comparing LTAG (2) Presentation by Philip John Gorinski Seminar Recent Advances in Parsing Technology Saarland University, Winter Term 2011/12 (1) Yves Shabes, Aravind K. Joshi, 1990 (2)

514 views • 36 slides
* 07/16/96 Plan for Today Shift-reduce parsing The problem with predictive top down parsing

* 07/16/96 Plan for Today Shift-reduce parsing The problem with predictive top down parsing

* 07/16/96 Plan for Today Shift-reduce parsing The problem with predictive top down parsing CS453 : Shift Reduce Parsing LR parsing: bottom up Unambiguous Grammars LR(0) and SLR Parse Tables Performs a right-most derivation in

316 views • 6 slides
1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G ,

1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G ,

1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G , to decide if w L ( G ), and if so, to construct a derivation or a parse tree for w . Parsing is studied in courses in compilers. To be efficient

519 views • 12 slides
Bottom-up parsing LR parsing Construct parse tree for input from leaves up LR( k ) parsing

Bottom-up parsing LR parsing Construct parse tree for input from leaves up LR( k ) parsing

Bottom-up parsing LR parsing Construct parse tree for input from leaves up LR( k ) parsing reducing a string of tokens to single start symbol L eft-to-right scan of input, R ightmost derivation (inverse of deriving a string of tokens from

413 views • 5 slides
Bottom Up Parsing Also known as Shift-Reduce parsing More powerful than top down Dont

Bottom Up Parsing Also known as Shift-Reduce parsing More powerful than top down Dont

9/26/2012 Bottom Up Parsing Also known as Shift-Reduce parsing More powerful than top down Dont need left factored grammars Can handle left recursion Bottom Up Parsing Attempt to construct parse tree from an input string

654 views • 6 slides
Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP

Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP

Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP February 2, 2015 Roadmap Dependency parsing Graph-based dependency parsing Maximum spanning tree CLE Algorithm Learning

899 views • 87 slides
Statistical Parsing Parsing context-free languages ar ltekin University of Tbingen

Statistical Parsing Parsing context-free languages ar ltekin University of Tbingen

Statistical Parsing Parsing context-free languages ar ltekin University of Tbingen Seminar fr Sprachwissenschaft November 8, 2016 Recap Parsing basics CKY Earley Closing Ingredients of a (natural language) parser .

1.17k views • 51 slides
Generalised Parsing and Combinator Parsing A Happy Marriage? L. Thomas van Binsbergen

Generalised Parsing and Combinator Parsing A Happy Marriage? L. Thomas van Binsbergen

Conventional Parser Combinators Aspects of Generalised Parsing Combinators for Generalised Parsing Generalised Parsing and Combinator Parsing A Happy Marriage? L. Thomas van Binsbergen ltvanbinsbergen@acm.org Royal Holloway, University of

452 views • 30 slides
Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs

Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs

Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs Semantic Parsing Dependency parsing models the syntactic structure between words in a sentence. Dependency Parsing vs Semantic Parsing

637 views • 31 slides
Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr

Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr

Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr Sprachwissenschaft November 2016 Recap/background Dependency grammar Dependency parsing Evaluation Summary Ingredients of a parser

1.44k views • 89 slides
Plan for Today Predictive parsing as a specific subclass of recursive descent parsing

Plan for Today Predictive parsing as a specific subclass of recursive descent parsing

Plan for Today Predictive parsing as a specific subclass of recursive descent parsing complexity comparisons with general parsing might have to left-factor necessary to remove left-recursion (MiniSVG modification as example) Working

346 views • 5 slides
Parse Trees Statistical NLP Spring 2011 Lecture 15: Parsing I The move followed a round of

Parse Trees Statistical NLP Spring 2011 Lecture 15: Parsing I The move followed a round of

Parse Trees Statistical NLP Spring 2011 Lecture 15: Parsing I The move followed a round of similar increases by other lenders, reflecting a continuing decline in that market Dan Klein UC Berkeley Phrase Structure Parsing Constituency

265 views • 5 slides
Certification of context-free grammar algorithms Denis Firsov Institute of Cybernetics at TUT

Certification of context-free grammar algorithms Denis Firsov Institute of Cybernetics at TUT

Certification of context-free grammar algorithms Denis Firsov Institute of Cybernetics at TUT August 31, 2016 1 / 1 Certification Certification refers to the confirmation of certain characteristics of an object, person, or organization. The

843 views • 31 slides
Weighted Context-Free Grammars over Bimonoids George Rahonis and Faidra Torpari Aristotle

Weighted Context-Free Grammars over Bimonoids George Rahonis and Faidra Torpari Aristotle

Weighted Context-Free Grammars over Bimonoids George Rahonis and Faidra Torpari Aristotle University of Thessaloniki, Greece WATA 2018 Leipzig, May 22, 2018 Faidra Torpari (Aristotle University of Thessaloniki) Weighted Context-Free Grammars

635 views • 19 slides
BU CS 332 Theory of Computation Lecture 13: Reading: Mid Semester Feedback Sipser Ch

BU CS 332 Theory of Computation Lecture 13: Reading: Mid Semester Feedback Sipser Ch

BU CS 332 Theory of Computation Lecture 13: Reading: Mid Semester Feedback Sipser Ch 4.1 Enumerators Decidable Languages Mark Bun March 16, 2020 What aspects of the course help you learn best? Examples in class

549 views • 29 slides
Parsing CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Fall 2018 Chinese University

Parsing CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Fall 2018 Chinese University

Parsing CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Fall 2018 Chinese University of Hong Kong 1/28 Context-free versus regular Every regular language is context-free regular expression NFA DFA 2/28 Write a CFG for the

868 views • 54 slides
CS20a: summary (Oct 24, 2002) Context-free languages Grammars G = (V, T, P, S)

CS20a: summary (Oct 24, 2002) Context-free languages Grammars G = (V, T, P, S)

CS20a: summary (Oct 24, 2002) Context-free languages Grammars G = (V, T, P, S) Pushdown automata N-PDA = CFG D-PDA < CFG Today What languages are context-free? Pumping lemma (similar to pumping lemma for regular

712 views • 41 slides
Chrobak normal form revisited, with applications Pawe Gawrychowski Institute of Computer

Chrobak normal form revisited, with applications Pawe Gawrychowski Institute of Computer

Chrobak normal form revisited, with applications Pawe Gawrychowski Institute of Computer Science, University of Wrocaw July 20, 2011 Pawe Gawrychowski Chrobak normal form... July 20, 2011 1 / 18 What is Chrobak normal form? We focus on

230 views • 22 slides
INF2080 Context-Free Langugaes Daniel Lupp Universitetet i Oslo 1st February 2016 Department

INF2080 Context-Free Langugaes Daniel Lupp Universitetet i Oslo 1st February 2016 Department

INF2080 Context-Free Langugaes Daniel Lupp Universitetet i Oslo 1st February 2016 Department of University of Informatics Oslo INF2080 Lecture :: 1st February 1 / 23 Repetition Weve looked at one of the simpler computational models:

859 views • 72 slides

More recommend