parsing parsers
play

Parsing Parsers Jenna Zeigen JSConf Hawaii 2/5/2020 - PowerPoint PPT Presentation

Jul 28, 2023 •339 likes •1.03k views

Parsing Parsers Jenna Zeigen JSConf Hawaii 2/5/2020 @zeigenvector jenna.is/at-jsconfhi Senior Frontend Engineer at Slack Organizer of EmpireJS Organizer of BrooklynJS @zeigenvector jenna.is/at-jsconfhi @zeigenvector

  1. Parsing Parsers Jenna Zeigen JSConf Hawaii 2/5/2020 @zeigenvector jenna.is/at-jsconfhi

  2. Senior Frontend Engineer at Slack Organizer of EmpireJS Organizer of BrooklynJS @zeigenvector jenna.is/at-jsconfhi

  3. @zeigenvector jenna.is/at-jsconfhi @zeigenvector jenna.is/at-jsconfhi

  4. parsing parsers! @zeigenvector jenna.is/at-jsconfhi

  5. 1. abcs of language @zeigenvector jenna.is/at-jsconfhi

  6. 1. abcs of language 2. hmm, actually, let's just step through a (small) parser @zeigenvector jenna.is/at-jsconfhi

  7. the abcs of language @zeigenvector jenna.is/at-jsconfhi

  8. the abcs of language "language" is a structured system of communication First you're up and you’re down And then between Oh I really want to know What do you mean? Ooh ♪♪♪ Justin Bieber - What Do You Mean? @zeigenvector https://en.wikipedia.org/wiki/Language jenna.is/at-jsconfhi

  9. the abcs of language "natural language" is a naturally evolved system that humans use to communicate with each other You speak And I know just what You're what you're sayin' ♪♪♪ @zeigenvector No Doubt - Don't Speak https://en.wikipedia.org/wiki/Language jenna.is/at-jsconfhi

  10. the abcs of language "formal languages" have an alphabet and words, which can be combined correctly based on specific rules I got new rules, I count 'em I got new rules, I count 'em ♪♪♪ 🎁 Dua Lipa - New Rules @zeigenvector https://en.wikipedia.org/wiki/Formal_language jenna.is/at-jsconfhi

  11. grammar school a language's grammar is the set of rules for that language Stop! Grammar time! ♪♪♪ MC Hammer - U Can't Touch This @zeigenvector https://en.wikipedia.org/wiki/Formal_grammar jenna.is/at-jsconfhi

  12. grammar school "formal grammars" put these rules in terms of replacement To the left, to the left To the left, to the left (Mmm) To the left, to the left Non-terminals in the spot to the left To the left, to the left The grammar tells us for what symbols 🎁 They are replaceable ♪♪♪ Beyoncé - Irreplaceable @zeigenvector https://en.wikipedia.org/wiki/Formal_grammar jenna.is/at-jsconfhi

  13. grammar school Sentence Noun Verb Phrase Verb Noun Phrase Direct Object Noun Jenna gave the talk @zeigenvector jenna.is/at-jsconfhi

  14. grammar school Sentence = Noun + Verb Phrase Verb Phrase = Verb + Noun Phrase Noun Phrase = Direct Object + Noun @zeigenvector jenna.is/at-jsconfhi

  15. grammar school Programming language grammars are defined in their spec thank u, spec thank u, spec thank u, spec I'm so very grateful for my spec ♪♪♪ Ariana Grande - thank u, next @zeigenvector https://www.ecma-international.org/ecma-262/10.0/index.html jenna.is/at-jsconfhi

  16. syntax city @zeigenvector jenna.is/at-jsconfhi

  17. syntax city javascript "front end" in:#random in: #general from:@jenna @zeigenvector jenna.is/at-jsconfhi

  18. syntax city javascript "front end" in:#random in: #general from:@jenna Query → Term Query → Term Query Query → Filter Query → Filter Query @zeigenvector jenna.is/at-jsconfhi

  19. syntax city javascript "front end" in:#random in: #general from:@jenna @zeigenvector jenna.is/at-jsconfhi

  20. ok, now parsers @zeigenvector jenna.is/at-jsconfhi

  21. moving parse the process of analyzing language against the rules of its grammar I got my rules up, And a bit of language Is its syntax okay? Yeah we're parsing in the USA ♪♪♪ Miley Cyrus - Party in the USA @zeigenvector https://en.wikipedia.org/wiki/Parsing jenna.is/at-jsconfhi

  22. moving parse a function that takes raw input and returns meaningful data created from the input, or an error All the beautiful inputs Are very, very meaningful You know, space is my favorite delimiter I felt so symbolic yesterday ♪♪♪ Counting Crows - Mr. Jones @zeigenvector https://dev.to/yelouafi/a-gentle-introduction-to-parser-combinators-21a0 jenna.is/at-jsconfhi

  23. moving parse parsers usually have two parts: the lexer and the parser lexer and parser making us a tree P-A-R-S-I-N-G ♪♪♪ @zeigenvector https://en.wikipedia.org/wiki/Parsing jenna.is/at-jsconfhi

  24. lex go the lexer takes the text and breaks it down into meaningful units, called "tokens" Reading through this code I've been asked to invoke Got a lexer out here first Made a nice short token ♪♪♪ Old Crow Medicine Show - Wagon Wheel @zeigenvector https://tomassetti.me/guide-parsing-algorithms-terminology/ jenna.is/at-jsconfhi

  25. lex go first, the "scanner" goes through and breaks the string of characters into the proper chunks, or "lexemes" I was born to lex (Yes) According to the spec What amazing tech, Having this effect (Woo) And soon the parser will turn These strings into objects (Money) ♪♪♪ Cardi B - Money @zeigenvector https://en.wikipedia.org/wiki/Lexical_analysis jenna.is/at-jsconfhi

  26. lex go coding time ♪♪♪ Semisonic - Closing Time https://github.com/jennazee/sparse/blob/master/sparse.js @zeigenvector https://jenna.is/sparse/sparse.html jenna.is/at-jsconfhi

  27. lex go const lexemes = 'Jenna gave the talk'.split(' '); @zeigenvector jenna.is/at-jsconfhi

  28. lex go const lexemes = "Jenna gave the talk" . split ( ' ' ) ; @zeigenvector jenna.is/at-jsconfhi

  29. lex go then, the "evaluator" combines the lexeme's type with its value to create the "token" I then begin to encounter with my parse, To split the text apart Break it down into sections Tokens from the lexemes ♪♪♪ The Notorious B.I.G. – Sky's The Limit @zeigenvector https://en.wikipedia.org/wiki/Lexical_analysis jenna.is/at-jsconfhi

  30. lex go coding time ♪♪♪ Semisonic - Closing Time https://github.com/jennazee/sparse/blob/master/sparse.js @zeigenvector https://jenna.is/sparse/sparse.html jenna.is/at-jsconfhi

  31. lex go const lexemes = "Jenna gave the talk" . split ( ' ' ) ; @zeigenvector jenna.is/at-jsconfhi

  32. lex go Keyword Identifier Punctuator String Punctuator Identifier Punctuator String Punctuator Punctuator @zeigenvector https://esprima.org/demo/parse.html# jenna.is/at-jsconfhi

  33. lex go [ { "type": "Keyword", "value": "const" }, { "type": "Identifier", "value": "lexemes" }, { "type": "Punctuator", "value": "=" }, { "type": "String", "value": "'Jenna gave a talk'" { "type": "Punctuator", "value": "." }, { "type": "Identifier", "value": "split" }, { "type": "Punctuator", "value": "(" }, { "type": "String", "value": "' '" }, { "type": "Punctuator", "value": ")" }, { "type": "Punctuator", "value": ";" } ] weird lex but ok @zeigenvector https://esprima.org/demo/parse.html# jenna.is/at-jsconfhi

  34. parse for the course the parser will check that the syntax is correct while creating a structural representation Every single word Is perfect as it can be And I put it in a tree ♪♪♪ CHVRCHES ft. Marshmello - Here With Me @zeigenvector https://www.geeksforgeeks.org/introduction-of-parsing-ambiguity-and-parsers-set-1/ jenna.is/at-jsconfhi

  35. parse for the course javascript "front end" in:#random in: #general from:@jenna Query Term Term InToken InToken FromToken @zeigenvector jenna.is/at-jsconfhi

  36. parse for the course I know who I want To read my code (It's you!) ♪♪♪ Semisonic - Closing Time https://github.com/jennazee/sparse/blob/master/sparse.js @zeigenvector https://jenna.is/sparse/sparse.html jenna.is/at-jsconfhi

  37. parse for the course Program VariableDeclaration VariableDeclarator CallExpression MemberExpression Arguments String Identifier String Identifier @zeigenvector https://esprima.org/demo/parse.html# jenna.is/at-jsconfhi

  38. parse for the course VariableDeclarator CallExpression MemberExpression Arguments String Identifier String Identifier const lexemes = 'Jenna gave the talk'.split(' '); @zeigenvector https://esprima.org/demo/parse.html# jenna.is/at-jsconfhi

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

LR Parsing Compiler Design CSE 504 Shift-Reduce Parsing 1 LR Parsers 2 SLR and LR(1) Parsers

LR Parsing Compiler Design CSE 504 Shift-Reduce Parsing 1 LR Parsers 2 SLR and LR(1) Parsers

LR Parsing Compiler Design CSE 504 Shift-Reduce Parsing 1 LR Parsers 2 SLR and LR(1) Parsers 3 Last modifled: Fri Mar 06 2015 at 13:50:06 EST Version: 1.7 16:58:46 2016/01/29 Compiled at 12:57 on 2016/02/26 Compiler Design LR Parsing

418 views • 16 slides
Compiling T echniques Lecture 7: Bottom-Up Parsing Christophe Dubach Overview Bottom-Up

Compiling T echniques Lecture 7: Bottom-Up Parsing Christophe Dubach Overview Bottom-Up

Compiling T echniques Lecture 7: Bottom-Up Parsing Christophe Dubach Overview Bottom-Up Parsing Finding Reductions Handle Pruning Shift-Reduce Parsers Parsing T echniques Top-down parsers (LL(1), recursive descent) Start at the root of

644 views • 30 slides
CS406: Compilers Spring 2020 Week 5: Parsers, AST, and Semantic Routines 1 Recap 2 3

CS406: Compilers Spring 2020 Week 5: Parsers, AST, and Semantic Routines 1 Recap 2 3

CS406: Compilers Spring 2020 Week 5: Parsers, AST, and Semantic Routines 1 Recap 2 3 Top-down Parsing predictive parsers 4 Suggested reading: https://en.wikipedia.org/wiki/LL_parser Top-down Parsing contd.. Also called

750 views • 46 slides
Objectives LL Parsing The topic for this lecture is a kind of grammar that works well with

Objectives LL Parsing The topic for this lecture is a kind of grammar that works well with

2 * 3 4 + + 2 * 3 4 Introduction LL Parsing Breaking LL Parsers Introduction LL Parsing Breaking LL Parsers Objectives LL Parsing The topic for this lecture is a kind of grammar that works well with recursive-descent parsing. Classify

425 views • 3 slides
CSC 4181 Compiler Construction Parsing 1 1 Outline Top-down v.s. Bottom-up Top-down parsing

CSC 4181 Compiler Construction Parsing 1 1 Outline Top-down v.s. Bottom-up Top-down parsing

CSC 4181 Compiler Construction Parsing 1 1 Outline Top-down v.s. Bottom-up Top-down parsing Bottom-up parsing Recursive-descent Shift-reduce parsers parsing LR(0) parsing LL(1) parsing LR(0) items LL(1) parsing

429 views • 29 slides
Predictive Parsers LL(k) Parsing Can we avoid backtracking? Yes, if for a given input symbol and

Predictive Parsers LL(k) Parsing Can we avoid backtracking? Yes, if for a given input symbol and

10/17/2012 Predictive Parsers LL(k) Parsing Can we avoid backtracking? Yes, if for a given input symbol and given non- LL(k) terminal, we can choose the alternative appropriately. L left to right scan L leftmost derivation

396 views • 7 slides
LL Parsing Dr. Mattox Beckman University of Illinois at Urbana-Champaign Department of Computer

LL Parsing Dr. Mattox Beckman University of Illinois at Urbana-Champaign Department of Computer

Introduction LL Parsing Breaking LL Parsers LL Parsing Dr. Mattox Beckman University of Illinois at Urbana-Champaign Department of Computer Science Introduction LL Parsing Breaking LL Parsers Objectives The topic for this lecture is a

565 views • 12 slides
Objectives Combinator Parsing Show how to build complex parsers by composing simpler parsers.

Objectives Combinator Parsing Show how to build complex parsers by composing simpler parsers.

Introduction Choices and Recursion Repeating and Composing Introduction Choices and Recursion Repeating and Composing Objectives Combinator Parsing Show how to build complex parsers by composing simpler parsers. Dr. Mattox Beckman

507 views • 5 slides
EVALB, Improving CKY Parsing, Hw3 Evaluating parsers Hw3 Optimization: tips and tricks Scott

EVALB, Improving CKY Parsing, Hw3 Evaluating parsers Hw3 Optimization: tips and tricks Scott

EVALB, Improving CKY Parsing, Hw3 Scott Farrar CLMA, University of Washington far- rar@u.washington.edu EVALB, Improving CKY Parsing, Hw3 Evaluating parsers Hw3 Optimization: tips and tricks Scott Farrar 1. Size of the grammar CLMA,

1.26k views • 87 slides
Parsing Parsing involves: determining if a string belongs to a language, and

Parsing Parsing involves: determining if a string belongs to a language, and

Parsing Parsing involves: determining if a string belongs to a language, and constructing structure of string if it belongs to language. Two approaches to constructing parsers: 1. Top down parsing: Our focus is on table-driven

428 views • 18 slides
Programming Languages: Parsing Onur Tolga S ehito glu Computer Engineering,METU 27 May

Programming Languages: Parsing Onur Tolga S ehito glu Computer Engineering,METU 27 May

Programming Languages:Parsing Programming Languages: Parsing Onur Tolga S ehito glu Computer Engineering,METU 27 May 2009 Programming Languages:Parsing Outline 1 Parsing Top-down Parsing Recursive Descent Parser LL Parsers

215 views • 8 slides
Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency

Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency

Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency Parsing Dependency Parsing Pure dependency parsing is only cubic [Eisner 99] Lexicalized parsers can be seen as producing dependency trees

314 views • 5 slides
Statistical Natural Language Parsing Gerald Penn [based on slides by Christopher Manning]

Statistical Natural Language Parsing Gerald Penn [based on slides by Christopher Manning]

Statistical Natural Language Parsing Gerald Penn [based on slides by Christopher Manning] Parsing examples Fed raises interest rates Parsing in the early 1990s The parsers produced detailed, linguistically rich representations

849 views • 46 slides
Plan for Today Recall Predictive Parsing when it works and when it doesnt necessary to

Plan for Today Recall Predictive Parsing when it works and when it doesnt necessary to

Plan for Today Recall Predictive Parsing when it works and when it doesnt necessary to remove left-recursion might have to left-factor Error recovery for predictive parsers Predictive parsing as a specific subclass of recursive

590 views • 10 slides
Parsing I: Earley Parser CMSC 35100 Natural Language Processing May 1, 2003 Roadmap

Parsing I: Earley Parser CMSC 35100 Natural Language Processing May 1, 2003 Roadmap

Parsing I: Earley Parser CMSC 35100 Natural Language Processing May 1, 2003 Roadmap Parsing: Accepting & analyzing Combining top-down & bottom-up constraints Efficiency Earley parsers Probabilistic CFGs

792 views • 36 slides
LR(0) Parsers CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Chinese University of

LR(0) Parsers CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Chinese University of

1/31 LR(0) Parsers CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Chinese University of Hong Kong Fall 2016 2/31 The alphabet of Java CFG consists of tokens like Parsing computer programs if (n == 0) { return x; } First phase of

729 views • 34 slides
1 Parse Trees Parse trees are a representation of derivations that is much more compact. Several

1 Parse Trees Parse trees are a representation of derivations that is much more compact. Several

1 Parse Trees Parse trees are a representation of derivations that is much more compact. Several derivations may correspond to the same parse tree. For example, in the balanced parenthesis grammar, the following parse tree: s s s ( s ) (

170 views • 6 slides
Parsing of Context-Free Grammars Bernd Kiefer { Bernd.Kiefer } @dfki.de Deutsches

Parsing of Context-Free Grammars Bernd Kiefer { Bernd.Kiefer } @dfki.de Deutsches

Parsing of Context-Free Grammars Bernd Kiefer { Bernd.Kiefer } @dfki.de Deutsches Forschungszentrum f ur k unstliche Intelligenz Introduction to CL - Context-Free Grammars p.1/32 Limits of Regular Expressions Assignment: Write a

1.62k views • 130 slides
Earley Parser Christopher Millar and Ekaterina Volkova Seminar fr Sprachwissenschaft

Earley Parser Christopher Millar and Ekaterina Volkova Seminar fr Sprachwissenschaft

Earley Parser Christopher Millar and Ekaterina Volkova Seminar fr Sprachwissenschaft Universitt Tbingen January 2007 Earley Parser: Bottom-up parsers In general, breadth-first bottom-up parsers are attractive since: they work

709 views • 59 slides
Parsing XML STAT 133 Gaston Sanchez Department of Statistics, UCBerkeley gastonsanchez.com

Parsing XML STAT 133 Gaston Sanchez Department of Statistics, UCBerkeley gastonsanchez.com

Parsing XML STAT 133 Gaston Sanchez Department of Statistics, UCBerkeley gastonsanchez.com github.com/gastonstat Course web: gastonsanchez.com/stat133 Parsing XML and HTML Content 2 Motivation In a nutshell Well cover a variety of

1.32k views • 64 slides
Constituency Parsing Karl Stratos Rutgers University Karl Stratos CS 533: Natural Language

Constituency Parsing Karl Stratos Rutgers University Karl Stratos CS 533: Natural Language

CS 533: Natural Language Processing Constituency Parsing Karl Stratos Rutgers University Karl Stratos CS 533: Natural Language Processing 1/37 Project Logistics 1. Proposal (due 3/24) 2. Milestone (due 4/15) 3. Presentation (tentatively

869 views • 37 slides
CS 103 Unit 14 Stringstreams and Parsing 2 I/O Streams '>>' operator used to

CS 103 Unit 14 Stringstreams and Parsing 2 I/O Streams '>>' operator used to

1 CS 103 Unit 14 Stringstreams and Parsing 2 I/O Streams '>>' operator used to read data from an input stream Always stops at whitespace '<<' operator used to write data to an output stream 'endl ' forces a

733 views • 28 slides
Lagrangian Based Approaches for Lexicalized Tree Adjoining Grammar Parsing Caio Corro

Lagrangian Based Approaches for Lexicalized Tree Adjoining Grammar Parsing Caio Corro

Lagrangian Based Approaches for Lexicalized Tree Adjoining Grammar Parsing Caio Corro Supervision: Adeline Nazarenko & Joseph Le Roux 9 march 2018 1 / 51 Syntax: description of structures in natural languages She walks the woman dog

1.77k views • 162 slides
LL1 Parsing a b $ S 1 A 4 B 3 2 a b b b 1. S --> a B

LL1 Parsing a b $ S 1 A 4 B 3 2 a b b b 1. S --> a B

LL1 Parsing a b $ S 1 A 4 B 3 2 a b b b 1. S --> a B Input 2. B --> 3. B --> A S 4. A --> b B Parse Stack Compare Input and Stacktop 1 LL1 Parsing a b $ S 1 A

694 views • 17 slides

More recommend