Packrat Parsin g: Sim ple, Powerfu l, Lazy, Lin ear Tim e Bryan - - PowerPoint PPT Presentation

▶
packrat parsin g
SMART_READER_LITE
LIVE PREVIEW

Packrat Parsin g: Sim ple, Powerfu l, Lazy, Lin ear Tim e Bryan - - PowerPoint PPT Presentation

Oct 02, 2023 45 likes •1.04k views

Packrat Parsin g: Sim ple, Powerfu l, Lazy, Lin ear Tim e Bryan Ford Massachusetts Institute of Technology In tern ation al Con feren ce on Fu n ction al Program m in g, October 2002 Overview Wh at Is Packrat Parsin

slide-1
SLIDE 1

Packrat Parsin g:

Sim ple, Powerfu l, Lazy, Lin ear Tim e

Bryan Ford Massachusetts Institute of Technology

In tern ation al Con feren ce on Fu n ction al Program m in g, October 2002

slide-2
SLIDE 2

Overview

  • Wh at Is Packrat Parsin g?
  • Wh at is it Good (an d n ot good) For?
  • Practical Experien ce
  • Related Work
  • Con clu sion
slide-3
SLIDE 3

Wh at is Packrat Parsin g?

Answe r: Top-down parsin g with backtrackin g – except: Uses m em oization to ach ieve lin ear parse tim e

slide-4
SLIDE 4

Example Grammar

Additive → Multitive '+' Additive | Multitive Multitive → Primary '*' Multitive | Primary Primary → '(' Additive ')' | Decimal Decimal → '0' | ... | '9'

slide-5
SLIDE 5

Recu rsive Descen t Parser

slide-6
SLIDE 6

Recu rsive Descen t Parser

data Result v = Parsed v String | NoParse

slide-7
SLIDE 7

Recu rsive Descen t Parser

data Result v = Parsed v String | NoParse

slide-8
SLIDE 8

Recu rsive Descen t Parser

data Result v = Parsed v String | NoParse

Semantic Value

slide-9
SLIDE 9

Recu rsive Descen t Parser

data Result v = Parsed v String | NoParse

Remainder String

slide-10
SLIDE 10

Recu rsive Descen t Parser

data Result v = Parsed v String | NoParse

slide-11
SLIDE 11

Recu rsive Descen t Parser

data Result v = Parsed v String | NoParse pAdditive :: String -> Result Int pMultitive:: String -> Result Int pPrimary :: String -> Result Int pDecimal :: String -> Result Int

slide-12
SLIDE 12

Recu rsive Descen t Parser

pAdditive :: String -> Result Int

slide-13
SLIDE 13

Recu rsive Descen t Parser

  • - Additive →

Multitive '+' Additive

  • |

Multitive pAdditive :: String -> Result Int

slide-14
SLIDE 14

Recu rsive Descen t Parser

  • - Additive →

Multitive '+' Additive

  • |

Multitive pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

slide-15
SLIDE 15

Recu rsive Descen t Parser

  • - Additive →

Multitive '+' Additive

  • |

Multitive pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

slide-16
SLIDE 16

Recu rsive Descen t Parser

  • - Additive →

Multitive '+' Additive

  • |

Multitive pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

slide-17
SLIDE 17

Recu rsive Descen t Parser

  • - Additive →

Multitive '+' Additive

  • |

Multitive pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

slide-18
SLIDE 18

Recu rsive Descen t Parser

  • - Additive →

Multitive '+' Additive

  • |

Multitive pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

slide-19
SLIDE 19

Recu rsive Descen t Parser

  • - Additive →

Multitive '+' Additive

  • |

Multitive pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

slide-20
SLIDE 20

Recu rsive Descen t Parser

  • - Additive →

Multitive '+' Additive

  • |

Multitive pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

slide-21
SLIDE 21

Parsin g Exam ple

pAdditive “2*(3+4)”

slide-22
SLIDE 22

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

slide-23
SLIDE 23

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)”

slide-24
SLIDE 24

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

slide-25
SLIDE 25

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

pPrim ary “2*(3+4)”

slide-26
SLIDE 26

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

pPrim ary “2*(3+4)” P → '(' A ')'

slide-27
SLIDE 27

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

pPrim ary “2*(3+4)” P → '(' A ')'

slide-28
SLIDE 28

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

pPrim ary “2*(3+4)” P → D

slide-29
SLIDE 29

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

pPrim ary “2*(3+4)” P → D

pDecim al “2*(3+4)”

slide-30
SLIDE 30

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

pPrim ary “2*(3+4)” P → D

pDecim al “2*(3+4)” ⇒ Parsed 2 “*(3+4)”

slide-31
SLIDE 31

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

pPrim ary “2*(3+4)” ⇒ Parsed 2 “*(3+4)”

slide-32
SLIDE 32

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

pPrim ary “2*(3+4)” ⇒ Parsed 2 “*(3+4)” pCh ar '*' “*(3+4)”

slide-33
SLIDE 33

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

pPrim ary “2*(3+4)” ⇒ Parsed 2 “*(3+4)” pCh ar '*' “*(3+4)” ⇒ Parsed () “(3+4)”

slide-34
SLIDE 34

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

pPrim ary “2*(3+4)” ⇒ Parsed 2 “*(3+4)” pCh ar '*' “*(3+4)” ⇒ Parsed () “(3+4)” pMu ltitive “(3+4)”

slide-35
SLIDE 35

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

pPrim ary “2*(3+4)” ⇒ Parsed 2 “*(3+4)” pCh ar '*' “*(3+4)” ⇒ Parsed () “(3+4)” pMu ltitive “(3+4)”

. . .

slide-36
SLIDE 36

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” M → P '*' M

pPrim ary “2*(3+4)” ⇒ Parsed 2 “*(3+4)” pCh ar '*' “*(3+4)” ⇒ Parsed () “(3+4)” pMu ltitive “(3+4)” ⇒ Parsed 7 “”

slide-37
SLIDE 37

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” ⇒ Parsed 14 “”

slide-38
SLIDE 38

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” ⇒ Parsed 14 “” pCh ar '+' “”

slide-39
SLIDE 39

Parsin g Exam ple

pAdditive “2*(3+4)” A → M '+' A

pMu ltitive “2*(3+4)” ⇒ Parsed 14 “” pCh ar '+' “”

slide-40
SLIDE 40

Parsin g Exam ple

pAdditive “2*(3+4)” A → M

slide-41
SLIDE 41

Parsin g Exam ple

pAdditive “2*(3+4)” A → M

pMu ltitive “2*(3+4)”

slide-42
SLIDE 42

Parsin g Exam ple

pAdditive “2*(3+4)” A → M

pMu ltitive “2*(3+4)”

. . .

slide-43
SLIDE 43

Th e Backtrackin g Problem

  • Can yield expon en tial worst-case parse

tim es

slide-44
SLIDE 44

Th e Backtrackin g Problem

  • Can yield expon en tial worst-case parse

tim es

  • T

ypic a l solution: avoid backtrackin g by

  • Prediction u sin g on e-token lookah ead
  • Hackin g th e gram m ar
  • Design in g th e lan gu age for easy parsin g
slide-45
SLIDE 45

Th e Backtrackin g Problem

  • Can yield expon en tial worst-case parse

tim es

  • T

ypic a l solution: avoid backtrackin g by

  • Prediction u sin g on e-token lookah ead
  • Hackin g th e gram m ar
  • Design in g th e lan gu age for easy parsin g
  • Alte rna te solution: allow backtrackin g;
  • m em oize all interm ediate res ults .
slide-46
SLIDE 46

Mem oization of Resu lts

Assu m ption s:

  • Parsin g fu n ction s depen d only on in pu t

strin g.

  • Parsin g fu n ction s yield at m os t one res ult.

Im plication :

  • Requ ires resu lts table of size (m
✁

(n+1))

✂

m = n u m ber of n on term in als/ parsin g fu n ction s

✂

n = len gth of in pu t strin g

slide-47
SLIDE 47

Bu ildin g a Packrat Parser

slide-48
SLIDE 48

Bu ildin g a Packrat Parser

data Result v = Parsed v String | NoParse pAdditive :: String -> Result Int pMultitive:: String -> Result Int pPrimary :: String -> Result Int pDecimal :: String -> Result Int

slide-49
SLIDE 49

Bu ildin g a Packrat Parser

data Result v = Parsed v String | NoParse pAdditive :: String -> Result Int pMultitive:: String -> Result Int pPrimary :: String -> Result Int pDecimal :: String -> Result Int

slide-50
SLIDE 50

Bu ildin g a Packrat Parser

data Result v = Parsed v Derivs | NoParse pAdditive :: Derivs -> Result Int pMultitive:: Derivs -> Result Int pPrimary :: Derivs -> Result Int pDecimal :: Derivs -> Result Int

slide-51
SLIDE 51

Bu ildin g a Packrat Parser

data Result v = Parsed v Derivs | NoParse data Derivs = Derivs { dvAdditive :: Result Int, dvMultitive :: Result Int, dvPrimary :: Result Int, dvDecimal :: Result Int, dvChar :: Result Char}

slide-52
SLIDE 52

parse :: String -> Derivs parse s = d where

Bu ildin g th e Derivs Stru ctu re

slide-53
SLIDE 53

Bu ildin g th e Derivs Stru ctu re

parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr

slide-54
SLIDE 54

Bu ildin g th e Derivs Stru ctu re

parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') []

  • > NoParse
slide-55
SLIDE 55

Bu ildin g th e Derivs Stru ctu re

parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') []

  • > NoParse
slide-56
SLIDE 56

Bu ildin g th e Derivs Stru ctu re

parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') []

  • > NoParse
slide-57
SLIDE 57

Bu ildin g th e Derivs Stru ctu re

parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') []

  • > NoParse
slide-58
SLIDE 58

Bu ildin g th e Derivs Stru ctu re

parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') []

  • > NoParse
slide-59
SLIDE 59

Bu ildin g th e Derivs Stru ctu re

parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') []

  • > NoParse
slide-60
SLIDE 60

Bu ildin g th e Derivs Stru ctu re

parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') []

  • > NoParse
slide-61
SLIDE 61

Bu ildin g th e Derivs Stru ctu re

parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') []

  • > NoParse

add = pAdditive d mult= pMultitive d prim= pPrimary d dec = pDecimal d

slide-62
SLIDE 62

Bu ildin g th e Derivs Stru ctu re

parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') []

  • > NoParse

add = pAdditive d mult= pMultitive d prim= pPrimary d dec = pDecimal d

slide-63
SLIDE 63

Modifyin g th e Parsin g Fu n ction s

pAdditive :: Derivs -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

slide-64
SLIDE 64

Modifyin g th e Parsin g Fu n ction s

pAdditive :: Derivs -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

slide-65
SLIDE 65

Modifyin g th e Parsin g Fu n ction s

pAdditive :: Derivs -> Result Int pAdditive = (do l <- dvMultitive char '+' r <- dvAdditive return (l + r)) <|> (do dvMultitive)

slide-66
SLIDE 66

Packrat Parsin g Exam ple

parse “2*(3+4)”

slide-67
SLIDE 67

Packrat Parsin g Exam ple

A

? ? ? ?

M P D C

?

slide-68
SLIDE 68

Packrat Parsin g Exam ple

A

parse “*(3+4)” ? ? ? ? 2 ●

M P D C

slide-69
SLIDE 69

Packrat Parsin g Exam ple

A

parse “(3+4)” ? ? ? ? 2 ●

M P D C

? ? ? ? * ●

slide-70
SLIDE 70

Packrat Parsin g Exam ple

A

parse “3+4)” ? ? ? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ●

slide-71
SLIDE 71

Packrat Parsin g Exam ple

A

parse “+4)” ? ? ? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ●

slide-72
SLIDE 72

Packrat Parsin g Exam ple

A

parse “4)” ? ? ? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ● ? ? ? ? + ●

slide-73
SLIDE 73

Packrat Parsin g Exam ple

A

parse “)” ? ? ? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ● ? ? ? ? + ● ? ? ? ? 4 ●

slide-74
SLIDE 74

Packrat Parsin g Exam ple

A

parse “” ? ? ? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ● ? ? ? ? + ● ? ? ? ? 4 ● ? ? ? ? ) ●

slide-75
SLIDE 75

Packrat Parsin g Exam ple

A

? ? ? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ● ? ? ? ? + ● ? ? ? ? 4 ● ? ? ? ? ) ● ? ? ? ?

✘

slide-76
SLIDE 76

Packrat Parsin g Exam ple

A

? ? ? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ● ? ? ? ? + ● ? ? ? ? 4 ● ? ? ? ? ) ● ? ? ? ?

✘

(eval) A → M + A

slide-77
SLIDE 77

Packrat Parsin g Exam ple

A

? ? ? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ● ? ? ? ? + ● ? ? ? ? 4 ● ? ? ? ? ) ● ? ? ? ?

✘

(eval) M → P * M

slide-78
SLIDE 78

Packrat Parsin g Exam ple

A

? ? ? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ● ? ? ? ? + ● ? ? ? ? 4 ● ? ? ? ? ) ● ? ? ? ?

✘

(eval) P → D

slide-79
SLIDE 79

Packrat Parsin g Exam ple

A

? ? ? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ● ? ? ? ? + ● ? ? ? ? 4 ● ? ? ? ? ) ● ? ? ? ?

✘

(eval) D → '2'

slide-80
SLIDE 80

Packrat Parsin g Exam ple

A

? ? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ● ? ? ? ? + ● ? ? ? ? 4 ● ? ? ? ? ) ● ? ? ? ?

✘

2 ● (eval) P → D

slide-81
SLIDE 81

Packrat Parsin g Exam ple

A

? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ● ? ? ? ? + ● ? ? ? ? 4 ● ? ? ? ? ) ● ? ? ? ?

✘

2 ● 2 ● (eval) M → P * M

slide-82
SLIDE 82

Packrat Parsin g Exam ple

A

? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ● ? ? ? ? + ● ? ? ? ? 4 ● ? ? ? ? ) ● ? ? ? ?

✘

2 ● 2 ● (eval) M → P * M

slide-83
SLIDE 83

Packrat Parsin g Exam ple

A

? ? 2 ●

M P D C

? ? ? ? * ● ? ? ? ? ( ● ? ? ? ? 3 ● ? ? ? ? + ● ? ? ? ? 4 ● ? ? ? ? ) ● ? ? ? ?

✘

2 ● 2 ● (eval) M → P * M

slide-84
SLIDE 84

Packrat Parsin g Exam ple

A

? ? 2 ●

M P D C

? ? ? ? * ● ? ? ( ● 3 ● ? ? ? ? + ● 4 ● ? ? ? ? ) ● ? ? ? ?

✘

2 ● 2 ● (eval) 7 ● 7 ● 3 ● 3 ● 3 ● 7 ● 4 ● 4 ● 4 ● 4 ●

slide-85
SLIDE 85

Packrat Parsin g Exam ple

A

? ? 2 ●

M P D C

? ? ? ? * ● ? ? ( ● 3 ● ? ? ? ? + ● 4 ● ? ? ? ? ) ● ? ? ? ?

✘

2 ● 2 ● (eval) 7 ● 7 ● 3 ● 3 ● 3 ● 7 ● 4 ● 4 ● 4 ● 4 ● M → P * M

slide-86
SLIDE 86

Packrat Parsin g Exam ple

A

? 2 ●

M P D C

? ? ? ? * ● ? ? ( ● 3 ● ? ? ? ? + ● 4 ● ? ? ? ? ) ● ? ? ? ?

✘

2 ● 2 ● (eval) 7 ● 7 ● 3 ● 3 ● 3 ● 7 ● 4 ● 4 ● 4 ● 4 ● 14 ● A → M + A

slide-87
SLIDE 87

Packrat Parsin g Exam ple

A

? 2 ●

M P D C

? ? ? ? * ● ? ? ( ● 3 ● ? ? ? ? + ● 4 ● ? ? ? ? ) ● ? ? ? ?

✘

2 ● 2 ● (eval) 7 ● 7 ● 3 ● 3 ● 3 ● 7 ● 4 ● 4 ● 4 ● 4 ● 14 ● A → M + A

slide-88
SLIDE 88

Packrat Parsin g Exam ple

A

? 2 ●

M P D C

? ? ? ? * ● ? ? ( ● 3 ● ? ? ? ? + ● 4 ● ? ? ? ? ) ● ? ? ? ?

✘

2 ● 2 ● (eval) 7 ● 7 ● 3 ● 3 ● 3 ● 7 ● 4 ● 4 ● 4 ● 4 ● 14 ● A → M

slide-89
SLIDE 89

Packrat Parsin g Exam ple

A

2 ●

M P D C

? ? ? ? * ● ? ? ( ● 3 ● ? ? ? ? + ● 4 ● ? ? ? ? ) ● ? ? ? ?

✘

2 ● 2 ● (eval) 7 ● 7 ● 3 ● 3 ● 3 ● 7 ● 4 ● 4 ● 4 ● 4 ● 14 ● 14 ●

slide-90
SLIDE 90

Wh at is Packrat Parsin g Good (an d n ot good) For?

slide-91
SLIDE 91

Th eoretical Properties

  • Form ally developed by Birm an in 1970s
  • Proved existen ce of lin ear-tim e parsin g

algorith m

  • ...but apparently never im plem ented
  • Recogn izable langu ages:
  • Strictly larger th an determ in istic parsin g

algorith m s: e.g., LL(k ), LR(k )

  • In com parable to class of con text-free

lan gu ages

slide-92
SLIDE 92

Scan n erless Parsin g

  • Tradition al lin ear-tim e parsers lim ited by

fixed (e.g., on e-token ) lookah ead

  • If we on ly h ave on e lookah ead token , th en

it's easier if token s are big.

  • Packrat parsers provide u n lim ited

lookah ead

  • No lon ger n eed to separate lexical an alysis
  • Wh y scan n erless parsin g?
  • Sim plicity: u n ified gram m ar for en tire

lan gu age

  • Power: lexical elem en ts with com plex syn tax
slide-93
SLIDE 93

Syn tactic Flexibility

  • Syn tactic predicates
  • Parse X on ly if Y also m atch es
  • Parse X on ly if followed by Y
  • Su btractive syn tax
  • Parse X on ly if Y doesn 't m atch
  • Parse X on ly if not followed by Y
  • Sem an tic predicates
  • Parse X if its sem an tic valu e satisfies

con dition

slide-94
SLIDE 94

Lim itation s

Wh at is a packrat parser not good for?

  • Gen eral CFG parsin g: e.g., am bigu ou s

gram m ars

(becau se of “at-m ost-on e resu lt” lim itation )

  • Parsin g h igh ly “statefu l” syn tax: e.g., C, C++

(m em oization depen ds on statelessn ess)

  • Parsin g in m in im al space

(LL/ LR parser grows with stack depth , n ot in pu t size)

slide-95
SLIDE 95

Practical Parsers

Exam ple packrat parser for th e J ava lan gu age:

  • Un ified (scan n erless) parser
  • Im plem en ted in Haskell
  • Th ree version s:
  • 1. Han d-coded with m on adic com bin ators
  • 2. Han d-coded with prim itive pattern -m atch in g
  • 3. Au tom atically bu ilt by prototype parser

gen erator

slide-96
SLIDE 96

Perform an ce Resu lts (Su m m ary)

Parse Tim e:

  • Reliably lin ear growth with in pu t size
  • 26-52KB/ s (600-1200 lin es/ sec)

(GHC 5.04, 1.2GHz Ath lon )

  • Com parable to Happy-gen erated LR parser

(faster for average-size J ava sou rces)

  • Heap u sage:
  • Reliably lin ear growth with in pu t size
  • 300-600
✁

expan sion ratio

slide-97
SLIDE 97

Related Work

Fu n ction al/ m on adic Parsin g:

  • Wadler, Fokker, Hu tton , Meijer, etc.

Scann erless Parsin g:

  • Tai, Salom on , Corm ack – NSLR(1)
✂

(lin ear tim e, bu t restrictive of gram m ar)

  • Visser et al. – Gen eralized-LR
✂

(n ot lin ear tim e)

Syn tactic & Sem an tic Predicates:

  • Parr, Qu on g – pred-LL(k )
slide-98
SLIDE 98

Con clu sion

Packrat parsing:

  • Uses m em oization to provide backtrackin g

an d u n lim ited lookah ead in a lin ear-tim e parser

  • Is easily expressed as a lazy data stru ctu re
  • Provides m ore flexibility th an LL or LR

parsin g

  • En ables practical scan n erless parsin g
  • Has su bstan tial storage cost, bu t often

reason able

slide-99
SLIDE 99

For More In form ation

Papers, Master's Th esis Prototype Packrat Parser Gen erator Sou rce Code for Exam ple Parsers Test Su ite for Exam ple J ava Parsers available at:

http://pdos.lcs.mit.edu/~baford/packrat/