Packrat Parsin g: Sim ple, Powerfu l, Lazy, Lin ear Tim e Bryan - PowerPoint PPT Presentation

Packrat Parsin g: Sim ple, Powerfu l, Lazy, Lin ear Tim e Bryan Ford Massachusetts Institute of Technology In tern ation al Con feren ce on Fu n ction al Program m in g, October 2002

� � � � � Overview Wh at Is Packrat Parsin g? Wh at is it Good (an d n ot good) For? Practical Experien ce Related Work Con clu sion

Wh at is Packrat Parsin g? Answe r: Top-down parsin g with backtrackin g – except: Uses m em oization to ach ieve lin ear parse tim e

Example Grammar → Additive Multitive '+' Additive | Multitive Multitive → Primary '*' Multitive | Primary → Primary '(' Additive ')' | Decimal → Decimal '0' | ... | '9'

Recu rsive Descen t Parser

Recu rsive Descen t Parser data Result v = Parsed v String | NoParse

Recu rsive Descen t Parser data Result v = Parsed v String | NoParse

Recu rsive Descen t Parser data Result v = Parsed v String | NoParse Semantic Value

Recu rsive Descen t Parser data Result v = Parsed v String | NoParse Remainder String

Recu rsive Descen t Parser data Result v = Parsed v String | NoParse

Recu rsive Descen t Parser data Result v = Parsed v String | NoParse pAdditive :: String -> Result Int pMultitive:: String -> Result Int pPrimary :: String -> Result Int pDecimal :: String -> Result Int

Recu rsive Descen t Parser pAdditive :: String -> Result Int

Recu rsive Descen t Parser -- Additive → Multitive '+' Additive | Multitive -- pAdditive :: String -> Result Int

Recu rsive Descen t Parser -- Additive → Multitive '+' Additive | Multitive -- pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

Recu rsive Descen t Parser -- Additive → Multitive '+' Additive | Multitive -- pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

Recu rsive Descen t Parser -- Additive → Multitive '+' Additive | Multitive -- pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

Recu rsive Descen t Parser -- Additive → Multitive '+' Additive | Multitive -- pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

Recu rsive Descen t Parser -- Additive → Multitive '+' Additive | Multitive -- pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

Recu rsive Descen t Parser -- Additive → Multitive '+' Additive | Multitive -- pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

Recu rsive Descen t Parser -- Additive → Multitive '+' Additive | Multitive -- pAdditive :: String -> Result Int pAdditive = (do l <- pMultitive char '+' r <- pAdditive return (l + r)) <|> (do pMultitive)

Parsin g Exam ple pAdditive “2*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” pMu ltitive “2*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)” pPrim ary “2*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)” P → '(' A ')' pPrim ary “2*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)” P → '(' A ')' pPrim ary “2*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)” P → D pPrim ary “2*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)” P → D pPrim ary “2*(3+4)” pDecim al “2*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)” P → D pPrim ary “2*(3+4)” ⇒ Parsed 2 “*(3+4)” pDecim al “2*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)” ⇒ Parsed 2 “*(3+4)” pPrim ary “2*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)” ⇒ Parsed 2 “*(3+4)” pPrim ary “2*(3+4)” pCh ar '*' “*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)” ⇒ Parsed 2 “*(3+4)” pPrim ary “2*(3+4)” ⇒ Parsed () “(3+4)” pCh ar '*' “*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)” ⇒ Parsed 2 “*(3+4)” pPrim ary “2*(3+4)” ⇒ Parsed () “(3+4)” pCh ar '*' “*(3+4)” pMu ltitive “(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)” ⇒ Parsed 2 “*(3+4)” pPrim ary “2*(3+4)” ⇒ Parsed () “(3+4)” pCh ar '*' “*(3+4)” pMu ltitive “(3+4)” . . .

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” M → P '*' M pMu ltitive “2*(3+4)” ⇒ Parsed 2 “*(3+4)” pPrim ary “2*(3+4)” ⇒ Parsed () “(3+4)” pCh ar '*' “*(3+4)” ⇒ Parsed 7 “” pMu ltitive “(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” ⇒ Parsed 14 “” pMu ltitive “2*(3+4)”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” ⇒ Parsed 14 “” pMu ltitive “2*(3+4)” pCh ar '+' “”

Parsin g Exam ple A → M '+' A pAdditive “2*(3+4)” ⇒ Parsed 14 “” pMu ltitive “2*(3+4)” pCh ar '+' “”

Parsin g Exam ple A → M pAdditive “2*(3+4)”

Parsin g Exam ple A → M pAdditive “2*(3+4)” pMu ltitive “2*(3+4)”

Parsin g Exam ple A → M pAdditive “2*(3+4)” pMu ltitive “2*(3+4)” . . .

� Th e Backtrackin g Problem Can yield expon en tial worst-case parse tim es

� � � � � Th e Backtrackin g Problem Can yield expon en tial worst-case parse tim es T ypic a l solution: avoid backtrackin g by Prediction u sin g on e-token lookah ead Hackin g th e gram m ar Design in g th e lan gu age for easy parsin g

� � � � � � � Th e Backtrackin g Problem Can yield expon en tial worst-case parse tim es T ypic a l solution: avoid backtrackin g by Prediction u sin g on e-token lookah ead Hackin g th e gram m ar Design in g th e lan gu age for easy parsin g Alte rna te solution: allow backtrackin g; m em oize all interm ediate res ults .

� � � ✁ ✂ ✂ Mem oization of Resu lts Assu m ption s: Parsin g fu n ction s depen d only on in pu t strin g. Parsin g fu n ction s yield at m os t one res ult . Im plication : Requ ires resu lts table of size ( m ( n +1)) m = n u m ber of n on term in als/ parsin g fu n ction s n = len gth of in pu t strin g

Bu ildin g a Packrat Parser

Bu ildin g a Packrat Parser data Result v = Parsed v String | NoParse pAdditive :: String -> Result Int pMultitive:: String -> Result Int pPrimary :: String -> Result Int pDecimal :: String -> Result Int

Bu ildin g a Packrat Parser data Result v = Parsed v String | NoParse pAdditive :: String -> Result Int pMultitive:: String -> Result Int pPrimary :: String -> Result Int pDecimal :: String -> Result Int

Bu ildin g a Packrat Parser data Result v = Parsed v Derivs | NoParse pAdditive :: Derivs -> Result Int pMultitive:: Derivs -> Result Int pPrimary :: Derivs -> Result Int pDecimal :: Derivs -> Result Int

Bu ildin g a Packrat Parser data Result v = Parsed v Derivs | NoParse data Derivs = Derivs { dvAdditive :: Result Int, dvMultitive :: Result Int, dvPrimary :: Result Int, dvDecimal :: Result Int, dvChar :: Result Char}

Bu ildin g th e Derivs Stru ctu re parse :: String -> Derivs parse s = d where

Bu ildin g th e Derivs Stru ctu re parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr

Bu ildin g th e Derivs Stru ctu re parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') [] -> NoParse

Bu ildin g th e Derivs Stru ctu re parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') [] -> NoParse

Bu ildin g th e Derivs Stru ctu re parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') [] -> NoParse

Bu ildin g th e Derivs Stru ctu re parse :: String -> Derivs parse s = d where d = Derivs add mult prim dec chr chr = case s of (c:s') -> Parsed c (parse s') [] -> NoParse