Syntactic list of tokens analysis Syntactic analyzer grammar: - - PowerPoint PPT Presentation

90

Syntactic analysis

Syntactic analyzer

• grammar: context free
• format: BNF
• (implementation:

pushdown (stack) automaton)

list of tokens parse tree (symbol table?)

91

Description

• f the

syntax

• Check that the program respects the

syntax

• Generate parse tree and (and maybe

start symbol table) for code generation

• Context free grammar

– Parsing is represented by rules – Most commonly using BNF (or EBNF) – Rules do not depend on the context in

which they appear.

92

BNF (Backus-Naur Form)

• Notation to describe syntax
• Widely used in many areas of computing
• Describe syntax by giving syntax rules
• n how things can be composed from
• ther things
• Terminal symbol: basic tokens coming

from lexical analysis (not composed of

• ther symbols)
• Non-terminal symbol: symbol created

by a syntax rule from other symbols (both terminal and non-terminal)

93

Simple examples of syntactic rules

<variable definition> ::= identifier identifier '=' <expression> ';' (type) (name) <iteration statement> ::= 'while' '(' <expression> ')' <statement> <assignment> ::= identifier '=' <expression> ';' <statement> ::= <iteration statement> <statement> ::= <assignment> <statement> ::= <compound statement> <compound statement> ::= '{' <statement seq> '}' <statement seq> ::= <statement> <statement seq> ::= <statement> <statement seq> Statements, Code block

94

Example of a grammar

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> ) expr, term, factor, number number, (, ), +, -, *, / expr BNF rules: Non-terminals terminals Beginning symbol

95

Deriving exp. 2 * ( 12 + 3 )

2 * ( 12 + 3 )

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> )

96

Deriving exp. 2 * ( 12 + 3 )

2 * ( 12 + 3 ) (lexer) = number * ( number + number )

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> )

97

Deriving exp. 2 * ( 12 + 3 )

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> )

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> )

98

Deriving exp. 2 * ( 12 + 3 )

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) = number * ( <term> + <term> )

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> )

99

Deriving exp. 2 * ( 12 + 3 )

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) = number * ( <term> + <term> ) = number * ( <expr> + <term> )

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> )

100

Deriving exp. 2 * ( 12 + 3 )

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) = number * ( <term> + <term> ) = number * ( <expr> + <term> ) = number * ( <expr> )

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> )

101

Deriving exp. 2 * ( 12 + 3 )

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) = number * ( <term> + <term> ) = number * ( <expr> + <term> ) = number * ( <expr> ) = number * <factor>

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> )

102

Deriving exp. 2 * ( 12 + 3 )

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) = number * ( <term> + <term> ) = number * ( <expr> + <term> ) = number * ( <expr> ) = number * <factor> = <factor> * <factor>

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> )

103

Deriving exp. 2 * ( 12 + 3 )

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) = number * ( <term> + <term> ) = number * ( <expr> + <term> ) = number * ( <expr> ) = number * <factor> = <factor> * <factor> = <term> * <factor>

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> )

104

Deriving exp. 2 * ( 12 + 3 )

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) = number * ( <term> + <term> ) = number * ( <expr> + <term> ) = number * ( <expr> ) = number * <factor> = <factor> * <factor> = <term> * <factor> = <term>

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> )

105

Deriving exp. 2 * ( 12 + 3 )

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) = number * ( <term> + <term> ) = number * ( <expr> + <term> ) = number * ( <expr> ) = number * <factor> = <factor> * <factor> = <term> * <factor> = <term> = <expr> (Success!)

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> )

106

Fc Tm Tm Tm E E Fc E Tm Fc Fc + num(3) num(12) num(2) * ( )

Expression 2 * ( 12 + 3 ) as a parse tree E = expression Tm = term Fc = factor

107

Ambiguous grammar

<expr> ::= <expr> + <expr> <expr> ::= <expr> - <expr> <expr> ::= <expr> * <expr> <expr> ::= <expr> / <expr> <expr> ::= ( <expr> ) <expr> ::= number

108

Ambiguity example

if E1 then if E2 then S1 else S2 if E1 then if E2 then S1 else S2

"Dangling else"

Grammar: <stmt> ::= <if_stmt> | ... <if_stmt> ::= if <expr> then <stmt> <if_stmt> ::= if <expr> then <stmt> else <stmt>

Pascal: (also C/C++)

109

”Dangling else” in parse trees

<if-stmt> if <expr> then <stmt> else <stmt> <if-stmt> if <expr> then <stmt> <if-stmt>

if <expr> then <stmt>

<if-stmt> if <expr> then <stmt> else <stmt> <stmt> ::= <if_stmt> | ... <if_stmt> ::= if <expr> then <stmt> <if_stmt> ::= if <expr> then <stmt> else <stmt>

110

Dangling

• else –

solution

if <expr> then <stmt-list> elsif <expr> then <stmt-list> ... else <stmt-list> end if if E1 then if E2 then S1; else S2; end if; end if; if (E1) { if E2 { S1; } else { S2; } Modula-2 etc.: Ada: C/C++/Java etc. (braces optional!):

111

Extended BNF- grammars (EBNF)

<expr> ::= <term> { ( ’+’ | ’-’ ) <term> } <term>::= <factor> { ( ’*’ | ’/’ ) <factor> } <factor> ::= '(' <expr> ')' | number <expr> ::= <term> { ( ’+’ | ’-’ ) <term> }* ... parentheses: grouping of parts braces: repetition 0, 1,... times brackets: optional part *: 0, 1,.. repetitions +: 1, 2, ... reps.