CSE443 Compilers Dr. Carl Alphonce alphonce@buffalo.edu 343 Davis - - PowerPoint PPT Presentation

CSE443 Compilers

• Dr. Carl Alphonce

alphonce@buffalo.edu 343 Davis Hall

Announcements

HW-01 posted PR-01 posted Team formation: what is current status?

Phases of a compiler

Figure 1.6, page 5 of text

Lexical structure

Bird's eye view

{ for, while, x, factorial, … }

language: a set of strings

G = (N, ∑, P, S)

grammar: rules for generating language

regular expression

regex: a form of grammar

finite automaton

a machine for language

C program

generated by FLEX

Formally, a grammar is defined by 4 items:

• 1. N, a set of non-terminals
• 2. ∑, a set of terminals
• 3. P, a set of productions
• 4. S, a start symbol

G = (N, ∑, P, S)

languages & grammars

N, a set of non-terminals ∑, a set of terminals (alphabet) N ∩ ∑ = {} P, a set of productions of the form (right linear) X -> a X -> aY X -> 𝜁 X ∈ N, Y ∈ N, a ∈ ∑, 𝜁 denotes the empty string S, a start symbol S ∈ N

languages & grammars

Lexical Analysis

Lexical structure described by regular grammar Deterministic finite state machine performs analysis

LANGUAGE operations

base cases { 𝜁 } is a regular language ∀ a ∈ ∑, { a } is a regular language

Recall, 𝜁 is the empty string

Li is L concatenated with itself i times: L0 = {𝜁}, by definition L1 = L L2 = LL L3 = LLL, etc. L* is the union of all these sets!

LANGUAGE operations

If L and M are regular, so are:

L ∪ M = { s | s ∈ L or s ∈ M } union LM = { st | s ∈ L and t ∈ M } concatenation L* = ∪i=0,∞ Li Kleene closure

Example of L*

Suppose L is {a, bb} L0 = {𝜁}, by definition L1 = L = {a, bb} L2 = LL = {aa, abb, bba, bbbb} L3 = LLL = {aaa, aabb, abba, abbbb, bbaa, bbbba, bbaa, bbabb, bbbba, bbbbbb, abbbb, bbabb} L4 = …and so so… L* = ∪i=0,∞ Li = {𝜁, a, bb, aa, abb, bba, bbbb, aaa, aabb, abba, abbbb, bbaa, bbbba, bbaa, bbabb, bbbba, bbbbbb, abbbb, bbabb, … }

Given an alphabet ∑ REGular EXpression (regex) Inductive definition

𝜁 is a regex 𝓜(𝜁) = {𝜁} For each a ∈ ∑, a is a regex 𝓜(a) = {a}

Regular expressions (regex) Inductive definition

Assume r and s are regexes. r|s is a regex denoting 𝓜(r)∪𝓜(s) rs is a regex denoting 𝓜(r)𝓜(s) r* is a regex denoting (𝓜(r))* (r) is a regex denoting 𝓜(r) Precedence: Kleene closure > concatenation > union Associativity: all left-associative (minimize use of parentheses: (r|s)|t = r|s|t )

Algebraic laws

Assume r and s are regexes. Commutativity r|s = s|r Associativity r|(s|t) = (r|s)|t and r(st) = (rs)t Disributivity r(s|t) = rs|rt and (s|t)r = sr|tr Identity 𝜁r = r𝜁 = r Idempotency r** = r*

We can describe a regular language using a regular expression

A regular expression can be recognized using a finite state machine. Machines: NFA non-deterministic finite automaton DFA deterministic finite automaton

Process of building lexical analyzer

language

1) spell out the language

Process of building lexical analyzer

language regex

2) formulate a regular expression

Process of building lexical analyzer

language regex NFA

3) build an NFA

Process of building lexical analyzer

DFA

4) transform NFA to DFA

language regex NFA

Process of building lexical analyzer

DFA

5) transform DFA to a minimal DFA

DFA language regex NFA

Process of building lexical analyzer

DFA character stream token stream

lexical analyzer 5) The minimal DFA is

• ur lexical analyzer

DFA language regex NFA

Focus for today

regex NFA

Nondeterministic Finite Automata (NFA)

A finite set of states S An alphabet ∑, 𝜁 ∉ ∑ 𝛆 ⊆ S X (∑ ∪ {𝜁}) X 𝒬(S) (transition function) s0 ∈ S (a single start state) F ⊆ S (a set of final or accepting states)

Deterministic Finite Automata (DFA)

A finite set of states S An alphabet ∑, 𝜁 ∉ ∑ 𝛆 ⊆ S X ∑ X S (transition function) s0 ∈ S (a single start state) F ⊆ S (a set of final or accepting states)

A state is a circle with its state number written inside.

Initial state has an arrow from nowhere pointing in. State 0 is

• ften the initial state.

1

A final state is drawn with a double circle.

… or a ∈ ∑.

1 𝜁 1 a for each a ∈ ∑

Arrows are labeled with 𝜁 …

Regex -> NFA

1 𝜁 1 a

N(s) N(t)

1 for each a ∈ ∑ S | t 𝜁 𝜁 𝜁 𝜁

Regex -> NFA

N(s) N(s)

1

N(t)

1 𝜁 𝜁 𝜁 𝜁 St S*

Simple example

static

Simple example

static 1 2 3 4 5 6 s t a t i c

Simple example

static struct 1 2 3 4 5 6 s t a t i c 7 8 9

10 11 12 13

s t r u c t i

F

𝜁 𝜁 𝜁 𝜁

Process of building lexical analyzer

DFA character stream token stream

lexical analyzer 5) The minimal DFA is

• ur lexical analyzer

DFA language regex NFA

Focus above: build a non-deterministic recognizer

regex NFA

DFA NFA

Next step: make recognizer deterministic

(a|b)*abb

first we construct an NFA from this regular expression

(a|b)*abb

a

(a|b)*abb

a b

(a|b)*abb

𝜁 𝜁 𝜁 𝜁 a b

(a|b)*abb

𝜁 𝜁 𝜁 𝜁 a b 𝜁 𝜁 𝜁 𝜁

(a|b)*abb

𝜁 𝜁 𝜁 𝜁 a b a 𝜁 𝜁 𝜁 𝜁

(a|b)*abb

𝜁 𝜁 𝜁 𝜁 a b a 𝜁 𝜁 𝜁 𝜁 b

(a|b)*abb

𝜁 𝜁 𝜁 𝜁 a b a 𝜁 𝜁 𝜁 𝜁 b b

(a|b)*abb

𝜁 𝜁 𝜁 𝜁 a b a 𝜁 𝜁 𝜁 𝜁 b b 1 2 3 4 5 6 7 8 9

10

Operations

𝜁-closure(t) is the set of states reachable from state t using only 𝜁-transitions. 𝜁-closure(T) is the set of states reachable from any state t ∈ T using only 𝜁- transitions. move(T,a) is the set of states reachable from any state t ∈ T following a transition on symbol a ∈ ∑.

NFA -> DFA algorithm

(set of states construction - page 153 of text)

INPUT: An NFA N = (S, ∑, 𝛆, s0, F) OUTPUT: A DFA D = (S', ∑, 𝛆', s0', F') such that ℒ(D)=ℒ(N) ALGORITHM: Compute s0' = 𝜁-closure(s0), an unmarked set of states Set S' = { s0' } while there is an unmarked T ∈ S' mark T for each symbol a ∈ ∑ let U = 𝜁-closure(move(T,a)) if U ∉ S', add unmarked U to S' add transition: 𝛆'(T,a) = U F' is the subset of S' all of whose members contain a state in F .