Continued CS 230 - Spring 2020 4-1 Scanning / Lexical Analysis - - PowerPoint PPT Presentation

CS 230 - Spring 2020 4-1

CS 230 – Introduction to Computers and Computer Systems Lecture 20 – CFGs and Tool Chain Continued

CS 230 - Spring 2020 4-2

Scanning / Lexical Analysis

 First step of compiler

 convert input string to tokens  use regular languages

 Conflicting rules: need priority

 use order of rules

 Usually: greedy quantification

 produce longest possible match

CS 230 - Spring 2020 4-3

Parsing / Syntax Analysis

 Second step of compiler  Clear and unambiguous description of

 valid sentences -> natural language  valid program -> programming language

 Need a rule set for classification of valid

sequences

 called a grammar

 Programming language – formal specification  English – informal specification

CS 230 - Spring 2020 4-4

Grammar

 Context-free grammar (CFG)

 formal description of a context free language  more powerful than a regular language

 Parsing

 compares a sequence of tokens with a CFG  build a parse tree

 if we can make a parse tree: sequence is in the language  if we cannot: sequence is not in the language

CS 230 - Spring 2020 4-5

Context-Free Grammar

 Formalism to specify languages

 simple and precise recursive definition  building block structure of languages  well-studied in theoretical computer science

 Context-free

 sequences do not depend on any external context

CS 230 - Spring 2020 4-6

CFG Components

 Token / Terminal

 atomic (undividable) symbol  usually from scanner phase

 Variable / Non-terminal

 abstract component  does not literally appear in input  designated non-terminal start symbol

 first non-terminal in grammar

 notation: angle brackets < ... >

CS 230 - Spring 2020 4-7

CFG Components

 Production / Rule:

 possible expansion of a non-terminal into

zero or more terminals and/or non-terminals

 more than one rule per non-terminal possible  describes how the language is put together  often recursive  build a parse tree by repeatedly applying

productions to the start symbol

 forms a tree that expands until only terminals remain

CS 230 - Spring 2020 4-8

Example – Basic Arithmetic CFG

<Expr>  <Expr> + <Term> <Expr>  <Expr> - <Term> <Expr>  <Term> <Term>  ( <Expr> ) <Term>  integer

CS 230 - Spring 2020 4-9

Example – Parse Tree

 Draw a parse tree for: 1 + 24 - 3

CS 230 - Spring 2020 4-10

Example – Parse Tree

 Draw a parse tree for: 1 + 24 - 3

<Expr>

CS 230 - Spring 2020 4-11

Example – Parse Tree

 Draw a parse tree for: 1 + 24 - 3

<Expr> <Expr> <Term>

CS 230 - Spring 2020 4-12

Example – Parse Tree

 Draw a parse tree for: 1 + 24 - 3

<Expr> <Expr> <Term>

• 3

CS 230 - Spring 2020 4-13

Example – Parse Tree

 Draw a parse tree for: 1 + 24 - 3

<Expr> <Expr> <Expr> <Term> <Term>

• +

3

CS 230 - Spring 2020 4-14

Example – Parse Tree

 Draw a parse tree for: 1 + 24 - 3

<Expr> <Expr> <Expr> <Term> <Term>

• +

3 24

CS 230 - Spring 2020 4-15

Example – Parse Tree

 Draw a parse tree for: 1 + 24 - 3

<Expr> <Expr> <Expr> <Term> <Term>

• +

<Term> 3 24

CS 230 - Spring 2020 4-16

Example – Parse Tree

 Draw a parse tree for: 1 + 24 - 3

<Expr> <Expr> <Expr> <Term> <Term>

• +

<Term> 3 24 1

CS 230 - Spring 2020 4-17

Example 2 – Parse Tree

 Draw a parse tree for: 1 + (24 – 3)

CS 230 - Spring 2020 4-18

Example 2 – Parse Tree

 Draw a parse tree for: 1 + (24 – 3)

<Expr> <Expr> <Term> +

CS 230 - Spring 2020 4-19

Example 2 – Parse Tree

 Draw a parse tree for: 1 + (24 – 3)

<Expr> <Expr> <Term> + <Term>

CS 230 - Spring 2020 4-20

Example 2 – Parse Tree

 Draw a parse tree for: 1 + (24 – 3)

<Expr> <Expr> <Term> + 1 <Term>

CS 230 - Spring 2020 4-21

Example 2 – Parse Tree

 Draw a parse tree for: 1 + (24 – 3)

<Expr> <Expr> <Term> + <Expr> 1 <Term> ( )

CS 230 - Spring 2020 4-22

Example 2 – Parse Tree

 Draw a parse tree for: 1 + (24 – 3)

<Expr> <Expr> <Term> + <Expr> 1 <Term> ( ) <Expr> <Term>

CS 230 - Spring 2020 4-23

Example 2 – Parse Tree

 Draw a parse tree for: 1 + (24 – 3)

<Expr> <Expr> <Term> + <Expr> 1 <Term> ( ) <Expr> <Term>

• <Term>

CS 230 - Spring 2020 4-24

Example 2 – Parse Tree

 Draw a parse tree for: 1 + (24 – 3)

<Expr> <Expr> <Term> + <Expr> 1 <Term> ( ) <Expr> <Term>

• 24

<Term>

CS 230 - Spring 2020 4-25

Example 2 – Parse Tree

 Draw a parse tree for: 1 + (24 – 3)

<Expr> <Expr> <Term> + <Expr> 1 <Term> ( ) <Expr> <Term>

• 24

<Term> 3

CS 230 - Spring 2020 4-26

Example 3 – Parse Tree

 Draw a parse tree for: 1 + 24 +

CS 230 - Spring 2020 4-27

Example 3 – Parse Tree

 Draw a parse tree for: 1 + 24 +

<Expr>

CS 230 - Spring 2020 4-28

Example 3 – Parse Tree

 Draw a parse tree for: 1 + 24 +

<Expr> <Expr> <Term> +

CS 230 - Spring 2020 4-29

Example 3 – Parse Tree

 Draw a parse tree for: 1 + 24 +

<Expr> <Expr> <Term> +

1 + 24 + is not in the language defined by this CFG

CS 230 - Spring 2020 4-30

Try it Yourself

Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5 <Expr>  <Expr> + <Term> <Expr>  <Expr> - <Term> <Expr>  <Term> <Term>  <Term> * <Factor> <Term>  <Term> / <Factor> <Term>  <Factor> <Factor>  ( <Expr> ) <Factor>  integer

CS 230 - Spring 2020 4-31

Try it Yourself

Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5

<Expr>

CS 230 - Spring 2020 4-32

Try it Yourself

Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5

<Expr> <Expr> <Term> +

CS 230 - Spring 2020 4-33

Try it Yourself

Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5

<Expr> <Expr> <Term> + <Term>

CS 230 - Spring 2020 4-34

Try it Yourself

Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5

<Expr> <Expr> <Term> + <Term> <Factor>

CS 230 - Spring 2020 4-35

Try it Yourself

Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5

<Expr> <Expr> <Term> + 8 <Term> <Factor>

CS 230 - Spring 2020 4-36

Try it Yourself

Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5

<Expr> <Expr> <Term> + 8 <Term> <Term> <Factor> * <Factor>

CS 230 - Spring 2020 4-37

Try it Yourself

Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5

<Expr> <Expr> <Term> + 8 <Term> <Term> <Factor> * <Factor> <Factor>

CS 230 - Spring 2020 4-38

Try it Yourself

Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5

<Expr> <Expr> <Term> + 8 <Term> 79 <Term> <Factor> * <Factor> <Factor>

CS 230 - Spring 2020 4-39

Try it Yourself

Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5

<Expr> <Expr> <Term> + 8 <Term> 79 5 <Term> <Factor> * <Factor> <Factor>

CS 230 - Spring 2020 4-40

Semantic Analysis

 Third step of compiler  Parsed tree is analyzed

 type checking  variables exist  function parameters match

 Build a symbol table

 list of variables, functions, classes, etc.

CS 230 - Spring 2020 4-41

Code Generation

 Fourth step of compiler  Convert parse tree and symbol table into target

language

 usually assembly language  need to make space in memory for variables based

• n types from symbol table

 parse tree tells us what order to do instructions

CS 230 - Spring 2020 4-42

Linking

 Code generation and assembler produce

machine code

 results are object files

 Linker combines object files into executables

 or libraries

 shared object files used by multiple programs

 part of classical tool chain

CS 230 - Spring 2020 4-43

Loading

 Last part of classical tool chain  Set up memory for new program  Loader loads executable file from HDD/SSD

 into main memory  also load any required libraries

 Start process

 put the program counter at the start of the program

CS 230 - Spring 2020 4-44

The End!

 CS 230 built up how a computer actually works

 from logic gates all the way up to compiling code