Continued CS 230 - Spring 2020 4-1 Scanning / Lexical Analysis - PowerPoint PPT Presentation

CS 230 – Introduction to Computers and Computer Systems Lecture 20 – CFGs and Tool Chain Continued CS 230 - Spring 2020 4-1

Scanning / Lexical Analysis  First step of compiler  convert input string to tokens  use regular languages  Conflicting rules: need priority  use order of rules  Usually: greedy quantification  produce longest possible match CS 230 - Spring 2020 4-2

Parsing / Syntax Analysis  Second step of compiler  Clear and unambiguous description of  valid sentences -> natural language  valid program -> programming language  Need a rule set for classification of valid sequences  called a grammar  Programming language – formal specification  English – informal specification CS 230 - Spring 2020 4-3

Grammar  Context-free grammar (CFG)  formal description of a context free language  more powerful than a regular language  Parsing  compares a sequence of tokens with a CFG  build a parse tree  if we can make a parse tree: sequence is in the language  if we cannot: sequence is not in the language CS 230 - Spring 2020 4-4

Context-Free Grammar  Formalism to specify languages  simple and precise recursive definition  building block structure of languages  well-studied in theoretical computer science  Context-free  sequences do not depend on any external context CS 230 - Spring 2020 4-5

CFG Components  Token / Terminal  atomic (undividable) symbol  usually from scanner phase  Variable / Non-terminal  abstract component  does not literally appear in input  designated non-terminal start symbol  first non-terminal in grammar  notation: angle brackets < ... > CS 230 - Spring 2020 4-6

CFG Components  Production / Rule:  possible expansion of a non-terminal into zero or more terminals and/or non-terminals  more than one rule per non-terminal possible  describes how the language is put together  often recursive  build a parse tree by repeatedly applying productions to the start symbol  forms a tree that expands until only terminals remain CS 230 - Spring 2020 4-7

Example – Basic Arithmetic CFG  <Expr> + <Term> <Expr>  <Expr> - <Term> <Expr>  <Term> <Expr>  ( <Expr> ) <Term>  integer <Term> CS 230 - Spring 2020 4-8

Example – Parse Tree  Draw a parse tree for: 1 + 24 - 3 CS 230 - Spring 2020 4-9

Example – Parse Tree  Draw a parse tree for: 1 + 24 - 3 <Expr> CS 230 - Spring 2020 4-10

Example – Parse Tree  Draw a parse tree for: 1 + 24 - 3 <Expr> <Expr> - <Term> CS 230 - Spring 2020 4-11

Example – Parse Tree  Draw a parse tree for: 1 + 24 - 3 <Expr> <Expr> - <Term> 3 CS 230 - Spring 2020 4-12

Example – Parse Tree  Draw a parse tree for: 1 + 24 - 3 <Expr> <Expr> - <Term> <Expr> + <Term> 3 CS 230 - Spring 2020 4-13

Example – Parse Tree  Draw a parse tree for: 1 + 24 - 3 <Expr> <Expr> - <Term> <Expr> + <Term> 3 24 CS 230 - Spring 2020 4-14

Example – Parse Tree  Draw a parse tree for: 1 + 24 - 3 <Expr> <Expr> - <Term> <Expr> + <Term> 3 <Term> 24 CS 230 - Spring 2020 4-15

Example – Parse Tree  Draw a parse tree for: 1 + 24 - 3 <Expr> <Expr> - <Term> <Expr> + <Term> 3 <Term> 24 1 CS 230 - Spring 2020 4-16

Example 2 – Parse Tree  Draw a parse tree for: 1 + (24 – 3) CS 230 - Spring 2020 4-17

Example 2 – Parse Tree  Draw a parse tree for: 1 + (24 – 3) <Expr> <Expr> + <Term> CS 230 - Spring 2020 4-18

Example 2 – Parse Tree  Draw a parse tree for: 1 + (24 – 3) <Expr> <Expr> + <Term> <Term> CS 230 - Spring 2020 4-19

Example 2 – Parse Tree  Draw a parse tree for: 1 + (24 – 3) <Expr> <Expr> + <Term> <Term> 1 CS 230 - Spring 2020 4-20

Example 2 – Parse Tree  Draw a parse tree for: 1 + (24 – 3) <Expr> <Expr> + <Term> <Term> ( <Expr> ) 1 CS 230 - Spring 2020 4-21

Example 2 – Parse Tree  Draw a parse tree for: 1 + (24 – 3) <Expr> <Expr> + <Term> <Term> ( <Expr> ) 1 <Expr> - <Term> CS 230 - Spring 2020 4-22

Example 2 – Parse Tree  Draw a parse tree for: 1 + (24 – 3) <Expr> <Expr> + <Term> <Term> ( <Expr> ) 1 <Expr> - <Term> <Term> CS 230 - Spring 2020 4-23

Example 2 – Parse Tree  Draw a parse tree for: 1 + (24 – 3) <Expr> <Expr> + <Term> <Term> ( <Expr> ) 1 <Expr> - <Term> <Term> 24 CS 230 - Spring 2020 4-24

Example 2 – Parse Tree  Draw a parse tree for: 1 + (24 – 3) <Expr> <Expr> + <Term> <Term> ( <Expr> ) 1 <Expr> - <Term> <Term> 3 24 CS 230 - Spring 2020 4-25

Example 3 – Parse Tree  Draw a parse tree for: 1 + 24 + CS 230 - Spring 2020 4-26

Example 3 – Parse Tree  Draw a parse tree for: 1 + 24 + <Expr> CS 230 - Spring 2020 4-27

Example 3 – Parse Tree  Draw a parse tree for: 1 + 24 + <Expr> <Expr> + <Term> CS 230 - Spring 2020 4-28

Example 3 – Parse Tree  Draw a parse tree for: 1 + 24 + <Expr> <Expr> + <Term> 1 + 24 + is not in the language defined by this CFG CS 230 - Spring 2020 4-29

Try it Yourself Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5  <Expr> + <Term> <Expr>  <Expr> - <Term> <Expr>  <Term> <Expr>  <Term> * <Factor> <Term>  <Term> / <Factor> <Term>  <Factor> <Term> <Factor>  ( <Expr> ) <Factor>  integer CS 230 - Spring 2020 4-30

Try it Yourself Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5 <Expr> CS 230 - Spring 2020 4-31

Try it Yourself Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5 <Expr> <Expr> + <Term> CS 230 - Spring 2020 4-32

Try it Yourself Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5 <Expr> <Expr> + <Term> <Term> CS 230 - Spring 2020 4-33

Try it Yourself Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5 <Expr> <Expr> + <Term> <Term> <Factor> CS 230 - Spring 2020 4-34

Try it Yourself Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5 <Expr> <Expr> + <Term> <Term> <Factor> 8 CS 230 - Spring 2020 4-35

Try it Yourself Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5 <Expr> <Expr> + <Term> <Term> <Term> * <Factor> <Factor> 8 CS 230 - Spring 2020 4-36

Try it Yourself Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5 <Expr> <Expr> + <Term> <Term> <Term> * <Factor> <Factor> <Factor> 8 CS 230 - Spring 2020 4-37

Try it Yourself Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5 <Expr> <Expr> + <Term> <Term> <Term> * <Factor> <Factor> <Factor> 8 79 CS 230 - Spring 2020 4-38

Try it Yourself Consider the following context-free grammar. Draw a parse tree for: 8 + 79 * 5 <Expr> <Expr> + <Term> <Term> <Term> * <Factor> <Factor> <Factor> 5 8 79 CS 230 - Spring 2020 4-39

Semantic Analysis  Third step of compiler  Parsed tree is analyzed  type checking  variables exist  function parameters match  Build a symbol table  list of variables, functions, classes, etc. CS 230 - Spring 2020 4-40

Code Generation  Fourth step of compiler  Convert parse tree and symbol table into target language  usually assembly language  need to make space in memory for variables based on types from symbol table  parse tree tells us what order to do instructions CS 230 - Spring 2020 4-41

Linking  Code generation and assembler produce machine code  results are object files  Linker combines object files into executables  or libraries  shared object files used by multiple programs  part of classical tool chain CS 230 - Spring 2020 4-42

Loading  Last part of classical tool chain  Set up memory for new program  Loader loads executable file from HDD/SSD  into main memory  also load any required libraries  Start process  put the program counter at the start of the program CS 230 - Spring 2020 4-43

The End!  CS 230 built up how a computer actually works  from logic gates all the way up to compiling code CS 230 - Spring 2020 4-44