Description of a programming language Syntax describes the - - PowerPoint PPT Presentation

▶
description of a programming language
SMART_READER_LITE
LIVE PREVIEW

Description of a programming language Syntax describes the - - PowerPoint PPT Presentation

Feb 10, 2023 97 likes •547 views

Description of a programming language Syntax describes the structure of a language given as grammatical rules which streams of symbols (characters) form a legal program Syntax checking (syntax analysis, parsing)

slide-1
SLIDE 1

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

1

Description of a programming language

  • Syntax

– describes the structure of a language – given as grammatical rules – which streams of symbols (characters) form a legal program

  • Syntax checking (syntax analysis, parsing)

– construction of a parse tree – does the input program follow the grammatical rules – checking requires program transformation from a character string into a stream of tokes (lexical analysis, scanning)

  • Semantics

– what is the meaning of a given legal program – which kind of computation does a legal program produce

slide-2
SLIDE 2

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

Phases of compilation

  • Compilation is usually divided to separate

phases: easier, simpler, clearer

  • Output of a previous phase is the input of

the next one

  • Symbol table collects information on user-

defined constructs (variables, functions, types, …)

slide-3
SLIDE 3

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

Analyses (check-ups)

  • Natural language levels:

– lexical: ”It iß seven o’clock.” – syntactic: ”It seven o’clock.” – semantic: ”It is thirty o’clock.”

  • Programming language levels:

3

e.g. type checking code generation parsing scanning Process contextual semantic syntactic lexical Analysis

slide-4
SLIDE 4

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

Compilation process

4

Source program Lexical analyzer Intermediate code generator

tokens target language (assembly) characters

Parse tree Syntax analyzer Semantic analyzer Optimization Code generator Intermediate code Symbol table Executable

maybe the same

slide-5
SLIDE 5

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

5

Source program Interpreter Input data Results

Interpretation process

slide-6
SLIDE 6

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

6

Source program Lexical analyzer Syntax analyzer Intermediate code generator Interpreter Results

tokens parse tree intermediate code

Input data

Hybrid process

slide-7
SLIDE 7

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

Preprocessor

7

#define MAX_LOOP 100 #define INCR ( a ) ( a ) ++ #define FOR_LOOP ( var, from, to ) \\ for ( var = from; var <= to; INCR ( var ) ) { #define END_FOR } #define NULL FOR_LOOP ( n, 1, MAX_LOOP ) NULL; END_FOR; An example of macros:

C-code:

slide-8
SLIDE 8

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

Lexical analysis

8

Lexer

  • grammar: regular
  • format: regular expressions
  • implementation: finite state machine

characters (source code) list of tokens

slide-9
SLIDE 9

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

9

Examples of regular expressions

<digit> → 0 | 1 | ... | 9 <letter> → a | ... | z | A | … | Z <unsigned int> → <digit>* <id> → <letter> | <id> <letter> | <id> <digit> Digits and letters Numbers Identifiers

slide-10
SLIDE 10

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

10

Lexical analysis

  • Grouping of input characters

– lexeme

  • a unit that can be detected from

a program text

– token

  • classification of lexemes
  • a name given to a lexeme
  • Lexeme

– terminal symbol index = 2 * count;

Lexeme Token index identifier = equal_sign 2 int_literal * mult_op count identifier ; semicolon

slide-11
SLIDE 11

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

11

Lexemes/tokens

  • Keywords

– reserved words

  • Identifiers

– names chosen by the programmer

  • Literals

– constant values

  • Operators

– acronyms for (e.g. aritmetic) functions

  • Separators

– characters and strings be- tween language constructs

  • Other things to be

considered in lexical analysis:

– comments – white spaces – indentations

slide-12
SLIDE 12

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

12

Lexical analysis

program gcd ( input, output ); var i, j: integer; begin read ( i, j ); while i <> j do if i > j then i := i – j else j := j – i; writeln ( i ) end. program gcd ( input ,

  • utput

) ; var i , j : integer ; begin read ( i , j ) ; while i <> j do if i > j then i := i – j else j := j – i ; writeln ( i ) end . Identifying the lexemes (pattern matching)

Pascal code:

slide-13
SLIDE 13

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

Syntactic analysis

13

Parser

  • grammar: context free
  • format: BNF
  • implementation: push-down (stack) automaton

list of tokens parse tree symbol table

slide-14
SLIDE 14

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

14

Describing syntax

G = ( N, T, P, S) N: set of nonterminals T: set of terminals P: set of productions (rules) S: start symbol, S ∈ N Context-free grammar: The rules do not depend on the context in which they appear.

slide-15
SLIDE 15

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

15

Examples of grammar rules (BNF)

<variable def> ::= <identifier> <identifier> = <expr> <iteration stmt> ::= while ( <expr> ) <stmt> <stmt> ::= <iteration stmt> <stmt> ::= <compound stmt> <compound stmt> ::= { <statement seq> } <statement seq> ::= <stmt> <statement seq> ::= <stmt> <statement seq> Variable definition while-loop Statements (name) (type)

slide-16
SLIDE 16

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

16

Example of a grammar

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= <integer> <factor> ::= ( <expr> ) expr, term, factor, integer (, ), +, -, *, / (and integer instances) expr Grammar rules in BNF Nonterminals Terminals Start symbol

slide-17
SLIDE 17

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

17

Derivation of 2 * ( 12 + 3 )

<expr> = <term> = <term> * <factor> = <factor> * <factor> = <integer> * <factor> = 2 * <factor> = 2 * ( <expr> ) = 2 * ( <expr> + <term> ) = 2 * ( <term> + <term> ) = 2 * ( <factor> + <factor> ) = 2 * ( <integer> + <integer> ) = 2 * ( 12 + 3 )

<expr> ::= <expr> + <term> <expr> ::= <expr> - <term> <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor>::= <integer> <factor>::= ( <expr> )

slide-18
SLIDE 18

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

18

Derivation as a parse tree

F T T T E F E I I F F I + 3 12 2 * ( )

E = expr T = term F = factor I = integer

E T

slide-19
SLIDE 19

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

19

Ambiguous grammar

  • There exist several parse trees for one input

– input can be derived in several ways

<expr> ::= <expr> + <expr> <expr> ::= <expr> - <expr> <expr> ::= <expr> * <expr> <expr> ::= <expr> / <expr> <expr> ::= ( <expr> ) <expr> ::= <integer>

slide-20
SLIDE 20

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

20

A real example of ambiguity

if E1 then if E2 then S1 else S2 if E1 then if E2 then S1 else S2 dangling else: Corresponding grammar: <stmt> ::= <if_stmt> | ... <if_stmt> ::= if <expr> then <stmt> <if_stmt> ::= if <expr> then <stmt> else <stmt>

Pascal- code:

slide-21
SLIDE 21

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

21

”Dangling else” in parse trees

<if-stmt> if <expr> then <stmt> else <stmt> <if-stmt> if <expr> then <stmt> <if-stmt> if <expr> then <stmt> <if-stmt> if <expr> then <stmt> else <stmt> <stmt> ::= <if_stmt> | ... <if_stmt> ::= if <expr> then <stmt> <if_stmt> ::= if <expr> then <stmt> else <stmt>

slide-22
SLIDE 22

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

22

Solutions for dangling else problem

  • (The same problems exists in C)
  • Semantic rule:

– else branch belongs to the latest condition that not yet has an else branch

  • Programmer can use compound statements
slide-23
SLIDE 23

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

23

Solutions in other languages

if <expr> then <stmt-list> elsif <expr> then <stmt-list> ... else <stmt-list> end if if E1 then if E2 then S1; else S2; end if; end if; if E1 then if E2 then S1; end if; else S2; end if; Comb-like structures e.g. Ada and Modula-2

Ada:

slide-24
SLIDE 24

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

24

Extended BNF (EBNF)

<expr> ::= <term> { ( ’+’ | ’-’ ) <term> } <term> ::= <factor> { ( ’*’ | ’/’ ) <factor> } <factor> ::= ’(’ <expr> ’)’ | <integer> <expr> ::= <term> { ( ’+’ | ’-’ ) <term> }* ... parenthesis: grouping curly brackets: repetition (0, 1, … times) brackets: optionality repetition: *: 0, 1,.. times +: 1, 2, ... times

slide-25
SLIDE 25

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

25

Syntax diagram

Term Term +

  • Expr
slide-26
SLIDE 26

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

26

Parsing

  • Top-down –parsing (LL)

– produces left derivation – parse tree is constructed in depth-first order – recursive-descent parsing – cannot handle left recursion

  • Bottom-up –parsing (LR)

– parse tree is constructed from leaves to root – produces right derivation

A → A + B A → B a A B → A b left recursion:

slide-27
SLIDE 27

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

27

Grammar example for LL-parsing

<Decl> ::= <VarDecl> | <TypeDecl> <VarDecl> ::= VAR <Variable> :: <Type> <VarInit>; <TypeDecl> ::= TYPE <OwnType> = <Type> ; <Variable> ::= identifier <Type> ::= identifier <OwnType> ::= identifier <VarInit> ::= := <Value> | ε <Value> ::= number Lexemes and tokens

VAR reservedVar TYPE reservedType identifier instance identifier number instance number :: varSep = typeSep := initSep ; semicolon

slide-28
SLIDE 28

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

28

Scanning as state machine

number : : = initSep semicolon ;

letter

typeSep identifier reservedVar reservedType varSep =

digit

VAR reservedVar TYPE reservedType identifier instance identifier number instance number :: varSep = typeSep := initSep ; semicolon

start

slide-29
SLIDE 29

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

29

Parse code (in pseudo code)

procedure ParseDecl ( ) if LookUp ( reservedVar ) then ParseVarDecl ( ); else if LookUp ( reservedType ) then ParseTypeDecl ( ); else Error ( ); end if; procedure ParseVarDecl ( ) Scan ( reservedVar ); ParseVariable ( ); Scan ( varSep ); ParseType ( ); ParseVarInit ( ); Scan ( semicolon ); procedure ParseVariable ( ) Scan ( identifier ); // identifier processing

Scan: scans (and checks) next token, moves cursor LookUp: looks up next token, but does not move cursor <Decl> ::= <VarDecl> | <TypeDecl> <VarDecl> ::= VAR <Variable> :: <Type> <VarInit> ; <TypeDecl> ::= TYPE <OwnType> = <Type> ; <Variable> ::= identifier

Grammar:

slide-30
SLIDE 30

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

<TypeDecl> ::= TYPE <OwnType> = <Type> ; <Type> ::= identifier <OwnType> ::= identifier <VarInit> ::= := <Value> | ε <Value> ::= number

30

More parser code (in pseudo code)

procedure ParseVarInit ( ) if LookUp ( initSep ) then Scan ( initSep ); ParseValue ( ); end if; procedure ParseTypeDecl ( ) Scan ( reservedType ); ParseOwnType ( ); Scan ( typeSep ); ParseType ( ); Scan ( semicolon ); procedure ParseType ( ) Scan ( identifier ); // identifier processing procedure ParseOwnType ( ) Scan ( identifier ); // identifier processing procedure ParseValue ( ) Scan ( number ); // number processing

slide-31
SLIDE 31

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

31

top_down_parsing is push the start symbol onto an empty stack; while the stack is not empty do /* let X be the top stack symbol */ /* let a be the current input token */ if X is a nonterminal symbol then pop X from the stack; push the components of X onto the stack in reverse order; else if X is a terminal symbol and X = a then pop X from the stack; scan a; else /* syntax error */ end if; end while; end top_down_parsing;

General algorithm for top-down parsing

slide-32
SLIDE 32

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

32

An example on LL-parsing

VAR x :: Integer := 10; reservedVar identifier varSep identifier initSep number semicolon Example program Tokens

slide-33
SLIDE 33

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

33

LR-parsing

  • Operations in LR-parsing

– shift

  • add input symbol (lexeme) onto the stack
  • move forward in the input text (move cursor)

– reduce

  • find the right side of a grammar rule from the top of

the stack (it may consist of several stack items) and replace (reduce) it with the left side of the rule

slide-34
SLIDE 34

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

34

An example on LR-parsing

  • 1. E → E + T
  • 2. E → T
  • 3. T → T * F
  • 4. T → F
  • 5. F → ( E )
  • 6. F → id

Grammar: id + id * id Input:

slide-35
SLIDE 35

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

35

Action Goto State id + * ( ) $ E T F S5 S4 1 2 3 1 S6 accept 2 R2 S7 R2 R2 3 R4 R4 R4 R4 4 S5 S4 8 2 3 5 R6 R6 R6 R6 6 S5 S4 9 3 7 S5 S4 10 8 S6 S11 9 R1 S7 R1 R1 10 R3 R3 R3 R3 11 R5 R5 R5 R5

LR-parsing table

  • 1. E → E + T
  • 2. E → T
  • 3. T → T * F
  • 4. T → F
  • 5. F → ( E )
  • 6. F → id
slide-36
SLIDE 36

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

36

Stack Input Action id + id * id $ shift 5 0, id5 + id * id $ reduce 6 [ 0, F ] 0, F3 + id * id $ reduce 4 [ 0, T ] 0, T2 + id * id $ reduce 2 [ 0, E ] 0, E1 + id * id $ shift 6 0, E1, +6 id * id $ shift 5 0, E1, +6, id5 * id $ reduce 6 [6, F ] 0, E1, +6, F3 * id $ reduce 4 [6, T ] 0, E1, +6, T9 * id $ shift 7 0, E1, +6, T9, *7 id $ shift 5 0, E1, +6, T9, *7, id5 $ reduce 6 [ 7, F ] 0, E1, +6, T9, *7, F10 $ reduce 3 [ 6, T ] 0, E1, +6, T9 $ reduce 1 [ 0, E ] 0, E1 $ accept

Course

  • f LR

parsing

slide-37
SLIDE 37

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

Semantic analysis

37

Semantic analyzer

  • contextual check / semantics
  • type definitions, type compatibility
  • function definitions, parameter compatibility
  • etc.

Often combined with intermediate code generation parse tree symbol table

slide-38
SLIDE 38

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

38

Describing semantics

  • Semantics

– What does the program do? – Does the program do what it is supposed do?

  • Semantics is usually described informally
  • Static semantics

– can be checked at compilation-time

  • type compatibility (in assignments, in parameter passing)
  • are variables declared before using them
  • Dynamic (run-time) semantics

– checked at run-time

slide-39
SLIDE 39

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

39

Formal descriptions (for dynamic semantics)

  • Operational semantics

– describes the operation of the program – meaning of a statement = change in the state of the machine (memory, registers)

  • Denotational semantics

– models the program functionality in recursive functions – meaning of a statement = the value of the function associated with the statement

  • Axiomatic semantics

– goal is to prove correctness – meaning of a statement = transformation in logical formulas that describe the state of computation

slide-40
SLIDE 40

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

40

An example on operational semantics

for ( expr1; expr2; expr3 ) { ... } expr1; loop: if expr2 = 0 goto out ... expr3; goto loop

  • ut:

for statement in C: Operational semantics:

slide-41
SLIDE 41

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

41

An example on denotational semantics

M ( ’0’ ) = 0 M ( ’1’ ) = 1 M ( <BinNum> ’0’ ) = 2 * M ( <BinNum> ) M ( <BinNum> ’1’ ) = 2 * M ( <BinNum> ) + 1 <BinNum> ::= 0 <BinNum> ::= 1 <BinNum> ::= <BinNum> 0 <BinNum> ::= <BinNum> 1 Grammar rules for binary numbers: Evaluation

  • f binary

numbers: M is a semantic function

slide-42
SLIDE 42

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

Final compilation phases

  • Intermediate code generation
  • Code optimization
  • Code generation
  • see:

https://www.tutorialspoint.com/compiler _design/index.htm

slide-43
SLIDE 43

Principles of programming languages Maarit Harsu / Matti Rintala / Henri Hansen

TUT Pervasive Computing

Code generation /

  • ptimization

MOV y R0

  • - load y into R0

ADD z R0

  • - add z to R0

MOV R0 x

  • - store R0 into x

a = b + c; d = a + e; MOV b R0 ADD c R0 MOV R0 a MOV a R0 ADD e R0 MOV R0 d MOV a R0 ADD #1 R0 MOV R0 a a = a + 1; INC a