CS453 Lecture Lexical Analysis with JLex 1
Plan for Lexical Analysis with Jlex and One Pass Code Gen
Overview of the MeggyJava Assignments PA2:Lexer/scanner in MJPA2.jar Expressing tokens with regular expressions
– regular expression syntax for JLex – using JLex with JavaCup
How do lexer generators work?
– Convert regular expressions to NFA – Converting an NFA to DFA – Implementing the DFA
PA2: Syntax-directed code generation (MJ.jar)
CS453 Lecture Introduction 2
Structure of the MeggyJava Compiler
sentences Synthesis Analysis character stream lexical analysis words tokens semantic analysis syntactic analysis AST AST and symbol table code gen Atmel assembly code PA1: Write test cases in MeggyJava, and AVR warmup PA2: MeggyJava scanner and setPixel PA3: add exps and control flow (AST) PA4: add methods (symbol table) PA5: add variables and objects PA6: add arrays and register allocation
PA2 Scanner/Lexer
Look at the assignment writeup and point out the tar ball. Look at the input files. Look at the output files. Look at MJPA2Driver.java. Look at mj.lex. Look at the Makefile.
CS453 Lecture Lexical Analysis with JLex 3 CS453 Lecture Lexical Analysis with JLex 4
Specifying Tokens with JLex
JLex example input file: package mjparser; import java_cup.runtime.Symbol; %% %line %char %cup %public %eofval{ return new Symbol(sym.EOF, newTokenValue("EOF", yyline, yychar));
%eofval} LETTER=[A-Za-z] DIGIT=[0-9] UNDERSCORE="_" LETT_DIG_UND={LETTER}|{DIGIT}|{UNDERSCORE} ID={LETTER}({LETT_DIG_UND})* ... %% "&&" {return new Symbol(sym.AND, newTokenValue(yytext(), yyline, yychar)); }
"+" {return new Symbol(sym.PLUS, ...); } "if" {return new Symbol(sym.IF,...); } {ID} {return new Symbol(sym.ID, new ... {EOL} { /* reset yychar */ … } {WS} { /* ignore */ }