Declarative Disambiguation of Deep Priority Conflicts Eduardo - - PowerPoint PPT Presentation

▶
declarative disambiguation of deep priority conflicts
SMART_READER_LITE
LIVE PREVIEW

Declarative Disambiguation of Deep Priority Conflicts Eduardo - - PowerPoint PPT Presentation

Apr 02, 2024 540 likes •997 views

Declarative Disambiguation of Deep Priority Conflicts Eduardo Souza, Timothee Haudebourg, Michael Steindorfer, Eelco Visser WG2.11 Kyoto June 4, 2018 Python Expression Syntax star_expr: '*' expr expr: xor_expr ('|' xor_expr)* xor_expr:

slide-1
SLIDE 1

Declarative Disambiguation of Deep Priority Conflicts

Eduardo Souza, Timothee Haudebourg, Michael Steindorfer, Eelco Visser

WG2.11 Kyoto June 4, 2018

slide-2
SLIDE 2

Python Expression Syntax

2

star_expr: '*' expr expr: xor_expr ('|' xor_expr)* xor_expr: and_expr ('^' and_expr)* and_expr: shift_expr ('&' shift_expr)* shift_expr: arith_expr (('<<'|'>>') arith_expr)* arith_expr: term (('+'|'-') term)* term: factor (('*'|'@'|'/'|'%'|'//') factor)* factor: ('+'|'-'|'~') factor | power power: atom_expr ['**' factor] atom_expr: [AWAIT] atom trailer* atom: ('(' [yield_expr|testlist_comp] ')' | '[' [testlist_comp] ']' | '{' [dictorsetmaker] '}' | NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False')

https://docs.python.org/3/reference/grammar.html

slide-3
SLIDE 3

Associativity and Precedence in YACC

3

%right '=' %left '+' '-' %left '*' '/' %% expr : expr '=' expr | expr '+' expr | expr '-' expr | expr '*' expr | expr '/' expr | NAME ;

slide-4
SLIDE 4

Semantics of Disambiguation in YACC

4

The precedences and associativities are used by Yacc to resolve parsing conflicts; they give rise to disambiguating rules. Formally, the rules work as follows:

  • 1. The precedences and associativities are recorded for those tokens and literals that have them.
  • 2. A precedence and associativity is associated with each grammar rule; it is the precedence and

associativity of the last token or literal in the body of the rule. If the %prec construction is used, it

  • verrides this default. Some grammar rules may have no precedence and associativity associated

with them.

  • 3. When there is a reduce/reduce conflict, or there is a shift/reduce conflict and either the input

symbol or the grammar rule has no precedence and associativity, then the two disambiguating rules given at the beginning of the section are used, and the conflicts are reported.

  • 4. If there is a shift/reduce conflict, and both the grammar rule and the input character have

precedence and associativity associated with them, then the conflict is resolved in favor of the action (shift or reduce) associated with the higher precedence. If the precedences are the same, then the associativity is used; left associative implies reduce, right associative implies shift, and nonassociating implies error.

http://dinosaur.compilertools.net/yacc/

slide-5
SLIDE 5

Disambiguation with Tree Automata

5

[Adams & Might, OOPSLA 2017]

slide-6
SLIDE 6

6

What is the direct semantics of associativity and priority?

slide-7
SLIDE 7

7

SDF2 Semantics

[Visser 1997]

slide-8
SLIDE 8

Declarative Disambiguation in SDF3

8

context-free syntax Exp.Add = Exp "+" Exp {left} Exp.Mul = Exp "*" Exp {left} Exp.Int = INT context-free priorities Exp.Mul > Exp.Add

slide-9
SLIDE 9

If Exp + Exp

SDF2 Semantics

9 Exp Exp + If context-free syntax Exp.Add = Exp "+" Exp Exp.If = "if" "(" Exp ")" Exp context-free priorities Exp.Add > Exp.If

slide-10
SLIDE 10

If Exp + Exp

SDF2 Semantics

10 Exp Exp + If context-free syntax Exp.Add = Exp "+" Exp Exp.If = "if" "(" Exp ")" Exp context-free priorities Exp.Add > Exp.If

Exp.If Exp.Add

Exp Exp if(e1) + e2 e3 Exp Exp + e2 e3 if(e1)

if(e1) e2 + e3

slide-11
SLIDE 11

If Exp + Exp

SDF2 Semantics

11 Exp Exp + If context-free syntax Exp.Add = Exp "+" Exp Exp.If = "if" "(" Exp ")" Exp context-free priorities Exp.Add > Exp.If

Exp.If Exp.Add

Exp Exp if(e1) + e2 e3 Exp Exp + e2 e3 if(e1)

if(e1) e2 + e3

slide-12
SLIDE 12

Exp.If Exp.Add

Exp Exp if(e1) + e2 e3 Exp Exp + e2 e3 if(e1)

if(e1) e2 + e3

If Exp + Exp

SDF2 Semantics

12 Exp Exp + If context-free syntax Exp.Add = Exp "+" Exp Exp.If = "if" "(" Exp ")" Exp context-free priorities Exp.Add > Exp.If

slide-13
SLIDE 13

13

context-free syntax Exp.Min = "-" Exp Exp.If = "if" "(" Exp ")" Exp Exp.Lt = Exp "<" Exp Exp.Add = Exp "+" Exp Exp.Seq = Exp ";" Exp ";" Exp.Fac = Exp "!" Exp = "(" Exp ")" Exp.Int = INT

slide-14
SLIDE 14

Exp.Fac Exp.Lt Exp.If Exp.Min Exp.Add Exp.Seq

if(e1) - e2 if(e1) e2 + e3 if(e1) e2; e3; e1 < - e2 e1 < e2 + e3 e1 < e2; e3; e1!; e2; e1! + e2

  • e1!

Exp Exp if(e1) e2

  • Exp

Exp if(e1) + e2 e3 Exp Exp + e2 e3 if(e1) Exp Exp if(e1) e2 ; e3; Exp Exp e2 if(e1) ; e3; e2 Exp Exp < e2

  • Exp

< e1 e2 Exp + e3 Exp Exp < e1 + e2 e3 ; e3; Exp Exp < e1 e2 Exp < e1 e2 Exp ; e3; Exp Exp e2

  • !

Exp Exp

  • e2

! Exp ! e1 e2 Exp + ; e2; Exp ! e1 Exp

14 Exp.Add > Exp.If Exp.If > Exp.Seq Exp.Add > Exp.Lt Exp.Lt > Exp.Seq Exp.Min > Exp.Fac context-free syntax Exp.Min = "-" Exp Exp.If = "if" "(" Exp ")" Exp Exp.Lt = Exp "<" Exp {right} Exp.Add = Exp "+" Exp {left} Exp.Seq = Exp ";" Exp ";" Exp.Fac = Exp "!" Exp = "(" Exp ")" {bracket} Exp.Int = INT context-free priorities Exp.Min > Exp.Lt > Exp.Add > Exp.If > Exp.Seq > Exp.Fac

slide-15
SLIDE 15

15

SDF2 Semantics is Unsafe

[Afroozeh et al. 2013]

slide-16
SLIDE 16

16

Exp.Add > Exp.If Exp.Add Exp.If

e1 + if(e2) e3

if(e2) e1 Exp Exp + e3

Exp.If Exp.Add

Exp Exp if(e1) + e2 e3 Exp Exp + e2 e3 if(e1)

if(e1) e2 + e3

If Exp + Exp Exp Exp + If

Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp "+" Exp

slide-17
SLIDE 17

17

Exp.Add > Exp.If Exp.Add Exp.If

e1 + if(e2) e3

if(e2) e1 Exp Exp + e3

Exp.If Exp.Add

Exp Exp if(e1) + e2 e3 Exp Exp + e2 e3 if(e1)

if(e1) e2 + e3

If Exp + Exp Exp Exp + If

Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp "+" Exp

slide-18
SLIDE 18

Exp.Fac Exp.Lt Exp.If Exp.Min Exp.Add Exp.Seq

if(e1) - e2 if(e1) e2 + e3 if(e1) e2; e3; e1 < - e2 e1 < e2 + e3 e1 < e2; e3; e1!; e2; e1! + e2

  • e1!

Exp Exp if(e1) e2

  • Exp

Exp if(e1) + e2 e3 Exp Exp + e2 e3 if(e1) Exp Exp if(e1) e2 ; e3; Exp Exp e2 if(e1) ; e3; e2 Exp Exp < e2

  • Exp

< e1 e2 Exp + e3 Exp Exp < e1 + e2 e3 ; e3; Exp Exp < e1 e2 Exp < e1 e2 Exp ; e3; Exp Exp e2

  • !

Exp Exp

  • e2

! Exp ! e1 e2 Exp + ; e2; Exp ! e1 Exp

18 Exp.Seq > Exp.Fac Exp.Add > Exp.If Exp.If > Exp.Seq Exp.Add > Exp.Lt Exp.Lt > Exp.Seq context-free syntax Exp.Min = "-" Exp Exp.If = "if" "(" Exp ")" Exp Exp.Lt = Exp "<" Exp {right} Exp.Add = Exp "+" Exp {left} Exp.Seq = Exp ";" Exp ";" Exp.Fac = Exp "!" Exp = "(" Exp ")" {bracket} Exp.Int = INT context-free priorities Exp.Min > Exp.Lt > Exp.Add > Exp.If > Exp.Seq > Exp.Fac Exp.Min > Exp.Fac Exp.Add > Exp.Fac

slide-19
SLIDE 19

19

Disambiguation should not reject unambiguous sentences

Disambiguation Safety

slide-20
SLIDE 20

20

Safe Semantics

slide-21
SLIDE 21

21

Exp.Add > Exp.If Exp.Add Exp.If

e1 + if(e2) e3

if(e2) e1 Exp Exp + e3

Exp.If Exp.Add

Exp Exp if(e1) + e2 e3 Exp Exp + e2 e3 if(e1)

if(e1) e2 + e3

If Exp + Exp Exp Exp + If

Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp "+" Exp

slide-22
SLIDE 22

22

Exp.Add > Exp.If Exp.Add Exp.If

e1 + if(e2) e3

if(e2) e1 Exp Exp + e3

Exp.If Exp.Add

Exp Exp if(e1) + e2 e3 Exp Exp + e2 e3 if(e1)

if(e1) e2 + e3

If Exp + Exp Exp Exp + If

Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp "+" Exp

slide-23
SLIDE 23

23

Exp.Add Exp.If

e1 + if(e2) e3

if(e2) e1 Exp Exp + e3

Exp.If Exp.Add

Exp Exp if(e1) + e2 e3 Exp Exp + e2 e3 if(e1)

if(e1) e2 + e3

Exp.Add > Exp.If

If Exp + Exp Exp Exp + If

Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp "+" Exp

slide-24
SLIDE 24

Exp.Fac Exp.Lt Exp.If Exp.Min Exp.Add Exp.Seq

if(e1) - e2 if(e1) e2 + e3 if(e1) e2; e3; e1 < - e2 e1 < e2 + e3 e1 < e2; e3; e1!; e2; e1! + e2

  • e1!

Exp Exp if(e1) e2

  • Exp

Exp if(e1) + e2 e3 Exp Exp + e2 e3 if(e1) Exp Exp if(e1) e2 ; e3; Exp Exp e2 if(e1) ; e3; e2 Exp Exp < e2

  • Exp

< e1 e2 Exp + e3 Exp Exp < e1 + e2 e3 ; e3; Exp Exp < e1 e2 Exp < e1 e2 Exp ; e3; Exp Exp e2

  • !

Exp Exp

  • e2

! Exp ! e1 e2 Exp + ; e2; Exp ! e1 Exp

24 Exp.Seq > Exp.Fac Exp.Add > Exp.If Exp.If > Exp.Seq Exp.Add > Exp.Lt Exp.Lt > Exp.Seq context-free syntax Exp.Min = "-" Exp Exp.If = "if" "(" Exp ")" Exp Exp.Lt = Exp "<" Exp {right} Exp.Add = Exp "+" Exp {left} Exp.Seq = Exp ";" Exp ";" Exp.Fac = Exp "!" Exp = "(" Exp ")" {bracket} Exp.Int = INT context-free priorities Exp.Min > Exp.Lt > Exp.Add > Exp.If > Exp.Seq > Exp.Fac Exp.Min > Exp.Fac Exp.Add > Exp.Fac

slide-25
SLIDE 25

Exp.Seq Exp.Add Exp.Min Exp.If Exp.Lt Exp.Fac

  • if(e1) e2
  • e2 < e3
  • e1!

e1 + if(e2) e3 e1 + e2 < e3 e1 + e2! e1; e2;! e1; e2; < e3 if(e1) e2; e3;

Exp Exp

  • e2

if(e1) e1 Exp Exp + e3 if(e2) Exp + e1 e2 Exp < e3 Exp Exp + e1 < e2 e3 ! Exp Exp + e1 e2 Exp + e1 e2 Exp ! Exp ; e2; e1 e3 Exp < ! Exp ; e2; e1 Exp Exp Exp e2 if(e1) ; e3; Exp Exp if(e1) e2 ; e3; Exp Exp

  • <

e2 e3 Exp Exp < e2 e3

  • Exp

Exp e2

  • !

Exp Exp

  • e2

!

25 Exp.Seq > Exp.Fac Exp.Add > Exp.If Exp.If > Exp.Seq Exp.Add > Exp.Lt Exp.Lt > Exp.Seq context-free syntax Exp.Min = "-" Exp Exp.If = "if" "(" Exp ")" Exp Exp.Lt = Exp "<" Exp {right} Exp.Add = Exp "+" Exp {left} Exp.Seq = Exp ";" Exp ";" Exp.Fac = Exp "!" Exp = "(" Exp ")" {bracket} Exp.Int = INT context-free priorities Exp.Min > Exp.Lt > Exp.Add > Exp.If > Exp.Seq > Exp.Fac Exp.Min > Exp.Fac Exp.Add > Exp.Fac

slide-26
SLIDE 26

26

Deep Conflicts

slide-27
SLIDE 27

27

e1 * if(e2) e3 + e4 Exp if(e2) Exp Exp Exp * e1 + e3 e4

context-free syntax Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp "+" Exp {left} Exp.Mul = Exp "*" Exp {left} Exp.Int = INT context-free priorities Exp.Mul > Exp.Add > Exp.If

e1 * [if(e2) e3 + e4]

slide-28
SLIDE 28

Exp Exp * Exp e1 if(e2) e3 Exp + Exp e4

28

e1 * if(e2) e3 + e4

context-free syntax Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp "+" Exp {left} Exp.Mul = Exp "*" Exp {left} Exp.Int = INT context-free priorities Exp.Mul > Exp.Add > Exp.If

Exp if(e2) Exp Exp Exp * e1 + e3 e4 e1 * [if(e2) e3 + e4] [e1 * if(e2) e3] + e4

slide-29
SLIDE 29

29

e1 * if(e2) e3 + e4 Exp Exp * Exp e1 if(e2) e3 Exp + Exp e4 Exp if(e2) Exp Exp Exp * e1 + e3 e4

context-free syntax Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp "+" Exp {left} Exp.Mul = Exp "*" Exp {left} Exp.Int = INT context-free priorities Exp.Mul > Exp.Add > Exp.If

e1 * [if(e2) e3 + e4] [e1 * if(e2) e3] + e4

Deep Priority Conflict!

slide-30
SLIDE 30

Operator-style Deep Conflict

30 Exp = α Exp Exp = Exp β Exp = α' Exp α' > β > α

α' α Exp β

Exp

α

Exp Exp

α'

Exp

β

Exp

β

Exp

α'

Exp

α

Exp

α' α [Exp β] [α' α Exp] β

slide-31
SLIDE 31

31

e1 * if(e2) e3 + e4 Exp if(e2) Exp Exp Exp * e1 + e3 e4

context-free syntax Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp "+" Exp {left} Exp.Mul = Exp "*" Exp {left} Exp.Int = INT context-free priorities Exp.Mul > Exp.Add > Exp.If

e1 * [if(e2) e3 + e4]

Deep Priority Conflict!

Exp Exp * Exp e1 if(e2) e3 Exp + Exp e4 [e1 * if(e2) e3] + e4

slide-32
SLIDE 32

32

e1 * if(e2) e3 + e4 Exp if(e2) Exp Exp Exp * e1 + e3 e4

context-free syntax Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp "+" Exp {left} Exp.Mul = Exp "*" Exp {left} Exp.Int = INT context-free priorities Exp.Mul > Exp.Add > Exp.If

e1 * [if(e2) e3 + e4]

Deep Priority Conflict!

Exp Exp * Exp e1 if(e2) e3 Exp + Exp e4 [e1 * if(e2) e3] + e4

slide-33
SLIDE 33

33

Contextual Grammars

slide-34
SLIDE 34

34

e1 * if(e2) e3 + e4 Exp Exp * Exp e1 if(e2) e3 Exp + Exp e4

context-free syntax Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp "+" Exp {left} Exp.Mul = Exp "*" Exp {left} Exp.Int = INT context-free priorities Exp.Mul > Exp.Add > Exp.If

[e1 * if(e2) e3] + e4

Deep Priority Conflict!

slide-35
SLIDE 35

35

e1 * if(e2) e3 + e4 Exp Exp * Exp e1 if(e2) e3 Exp + Exp e4

context-free syntax Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp{If} "+" Exp {left} Exp.Mul = Exp "*" Exp {left} Exp.Int = INT context-free priorities Exp.Mul > Exp.Add > Exp.If

[e1 * if(e2) e3] + e4

Deep Priority Conflict!

slide-36
SLIDE 36

36 context-free syntax Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp{If} "+" Exp {left} Exp.Mul = Exp "*" Exp {left} Exp.Int = INT Exp{If}.Add = Exp{If} "+" Exp{If} {left} Exp{If}.Mul = Exp "*" Exp{If} {left} Exp{If}.Int = INT context-free priorities Exp.Mul > Exp.Add > Exp.If

Exp Exp * Exp e1 if(e2) e3 Exp + Exp e4 [e1 * if(e2) e3] + e4 e1 * if(e2) e3 + e4

Deep Priority Conflict!

slide-37
SLIDE 37

37 context-free syntax Exp.If = "if" "(" Exp ")" Exp Exp.Add = Exp{If} "+" Exp {left} Exp.Mul = Exp{If} "*" Exp {left} Exp.Int = INT Exp{If}.Add = Exp{If} "+" Exp{If} {left} Exp{If}.Mul = Exp{If} "*" Exp{If} {left} Exp{If}.Int = INT context-free priorities Exp.Mul > Exp.Add > Exp.If

Exp Exp * Exp e1 if(e2) e3 Exp + Exp e4 [e1 * if(e2) e3] + e4 Exp if(e2) Exp Exp Exp * e1 + e3 e4 e1 * [if(e2) e3 + e4] e1 * if(e2) e3 + e4

Deep Priority Conflict!

slide-38
SLIDE 38

38

Spoofax Integration

slide-39
SLIDE 39

39

slide-40
SLIDE 40

40

slide-41
SLIDE 41

41

slide-42
SLIDE 42

Status

  • Safe and complete semantics for disambiguation with

priority and associativity in SDF3 complete for operator- style ambiguities [revision pending]

  • Implementation 1: disambiguation of deep conflicts by

transformation to contextual grammar [revision pending]

  • Implementation 2: data-dependent extension of SGLR;
  • nly requires context annotations in original grammar

productions [<Programming> 2018]

  • Available in SDF3+SGLR in current Spoofax LWB
  • Applied in grammars for OCaml, Java, …
  • Study: deep conflicts in the wild (they exist) [SLE 2017]

42