Integrated Language Definition Testing Lennart Kats Delft - - PowerPoint PPT Presentation

▶
integrated language definition testing
SMART_READER_LITE
LIVE PREVIEW

Integrated Language Definition Testing Lennart Kats Delft - - PowerPoint PPT Presentation

May 02, 2023 433 likes •875 views

Integrated Language Definition Testing Lennart Kats Delft University of Technology Rob Vermaas LogicBlox Eelco Visser Delft University of Technology Language Workbenches Testing DOES THE TYPE CHECKER CATCH THIS? Testing DOES THIS

slide-1
SLIDE 1

Integrated Language Definition Testing

Lennart Kats Rob Vermaas Eelco Visser Delft University of Technology LogicBlox Delft University of Technology

slide-2
SLIDE 2

Language Workbenches

slide-3
SLIDE 3

Testing

“DOES THE TYPE CHECKER CATCH THIS?”

slide-4
SLIDE 4

Testing

“DOES THIS HYPERLINK POINT TO THE RIGHT PLACE?”

slide-5
SLIDE 5

Syntax Semantics IDE parsing abstract syntax syntax error marking type checking compilation interpretation execution errors/warnings reference resolving content completion refactoring views

slide-6
SLIDE 6

How to test language services?

slide-7
SLIDE 7

IMPLEMENT FEATURE, TEST WITH EXAMPLE DOESN’T WORK CLOSE ENOUGH. DISCARD TEST

slide-8
SLIDE 8

How can we systematically test language definitions?

slide-9
SLIDE 9

General-Purpose Testing Tools?

slide-10
SLIDE 10

A Test Input

module Example function foo() { bar(); } function bar() { }

slide-11
SLIDE 11

Another Test Input

module Example function foo() { foo(); } function bar() { }

slide-12
SLIDE 12

A Negative Test Case

module Example function foo() { baz(); } function bar() { }

slide-13
SLIDE 13

Automated Testing Infrastructure

Test input Test script

Parser Compiler IDE + simple – language-specific script – limited expressiveness – boilerplate code – ...

slide-14
SLIDE 14

Can we design a general solution for specifying language tests?

slide-15
SLIDE 15

Yes We Can

Generic test specification language + Parametrization

slide-16
SLIDE 16

Language-Parametric Testing Language (LPTL)

module my-tests language mobl test Cannot assign an integer to a string [[ module Example <test input> var s : String = 1; } ]] 1 error module Example function test() { var s : String = 1; }

slide-17
SLIDE 17

LPTL Opportunities

Expressiveness Tool support

slide-18
SLIDE 18

LPTL Opportunities (1)

Module system + GUI Test Runner

slide-19
SLIDE 19

LPTL Opportunities (2)

IDE support for test specifications

slide-20
SLIDE 20

LPTL Opportunities (3)

Immediate test evaluation

slide-21
SLIDE 21

LPTL Opportunities (4)

IDE support for test inputs

slide-22
SLIDE 22

LPTL Opportunities (5)

setup [[ module Example imports stuff function test() { [[...]] } ]] test Cannot assign ... [[ var s : String = 1; ]] 1 error

Reduced boilerplate

slide-23
SLIDE 23

LPTL Opportunities (6)

Wide set of test conditions

1 error 2 warnings /expected here/ parse fails complete ... to ... resolve ... to ... refactor ... to ... build ... run ...

slide-24
SLIDE 24

Testing Syntax (1)

test Proper declaration [[ var s : String = "a"; ]] parse test Java-like declaration [[ String s = "a"; ]] parse fails

slide-25
SLIDE 25

Testing Syntax (2)

test Proper declaration [[ var s : String = "a"; ]] parse to VarDecl("s", _) test Precedence [[ 3 + 1 * 2 ]] parse to [[ 3 + (1 * 2) ]]

slide-26
SLIDE 26

Testing Error Markers

test Variable declaration [[ var s : String = "a"; ]] 0 errors test Bad variable declaration [[ var s : String = 25; ]] 1 error /wrong type/

slide-27
SLIDE 27

Testing References

test [[ module Example function foo() { bar(); } function bar() { } ]] resolve ... to ... test [[ module Example function foo() { [[bar]](); } function [[bar]]() { } ]] resolve #1 to #2

slide-28
SLIDE 28

Testing Code Generation..?

test [[ function foo() { return 3; } ]] build generate-javascript to [[ [[bar]](); function bar() { } ]] to [[ var foo = function foo() { return 3; }; ]]

slide-29
SLIDE 29

setup [[ application execution function test() : Num { // init [[...]] } ]] test Arithmetic [[ return 1 + 1; ]] run run-test to 2

Testing Execution

slide-30
SLIDE 30

Implementation

Spoofax Testing Language (spoofax.org)

slide-31
SLIDE 31

Implementation Techniques

Language embedding Dynamic instantiation of language services

slide-32
SLIDE 32

Language Embedding: Syntax

module tests language mobl test Java-like declaration [[ String s = "a"; ]] parse fails

1 2 3

slide-33
SLIDE 33

Language Embedding: Semantics & IDE (1)

test A function call [[

function foo() { }

fo| ]]

Content complete Mobl

slide-34
SLIDE 34

Language Embedding: Semantics & IDE (2)

test A function call [[

function foo() { }

notfoo(); ]]

No condition; error not expected Mobl

slide-35
SLIDE 35

Dynamic Language Service Instantiation

Mobl

slide-36
SLIDE 36

Dynamic Language Service Instantiation

Language registry Language consists of services Services have functional interfaces

slide-37
SLIDE 37

Test input

Dynamic Language Service Instantiation

Completion service Language registry Mobl language Parser

“mobl”

slide-38
SLIDE 38

Reflection

+ simple + no scripting required + IDE helps avoid errors + little boilerplate code + expressiveness

slide-39
SLIDE 39

Conclusions

  • General abstraction for language testing
  • Explored opportunities in expressiveness

and tool support www.spoofax.org

slide-40
SLIDE 40

ADDITIONAL SLIDES

slide-41
SLIDE 41

Related: Automatic Test Generation

  • Generate tests from grammar
  • Requires oracle
  • Complementary to our approach
slide-42
SLIDE 42

Self-Application

language Spoofax-Testing test Testing testing [[[ language Mobl test Testing [[ module y ]] ]]]

slide-43
SLIDE 43

The Spoofax Language Workbench

  • Integrated environment for language

definition

  • Define syntax, semantics, IDE
  • Based on Eclipse

www.spoofax.org [OOPSLA 2010]