Testing Xtext Languages Lorenzo Bettini Dipartimento di - - PowerPoint PPT Presentation

▶
testing xtext languages
SMART_READER_LITE
LIVE PREVIEW

Testing Xtext Languages Lorenzo Bettini Dipartimento di - - PowerPoint PPT Presentation

Jul 09, 2023 418 likes •972 views

Testing Xtext Languages Lorenzo Bettini Dipartimento di Informatica, Universit di Torino, Italy itemis GmbH, Zurich, Switzerland EclipseCon France 2015 Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 1/43 Wrong Workfmow

slide-1
SLIDE 1

Testing Xtext Languages

Lorenzo Bettini

Dipartimento di Informatica, Università di Torino, Italy itemis GmbH, Zurich, Switzerland

EclipseCon France 2015

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 1/43

slide-2
SLIDE 2

Wrong Workfmow

Modify the language Start an Eclipse runtime instance Manually check that “everything” works in the editor Write tests instead! The example for this talk http://github.com/LorenzoBettini/ecefrance2015-xtext-example

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 2/43

slide-3
SLIDE 3

Wrong Workfmow

Modify the language Start an Eclipse runtime instance Manually check that “everything” works in the editor Write tests instead! The example for this talk http://github.com/LorenzoBettini/ecefrance2015-xtext-example

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 2/43

slide-4
SLIDE 4

Example DSL: simple expressions

Write simple variables and expressions and generate Java code

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 3/43

slide-5
SLIDE 5

Initial Grammar

grammar org.eclipsecon.expdsl.Expressions with org.eclipse.xtext.common.Terminals generate expressions "http://www.eclipsecon.org/expdsl/Expressions" ExpressionsModel: elements += AbstractElement*; AbstractElement: Variable | Expression ; Variable: declaredType=Type name=ID '=' expression=Expression; Type: {IntType} 'int' | {BoolType} 'bool'; Expression: {IntConstant} value=INT | {StringConstant} value=STRING | {BoolConstant} value=('true'|'false') | {VariableRef} variable=[Variable];

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 4/43

slide-6
SLIDE 6

Initial Problems: “left recursion”

Solution Apply Left Factoring technique But let’s start testing right away! To check that we get associativity/priority right

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 5/43

slide-7
SLIDE 7

Projects Structure

Xtext already generates a testing project With injector providers (headless and UI tests)

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 6/43

slide-8
SLIDE 8

Junit tests

Most parts can be tested as Junit tests (no UI required) This is much faster!

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 7/43

slide-9
SLIDE 9

Typical TestCase (Xtend)

Run with XtextRunner InjectWith the generated injector provider Inject Xtext testing utility classes

package org.eclipsecon.expdsl.tests import static extension org.junit.Assert.* @RunWith(typeof(XtextRunner)) @InjectWith(typeof(ExpressionsInjectorProvider)) class ExpressionsParserTest { @Inject extension ParseHelper<ExpressionsModel> @Inject extension ValidationTestHelper @Test def void testSimpleExpression() { "10".parse.assertNoErrors } }

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 8/43

slide-10
SLIDE 10

Testing the Parser

Tests that the program is parsed without syntax errors Test that the AST is created as expected:

expected associativity expected operator precedence

ParseHelper: parses a string and returns the AST Additional Benefjts After parsing, the “real” language developer’s job starts: Validate the AST, generate code, etc. So it’s better that the AST is created the way you expect it

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 9/43

slide-11
SLIDE 11

Testing the Parser

Tests that the program is parsed without syntax errors Test that the AST is created as expected:

expected associativity expected operator precedence

ParseHelper: parses a string and returns the AST Additional Benefjts After parsing, the “real” language developer’s job starts: Validate the AST, generate code, etc. So it’s better that the AST is created the way you expect it

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 9/43

slide-12
SLIDE 12

Testing the Parser

Given the AST we create a string representation that highlights precedence/associativity Check that it is as expected

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 10/43

slide-13
SLIDE 13

Testing the Parser

Given the AST we create a string representation that highlights precedence/associativity Check that it is as expected

def String stringRepr(Expression e) { switch (e) { Plus: "(" + e.left.stringRepr+ " + " + e.right.stringRepr + ")" // other cases (omissis) Not: "!(" + e.expression.stringRepr ")" IntConstant: "" + e.value } }

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 10/43

slide-14
SLIDE 14

Testing the Parser

@Test def void testPlusMulPrecedence() { "10 + 5 * 2 - 5 / 1".assertRepr("((10 + (5 * 2)) - (5 / 1))") } @Test def void testComparison() { "10 <= 5 < 2 > 5".assertReprNoValidation("(((10 <= 5) < 2) > 5)") } @Test def void testEqualityAndComparison() { "true == 5 <= 2".assertRepr("(true == (5 <= 2))") } @Test def void testAndOr() { "true || false && 1 < 0".assertRepr("(true || (false && (1 < 0)))") } def assertRepr(CharSequence input, CharSequence expected) { input.parse => [ expected.assertEquals( (elements.last as Expression).stringRepr ) ] }

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 11/43

slide-15
SLIDE 15

Validation

We want to avoid forward variable references

int i = j // This should be an error! int j = 0

Let’s write a utility class, ExpressionsModelUtils, that

Given any element returns the list of variable declared before (we have to walk in the AST model)

Let’s test such class separately i.e., before writing the validator itself

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 12/43

slide-16
SLIDE 16

Validation

We want to avoid forward variable references

int i = j // This should be an error! int j = 0

Let’s write a utility class, ExpressionsModelUtils, that

Given any element returns the list of variable declared before (we have to walk in the AST model)

Let’s test such class separately i.e., before writing the validator itself

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 12/43

slide-17
SLIDE 17

Testing Forward References

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 13/43

slide-18
SLIDE 18

Let’s “code coverage” right away (Eclemma/Jacoco)

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 14/43

slide-19
SLIDE 19

Let’s “code coverage” right away (Eclemma/Jacoco)

Even if it’s generated Java code (by Xtend)

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 15/43

slide-20
SLIDE 20

Now we can write the Validator

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 16/43

slide-21
SLIDE 21

Testing the Validator

And now we test that such error is correctly generated

On the expected element of the program

We use the utility class ValidationTestHelper We can do that headlessly:

There’s no need to run the workbench

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 17/43

slide-22
SLIDE 22

Testing the Validator

We use the utility class ValidationTestHelper

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 18/43

slide-23
SLIDE 23

Type System & Type Checking

Check that expressions are correct with respect to types

subexpressions of a multiplication are integer in a variable declaration, initialization expression can be assigned to the variable declared type …

Implement the type system and test it Implement type checking in the validator, using the type system, and test it (not shown here: please have a look at the code)

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 19/43

slide-24
SLIDE 24

Code Generation

For each expdsl fjle we generate:

A Java class with the same name of the expdsl fjle in the package expressions With an eval() method

For each variable declaration we generate a Java variable declaration For each standalone expression we generate a System.out.println

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 20/43

slide-25
SLIDE 25

Testing the Compiler

Add the dependency org.eclipse.xtext.xbase.junit Add the dependency org.eclipse.jdt.core Use CompilationTestHelper Small pitfall: use a custom injector provider for missing Google Guice bindings expected by Xbase classes (see the code)

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 21/43

slide-26
SLIDE 26

Testing the generated Java code is as expected

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 22/43

slide-27
SLIDE 27

Testing the generated Java code is correct Java code

The test will fail if the generated Java code contains Java errors.

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 23/43

slide-28
SLIDE 28

Testing the generated Java executes correctly

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 24/43

slide-29
SLIDE 29

Testing UI

The content assist The integration with the workbench The outline

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 25/43

slide-30
SLIDE 30

Testing the Content Assist

Add the dependency org.eclipse.xtext.common.types.ui Use as base class

  • .e.x.xbase.junit.ui.AbstractContentAssistTest

Inject your test case with ExpressionsUiInjectorProvider This must be run as a “Junit Plug-in Test” Set memory arguments in the launch confjguration:

  • Xms256m -Xmx1024m -XX:MaxPermSize=256m

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 26/43

slide-31
SLIDE 31

Testing the Content Assist

Add the dependency org.eclipse.xtext.common.types.ui Use as base class

  • .e.x.xbase.junit.ui.AbstractContentAssistTest

Inject your test case with ExpressionsUiInjectorProvider This must be run as a “Junit Plug-in Test” Set memory arguments in the launch confjguration:

  • Xms256m -Xmx1024m -XX:MaxPermSize=256m

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 26/43

slide-32
SLIDE 32

Testing the Content Assist

Use as base class

  • .e.x.xbase.junit.ui.AbstractContentAssistTest

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 27/43

slide-33
SLIDE 33

Testing the Content Assist

Use as base class

  • .e.x.xbase.junit.ui.AbstractContentAssistTest

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 27/43

slide-34
SLIDE 34

Tweaking the Content Assist

Don’t propose forward references (e.g., i and j) Don’t propose references of the wrong type (e.g., s)

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 28/43

slide-35
SLIDE 35

Testing the modifjed Content Assist

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 29/43

slide-36
SLIDE 36

Testing the Workbench Integration

How is my DSL integrated with the workbench? Does a correct fjle lead to the generation of a Java fjle into src-gen folder? Does everything compile fjne? We use

  • .e.x.junit4.ui.AbstractWorkbenchTest as the base

class utility methods from JavaProjectSetupUtil and IResourcesSetupUtil methods for accessing programmatically the workbench (from

  • .e.core.resources)

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 30/43

slide-37
SLIDE 37

Testing the Workbench Integration

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 31/43

slide-38
SLIDE 38

Outline

Let’s customize it like that

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 32/43

slide-39
SLIDE 39

Outline

Let’s customize it like that

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 32/43

slide-40
SLIDE 40

Testing the Outline

See the code for AbstractOutlineWorkbenchTest

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 33/43

slide-41
SLIDE 41

Required Time for Junit tests

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 34/43

slide-42
SLIDE 42

Required Time for Junit Plug-in tests

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 35/43

slide-43
SLIDE 43

Headless Build - No UI

Or use maven-surefire-plugin if you experience problems with the classloader.

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 36/43

slide-44
SLIDE 44

Headless Build - No UI

Or use maven-surefire-plugin if you experience problems with the classloader.

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 36/43

slide-45
SLIDE 45

Headless Build - UI

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 37/43

slide-46
SLIDE 46

Testing other aspects

If some aspects are not covered by Xtext testing framework you can have a look at Xtext sources and get some inspiration…

This is what I did for the outline tests

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 38/43

slide-47
SLIDE 47

Other Testing Frameworks

Xpect: http://www.xpect-tests.org SWTBot, RCP Testing Tool, Jubula for functional tests

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 39/43

slide-48
SLIDE 48

Use CI, e.g., Jenkins

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 40/43

slide-49
SLIDE 49

Use CI, e.g., Jenkins

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 41/43

slide-50
SLIDE 50

…or Travis

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 42/43

slide-51
SLIDE 51

Small Advertising O:-)

http://www.packtpub.com/implementing-domain-specifjc-languages-with- xtext-and-xtend/book

An entire chapter on testing Please evaluate this session

Thanks! and Questions?

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 43/43

slide-52
SLIDE 52

Small Advertising O:-)

http://www.packtpub.com/implementing-domain-specifjc-languages-with- xtext-and-xtend/book

An entire chapter on testing Please evaluate this session

Thanks! and Questions?

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 43/43

slide-53
SLIDE 53

Small Advertising O:-)

http://www.packtpub.com/implementing-domain-specifjc-languages-with- xtext-and-xtend/book

An entire chapter on testing Please evaluate this session

Thanks! and Questions?

Lorenzo Bettini Testing Xtext Languages EclipseCon France 2015 43/43