Tests for free Fully Automatic Generator-Driven Tests of XSL - - PowerPoint PPT Presentation

▶
tests for free
SMART_READER_LITE
LIVE PREVIEW

Tests for free Fully Automatic Generator-Driven Tests of XSL - - PowerPoint PPT Presentation

Jun 29, 2023 371 likes •917 views

Tests for free Fully Automatic Generator-Driven Tests of XSL Transformations Adam Duracz Walid Taha Halmstad University July 16, 2012 Motivation XML Popular to specify XML formats using XML Schema XSL Transformations (XSLT)

slide-1
SLIDE 1

Tests for free

Fully Automatic Generator-Driven Tests of XSL Transformations

Adam Duracz Walid Taha Halmstad University

July 16, 2012

slide-2
SLIDE 2

Motivation

  • XML
  • Popular to specify XML formats using XML Schema
  • XSL

Transformations (XSLT)

  • Language for expressing transforms of XML docs
  • Complete XML Schema-based type checking of full

XSLT is not possible *

  • Common to test XSLTs by hand-crafting input data
  • Expensive / error-prone / tedious

* Alon et al 2001

slide-3
SLIDE 3

Initial idea

Since type checking of XSLTs isn’t possible, can we at least use generator-driven testing?

slide-4
SLIDE 4

XSLT - Transformations from XML to XML

  • Common application areas
  • Systems integration , data migration, publishing
  • Often XML Schema for I/O formats are available
  • Tells us how valid input should look
  • Allows us to automatically validate output

Sin XML Document XML Document XSLT Sout

slide-5
SLIDE 5

Generator-driven test method

Sin Generator Validator T Sout d T(d)

slide-6
SLIDE 6

How XML Schema define XML documents

<people> <person id="P001"> <name>Anna</name> <age>66</age> </person> <person id="P002"> <name>Jan</name> <age>35</age> </person> </people> <xsd:element name="people" type="PeopleType"/> <xsd:complexType name="PeopleType"> <xsd:sequence> <xsd:element name="person" type="PersonType" minOccurs="2"/> </xsd:sequence> </xsd:complexType>

slide-7
SLIDE 7

How XML Schema define XML documents

<people> <person id="P001"> <name>Anna</name> <age>66</age> </person> <person id="P002"> <name>Jan</name> <age>35</age> </person> </people> <xsd:element name="people" type="PeopleType"/> <xsd:complexType name="PeopleType"> <xsd:sequence> <xsd:element name="person" type="PersonType" minOccurs="2"/> </xsd:sequence> </xsd:complexType>

slide-8
SLIDE 8

How XML Schema define XML documents

<people> <person id="P001"> <name>Anna</name> <age>66</age> </person> <person id="P002"> <name>Jan</name> <age>35</age> </person> </people> <xsd:element name="people" type="PeopleType"/> <xsd:complexType name="PeopleType"> <xsd:sequence> <xsd:element name="person" type="PersonType" minOccurs="2"/> </xsd:sequence> </xsd:complexType>

slide-9
SLIDE 9

How XML Schema define XML documents

<people> <person id="P001"> <name>Anna</name> <age>66</age> </person> <person id="P002"> <name>Jan</name> <age>35</age> </person> </people> <xsd:element name="people" type="PeopleType"/> <xsd:complexType name="PeopleType"> <xsd:sequence> <xsd:element name="person" type="PersonType" minOccurs="2"/> </xsd:sequence> </xsd:complexType>

slide-10
SLIDE 10

How XML Schema define XML documents

<people> <person id="P001"> <name>Anna</name> <age>66</age> </person> <person id="P002"> <name>Jan</name> <age>35</age> </person> </people> <xsd:element name="people" type="PeopleType"/> <xsd:complexType name="PeopleType"> <xsd:sequence> <xsd:element name="person" type="PersonType" minOccurs="2"/> </xsd:sequence> </xsd:complexType>

slide-11
SLIDE 11

Generator-driven test method

Sin Generator Validator T Sout d T(d)

slide-12
SLIDE 12

XML Schema Instance Generation

<xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ...

slide-13
SLIDE 13

Parsing

<xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ...

Internal Repr.

Element

XML Schema

slide-14
SLIDE 14

Parsing

<xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ...

Internal Repr.

Element ComplexType

XML Schema

slide-15
SLIDE 15

Parsing

<xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ...

Internal Repr.

Element ComplexType Sequence

XML Schema

slide-16
SLIDE 16

Parsing

XML Schema

<xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ...

Internal Repr.

Element ComplexType Sequence Element Element

Internal Repr.

Element ComplexType Sequence Element Element

slide-17
SLIDE 17

Generator construction

Generator

genElement genComplexType genSequence genElement genElement

Internal Repr.

Element ComplexType Sequence Element Element

slide-18
SLIDE 18

Generator

genElement genComplexType genSequence genElement genElement

Instance generation

Schema instance

?

XML Schema

<xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ...

slide-19
SLIDE 19

Instance generation

Generator Schema instance XML Schema

<xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ... genElement genComplexType genSequence genElement genElement

slide-20
SLIDE 20

Instance generation

Generator Schema instance XML Schema

<xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ... <family> </family> genElement genComplexType genSequence genElement genElement

slide-21
SLIDE 21

Instance generation

Generator Schema instance XML Schema

<xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ... <family> </family> genElement genComplexType genSequence genElement genElement

slide-22
SLIDE 22

Instance generation

Generator Schema instance XML Schema

<xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ... <family> </family> genElement genComplexType genSequence genElement genElement

slide-23
SLIDE 23

Instance generation

Generator Schema instance XML Schema

<xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ... genElement genComplexType genSequence genElement genElement <family> </family>

slide-24
SLIDE 24

Instance generation

Generator Schema instance XML Schema

<xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ... genElement genComplexType genSequence genElement genElement <family> </family> <person name="Anna"/> <person name="Jan"/>

slide-25
SLIDE 25

Instance generation

Generator Schema instance XML Schema

<family> <person name="Anna"/> <person name="Jan"/> </family> <xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ... genElement genComplexType genSequence genElement genElement

slide-26
SLIDE 26

Instance generation

Generator Schema instance XML Schema

<family> <person name="Anna"/> <person name="Jan"/> </family> <xsd:element name="family" <xsd:complexType> <xsd:sequence> <xsd:element name="person" type="..." minOccurs="2" <xsd:element name="pet" ... genElement genComplexType genSequence genElement genElement <pet name="Moxy" species="rabbit"/>

slide-27
SLIDE 27

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

XML Schema Pattern facets

Instance generation: Regular Expressions

slide-28
SLIDE 28

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Instance generation: Regular Expressions

Generator Schema instance excerpt XML Schema Pattern facet

slide-29
SLIDE 29

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch

slide-30
SLIDE 30

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genBranch

slide-31
SLIDE 31

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genBranch genPiece genPiece genPiece genPiece genPiece

slide-32
SLIDE 32

genPiece genPiece genPiece genPiece

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genPiece genBranch genAtom

slide-33
SLIDE 33

genPiece genPiece genPiece genPiece

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genPiece genBranch genAtom genMatch

slide-34
SLIDE 34

genAtom genPiece genPiece genPiece genPiece

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genPiece genBranch genAtom genMatch genAtom

...

slide-35
SLIDE 35

genAtom genPiece genPiece genPiece genPiece

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genPiece genBranch genAtom genMatch genAtom

...

19

slide-36
SLIDE 36

genPiece genPiece genPiece

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genPiece genBranch

19

genPiece

slide-37
SLIDE 37

genPiece genPiece genPiece

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genPiece genBranch genAtom

19

genPiece

slide-38
SLIDE 38

genPiece genPiece genPiece

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genPiece genBranch genAtom

19

genPiece genCharClassEsc

...

slide-39
SLIDE 39

genPiece genPiece genPiece

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genPiece genBranch genAtom

19

genPiece genCharClassEsc

8

...

slide-40
SLIDE 40

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genBranch

19

genPiece genPiece genPiece genPiece genPiece

8

slide-41
SLIDE 41

genPiece genPiece

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genPiece genBranch

19

genPiece genPiece

8

genAtom

slide-42
SLIDE 42

genPiece genPiece

Instance generation: Regular Expressions

<xsd:simpleType name="SsnType"> <xsd:restriction base="xsd:string"> <xsd:pattern value=" (19|20)?\d\d (0[1-9]|1[012]) (0[1-9]|[12]\d|3[01]) \d{4} "/> </xsd:restriction> </xsd:simpleType>

Generator Schema instance excerpt XML Schema Pattern facet

genMatch genPiece genBranch

19

genPiece genPiece

8

genAtom genCharClassEsc

81

...

slide-43
SLIDE 43

Secret sauce: generator combinators

sizedListOf :: Occurs -> Occurs -> Gen a -> Gen [a] elements :: [a] -> Gen a listOf :: Gen a -> Gen [a] listOf . elements :: [a] -> Gen [a]

slide-44
SLIDE 44

Generator combinator: sizedListOf

  • XML Schema excerpt
  • Generator combinator type signatrue

<xsd:element name="person" type="i:PersonType" minOccurs="2" maxOccurs="unbounded"/> sizedListOf :: Occurs -> Occurs -> Gen e -> Gen [e]

slide-45
SLIDE 45

A rich and realistic generated input document

<?xml version="1.0" encoding="UTF-8" ?> <i:people xmlns:i="input"> <person> <location>stockholm</location> <ssn>195510319244</ssn> </person> <person> <location>gothenburg</location> <ssn>199410182576</ssn> </person> </i:people>

slide-46
SLIDE 46

Proof-of-concept tool

slide-47
SLIDE 47

Implementation approach

Sin QuickCheck Generators Validator T Sout d T(d)

slide-48
SLIDE 48

QuickCheck

  • Framework for Property-Based

Testing

  • Properties are predicates in terms of functions under test
  • Tests are automatically derived from properties
  • Generators of function parameter types are sampled to

create input data

  • Example property:

prop_RevRev (xs::[Int]) = reverse (reverse xs) == xs

slide-49
SLIDE 49

One property

Sin Generator Validator T Sout d T(d)

Static XSLT validity *

d ∈ Doc ⇒ T(d) ∈ DocSout

Sin

* Tozawa 2001 / Møller, Kuula 2006

slide-50
SLIDE 50

Shrinking

<people> <person id="P001"> <name>Anna</name> <age>-1</age> </person> <person id="P002"> <name>Andrzej</name> <age>67</age> </person> <person id="P003"> <name>Jan</name> <age>35</age> </person> </people> <people> <person id="P001"> <name>Anna</name> <age>-1</age> </person> <person id="P002"> <name>Andrzej</name> <age>67</age> </person> </people>

⇒

slide-51
SLIDE 51

Guided shrinking

<people> <person id="P001"> <name>Anna</name> <age>-1</age> </person> <person id="P002"> <name>Andrzej</name> <age>67</age> </person> <person id="P003"> <name>Jan</name> <age>35</age> </person> </people>

Failing test case input Metadata

ElementWithComplexTypeDecl "people" ... ComplexTypeSequence "PersonType" .. ElementWithSimpleTypeDecl "name" ... ElementWithSimpleTypeDecl "age" ... ...

slide-52
SLIDE 52

Take-home message

In the presence of detailed formal specifications, look for opportunities for automatic tests based on generators!

slide-53
SLIDE 53

Questions?