A .NET-based Test-Data Generator for Combinatorial Grammar- and - - PowerPoint PPT Presentation

▶
a net based test data generator for combinatorial grammar
SMART_READER_LITE
LIVE PREVIEW

A .NET-based Test-Data Generator for Combinatorial Grammar- and - - PowerPoint PPT Presentation

Jan 15, 2023 272 likes •451 views

A .NET-based Test-Data Generator for Combinatorial Grammar- and Schema-based Testing Vadim Zaytsev with: Ralf L ammel (VU), Wolfram Schulte (MSR) 14 April 2004 Grammar ware grammars grammar-dependent software In this project:

slide-1
SLIDE 1

A .NET-based Test-Data Generator for Combinatorial Grammar- and Schema-based Testing

Vadim Zaytsev with: Ralf L¨ ammel (VU), Wolfram Schulte (MSR) 14 April 2004

slide-2
SLIDE 2

Grammarware

  • grammars
  • grammar-dependent software
  • In this project:

– XML Validators – W3C XML Schemata as grammars

http://www.cs.vu.nl/grammarware/

1

slide-3
SLIDE 3

Scenarios for grammar-based testing

  • testing virtual processors

– virtual machines – just-in-time compilers

  • testing front-ends

– automated software modification & analysis

  • testing implementations

– optimisation of XPath

2

slide-4
SLIDE 4

Combinatorial exploration

Explosion

...

Term

Grammar

Term Term Term Term Term Term Term Term Term Term Term Term

. . .

Adversary of stochastic testing

3

slide-5
SLIDE 5

Explosion examples

Cardinalities per depth

4

slide-6
SLIDE 6

Controlled explosion

...

Term

Grammar

Recursion control Depth control

Term Term Term Term Term Term Term Term Term Term Term Term

. . .

+ other mechanisms

5

slide-7
SLIDE 7

Control mechanisms

  • depth control

– maximum “length” of terms

  • recursion control

– nested constructor application

  • equivalence control

– build equivalence classes

6

slide-8
SLIDE 8

Control mechanisms (contd.)

  • balance control

– limit the preceding levels

  • combination control

– limit Cartesian product – pair-wise testing

  • context control

– enforce context conditions

7

slide-9
SLIDE 9

Emphases in this project

  • The case study of XSD usage in testing XML Validators
  • Implementing and using control mechanisms for test data

generation

  • Developing a tool to support combinatorial testing

8

slide-10
SLIDE 10

Problem

System Under Test

XML Validator XSD NO YES

9

slide-11
SLIDE 11

Solution

Stage A:

YES GOOD/BAD YES Oracle Validator XSD XML TDGenerator NO

10

slide-12
SLIDE 12

Solution

Stage B:

NO Validator XML TDGenerator YES YES Oracle GOOD/BAD XSD XSD

11

slide-13
SLIDE 13

What kind of Oracle?

  • Differential testing

– run two or more against one another – if the outputs are different, something has to be wrong – in our case: different XML Validators ∗ using Microsoft .NET API ∗ Sun Multi-Schema XML Validator (JAXB) ∗ Ant Validation Task (JBind)

12

slide-14
SLIDE 14

The Tool we have

grammar terms Tree Nil = Nil Node-1(Nil,Nil) | Node(Tree,Tree) ... ; Terms as objects Constructors of all sorts +Serialisation

13

slide-15
SLIDE 15

Solution proposition

Mapping XSD Terms Constructors Parsing Serialisation Mapping XML The Tool grammar in BNF terms in text grammar test data XSD XML Sorts and

14

slide-16
SLIDE 16

Problems underway

  • XSD is not meant to be implemented

– (as a whole)

  • YACCification
  • how to deal with XML attributes
  • implementing control mechanisms
  • . . .

15

slide-17
SLIDE 17

Conclusion

The buzzwords are:

  • Test data generation
  • Combinatorial testing
  • Controlled explosion
  • Differential testing
  • The .NET Framework

16