M ODEL T RANSFORMATION T ESTING , T HE S TATE OF THE A RT Gehan - - PowerPoint PPT Presentation

MODEL TRANSFORMATION TESTING, THE STATE OF THE ART

Gehan Selim, James Cordy, Juergen Dingel,

Presented by: Lobna AbuSerrieh

I

NT RODUCT ION

Mo d e l Drive n De ve lo pme nt

2 Mode l Code

T ransformations

T

RANSF ORMAT ION C ORRE CT NE SS

F

• rma l Me tho d s : He a vywe ig ht

T e sting :

• e xe c ute s a tra nsfo rma tio n o n input mo de ls the n

va lida te s the a c tua l o utput ma tc he s the e xpe c te d

• utput.
• Auto ma ta b le te st a c tivitie s
• L

ig htwe ig ht, L

• w c o mputa tio na l c o mple xity

3

PHASE

S OF MODE L

T

RANSF ORMAT ION T E ST ING

• 1. T

e st Ca se Ge ne ra tion

• 2. T

e st Suite Asse ssme nt

• 3. Building the Ora c le
• 4. E

xe c ute a nd e va lua te

4

PHASE 1: T

E ST

CASE GE

NE RAT ION

 De fine te st a de q ua c y c rite ria , the n Build te st c a se s

tha t a c hie ve s its c o ve ra g e . And it c a n b e do ne b y using :

−

Bla c k- Box te sting : b a se d o n tra nsfo rma tio n

spe c ific a tio n.

−

Gr a y- box te sting : b a se d o n the a c c e ssib le pa rts o f

tra nsfo rma tio n imple me nta tio n.

−

White - Box te sting : b a se d o n tra nsfo rma tio n

imple me nta tio n

5

BL

ACK- BOX T E ST

CASE GE

NE RAT ION

ME

T AMODE L COVE RAGE

 Ad e q ua c y c rite ria fo r Cla ss d ia g ra ms

− Asso c ia tio n e nd multiplic ity c rite rio n − Ge ne ra liza tio n c rite rio n − Cla ss a ttrib ute c rite rio n 6

BL

ACK- BOX T E ST

CASE GE

NE RAT ION

ME

T AMODE L COVE RAGE

 Ad e q ua c y c rite ria fo r Inte ra c tio n d ia g ra ms

− E

a c h me ssa g e o n a link

− All me ssa g e pa th − Co lle c tio n c o ve ra g e − Co nditio n c o ve ra g e − F

ull pre dic a te c o ve ra g e

− T

ra nsitio n c o ve ra g e

7

BL

ACK- BOX T E ST

CASE GE

NE RAT ION

ME

T AMODE L COVE RAGE

 Ad e q ua c y c rite ria fo r sta te c ha rts

− F

ull pre dic a te c o ve ra g e

− All c o nte nt- de pe nde nc y re la tio nships − T

ra nsitio n c o ve ra g e

− tra nsitio n- pa ir c o ve ra g e − Co mple te se q ue nc e c o ve ra g e − All c o nfig ura tio ns tra nsitio n c o ve ra g e 8

BL

ACK- BOX T E ST

CASE GE

NE RAT ION

C ONT

RACT

COVE

RAGE

Ac hie ving input c o ntra c ts o f Mo de l tra nsfo rma tio n

 Co nstruc ting me ta mo de l o f o nly tho se e le me nts a re

a c tua lly use d in pre / po st c o nditio ns o f tra nsfo rma tio n

 Co mb ine c o ntra c t-b a se d a nd me ta mo de l b a se d.

And fo o tprints(numb e r o f time s te st mo de l c o ve rs e a c h c rite rio n).

9

WHIT

E- BOX T E ST

CASE GE

NE RAT ION

 Mo st o f the Studie s a re do ne witho ut c a se studie s

a nd no de ta ile d re sults.

 T

ra nsfo rming rule s to a so urc e me ta mo de l te mpla te .

 Asse ssing AT

L rule s b y pro filing :

• 1. Co mpila tio n re sulte d XML

file to e xtra c t the rule s.

• 2. T

ra nsfo rma tio n to b e e xe c ute d. And using the re sulte d lo g file to a sse ss the c o ve ra g e (rule , instruc tio n, de c isio n).

 Gra mma r te sting , E

a c h rule to b e trig g e re d in e ve ry po ssib le c o nte xt.

10

PHASE 2: T

E ST

SUIT

E ASSE SSME NT

 Ac hie ve d Co ve ra g e to a sse ss the te st suite q ua lity.  Muta tio n a na lysis, e va lua te the se nsitivity o f the te st

c a se to fa ults in tra nsfo rma tio n.

 Inje c ting fa ults b y a pplying muta tio n o pe ra to rs a nd

g e ne ra te muta nts.

− Diffe re nt re sults: K

ille d muta nt.

− No fa ults: the muta nt is a live

11

PHASE 3: BUIL

DING T HE ORACL E F UNCT ION

Co mpa re s T he a c tua l o utput with e xpe c te d o ne .

 if the e xpe c te d o utput is a va ila b le , the n Co mpa re .  If it is no t a va ila b le , va lida te s the re sulte d o utput with

the pre de fine d o utput pro pe rtie s o r c o ntra c ts

12

PHASE 3: BUIL

DING T HE ORACL E F UNCT ION

C OMPARISON

if the e xpe c te d o utput is a va ila b le , the n Co mpa re :

13

Test Case Constructor

(Input Model, Expected Output, Transformation Strategy)

Test Engine

Execute, Compare

Test Analyzer

Visualize using colors and shapes

A fra me wo rk use s Mo de l c o mpa riso n

PHASE 3: BUIL

DING T HE ORACL E F UNCT ION

C ONT

RACT S

If the e xpe c te d o utput is no t a va ila b le , va lida te s the re sult with the pre de fine d o utput pro pe rtie s o r c o ntra c ts.

 T

ra c ts, se t o f OCL c o nstra ints a nd a tra c t te st suite .

 Impro ving T

ra nsfo rma tio n c o ntra c ts:

• 1. Vig ila nc e : dyna mic a lly de te c t e rro rs
• 2. Dia g no sa b ility: e ffo rt to lo c a te a fa ult

14

PHASE 3: BUIL

DING T HE ORACL E F UNCT ION

C ONT

RACT S

 Vig ila nc e c a n b e impro ve d b y Ana lyzing a te st suit a nd

re pe a te dly using muta tio n a na lysis, until a c hie ving a n a c c e pta b le muta tio n sc o re .

 Othe r pro po se d a n impro ve d vig ila nc e a nd dia g no sa b iliy

b y using ma the ma tic a l mo de ling .

15

Q UE

ST IONS

 Gra y-Bo x T

e sting , is it fe a sib le to de pe nd o n pa rtia l imple me nta tio n while c o nside ring o the r pa rts a s b la c k b o x te sting ?

 Cla ss dia g ra ms, sta te c ha rts, a nd se q ue nc e dia g ra ms a re

the c o mmo n use d while te sting tra nsfo rma tio n, wha t a b o ut

• the r type s o f dia g ra ms?

 I

s Mo de l c o mpa riso n a s o ra c le func tio n c le a r e no ug h?

 Sinc e 2012 whe n this pa pe r wa s writte n, a nd ma ny re la te d

studie s we re witho ut c a se studie s o r re lia b le re sults, a ny ne w upda te s we re a dde d to te sting MDT ?

16