Beyond Code Coverage an Approach for Test Suite Assessment and - - PowerPoint PPT Presentation

▶
beyond code coverage an approach for test suite
SMART_READER_LITE
LIVE PREVIEW

Beyond Code Coverage an Approach for Test Suite Assessment and - - PowerPoint PPT Presentation

Apr 26, 2023 271 likes •465 views

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS UNIVERSITY OF SZEGED D epartment of Software Engineering Beyond Code Coverage an Approach for Test Suite Assessment and Improvement Dvid Tengeri* , rpd Beszdes*, Tams Gergely*, Lszl Vidcs

slide-1
SLIDE 1

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS UNIVERSITY OF SZEGED

Department of Software Engineering Beyond Code Coverage – an Approach for Test Suite Assessment and Improvement

Dávid Tengeri*, Árpád Beszédes*, Tamás Gergely*, László Vidács, Dávid Havas* and Tibor Gyimóthy*

*Department of Software Engineering, University of Szeged, Hungary

MTA-SZTE Research Group on Artificial Intelligence, University of Szeged,

Hungary

IEEE TAICPART 2015, Graz, Austria

slide-2
SLIDE 2

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 2

Industrial Motivation

WebKit

Source code Test suite

slide-3
SLIDE 3

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 3

Industrial Motivation

WebKit

Source code Test suite

WebKit

  • Open source web browser engine
  • ~ 2.2 million lines of code (mostly C++)
  • More than 27 000 test cases
slide-4
SLIDE 4

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 4

Industrial Motivation

WebKit

Source code Test suite Test suite

  • Challenge with the test suite:
  • Understand and maintain the test suite
  • What is its “quality”?
slide-5
SLIDE 5

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 5

Motivation and Goals

“Quality” of a (regression) test suite:

■ What is the likelihood of defect detection? ■ How efficient, modular, etc. it is?

How can the quality be improved? Quality assessment of source code – problem solved (?) Quality assessment of test suites?

Test suite Assessment and Improvement Method (TAIME)

slide-6
SLIDE 6

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

Determine functional units

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 6

features statements, functions subset of test suite

slide-7
SLIDE 7

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

Assessment of WebKit

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 7

Function level coverage of groups in WebKit

slide-8
SLIDE 8

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

The TAIME approach

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 8

slide-9
SLIDE 9

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

The TAIME approach

Measure How good the tests are overall in …

■ COV: … executing all parts of the software ■ PART: … being able to localize defects ■ TpP: … being relatively few of them yet effective

How good the test groups are in …

■ SPEC: … specializing to their code group compared to other test groups ■ UNIQ: … uniquely covering their code group compared to other code

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 9

slide-10
SLIDE 10

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 10

Use cases

One-shot refactoring

■ Delete, reorganize, rewrite, create test cases ■ To improve test suite quality

Change-oriented test suite evolution

■ Create new test cases or possibly remove test cases ■ To preserve or improve test suite quality

White-box test design

■ Monitor and maintain test suite quality during design ■ Use specific measurements as white-box criteria

Assessment

■ Detect any issues that require further investigation ■ Helps find the initial goal in the improvement phases

slide-11
SLIDE 11

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 11

Improvement of SoDA

Software Development Analysis Framework

■ http://soda.sed.hu ■ Platform independent ■ Plugin based ■ TAIME support (with GUI)

SoDA Repository

■ Benchmark programs (SIR, WebKit, GCC) ■ Set of measurement results

slide-12
SLIDE 12

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

Improvement of SoDA

White-box test design

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 12

slide-13
SLIDE 13

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

Improvement of SoDA

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 13

slide-14
SLIDE 14

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

Improvement of the cluster unit

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 14

Improving the coverage of cluster unit

slide-15
SLIDE 15

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

Improvement of the cluster unit

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 15

Improving the coverage of

  • ther units
slide-16
SLIDE 16

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

Improvement of the cluster unit

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 16

Removing the usage of cluster unit in

  • ther units
slide-17
SLIDE 17

UNIVERSITY OF SZEGED

Department of Software Engineering

UNIVERSITAS SCIENTIARUM SZEGEDIENSIS

17th April, 2015 IEEE TAICPART 2015, Graz, Austria 17

http://soda.sed.hu

One-shot refactoring Change-

  • riented test

suite evolution White-box test design Assessment

TAIME