STARTS: STARTS: STARTS: STARTS: STAtic STAtic Regression Test - - PowerPoint PPT Presentation

▶

Jul 29, 2023 358 likes •484 views

STARTS: STARTS: STARTS: STARTS: STAtic STAtic Regression Test Selection Regression Test Selection STAtic STAtic Regression Test Selection Regression Test Selection Owolabi Legunsen , August Shi, Darko Marinov ASE 2017 Urbana-Champaign,

SLIDE 1

STARTS: STARTS: STARTS: STARTS: STAtic STAtic STAtic STAtic Regression Test Selection Regression Test Selection Regression Test Selection Regression Test Selection

Owolabi Legunsen, August Shi, Darko Marinov

ASE 2017 Urbana-Champaign, Illinois November 1, 2017

CCF-1409423 CCF-1421503

SLIDE 2

Regression Testing

T1 T2 T3 T4 A B C D E F T1 T2 T3 T4 C

Problem: RetestAll can be slow especially when there are many tests!

Rerun tests after every change to check that

existing functionality is not broken

RetestAll: run all tests after each change

SLIDE 3

Regression Test Selection (RTS)

RTS speeds up regression testing by rerunning
nly tests that are affected by the code changes

T1 T2 T3 T4 A B C D E F C T1 T2

Run All Tests (RetestAll) Find Dependencies Analyze Run Affected Tests

Time Savings End-to-End Time for RTS

End-to-end time for RTS steps must be less than

time to rerun all tests

SLIDE 4

STARTS

STARTS is an RTS tool that finds dependencies statically
Dynamic may be too slow, or infeasible in some settings
We implemented STARTS as a Maven plugin
STARTS source code is publicly-available on GitHub:
https://github.com/TestingResearchIllinois/starts

SLIDE 5

STARTS Architecture

Old Checksums Old Dependencies New classfiles List of all test classes Checksum Comparator Checksum Finder New Checksums Impacted Test Finder Test Runner Impacted Tests Dependency Updater Type-Dependency Graph Type-level Dependencies Changed Types yasgl mvn starts:select mvn starts:starts mvn starts:diff New Checksums New Dependencies jdeps

SLIDE 6

STARTS: Finding Changes

mvn starts:diff Find changes since last time STARTS was run

A B C D E F T1 T2 T3 T4

Old Checksums Old Dependencies Changed Types

Checksum Comparator

New Checksums

Checksum Finder

New Classfiles

SLIDE 7

STARTS: Selecting Impacted Tests

T1 T2 T3 T4 A B C D E F C T1 T2

mvn starts:select Select impacted tests without running them

Old Checksums Old Dependencies List of all Test Classes Impacted Test Finder Changed Types Impacted Tests

SLIDE 8

STARTS: Running Impacted Tests

T1 T2 T3 T4 A B C D E F C T1 T2

mvn starts:starts Select and run impacted tests

Old Checksums Old Dependencies List of all Test Classes Impacted Test Finder Changed Types Impacted Tests Test Runner

SLIDE 9

STARTS: Updating for the Next Run

New Classfiles jdeps yasgl Type-Dependency Graph Type-Level Dependencies New Checksums Dependency Updater New Checksums New Dependencies

yasgl is our custom graph library for computing the transitive closure jdeps is part of the standard JDK and quickly finds direct class dependencies

SLIDE 10

Results on 840 versions of 32 GitHub projects

Tests (#) Selected Tests (#) Selected Tests (%) RetestAll Time (s) STARTS Time (%) Avg (SHORT) 58.0 16.4 32.4 17.6 87.8 Avg (LONG) 155.9 54.1 40.5 236.8 68.2 Avg (OVERALL) 91.7 29.4 35.2 93.0 81.0

STARTS Breakdown (%)

analysis execution graph constr. Compilation Avg (SHORT) 1.0 25.9 8.7 64.4 Avg (LONG) 0.8 70.8 2.3 26.2 Avg (OVERALL) 0.9 41.3 6.5 51.3

STARTS is more effective for longer-running projects STARTS analysis and graph construction time is relatively efficient

SLIDE 11

Conclusions

STARTS is a publicly-available, purely static, class-level

regression test selection tool

We are investigating ways to make STARTS safer
Maybe also more precise?
STARTS is available on GitHub
https://github.com/TestingResearchIllinois/starts

legunse2@illinois.edu