An Empirical Study Meta and Hyper Heuristic Search for - - PowerPoint PPT Presentation

an empirical study
SMART_READER_LITE
LIVE PREVIEW

An Empirical Study Meta and Hyper Heuristic Search for - - PowerPoint PPT Presentation

An Empirical Study Meta and Hyper Heuristic Search for Multi-Objective Release Planning Yuanyuan Zhang Mark Harman CREST, UCL, UK CREST, UCL, UK Guenther Ruhe Gabriela Ochoa Sjaak Brinkkemper University of Calgary, Canada University of


slide-1
SLIDE 1

An Empirical Study

Yuanyuan Zhang

CREST, UCL, UK

Mark Harman

CREST, UCL, UK

Gabriela Ochoa

University of Stirling, UK

Meta and Hyper Heuristic Search for Multi-Objective Release Planning

Guenther Ruhe

University of Calgary, Canada

Sjaak Brinkkemper

Utrecht University, The Netherlands

slide-2
SLIDE 2

Agenda

Contributions Background Data sets Fitness functions Algorithms RQs Results & analysis

slide-3
SLIDE 3

A Thorough Empirical Study Fitness Functions Data Sets Algorithms

slide-4
SLIDE 4

A Thorough Empirical Study Fitness Functions Data Sets Algorithms

slide-5
SLIDE 5

A Thorough Empirical Study Fitness Functions Algorithms

10 Real World Data Sets

Data Sets

slide-6
SLIDE 6

A Thorough Empirical Study Algorithms Data Sets

10 Real World Data Sets Scenario Based Objectives

Fitness Functions

slide-7
SLIDE 7

A Thorough Empirical Study Fitness Functions Data Sets

10 Real World Data Sets Scenario Based Objectives

A Wider Spectrum of Algorithmic Behaviours

Algorithms

slide-8
SLIDE 8

Release Planning

Repository

requirements and change requests

slide-9
SLIDE 9

Release Planning

Repository

requirements and change requests

slide-10
SLIDE 10

Release Planning

Repository

Release 1 Release 2 Release 3 Product or Service

slide-11
SLIDE 11

ORP deals with how to assign developers to the tasks to be performed. Operational Release Planning (ORP) Strategic Release Planning (SRP) SRP is concerned with how to select and assign requirements to multiple subsequent releases.

slide-12
SLIDE 12

Models

Stakeholders Stakeholders Number (M) Stakeholders Weight (W)

slide-13
SLIDE 13

Models

Requirements Cost (C) Value (V) Time to market (T) Risk (R) Frequency of use (F)

slide-14
SLIDE 14

Models

Requirements Dependence (D) And Or Precedence Value-related Cost-related

slide-15
SLIDE 15

Models

Releases Release Number (K) Release Importance (I)

Release 1 Release 2 Release 3

slide-16
SLIDE 16

1 3 1 2 2 3 1 3

A set of requirements

RQ1 RQ2 RQn

. . . . . . . . . . . . . . . . . . .

Release1 Release 2 Not included

Data Representation

slide-17
SLIDE 17

A Thorough Empirical Study Fitness Functions Data Sets

10 Real World Data Sets Scenario Based Objectives

A Wider Spectrum of Algorithmic Behaviours

Algorithms

slide-18
SLIDE 18

Data Sets

10 Real World

slide-19
SLIDE 19
slide-20
SLIDE 20

VALUE I

i,k i =1 N

Maximize f ( x ) =

k i,k i =1 N

Minimize f ( x ) = COST FREQUENCY, IMPORTANCE, … IMPACT, RISK, … Scenario-based Fitness Functions

slide-21
SLIDE 21

A Wider Spectrum of Algorithmic Behaviours

Local Hill Climbing Global NSGA-II In-Between Simulated Annealing

slide-22
SLIDE 22

A Wider Spectrum of Algorithmic Behaviours

In-Between Simulated Annealing Local Hill Climbing Global NSGA-II Meta Hyper HHC HSA HNSGA-II

slide-23
SLIDE 23

Meta-heuristics Hyper-heuristics Random Hill Climbing Simulated Annealing NSGA-II HHC HSA HNSGA-II

A Wider Spectrum of Algorithmic Behaviours

slide-24
SLIDE 24

10 Hyper-Heuristic Operators

1 Random 2 Swap 3 Delete_Add 4 Delete_Add_Best 5 Delete_Worst_Add 6 Delete_Worst_Add_Best 7 Delay_Ahead 8 Delay_Ahead_Best 9 Delay_Worst_Ahead 10 Delay_Worst_Ahead_Best

Ruin & Recreate

Delete_Add_Best

slide-25
SLIDE 25

Operator: Delete_Add_Best

1 2 3 1 2 3 1 3

delete a requirement from the release with uniform probability

2 1 2 3 1 3 1 3

slide-26
SLIDE 26

Operator: Delete_Add_Best

add the best requirement (based on one

  • f fitness values) to one release

1 2 3 1 3 1 3

find the best requirement

1 2 2 3 1 3 1 3

slide-27
SLIDE 27

Adaptive Operator Selection

Credit assignment Extreme value credit assignment Fitness improvement: hypervolume difference Reference value: the fitness of the parents Operator selection Probability matching

slide-28
SLIDE 28

Performance Metrics

Quality Convergence Hypervolume Contribution Unique Contribution Diversity Speed

All the metrics were normalised between 0.0 and 1.0 and converted to ‘Maximising metrics’.

is only interesting if the algorithm’s quality is strong

slide-29
SLIDE 29

Research Questions

RQ 1 - Quality: Which algorithm performs best? RQ 2 - Diversity: What is the diversity of the solutions produced by each algorithm? RQ 3 - Speed: How fast can the algorithm produce the solutions? RQ 4 - Scalability: What is the scalability of each algorithm with regard to solution quality, diversity and speed?

slide-30
SLIDE 30

Results & Analysis

RQ 1 - Quality

slide-31
SLIDE 31

RQ 1 - Quality

For the meta-heuristic algorithms, NSGA-II performs best overall for quality on smaller datasets SA performs noticeably better on the three larger datasets The three hyper-heuristic algorithms outperform their meta-heuristic counterparts; HNSGA-II is beaten by its meta-heuristic counterpart only on the Ericsson dataset.

slide-32
SLIDE 32

Results & Analysis

RQ 2 - Diversity

slide-33
SLIDE 33

RQ 2 - Diversity

Random search perform very well, but the solutions are largely suboptimal Of the Hyper-heuristic algorithms, HNSGA-II exhibits the best diversity NSGA-II significantly outperforms HNSGA-II for Ericsson dataset HNSGA-II significantly outperforms NSGA-II on Mozilla and Gnome

slide-34
SLIDE 34

Results & Analysis

RQ 3 - Speed

slide-35
SLIDE 35

RQ 3 - Speed

The speed of random search is worse than all other algorithms for the larger datasets HNSGA-II is fastest overall

slide-36
SLIDE 36

Results & Analysis

RQ 4 - Scalability

slide-37
SLIDE 37

RQ 4 - Scalability

The quality of solutions NSGA-II produced decrease as the problem size increase NSGA-II’s contribution to the reference front decrease, as the number of requirements increase A negative correlation between the number of requirements and convergence of NSGA-II For the other algorithms, there is no negative correlation between problem size and solution quality The algorithms increase their diversity as the scale of the problem increase

slide-38
SLIDE 38