An Empirical Study Meta and Hyper Heuristic Search for - - PowerPoint PPT Presentation
An Empirical Study Meta and Hyper Heuristic Search for Multi-Objective Release Planning Yuanyuan Zhang Mark Harman CREST, UCL, UK CREST, UCL, UK Guenther Ruhe Gabriela Ochoa Sjaak Brinkkemper University of Calgary, Canada University of
CREST, UCL, UK
CREST, UCL, UK
University of Stirling, UK
University of Calgary, Canada
Utrecht University, The Netherlands
Contributions Background Data sets Fitness functions Algorithms RQs Results & analysis
A Wider Spectrum of Algorithmic Behaviours
Repository
requirements and change requests
Repository
requirements and change requests
Repository
Release 1 Release 2 Release 3 Product or Service
ORP deals with how to assign developers to the tasks to be performed. Operational Release Planning (ORP) Strategic Release Planning (SRP) SRP is concerned with how to select and assign requirements to multiple subsequent releases.
Release 1 Release 2 Release 3
1 3 1 2 2 3 1 3
RQ1 RQ2 RQn
Release1 Release 2 Not included
A Wider Spectrum of Algorithmic Behaviours
i,k i =1 N
k i,k i =1 N
A Wider Spectrum of Algorithmic Behaviours
A Wider Spectrum of Algorithmic Behaviours
A Wider Spectrum of Algorithmic Behaviours
1 Random 2 Swap 3 Delete_Add 4 Delete_Add_Best 5 Delete_Worst_Add 6 Delete_Worst_Add_Best 7 Delay_Ahead 8 Delay_Ahead_Best 9 Delay_Worst_Ahead 10 Delay_Worst_Ahead_Best
Delete_Add_Best
1 2 3 1 2 3 1 3
2 1 2 3 1 3 1 3
1 2 3 1 3 1 3
1 2 2 3 1 3 1 3
Credit assignment Extreme value credit assignment Fitness improvement: hypervolume difference Reference value: the fitness of the parents Operator selection Probability matching
Quality Convergence Hypervolume Contribution Unique Contribution Diversity Speed
All the metrics were normalised between 0.0 and 1.0 and converted to ‘Maximising metrics’.
is only interesting if the algorithm’s quality is strong
RQ 1 - Quality: Which algorithm performs best? RQ 2 - Diversity: What is the diversity of the solutions produced by each algorithm? RQ 3 - Speed: How fast can the algorithm produce the solutions? RQ 4 - Scalability: What is the scalability of each algorithm with regard to solution quality, diversity and speed?
RQ 1 - Quality
For the meta-heuristic algorithms, NSGA-II performs best overall for quality on smaller datasets SA performs noticeably better on the three larger datasets The three hyper-heuristic algorithms outperform their meta-heuristic counterparts; HNSGA-II is beaten by its meta-heuristic counterpart only on the Ericsson dataset.
RQ 2 - Diversity
Random search perform very well, but the solutions are largely suboptimal Of the Hyper-heuristic algorithms, HNSGA-II exhibits the best diversity NSGA-II significantly outperforms HNSGA-II for Ericsson dataset HNSGA-II significantly outperforms NSGA-II on Mozilla and Gnome
RQ 3 - Speed
The speed of random search is worse than all other algorithms for the larger datasets HNSGA-II is fastest overall
RQ 4 - Scalability
The quality of solutions NSGA-II produced decrease as the problem size increase NSGA-II’s contribution to the reference front decrease, as the number of requirements increase A negative correlation between the number of requirements and convergence of NSGA-II For the other algorithms, there is no negative correlation between problem size and solution quality The algorithms increase their diversity as the scale of the problem increase