[PPT] - Metaheuristic models for decision support in the software PowerPoint Presentation

SLIDE 1

Metaheuristic models for decision support in the software construction process

– Ph.D. Thesis –

Aurora Ramírez Quesada

Dept. Computer Science and Numerical Analysis

University of Córdoba Supervisors:

Dr. José Raúl Romero Salguero
Dr. Sebastián Ventura Soto

24th September 2018

SLIDE 2

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

1

Introduction

2

Evolutionary discovery of software architectures

3

The multi- and many-objective perspectives

4

The human-in-the-loop approach

5

Conclusions and future work

Aurora Ramírez Quesada Ph.D. Thesis 2 / 50

SLIDE 3

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Introduction

Aurora Ramírez Quesada Ph.D. Thesis 3 / 50

SLIDE 4

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Background - Software architectures

A software architecture represents “the fundamental concepts or properties of a system in its environment embodied in its elements, relationships, and in the principles of its design and evolution” (ISO/IEC Std. 42010) A bridge between requirements and implementation Essential in early software conception and its subsequent evolution Special focus on non-functional requirements: availability, performance, security... Software architectures are fundamental for: Understanding Reuse Construction Evolution Analysis Management

Aurora Ramírez Quesada Ph.D. Thesis 4 / 50

SLIDE 5

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Background - Metaheuristics

“Intelligent” exploration of the search space Optimality is not guaranteed Intensification and diversification Properties:

1

Iterative

2

Stochastic

3

Memory-based

4

Bio-inspired

Aurora Ramírez Quesada Ph.D. Thesis 5 / 50

SLIDE 6

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Background - Evolutionary computation

“It is not the strongest of the species that survives, nor the most intelligent that

survives. It is the one that is most adaptable to change.” (Charles Darwin)

Phenotype/genotype mapping The fitness function

Aurora Ramírez Quesada Ph.D. Thesis 6 / 50

SLIDE 7

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Background - Optimisation with multiple objectives

Multi-objective optimisation problem: x = [x1, x2, ..., xn] subject to gj(x) ≤ 0 j = 1, ..., p f(x) = [f1(x), f2(x), ..., fm(x)] 2-3 objectives, often in conflict (≥ 4 → many-objective) The concept of Pareto dominance No single optimal solution ⇒ Pareto Set (PS) In the objective space ⇒ Pareto Front (PF) The goals of a multi-/many-objective evolutionary algorithm (MOEA/MaOEA):

1 To find a good approximation of the true PF 2 To generate a well-spread and uniform PF Aurora Ramírez Quesada Ph.D. Thesis 7 / 50

SLIDE 8

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Background - Interactive optimisation

Full automation is not always realistic: Uncertainty scenarios Creative tasks Any optimisation method, including metaheuristics, in which the human actively participates in any step of the process to provide feedback Human is put in the loop: To provide problem knowledge To increase trust on automatic results To meet user’s expectations Design and implementation issues: Selection of solutions Type of feedback Frequency of interaction Information lifetime

Aurora Ramírez Quesada Ph.D. Thesis 8 / 50

SLIDE 9

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Background - Search-based Software Engineering

Origin and characteristics SBSE was formalised as research field by Harman and Jones in 2001 Software engineering activities can be reformulated as optimisation problems Examples along the software lifecycle: project planning, requirement priorisation, selection of test cases, refactoring... Software metrics are adopted to define the fitness function Evolutionary computation (EC) stands out as the most popular technique Search-based software design (SBSD) The need of creativity and intuition makes it particularly challenging Applicable to different paradigms (object-oriented, services...) Problem encoding and evaluation are hard to define

Aurora Ramírez Quesada Ph.D. Thesis 9 / 50

SLIDE 10

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Background - Software architecture optimisation

Scope and characteristics Automatic specification and improvement of architectural models Often guided by more than one non-functional requirement Problems can be defined at different levels of abstraction Current approaches Architecture synthesis from requirements or use cases Selection and assembling of black-box components, e.g. COTS Recovery of software components from code Architecture deployment (reconfiguration and allocation)

Aurora Ramírez Quesada Ph.D. Thesis 10 / 50

SLIDE 11

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Objectives

General objective: Development of search models based on metaheuristic algorithms to provide semi-automatic support to software engineers in early stages of the software development process Subobjectives: O1: Analysis of the state of the art in SBSE, especially in the areas of SBSD and architecture optimisation O2: Design and development of a metaheuristic model based on EC to support the discovery of component-based software architectures O3: Design and development of multi-objective approaches and hybrid techniques to study their suitability to the decision support process O4: Design and development of an interactive approach to incorporate human expertise in the context of software architecture optimisation

Aurora Ramírez Quesada Ph.D. Thesis 11 / 50

SLIDE 12

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Work plan

1 Formulation of software architecture discovery as an optimisation problem 2 A first evolutionary model to validate the idea 3 Exploration of diverse search techniques to cope with additional decision factors 4 Participation of the engineer to complete the scenario Aurora Ramírez Quesada Ph.D. Thesis 12 / 50

SLIDE 13

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Problem description

Software architecture discovery To identify the underlying architecture of the system from the analysis information contained in a UML class diagram The elements of the component-based software architecture are obtained as follows: Component A cohesive group of classes that work together to satisfy the expected behaviour

f the component

Interface A directed relationship between classes in the analysis model belonging to different components Connector The linkage between a pair of required / provided interfaces interconnecting different components

Aurora Ramírez Quesada Ph.D. Thesis 13 / 50

SLIDE 14

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Evolutionary discovery of software architectures

Aurora Ramírez Quesada Ph.D. Thesis 14 / 50

SLIDE 15

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Motivation

Current approaches recover components from low-level artefacts, e.g. code The process cannot be performed until the system is built The system structure could be different from that originally conceived Analysis information is omitted

Aurora Ramírez Quesada Ph.D. Thesis 15 / 50

SLIDE 16

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Research questions

RQ: Can single-objective EAs help the software engineer to iden- tify an initial candidate architecture of a system at a high level of abstraction? RQ: How does the configuration of the algorithm influence both the evolutionary performance and the quality of the returned solution?

Aurora Ramírez Quesada Ph.D. Thesis 16 / 50

SLIDE 17

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Solution encoding

Solutions are encoded using tree structures: Hierarchical decomposition Comprehensive for software engineers Architectural solutions of different size

A_prov_E E_req_A

Phenotype

Architecture Components Connectors

Component_1 Component_2 Connector_1 Classes Classes Provided interfaces Required interfaces Provided interfaces Required interfaces Provided interface Required interface B A_prov_E E_req_A A_prov_E E_req_A A C D F E G H Genotype

Aurora Ramírez Quesada Ph.D. Thesis 17 / 50

SLIDE 18

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Fitness function

Intra-modular Coupling Density (ICD) ICDi = #classestotal − #classesi #classestotal · CIin

i

CIin

i + CIout i

ICD = 1 n ·

n

i=1

ICDi External Relations Penalty (ERP) ERP =

n

i=1

n

j=i+1

(was · nasij + wag · nagij + wco · ncoij + wge · ngeij ) Groups/Components Ratio (GCR) GCR = #cgroups n fitness(ind) =

r(ICD(ind)) + r(ERP(ind)) + r(GCR(ind))

if ind is feasible #individuals · #metrics + 1 if ind is infeasible

Aurora Ramírez Quesada Ph.D. Thesis 18 / 50

SLIDE 19

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Mutation operator

Algorithm 1: Add

1 Choose contributors; 2 Extract small groups; 3 Create new component; 4 Add selected classes; 5 Set interfaces;

Algorithm 2: Remove

1 Find highly-coupled components; 2 Choose component; 3 Distribute classes; 4 Set interfaces;

Algorithm 3: Move

1 Choose origin; 2 Choose class; 3 Choose target; 4 Move class; 5 Set interfaces;

Algorithm 4: Merge

1 Find highly-coupled components; 2 Choose candidate 1; 3 if found>2 then 4

Choose candidate 2;

5 else 6

Choose component 2;

7 Join classes; 8 Set interfaces;

Algorithm 5: Split

1 Find components with > 1 group; 2 if found then 3

Choose candidate;

4

Distribute groups;

5 else 6

Choose component;

7

Distribute classes;

8 Set interfaces

Aurora Ramírez Quesada Ph.D. Thesis 19 / 50

SLIDE 20

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Illustrative example

(a) Input class diagram

Component_3 Component_2 Component_1

E_req_A G_req_H A_prov_E H_prov_G ICD=0.17 GCR=2.00 Fitness=7.00 ERP=13.00

(b) Initial population

Fitness=5.00 ERP=5.00 ICD=0.35 GCR=1.50 A_prov_E E_req_A Component_1 Component_2

(c) 5th Generation

Fitness=3.00 ERP=0.00 ICD=0.38 GCR=1.00 A_prov_E E_req_A Component_1 Component_2

(d) 10th Generation

Aurora Ramírez Quesada Ph.D. Thesis 20 / 50

SLIDE 21

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Experimental framework

Implementation details JCLEC as the metaheuristic framework to code the algorithm SDMetrics Open Core to parse XMI files Datapro4j to preprocess input data and generate reports Problem instances Problem #Classes #Relationships #Interfaces As De Ag Co Ge Aqualush 58 69 6 20 74 Borg 167 44 109 36 38 90 300 Datapro4j 59 3 4 3 2 49 12 Java2HTML 53 20 66 15 15 170 JSapar 46 7 33 21 9 19 80 Marvin 32 5 11 22 5 8 28 NekoHTML 47 6 17 15 18 17 46

Aurora Ramírez Quesada Ph.D. Thesis 21 / 50

SLIDE 22

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Results

Parameter study Selection and replacement strategies: Binary tournament, 10% elitism Mutation weights (126 combinations): wadd = 0.2, wremove = 0.1, wmerge = 0.1, wsplit = 0.3, wmove = 0.3 Population size and number of evaluations: 150, 20,000-24,000 Final results Problem ICD (↑) ERP (↓) GCR (↓) Time (ms) Aqualush 0.41240.0604 6.23333.8443 1.08330.1708 116.10198.4891 Borg 0.28200.0689 3.93332.4626 1.16670.2274 2, 489.1223171.2028 Datapro4j 0.64250.0356 33.600014.7436 2.39890.6744 37.11012.8308 Java2HTML 0.25930.0000 0.00000.0000 1.00000.0000 250.45727.4674 JSapar 0.37510.0307 9.00001.5492 1.16670.2981 94.88915.1418 Marvin 0.50800.0187 3.16671.0980 1.67500.1146 14.11940.6885 NekoHTML 0.45940.0345 3.26675.3037 1.23890.3768 57.11503.5524

Aurora Ramírez Quesada Ph.D. Thesis 22 / 50

SLIDE 23

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Lessons learned

◮ Lack of software metrics suitable for fitness function definition ◮ Need of domain-specific operators to improve performance ◮ Ability to obtain solutions close to manual designs ◮ Influence of the number and type of UML relations in the hardness of the problem Aurora Ramírez Quesada Ph.D. Thesis 23 / 50

SLIDE 24

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Associated publications

Main publication:

A. Ramírez, J.R. Romero, S. Ventura. “An approach for the evolutionary discovery of

software architectures”. Information Sciences, vol. 305, pp. 234-255, 2015. Additional publications:

1

A. Ramírez, J.R. Romero, S. Ventura. “Análisis de la aplicabilidad de medidas software para el

diseño semi-automático de arquitecturas”. Proceedings of the XIX Jornadas en Ingeniería del Software y Bases de Datos (JISBD), pp. 307-320, 2014.

2

A. Ramírez, J.A. Molina, J.R. Romero, S. Ventura. “Estudio de mecanismos de hibridación para

el descubrimiento evolutivo de arquitecturas”. Proceedings of the XXI Jornadas en Ingeniería del Software y Bases de Datos (JISBD), pp. 481-494, 2016.

3

A. Ramírez, R. Barbudo, J.R. Romero, S. Ventura. “Memetic Algorithms for the Automatic Dis-

covery of Software Architectures”. Proceedings of the 16th International Conference on Intelli- gent Systems Design and Applications (ISDA), vol. 557 AISC, pp. 437-447, 2016.

Aurora Ramírez Quesada Ph.D. Thesis 24 / 50

SLIDE 25

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

The multi- and many-objective perspectives

Aurora Ramírez Quesada Ph.D. Thesis 25 / 50

SLIDE 26

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Motivation

Evidence of conflict between ICD and ERP Additional quality criteria usually guide the discovery process Each system might require the optimisation of a different number of metrics The algorithm could provide a set of alternative solutions for engineer’s choice

Aurora Ramírez Quesada Ph.D. Thesis 26 / 50

SLIDE 27

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Research questions

RQ: How do a larger number and combination of metrics influence the search-based discovery of software architectures? RQ: How do state-of-the-art MOEAs and MaOEAs perform when multiple objectives guide the discovery process?

Aurora Ramírez Quesada Ph.D. Thesis 27 / 50

SLIDE 28

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Objective functions

Maintainability is “the degree to which the software product can be modified", where modifications “may include corrections, improvements or adaptation of the software to changes in the environment, and in requirements and functional specifications" (ISO/IEC Std. 25000) Modularity: “the degree to which a system is composed of discrete components such that a change to one component has minimal impact on other components" Reusability: “the degree to which an asset can be used in more than one software system or in building other assets" Analysability: “the degree to which the parts of the software to be modified can be identified"

Metric Min/Max Quality attribute Range of values Design goals ICD max modularity [0, 1] Small components with high cohesion ERP min modularity [0, ∗] Large components with low coupling INS min modularity [0, 1] Components with few interactions ENC max modularity [0, 1] Components with hidden classes CS min modularity [0, n] Small or medium-sized components CL min modularity [0, n] Components with few provided interfaces GCR min reusability [1, ∗] Connected classes within each component ABS max reusability [0, 1] Components with abstract classes CB max analisability [0, 1] Equal-sized components

Aurora Ramírez Quesada Ph.D. Thesis 28 / 50

SLIDE 29

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Multi/Many-objective evolutionary algorithms

1 Multi-objective algorithms (Pareto-based)

SPEA2 NSGA-II

2 Decomposition algorithms

MOEA/D

3 Algorithms based on reference set

NSGA-III

4 Algorithms based on landscape partition

ǫ-MOEA GrEA

5 Indicator-based algorithms

IBEA HypE

Aurora Ramírez Quesada Ph.D. Thesis 29 / 50

SLIDE 30

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Experimental framework

Development of JCLEC-MO Independence between the metaheuristic and the multi-objective techniques Experimental utilities Parameters and analysis All possible combinations of 2, 4, 6, 8 and 9 metrics (256 in total) The number of software systems is increased up to 10 (32 - 256 classes) Quality indicators: hypervolume, spacing Statistical analysis: Friedman&Holm, Cliff’s Delta (effect size) Parameter Value Population size 100, 120, 126, 330, 495

Max. evaluations

10000, 15000, 20000, 66000, 99000 Min-max. components 2-8 Mutator weights wadd = 0.2, wremove = 0.3, wmerge = 0.2 wsplit = 0.1, wmove = 0.2 ERP weights was = 2, wag = 3, wco = 3, wge = 5 CS threshold 0.3 CL threshold 8

Algorithms configured following authors’ recommendations / preliminary experiments.

Aurora Ramírez Quesada Ph.D. Thesis 30 / 50

SLIDE 31

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Results

Hypervolume Algorithm 2 4 6 8 9 SPEA2 4.35 ± 0.78 4.66 ± 0.86 5.64 ± 0.29 6.33 ± 0.28 6.70 ± 0.00 NSGA-II 2.02 ± 0.59 1.39 ± 0.40 1.92 ± 0.54 2.66 ± 0.46 2.90 ± 0.00 MOEA/D 3.90 ± 0.81 4.41 ± 0.68 3.75 ± 0.47 3.48 ± 0.43 3.30 ± 0.00 NSGA-III 6.75 ± 0.49 6.62 ± 0.21 6.49 ± 0.16 6.01 ± 0.29 5.90 ± 0.00 ǫ-MOEA 3.88 ± 1.12 2.78 ± 1.15 1.51 ± 0.55 1.08 ± 0.09 1.00 ± 0.00 GrEA 4.86 ± 0.79 5.30 ± 0.75 5.80 ± 0.41 5.83 ± 0.11 5.50 ± 0.00 IBEA 7.21 ± 0.31 7.64 ± 0.15 7.76 ± 0.06 7.79 ± 0.05 7.90 ± 0.00 HypE 3.04 ± 0.58 3.20 ± 0.75 3.20 ± 0.55 2.82 ± 0.31 2.80 ± 0.00 Spacing Algorithm 2 4 6 8 9 SPEA2 3.65 ± 1.25 2.09 ± 1.08 1.10 ± 0.17 1.37 ± 0.37 1.70 ± 0.00 NSGA-II 4.00 ± 0.82 4.59 ± 0.91 2.77 ± 0.85 1.97 ± 0.34 1.80 ± 0.00 MOEA/D 3.23 ± 1.10 3.16 ± 0.79 5.21 ± 0.50 5.69 ± 0.23 5.30 ± 0.00 NSGA-III 4.35 ± 1.45 4.02 ± 1.09 3.53 ± 0.51 3.13 ± 0.16 3.20 ± 0.00 ǫ-MOEA 5.58 ± 1.04 3.02 ± 1.49 3.41 ± 0.69 4.06 ± 0.37 4.60 ± 0.00 GrEA 5.37 ± 0.52 6.78 ± 0.45 7.29 ± 0.17 7.28 ± 0.14 6.60 ± 0.00 IBEA 5.73 ± 1.51 7.43 ± 0.42 7.60 ± 0.04 7.63 ± 0.07 7.80 ± 0.00 HypE 4.09 ± 0.72 4.91 ± 0.95 5.11 ± 0.82 4.88 ± 0.57 5.00 ± 0.00

Aurora Ramírez Quesada Ph.D. Thesis 31 / 50

SLIDE 32

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Results

9-objective Pareto fronts

(a) SPEA2 (b) NSGA-II (c) MOEA/D (d) NSGA-III (e) ǫ-MOEA (f) GrEA (g) IBEA (h) HypE

Aurora Ramírez Quesada Ph.D. Thesis 32 / 50

SLIDE 33

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Lessons learned

◮ Not all many-objective algorithms are effective in this context ◮ Some metrics do not present real conflict, others are highly opposed ◮ The number and combination of metrics might depend on the system under study ◮ Practical implications in decision-making: execution time, number of solutions Aurora Ramírez Quesada Ph.D. Thesis 33 / 50

SLIDE 34

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Associated publications

Main publication:

A. Ramírez, J.R. Romero, S. Ventura. “A comparative study of many-objective

evolutionary algorithms for the discovery of software architectures”. Empirical Software Engineering, vol. 21, no. 6, pp. 2546-2600, 2016. Additional publications:

1

A. Ramírez, J.R. Romero, S.Ventura. “On the Performance of Multiple Objective Evolutionary

Algorithms for Software Architecture Discovery”. Proceedings of the 16th Genetic and Evolu- tionary Computation Conference (GECCO), pp. 1287-1294, 2014. Best paper of the SBSE track and nominated to best paper of GECCO’14.

2

A. Ramírez, J.R. Romero, S.Ventura. “Estudio preliminar del rendimiento de familias de algo-

ritmos multiobjetivo en diseño arquitectónico”. Proceedings of the X Congreso Español sobre Metaheurísticas, Algoritmos Evolutivos y Bioinspirados (MAEB), pp. 173-180, 2015.

3

A. Ramírez, J.R. Romero, S.Ventura. “An Extensible JCLEC-based Solution for the Implemen-

tation of Multi-Objective Evolutionary Algorithms”. Proceedings of the Companion Publication

f 17th Genetic and Evolutionary Computation Conference (GECCO Companion), pp. 1085-

1092, 2015.

4

A. Ramírez, J.R. Romero, S. Ventura. “On the effect of local search in the multi-objective evolu-

tionary discovery of software architectures”. Proceedings of the IEEE Congress on Evolutionary Computation (IEEE CEC), pp. 2038-2045, 2017.

5

A. Ramírez, J.R. Romero, C. García-Martínez, S.Ventura. “JCLEC-MO: A Java suite for solving

many-objective optimization engineering problems”. Under review, 2018. (Indexed journal, Q1).

6

A. Ramírez, J.R. Romero, S.Ventura. “A survey of many-objective optimisation in search-based

software engineering”. Under review, 2018. (Indexed journal, Q1).

Aurora Ramírez Quesada Ph.D. Thesis 34 / 50

SLIDE 35

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

The human-in-the-loop approach

Aurora Ramírez Quesada Ph.D. Thesis 35 / 50

SLIDE 36

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Motivation

Software design is a human-centred task Qualitative aspects are difficult to quantify The engineer should be further involved in the optimisation process Sometimes it is easier to identify a bad solution rather than a good one

Aurora Ramírez Quesada Ph.D. Thesis 36 / 50

SLIDE 37

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Research questions

RQ: How can the qualitative judgement of the engineer be inte- grated into the evolutionary discovery of software architectures? RQ: Does putting the human in the loop involve a significant im- provement compared with not considering him/her along the opti- misation process?

Aurora Ramírez Quesada Ph.D. Thesis 37 / 50

SLIDE 38

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

The algorithm

Steady-state algorithm: two offspring are produced in each generation Binary tournament selection from population and archive Offspring replace population members with worst fitness Archive to store a small set of representative solutions

Aurora Ramírez Quesada Ph.D. Thesis 38 / 50

SLIDE 39

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

The fitness function

A novel fitness function that combines objective and subjective evaluation criteria fitness(s) = wobj · fobj(s) + wsub · fsub(s) Quantitative: Maximin function to quantify both dominance and diversity (software metrics) fobj(s) = 1 + maxz=s(mink(f s

k − f z k ))

2 ∀z ∈ Z Qualitative: Engineer’s preferences on phenotypic aspects of the solution (architectural preferences) fsub(s) = 1 − 1 P ·

P

p=1

wp · prefp(s)

Aurora Ramírez Quesada Ph.D. Thesis 39 / 50

SLIDE 40

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Architectural preferences

Preference Description Best component Similarity to the set of classes Worst component Dissimilarity to the set of classes Best provided interface Similarity to the set of operations Worst provided interface Dissimilarity to the set of operations Number of components Distance to the preferred number Metric in range Distance to the midrange Aspiration levels Weighted distance to the reference point

Aurora Ramírez Quesada Ph.D. Thesis 40 / 50

SLIDE 41

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Interaction mechanism

Interactions are scheduled at regular intervals Solutions are selected from the current population using a clustering method The software engineer rewards or penalises some aspect of the solution by choosing one preference Additional actions:

Freeze one component Add to the archive Remove from the population Stop the search

Aurora Ramírez Quesada Ph.D. Thesis 41 / 50

SLIDE 42

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Empirical study

Methodology Case study, 9 participants 3 interactions, 3 solutions each Log files and questionnaires

00

01 02 03 04 05

i1s1 i1s2 i1s3 i2s1 i2s2 i2s3 i3s1 i3s2 i3s3 Step of interaction Evaluation time (minutes)

Impact of interaction Cohesion is indirectly improved Different number of components Components similar or equal to the manual design Actions (add, freeze) contribute to find better solutions

10 20 30 40 50 60 70 80 2 3 4 5 6 Number of components Number of solutions Without interaction With interaction

Aurora Ramírez Quesada Ph.D. Thesis 42 / 50

SLIDE 43

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Empirical study

Analysis of architectural preferences More interest in the internal structure Negative preferences are more frequently applied at the beginning Use of “No preference”

5 10 15 20 25 Frequency N

p

r e f e r e n c e B e s t c

m

p

n

e n t B e s t i n t e r f a c e W

r

s t c

m

p

n

e n t W

r

s t i n t e r f a c e N

.

C

m

p

n

e n t s M e a s u r e i n r a n g e A s p i r a t i

n

l e v e l s

Architectural Selected Usefulness Intuitiveness Preference (%) [1-8] [1-8] No preference 22.22 6.44 7.33 Best component 29.63 7.44 7.44 Worst component 23.46 7.22 7.33 Best provided interface 2.47 5.29 6.38 Worst provided interface 0.00 4.71 6.38 Number of components 17.28 7.50 7.33 Metric in range 2.47 4.17 5.44 Aspiration levels 2.47 5.80 5.22

Aurora Ramírez Quesada Ph.D. Thesis 43 / 50

SLIDE 44

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Lessons learned

◮ Architecture preferences as a novel mechanism for subjective evaluation ◮ Design metrics are still needed at the beginning ◮ Negative opinions are useful too ◮ Humans are willing to participate (even more) in the optimisation Aurora Ramírez Quesada Ph.D. Thesis 44 / 50

SLIDE 45

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Associated publications

Main publication:

A. Ramírez, J.R. Romero, and S. Ventura. “Interactive multi-objective optimization of

software architectures”. Information Sciences, vol. 463-464, pp. 92-109, 2018. Additional publications:

1

A. Ramírez, J.R. Romero, S.Ventura. “Interactividad en el descubrimiento evolutivo de arquitec-

turas”. Proceedings of the XX Jornadas en Ingeniería del Software y Bases de Datos (JISBD), 2015.

2

A. Ramírez, R. Barbudo, J.R. Romero, S. Ventura. “Herramienta basada en computación evo-

lutiva interactiva para arquitectos software”. Proceedings of the XI Congreso Español sobre Metaheurísticas, Algoritmos Evolutivos y Bioinspirados (MAEB), pp. 387-396, 2016.

3

A. Ramírez, J.R. Romero, S. Ventura. “Búsqueda coevolutiva interactiva aplicada al diseño

de software”. Proceedings of the XXII Jornadas en Ingeniería del Software y Bases de Datos (JISBD), 2017.

4

A. Ramírez, J.R. Romero, S. Ventura.“API para el desarrollo de algoritmos interactivos en inge-

niería del software basada en búsqueda”. XXIII Jornadas en Ingeniería del Software y Bases de Datos (JISBD), 2018.

5

A. Ramírez, J.R. Romero, C. Simons. “A Systematic Review of Interaction in Search-Based

Software Engineering”. IEEE Transactions on Software Engineering, pp. 1-22. 2018. In press.

Aurora Ramírez Quesada Ph.D. Thesis 45 / 50

SLIDE 46

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Conclusions and future work

Aurora Ramírez Quesada Ph.D. Thesis 46 / 50

SLIDE 47

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Concluding remarks

1 Definition of software architecture discovery as an optimisation problem 2 Design of a comprehensible evolutionary model to discover architectures 3 Analysis of the influence of software metrics in a many-objective space 4 Exploration of hybrid approaches to improve the performance 5 Development of a general interactive model to integrate human expertise

Literature Problem Algorithm Experimental Model analysis formulation implementation validation integration O1 TSE UnderReview1 O2 JISBD’14 INFSCI O3 GECCO’14, MAEB’15, EMSE GECCO’15 JISBD’16, ISDA’16, CEC’17 UnderReview2 JISBD’16 ESwA O4 JISBD’15 INFSCI MAEB’16 JISBD’17 JISBD’18

Overview of publications derived from the Ph.D. Thesis

Aurora Ramírez Quesada Ph.D. Thesis 47 / 50

SLIDE 48

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Future lines of research

1 Extensions to the problem formulation

Design constraints Evaluation and semantic aspects

2 Advanced interactive models

Cooperative coevolution + interaction Learning from user interaction

3 New optimisation problems in SBSD

Service-oriented architectures Cloud computing

4 Industrial acceptance

Integrated process Software tools

Aurora Ramírez Quesada Ph.D. Thesis 48 / 50

SLIDE 49

Introduction Architecture discovery Multi/Many perspectives Human-in-the-loop approach Conclusions

Thesis quality indicators

Compendium of publications A minimum of three articles published in journals ranked in the Journal Citation Report (Q1) International mention Research stay (three months) in a foreign institution: University of the West of England, under the supervision of

Dr. Christopher L. Simons

Positive evaluation reports by two researchers from a different institution Additional international publications

Aurora Ramírez Quesada Ph.D. Thesis 49 / 50

SLIDE 50

Metaheuristic models for decision support in the software construction process

– Ph.D. Thesis –

Aurora Ramírez Quesada

Dept. Computer Science and Numerical Analysis

University of Córdoba Supervisors:

Dr. José Raúl Romero Salguero
Dr. Sebastián Ventura Soto

24th September 2018

SLIDE 51

Mutation operator

Algorithm 1 Add a component Require: Ensure:

1:

✁

2: for all component in

do

3:

candidates

✁

4:

if (random(0,1)> Prob(component)) then

5:

allGroups

✁ getGroups(component)

6:

if (size(groupsOf Classes) > 1) t hen

7:

for all groupOf Classes in allGroups do

8:

if (size(groupOf Classes) = min) then

9:

✁ groupOf Classes

10:

end if

11:

end for

12:

end if

13:

end if

14:

newComp

✁ randomGroup(

)

15:

✁ component ✂

16:

✁

17: end for 18: if (newComp =

) then

19:

✁

20:

for all component in do

21:

for all class in component do

22:

if (random(0,1) > 0.5) then

23:

✁ class

24:

end if

25:

end for

26:

✁ component ✂

27:

newComp

✁

28:

✁

29:

end for

30: end if 31:

✁ newComp

32:

)

33: return

ffspring
ffspring
ffspring
ffspring
ffspring
ffspring

candidates candidates candidates candidates candidates candidates candidates candidates

ffspring

parent parent parent

ffspring

setInterfacesAndConnectors( Algorithm 2 Split a component Require: Ensure:

1:

✁

2:

✁

3: for all component in

do

4:

if (numberOfGroups(component) > 1) then

5:

✁ component

6:

end if

7: end for 8: if (size(

) > 0) then

9:

✁ randomComponent(

)

10:

for all in do

11:

if (random(0,1) > 0.5) then

12:

component1

✁

13:

else

14:

component2

✁

15:

end if

16:

end for

17: else 18:

✁ randomComponent(

)

19:

for all class in do

20:

if (random(0,1) > 0.5) then

21:

component1

✁ class

22:

else

23:

component2

✁ class

24:

end if

25:

end for

26: end if 27:

✁ ✂

28:

✁ component1

29:

✁ component2

30:

)

31: return

parent parent parent parent

ffspring
ffspring
ffspring
ffspring
ffspring
ffspring
ffspring

groupOfClasses groupOfClasses groupOfClasses compToSplit compToSplit compToSplit candidates candidates candidates candidates compToSplit compToSplit setInterfacesAndConnectors( Aurora Ramírez Quesada Ph.D. Thesis 1 / 7

SLIDE 52

Mutation operator

Algorithm 3 Remove a component Require: Ensure:

1:

✁

2:

✁

3: maxRel

✁ maxNumExtRel(

)

4: for all component in

do

5:

if (numExtRel(component)=maxRel) then

6:

✁ component

7:

end if

8: end for 9: compToRemove

✁ randomComponent(

)

10:

✁ ✂ compToRemove

11: for all class in compToRemove do 12:

randomComponent( )

✁ class

13: end for 14:

)

15: return

parent parent

ffspring
ffspring

parent

ffspring
ffspring
ffspring

candidates candidates candidates parent

ffspring

setInterfacesAndConnectors( Algorithm 5 Move a class Require: Ensure:

1:

✁

2:

✁ randomComponent(

)

3: destination

✁ randomComponent(

)

4:

✁ randomClass(

)

5: removeClass(

, )

6: addClass(

, )

7:

✁ ✂

8:

✁

9:

✁

10:

)

11: return

ffspring
ffspring
ffspring
ffspring
ffspring
ffspring
ffspring

parent parent parent parent

rigin
rigin
rigin

(origin

rigin

destination destination) destination class class class setInterfacesAndConnectors( ∪ Algorithm 4 Merge two components Require: Ensure:

1:

✁

2:

✁

3: maxRel

✁ maxNumExtRel(

)

4: for all component in

do

5:

if (numExtRel(component)=maxRel) then

6:

✁ component

7:

end if

8: end for 9: component1

✁ randomComponent(

)

10: if (size(

) > 1) then

11:

component2

✁ randomComponent(

)

12: else 13:

component2

✁ randomComponent(

)

14: end if 15:

✁ ✂

com ponent1 component2

16:

✁ component1∪component2

17:

)

18: return

ffspring
ffspring
ffspring
ffspring
ffspring
ffspring

parent parent parent parent parent candidates candidates candidates candidates candidates ( ∪ ) setInterfacesAndConnectors( Aurora Ramírez Quesada Ph.D. Thesis 2 / 7

SLIDE 53

Mutation operator

Initial individual

Architecture Components Connectors

Component_1 Component_2 Connector_1 Classes Classes Provided interfaces Required interfaces Provided interfaces Required interfaces Provided interface Required interface A_prov_E E_req_A A_prov_E E_req_A A C D E G F H B Provided interface Required interface Connector_2 D_prov_B B_req_D D_prov_B B_req_D

Component_1 Component_2 A_prov_E E_req_A D_prov_B B_req_D

Add a component

Architecture Components Connectors

Component_1 Component_2 Connector_1 Classes Classes Provided interfaces Required interfaces Provided interfaces Required interfaces Provided interface Required interface A_prov_E E_req_A A_prov_E E_req_A A C D E G F H B Provided interface Required interface Connector_2 D_prov_B B_req_D D_prov_B B_req_D Component_3 Classes Provided interfaces Required interfaces B F B_req_D

A_prov_E E_req_A D_prov_B B_req_D Component_3 Component_1 Component_2

Move a class

Architecture Components Connectors

Component_1 Component_2 Connector_1 Classes Classes Provided interfaces Required interfaces Provided interfaces Required interfaces Provided interface Required interface A_prov_E E_req_A A_prov_E E_req_A A C E G F H B Provided interface Required interface Connector_2 D_prov_B B_req_D D_prov_B B_req_D B D

Component_1 Component_2 A_prov_E E_req_A

Aurora Ramírez Quesada Ph.D. Thesis 3 / 7

SLIDE 54

Objective functions

Instability INSi = ECi ECi + ACi INS = 1 n ·

n

i=1

INSi Encapsulation ENCi = #classesinner #classestotal ENC = 1 n ·

n

i=1

ENCi Abstractness ABSi = #classesabstract #classestotal ABS = 1 n ·

n

i=1

ABSi

Aurora Ramírez Quesada Ph.D. Thesis 4 / 7

SLIDE 55

Objective functions

Critical size CCsize

i

=

1

if size(i) > threshold

therwise

CS =

n

i=1

CCsize

i

Critical link CClink

i

=

1

if #provided interfacesi > threshold

therwise

CL =

n

i=1

CClink

i

Component balance SB(n) =         

n−1 µ−1

if n < µ 1 − n−µ

ω−µ

if µ < n < ω if n ≥ ω CSU(C) = 1 − Gini({volume(c) : c ∈ C}) CB = SB(|C|) · CSU(C)

Aurora Ramírez Quesada Ph.D. Thesis 5 / 7

SLIDE 56

Architectural preferences

Best component prefbc = max{J(classes(c), classes(c+))} ∀c ∈ [1, n] J(A, B) = |A ∩ B|/|A ∪ B| Worst component prefwc = max{1 − J(classes(c), classes(c−))} ∀c ∈ [1, n] Best provided interface prefbi = max{J(operations(interface(c)), operations(p+))} ∀c ∈ [1, n] Worst provided interface prefwi = max{1 − J(operations(interface(c)), operations(p−))} ∀c ∈ [1, n]

Aurora Ramírez Quesada Ph.D. Thesis 6 / 7

SLIDE 57

Architectural preferences

Number of components prefnc =

(n − nmin)/(n+ − nmin)

if n < n+ 1 − ((n − n+)/(nmax − n)) if n ≥ n+ Metric in a range prefmr =      if ms < mmin 1 − (ms − mmid)/mmid if ms ∈ [mmin, mmax] if ms > mmax mmid = (mmax − mmin)/2 Aspiration levels prefal =

1

if ASF ≤ 0 1 − ASF if ASF > 0 ASF = max{wk · (f s

k − z∗ k )}

Aurora Ramírez Quesada Ph.D. Thesis 7 / 7