Answering the Call of the Wild?: Thoughts on the Elusive Quest for - - PowerPoint PPT Presentation

answering the call of the wild thoughts on the elusive
SMART_READER_LITE
LIVE PREVIEW

Answering the Call of the Wild?: Thoughts on the Elusive Quest for - - PowerPoint PPT Presentation

Answering the Call of the Wild?: Thoughts on the Elusive Quest for Ecological Validity in Variability Modelling Seiede Reyhane Kamali, Shirin Kasaei, Roberto E. Lopez-Herrejon cole de technologie suprieure, Universit du Qubec, Canada


slide-1
SLIDE 1

Answering the Call of the Wild?: Thoughts on the Elusive Quest for Ecological Validity in Variability Modelling

Seiede Reyhane Kamali, Shirin Kasaei, Roberto E. Lopez-Herrejon École de technologie supérieure, Université du Québec, Canada introduction

MODEVAR @ SPLC19

1

slide-2
SLIDE 2

Ecological Validity – Form of External Validity

MODEVAR @ SPLC19

2

} Poses the question:

} Is a study, method, artifacts, setting an approximation of what

happens in real-life?

In the wild … In the lab …

slide-3
SLIDE 3

In SPL, what is “real-life”?

Industry

MODEVAR @ SPLC19

3

} Limited access } Unfeasible reproducibility } Transparent access } Enables reproducibility

slide-4
SLIDE 4

Study description

MODEVAR @ SPLC19

4

} General question:

} What is the impact of open source projects in SPL research?

} Our paper focus:

} Variability Modeling

} Starting point:

} José A. Galindo, David Benavides, Pablo Trinidad, Antonio Manuel Gutiérrez Fernández, and

Antonio Ruiz-Cortés. 2019. Automated analysis of feature models: Quo vadis? Computing 101, 5 (2019), 387–433.

} Corpus of 242 articles from 2017 to 2017

} Open source project – less restrictive interpretation:

} Publicly available code repository (URL) } More than one author } No academic projects

slide-5
SLIDE 5

First findings

MODEVAR @ SPLC19

5

}Primary sources

selected: 41

}Open source projects

identified: 43

}Publications per year:

} Median 5

slide-6
SLIDE 6

Projects and primary sources

MODEVAR @ SPLC19

6

5 10 15 20 25 Linux eCos FreeBSD BusyBox uClibc uClinux Fiasco BerkeleyDB Drupal axTLS ToyBox ArgoUml Eclipse BuildRoot FraSCAti EmbToolkit Freetz coreboot LLVM SQLite Apache http… Ubuntu OpenSSL JDK Buffer library JavaGeom Gantt x264 Cherokee Gnuplot Parrot Postgresql OpenVPN Libxml2 Sendmail Vim73 Xterm Qemu Xfig Elevator Gimp Gnumeric H264 Subversion

Linux rules with 23 and 43 versions 21 projects with 1 primary source 9 projects with 2 primary sources } Inconsistent or

incomplete report of:

} Version numbers } Number of features

} Projects with more

than one version reported:

} Eclipse (7) } BusyBox & FraSCati (3) } Drupal & uClibc (2)

eCos with 13 but 1 version

slide-7
SLIDE 7

Recency – gap between versions analyzed and released

MODEVAR @ SPLC19

7

20 40 60 80 100 120 140 160 180 200

JDK buffer library Ubuntu coreboot x264 Eclipse Freetz axTLS BuildRoot ToyBox FreeBSD OpenSSL BusyBox Drupal Linux

Months Versions

3 years

slide-8
SLIDE 8

Feature Models – What for?

MODEVAR @ SPLC19

8

33 1 20 8 7 4 2 10 30

Domain Requirements Engineering Domain Design Domain Realisation Domain Testing Application Requirements Engineering Application Design Application Realisation Application Testing Maintenance and Evolution Tooling

5 10 15 20 25 30 35

slide-9
SLIDE 9

Availability for replication

MODEVAR @ SPLC19

9

22 2 10 7

5 10 15 20 25

Full Partial None Undetermined

slide-10
SLIDE 10

Model Formats

MODEVAR @ SPLC19

10 12 12 5 4 2 2 2 1 1 1 1 1 1 1 1

2 4 6 8 10 12

KConfig DIMACS CDL FAMA FAMILIAR SXFM Adhoc Metamodel CNF BDD XML Prolog FDL CIDE SPLConqueror EvoFM

slide-11
SLIDE 11

Analysis (1)

MODEVAR @ SPLC19

11

} Predominance of Linux and KConfig

} Number of primary sources, number of versions, most recent, … } T

  • ol ecosystem built around it

} Aging and outdated datasets

} Only Linux has been updated but still at a lower rate than releases

} Artifacts beyond feature models and source code

} Open sources projects are used for multiple purposes hence the need

to study other types of artifacts à e.g. faults, test cases, test code, …

slide-12
SLIDE 12

Analysis (2)

MODEVAR @ SPLC19

12

} Replication as a pillar for empirical SPL research

} Reproducibility should be the norm not the exception

} Building bridges to open source communities

} Linux has had enormous impact on SPL research, but how about the

  • ther way around?
slide-13
SLIDE 13

Future Work

MODEVAR @ SPLC19

13

} Expand the focus of our study on impact of open source

projects in SPL research to:

} Primary sources beyond Galindo et al.’s work } Other artifacts à related to testing activities

} Other open source projects

} Based on KConfig } Mining Repositories community at large } Open source communities at large

slide-14
SLIDE 14

The End

14

Any questions?

Roberto Erick Lopez Herrejon École de technologie supérieure Université du Québec Montreal, Canada Roberto.lopez@etsmtl.ca

MODEVAR @ SPLC19

Natural Sciences and Engineering Research Council of Canada