Answering the Call of the Wild?: Thoughts on the Elusive Quest for - - PowerPoint PPT Presentation
Answering the Call of the Wild?: Thoughts on the Elusive Quest for Ecological Validity in Variability Modelling Seiede Reyhane Kamali, Shirin Kasaei, Roberto E. Lopez-Herrejon cole de technologie suprieure, Universit du Qubec, Canada
Answering the Call of the Wild?: Thoughts on the Elusive Quest for Ecological Validity in Variability Modelling
Seiede Reyhane Kamali, Shirin Kasaei, Roberto E. Lopez-Herrejon École de technologie supérieure, Université du Québec, Canada introduction
MODEVAR @ SPLC19
1
MODEVAR @ SPLC19
2
} Poses the question:
} Is a study, method, artifacts, setting an approximation of what
In the wild … In the lab …
Industry
MODEVAR @ SPLC19
3
} Limited access } Unfeasible reproducibility } Transparent access } Enables reproducibility
MODEVAR @ SPLC19
4
} General question:
} What is the impact of open source projects in SPL research?
} Our paper focus:
} Variability Modeling
} Starting point:
} José A. Galindo, David Benavides, Pablo Trinidad, Antonio Manuel Gutiérrez Fernández, and
Antonio Ruiz-Cortés. 2019. Automated analysis of feature models: Quo vadis? Computing 101, 5 (2019), 387–433.
} Corpus of 242 articles from 2017 to 2017
} Open source project – less restrictive interpretation:
} Publicly available code repository (URL) } More than one author } No academic projects
MODEVAR @ SPLC19
5
} Median 5
MODEVAR @ SPLC19
6
5 10 15 20 25 Linux eCos FreeBSD BusyBox uClibc uClinux Fiasco BerkeleyDB Drupal axTLS ToyBox ArgoUml Eclipse BuildRoot FraSCAti EmbToolkit Freetz coreboot LLVM SQLite Apache http… Ubuntu OpenSSL JDK Buffer library JavaGeom Gantt x264 Cherokee Gnuplot Parrot Postgresql OpenVPN Libxml2 Sendmail Vim73 Xterm Qemu Xfig Elevator Gimp Gnumeric H264 Subversion
Linux rules with 23 and 43 versions 21 projects with 1 primary source 9 projects with 2 primary sources } Inconsistent or
incomplete report of:
} Version numbers } Number of features
} Projects with more
than one version reported:
} Eclipse (7) } BusyBox & FraSCati (3) } Drupal & uClibc (2)
eCos with 13 but 1 version
Recency – gap between versions analyzed and released
MODEVAR @ SPLC19
7
20 40 60 80 100 120 140 160 180 200
JDK buffer library Ubuntu coreboot x264 Eclipse Freetz axTLS BuildRoot ToyBox FreeBSD OpenSSL BusyBox Drupal Linux
Months Versions
3 years
MODEVAR @ SPLC19
8
33 1 20 8 7 4 2 10 30
Domain Requirements Engineering Domain Design Domain Realisation Domain Testing Application Requirements Engineering Application Design Application Realisation Application Testing Maintenance and Evolution Tooling
5 10 15 20 25 30 35
MODEVAR @ SPLC19
9
5 10 15 20 25
Full Partial None Undetermined
MODEVAR @ SPLC19
10 12 12 5 4 2 2 2 1 1 1 1 1 1 1 1
2 4 6 8 10 12
KConfig DIMACS CDL FAMA FAMILIAR SXFM Adhoc Metamodel CNF BDD XML Prolog FDL CIDE SPLConqueror EvoFM
MODEVAR @ SPLC19
11
} Predominance of Linux and KConfig
} Number of primary sources, number of versions, most recent, … } T
} Aging and outdated datasets
} Only Linux has been updated but still at a lower rate than releases
} Artifacts beyond feature models and source code
} Open sources projects are used for multiple purposes hence the need
to study other types of artifacts à e.g. faults, test cases, test code, …
MODEVAR @ SPLC19
12
} Replication as a pillar for empirical SPL research
} Reproducibility should be the norm not the exception
} Building bridges to open source communities
} Linux has had enormous impact on SPL research, but how about the
MODEVAR @ SPLC19
13
} Expand the focus of our study on impact of open source
} Primary sources beyond Galindo et al.’s work } Other artifacts à related to testing activities
} Other open source projects
} Based on KConfig } Mining Repositories community at large } Open source communities at large
14
Roberto Erick Lopez Herrejon École de technologie supérieure Université du Québec Montreal, Canada Roberto.lopez@etsmtl.ca
MODEVAR @ SPLC19
Natural Sciences and Engineering Research Council of Canada