Human factors: Intertwining of formal and empirical methods in - - PowerPoint PPT Presentation

1 Intertwining of formal and empirical methods in software engineering

Human factors:

Intertwining of formal and empirical methods in software engineering

HaPoC Symposium at IACAP-14

Conference of the International Association for Computing and Philosophy

Thessaloniki, July 3rd 2014 Gonzalo Génova ggenovaf@gmail.com Knowledge Reuse Group Universidad Carlos III de Madrid Universidad de Santiago de Chile

2 Intertwining of formal and empirical methods in software engineering

Introduction

• Question: What is the science in Computer Science?

– A very broad term. – A plurality of research areas and methods. – Both in "pure" science and in applied science (i.e. engineering). – Empirical methods are not enough to account for all kinds of scientific activity in computing.

• A three dimensional categorization of research methods in CS:

– Formal vs Empirical – Natural vs Artificial, or Science vs Engineering – Things vs People, or Product vs Process

• Focus on human factors in software engineering research.

3 Intertwining of formal and empirical methods in software engineering

Gonzalo Génova, Juan Llorens, Jorge Morato Software engineering research: The need to strengthen and broaden the classical scientific method IGI-Global, 2012

Reference

4 Intertwining of formal and empirical methods in software engineering

It is often heard… The experimental-scientific method is the preferred way, even the only way, to attain knowledge of reality. A work only deserves to be qualified as “scientific” if it is supported by “empirical evidence”. But… too simplistic in general, and specifically for CS.

5 Intertwining of formal and empirical methods in software engineering

1st dimension: Formal vs Empirical Can we experiment with formal systems? The dice ...

Formal Sciences Empirical Sciences deductive method definitions, axioms, theorems laws are established inductive method

• bservation, experimentation

laws are discovered modeling languages cryptographic algorithms neural networks inheritance laws in OO network traffic convergence of genetic algorithms frequency of “bugs” improving productivity with OO

6 Intertwining of formal and empirical methods in software engineering

2nd dimension: Natural vs Artificial

• Herbert Simon. The Sciences of the Artificial, 1969.
• Artifacts:

– physical machines – conceptual or social systems (information structures, methods, processes, organizations…) Analysis (pure science) Design (scientific engineering)

• bservation of natural phenomena

hypothesis formulation experimental confirmation / refutation identification of needs artifact construction artifact evaluation in search for truth (know) in search for utility (transform)

7 Intertwining of formal and empirical methods in software engineering

Combining both dimensions

Formal Empirical Analysis (Observation) formal analysis empirical analysis Design (Construction) formal design empirical design

8 Intertwining of formal and empirical methods in software engineering

3rd dimension: Things vs People

• Two very different types of indeterminism

– Inanimate beings: inherent vs improper indeterminism – Human beings: able to change behavior through cognition and free will

Inanimate products Human processes pieces of information inanimate entities (machinery and) humans involved developers and users physics, chemistry, biology economics, psychology, pedagogy

9 Intertwining of formal and empirical methods in software engineering

Plurality of research methods

Formal Empirical Analysis (Observation of nature) Entities formal analysis

• f natural entities

empirical analysis

• f natural entities

Processes formal analysis

• f natural processes

empirical analysis

• f natural processes

Design (Construction of artifacts) Products formal design

• f artificial products

empirical design

• f artificial products

Processes formal design

• f artificial processes

empirical design

• f artificial processes
• Human Factors in the design of processes to improve:

– Production (developers viewpoint) – Interaction (users viewpoint)

10 Intertwining of formal and empirical methods in software engineering

Difficulties

• My area of interest:

– Lack of adequate consideration of human factors in the justification of new artifacts that supposedly improve the software development process.

• Difficulties in the empirical validation of production artifacts:

– Are developers less demanding and more gullible than end users? – Fallacies against empirical methods (they are expensive...) – Distinguish experimentation and application.

• Too naïve experiments:

– Influence of culture and education. – People learn and change, they are not inanimate atoms. – Repeatability: we cannot isolate variables, or rewind history. – The Hawthorne effect.

11 Intertwining of formal and empirical methods in software engineering

Solutions?

• Increase and vary the sample

– Still, little predictive value in individual cases

• Once again: research subjects are rational beings

– Need of re-education and rational persuasion – Qualitative rather than quantitative arguments

• Who am I?

– Formal or empirical scientist? – Observation or construction scientist? – Things or people scientist?

12 Intertwining of formal and empirical methods in software engineering

Conclusions

• Don’t let you be measured with other’s rule:

– Particularly, empirical methods for inanimate beings. – Study and strengthen your own methods.

• Increase knowledge of methods in social sciences:

– How to isolate variables, how to repeat experiments. – Less ambitious experiments ( HCI).

• Adapting empirical methods is not enough.

“Not everything that can be counted counts, and not everything that counts can be counted.”

• Questions? Feel free! (ggenovaf@gmail.com)