[PDF] - SoberIT Software Business and Engineering Institute Research PDF Document

SLIDE 1

1

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

T-0.050: Introduction to Postgraduate Studies in Computer Science

Research Approaches

Dr. Jyrki Kontio

Professor of Software Product Business

http://www.soberit.hut.fi/T-0.050/

T-0.050 Introduction to Postgraduate Studies in Computer Science

2

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

What Type of Problems Are You Solving

Building something

Constructive research

Observation to understand a phenomenon

Classification, taxonomies

Discovery of patterns to formulate theories

Qualitative studies

Evaluation and validation of theories

Controlled experiments

SLIDE 2

2

T-0.050 Introduction to Postgraduate Studies in Computer Science

3

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Research Approaches

Constructive research Case studies

Action research Ethnonography/observation Planned/controlled case study

Interviews

Focus groups Individual

Open ended Structured

Surveys Experiments

Empirical research

T-0.050 Introduction to Postgraduate Studies in Computer Science

4

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Constructive research Aims at producing novel solutions to practically and theoretically

relevant problems

Managerial problem solving through the construction of models,

diagrams, plans, organizations, etc.

Widely used in software engineering and computer science, rarely

in management and social sciences

The engineering research tradition

Steps:

1. Find a practically relevant problem
2. Obtain an understanding of the topic and the problem
3. Innovate, i.e., construct a solution idea

heuristic process theoretical justification and testing come later

4. Demonstrate that the solution works
5. Show theoretical connections and research contribution
6. Examine the scope of applicability

SLIDE 3

3

T-0.050 Introduction to Postgraduate Studies in Computer Science

5

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Steps of Empirical Study Design

1. Define research objectives

Scope, research questions, hypotheses

2. Experimental design

Design and plan the experiment: define variables, plan data collection, # of participants, # of runs, # of repeats

3. Piloting

Test the design and practical arrangements

4. Execution

Conduct the experiment, collect data

5. Analysis

Analyze the results, draw your conclusions, archive the data

6. Publishing

Document the study, publish the results

T-0.050 Introduction to Postgraduate Studies in Computer Science

6

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Purpose of Empirical Studies

(adapted from Basili 1996)

Descriptive

what is the phenomenon, what are its key attributes,

are there any patterns

Correlational

changes in some parameters is related to other

parameters

Cause-effect

the independent variables cause the change in

dependent variables

SLIDE 4

4

T-0.050 Introduction to Postgraduate Studies in Computer Science

7

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Types of Research Questions

(adapted from Basili’s GQM)

Characterize: to record or measure features or attributes in

the entity being studied.

Describe: Collect relevant information to document information

about an object.

Monitor: Use measurement data to track progress or status.
Understand: Recognize patterns or propose hypotheses, theories
r models.

Evaluate:

Assess: Evaluate against a well-defined standard or baseline.
Compare: Evaluate two or more alternatives against each other.
Validate: Evaluate feasibility.

Appraise: Evaluate effectiveness or usefulness

T-0.050 Introduction to Postgraduate Studies in Computer Science

8

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Types of Empirical Studies

(from Votta et al. 1995 & Basili, 1996)

Individual vs. group

Individual: easier replication, individual differences, mental not

bservable

Group: social phenomenon, many parameters

Novices vs. experts

Novices: easily available (e.g. students), not typical subjects Experts: hard to make available

In vivo vs. in vitro

In vivo: real context but little control In vitro: some control but many real parameters biased

Experiment vs. case study vs. observation

Experiment: controlled variables exist Case study: limited possibilities for control Observation: no treatment or controlled variables

SLIDE 5

5

T-0.050 Introduction to Postgraduate Studies in Computer Science

9

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Types of Studies

Controlled experiments

carefully planned studies, where (all) relevant

variables can be controlled

Quasi experiments

partially controlled studies

Case studies

little or no control of the situation, use of qualitative

research techniques

Surveys

no control of situation but large number of data

points

T-0.050 Introduction to Postgraduate Studies in Computer Science

10

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Case studies

Action research

Researcher is an active agent Deeper insight through personal involvement Researcher bias

Ethnonography/observation

Researcher spends time with the object of research

Planned/controlled case study

Some elements of the case are preplanned and

controlled

SLIDE 6

6

T-0.050 Introduction to Postgraduate Studies in Computer Science

11

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Interviews

Focus groups

1-n interviews with 3-12 participants An open-ended, structured discussion with a

representative group

A widely used qualitative market research method Emerged in 1950’s Used in marketing research Suited for exploration, collecting initial feedback

Dynamic setting

Individual

Open ended Structured

T-0.050 Introduction to Postgraduate Studies in Computer Science

12

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Surveys

A standard questionnaire is sent to large number

f subjects

Data is collected and analyzed statistically Trends, classifications, correlations can be

revealed

Examples:

Survey of the Finnish Software Product Industry

http://www.soberit.hut.fi/oskari/

SLIDE 7

7

T-0.050 Introduction to Postgraduate Studies in Computer Science

13

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Experiments

An empirical study where the researcher can

control independent variables measure dependent variables

Variation is intentionally created to test

hypotheses

Data is statistically analyzed to discover

correlations and causality

Examples:

Basili’s inspection studies Kontio’s risk mgmt experiments

T-0.050 Introduction to Postgraduate Studies in Computer Science

14

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Positioning of Research Approaches

Degree of control Type

f

data

Low High Qualitative Quantitative Experiment Survey

Postmortem data analysis

Case study Action research Controlled case study Replicated case studies Observation Interview

SLIDE 8

8

T-0.050 Introduction to Postgraduate Studies in Computer Science

15

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Research Approaches and Problems

Experiments

Surveys
Interviews
Case studies
Constructive

research Evaluate Discover Understand Build something

T-0.050 Introduction to Postgraduate Studies in Computer Science

16

SoberIT

Software Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY