SoberIT Software Business and Engineering Institute T-0.050: Introduction to Postgraduate Studies in Computer Science Research Approaches Dr. Jyrki Kontio Professor of Software Product Business http://www.soberit.hut.fi/T-0.050/ 1 HELSINKI UNIVERSITY OF TECHNOLOGY SoberIT Software Business and Engineering Institute What Type of Problems Are You Solving � Building something � Constructive research � Observation to understand a phenomenon � Classification, taxonomies � Discovery of patterns to formulate theories � Qualitative studies � Evaluation and validation of theories � Controlled experiments 2 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science 1
SoberIT Software Business and Engineering Institute Research Approaches � Constructive research � Case studies � Action research � Ethnonography/observation � Planned/controlled case study � Interviews � Focus groups Empirical research � Individual � Open ended � Structured � Surveys � Experiments 3 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science SoberIT Software Business and Engineering Institute Constructive research � Aims at producing novel solutions to practically and theoretically relevant problems � Managerial problem solving through the construction of models, diagrams, plans, organizations, etc. � Widely used in software engineering and computer science, rarely in management and social sciences � The engineering research tradition � Steps: 1. Find a practically relevant problem 2. Obtain an understanding of the topic and the problem 3. Innovate, i.e., construct a solution idea � heuristic process � theoretical justification and testing come later 4. Demonstrate that the solution works 5. Show theoretical connections and research contribution 6. Examine the scope of applicability 4 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science 2
SoberIT Software Business and Engineering Institute Steps of Empirical Study Design 1. Define research objectives � Scope, research questions, hypotheses 2. Experimental design � Design and plan the experiment: define variables, plan data collection, # of participants, # of runs, # of repeats 3. Piloting � Test the design and practical arrangements 4. Execution � Conduct the experiment, collect data 5. Analysis � Analyze the results, draw your conclusions, archive the data 6. Publishing � Document the study, publish the results 5 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science SoberIT Software Business and Engineering Institute Purpose of Empirical Studies (adapted from Basili 1996) � Descriptive � what is the phenomenon, what are its key attributes, are there any patterns � Correlational � changes in some parameters is related to other parameters � Cause-effect � the independent variables cause the change in dependent variables 6 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science 3
SoberIT Software Business and Engineering Institute Types of Research Questions (adapted from Basili’s GQM) � Characterize : to record or measure features or attributes in the entity being studied. • Describe: Collect relevant information to document information about an object. • Monitor : Use measurement data to track progress or status. • Understand: Recognize patterns or propose hypotheses, theories or models. � Evaluate : • Assess : Evaluate against a well-defined standard or baseline. • Compare : Evaluate two or more alternatives against each other. • Validate : Evaluate feasibility. � Appraise : Evaluate effectiveness or usefulness 7 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science SoberIT Software Business and Engineering Institute Types of Empirical Studies (from Votta et al. 1995 & Basili, 1996) � Individual vs. group � Individual: easier replication, individual differences, mental not observable � Group: social phenomenon, many parameters � Novices vs. experts � Novices: easily available (e.g. students), not typical subjects � Experts: hard to make available � In vivo vs. in vitro � In vivo: real context but little control � In vitro: some control but many real parameters biased � Experiment vs. case study vs. observation � Experiment: controlled variables exist � Case study: limited possibilities for control � Observation: no treatment or controlled variables 8 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science 4
SoberIT Software Business and Engineering Institute Types of Studies � Controlled experiments � carefully planned studies, where (all) relevant variables can be controlled � Quasi experiments � partially controlled studies � Case studies � little or no control of the situation, use of qualitative research techniques � Surveys � no control of situation but large number of data points 9 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science SoberIT Software Business and Engineering Institute Case studies � Action research � Researcher is an active agent � Deeper insight through personal involvement � Researcher bias � Ethnonography/observation � Researcher spends time with the object of research � Planned/controlled case study � Some elements of the case are preplanned and controlled 10 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science 5
SoberIT Software Business and Engineering Institute Interviews � Focus groups � 1-n interviews with 3-12 participants � An open-ended, structured discussion with a representative group � A widely used qualitative market research method � Emerged in 1950’s � Used in marketing research � Suited for exploration, collecting initial feedback Dynamic setting � Individual � Open ended � Structured 11 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science SoberIT Software Business and Engineering Institute Surveys � A standard questionnaire is sent to large number of subjects � Data is collected and analyzed statistically � Trends, classifications, correlations can be revealed � Examples: � Survey of the Finnish Software Product Industry http://www.soberit.hut.fi/oskari/ 12 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science 6
SoberIT Software Business and Engineering Institute Experiments � An empirical study where the researcher can � control independent variables � measure dependent variables � Variation is intentionally created to test hypotheses � Data is statistically analyzed to discover correlations and causality � Examples: � Basili’s inspection studies � Kontio’s risk mgmt experiments 13 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science SoberIT Software Business and Engineering Institute Positioning of Research Approaches Quantitative Postmortem Survey Experiment data analysis Type Interview of data Observation Controlled Replicated Case study case study case studies Action research Qualitative Degree of control Low High 14 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science 7
SoberIT Software Business and Engineering Institute Research Approaches and Problems Build Understand Discover Evaluate something Constructive � � � � research Case studies � � � � � � Interviews � � � � � Surveys � � � Experiments � � 15 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science SoberIT Software Business and Engineering Institute Empirical Research Resources � ISERN: http://www.iese.fhg.de/ISERN/ International Software Engineering Research Network � Network of leading empirical researcher in software engineering � Annual meetings, tutorials � ISESE: http://www.isese.org/ International Symposium on Empirical Software Engineering � A conference focusing on empirical software engineering research � Empirical Software Engineering: An International Journal http://www.kluweronline.com/issn/1382-3256 � Peer reviewed journal � International Software Metrics Symposium http://swmetrics.org/ � Conference focusing on software metrics based research 16 HELSINKI UNIVERSITY OF TECHNOLOGY T-0.050 Introduction to Postgraduate Studies in Computer Science 8
Recommend
More recommend
