Zooming in and zooming out: Mixing qualitative and quantitative - PowerPoint PPT Presentation

“Zooming in and zooming out”: Mixing qualitative and quantitative approaches to the analysis of textual data Graham Hughes QUIC Conference 2011 – University of Surrey 4 th May 2011 n.hughes@surrey.ac.uk

Focus of this session • Analysis of responses to open ended questions asked in survey situations. • An important example of mixing qualitative and quantitative methods • Relevant, because these types of questions are being used more often with internet surveys

Analyse by case or by question? • Case by Case analysis is routine in qualitative work. • Question by Question analysis is routine in quantitative work. Case Perspective Variable Perspective Header: Respondent Name/ID - (e.g. Case 1234) Header: Question Name/ID - (e.g. Question 31) Question 27 Response by Case 1234 to Q27 Case 1000 Response by Case 1000 to Q31 Question 31 Response by Case 1234 to Q31 Case 1005 Response by Case 1005 to Q31 Question 32 Response by Case 1234 to Q32 Case 1234 Response by Case 1234 to Q31 Question 44 Response by Case 1234 to Q44 Case 1235 Response by Case 1235 to Q31 etc. etc.

Data collection methods and OEQ data quality Interview Type Life History Survey Interview Interview Flexibility to Respondent Self- Online or paper Face to completion Face Data Collection Recorded & Interviewer transcribed mediation Questionnaire Design OEQs grouped OEQs spread Strong theme or at start varied topics around

Data used in these examples • Professional survey of flooding experiences • Households in England • Events in 2000 • 1,250 respondents • Face to Face using CAPI • Mainly closed questions, but 8 OEQs • Interviewers typed responses in real time

Who might be asked to analyse it? • Quantitative survey analyst: – Because you are analysing the rest of the data • Qualitative research analyst: – Because it’s qualitative and that’s what you do • Online research analyst: – Because it is easier to phrase open ended questions

Why use CAQDAS? • ATLAS.ti, NVivo, MAXqda, QDA Miner • Speed • Accuracy • Transparency • Replicable • Tools designed for the task

Four main phases of work • Data preparation & import into CAQDAS • Development & application of themes (coding) • Exploration of relationships between codes, respondent attributes, and the texts • Export of selected coding data to SPSS for further analysis (“Quantitising”)

NVivo 8 Main Screen

MAXqda Main Screen

Developing Themes • A range of possibilities from fully manual to fully automatic • Our own prejudices and preconceptions may hold us back • Range of data quantities and qualities should be considered • Inductive approach based on the data

ATLAS.ti v6 – Word Frequency Table

NVivo – Word Frequency Tag Cloud

QDA Miner/Wordstat – Phrase Finder

Manual coding in MAXqda

Autocoding in MAXqda

Fuzzy Search – Query by Example in QDA Miner

Survey Import in NVivo9

Survey Import in ATLAS.ti v6

Reporting and Checking Codes • “X% of respondents mentioned Y” – Reference coding - (“signposts”) – Substitution coding – (“exact”) • So, check for errors – Code definition – Incorrect use – Omissions • And be prepared to merge, split or change code applications

Checking omissions in MAXqda

• But also, look for difference. • Make rarity a virtue! • “My personal experience was being told my family had been evacuated when they had not and not being allowed access to my own road when my family were trapped inside” (R:12284 – male, aged 35/44, working full time, house was flooded) – response to a question about better ways to warn people.

MAXqda – Attributes Table

Crosstabs in MAXqda

NVivo – Matrix Coding Query

QDA Miner – Coding by Variable Heatmap

Data collection methods and CAQDAS programs Interview Type Life History Survey Interview Interview Flexibility to Respondent Self- Online or paper Face to completion Face Data Collection Recorded & Interviewer transcribed mediation Questionnaire Design OEQs grouped OEQs spread Strong theme or at start varied topics around