Integrating the Web Mode in Marc Plate Romana Riegler the Austrian - - PowerPoint PPT Presentation

www.statistik.at We provide information

Integrating the Web Mode in the Austrian Household Budget Survey 2014/15

First Experiences with a New IT System for Data Collection in a Mixed Mode Design Marc Plate Romana Riegler Statistics Austria Directorate Social Statistics Brussels 12 Mar 2015

www.statistik.at slide 2 | 12 March 2015

Introduction

• „EISNeu“ (SurveyInfrastructureNew)
• A new electronic survey infrastructure tool
• Ongoing in-house development since 2013, integrates data

collection and field work management of mixed mode surveys: CAWI, CAPI, CATI

• Austrian Household Budget Survey 2014/15
• Field work: Oct. 2014 – Nov. 2015
• Probability sample of addresses from population register
• Voluntary, 50 € voucher as incentive
• Modes: CAWI (first time ever in Austrian HBS), PAPI/CAPI
• All presented findings are preliminary and rely on a limited

number of observations from the first months of field work

www.statistik.at slide 3 | 12 March 2015

Participation procedure & different modes

• Concerns beforehand:
• Are households even willing to participate via CAWI (CAWI response

rate)?

• Will CAWI result in high break-off rates (no personal contact)?
• Importance of the notification letter / process (ongoing experimentation)

Notification letter CAWI self-registration, if not: automatic mode switch to CAPI/PAPI (visit by interviewer) 3 phases of the HBS:

‚ Questionnaires part I: household/personal ‚ 14 days of diary keeping (reporting period) ‚ Questionnaires part II: household/personal

www.statistik.at slide 4 | 12 March 2015

Reporting periods 5-10 Gross sample=2853 households (~476 per reporting period)

CAWI complete sampling losses, not quality neutral sampling losses, quality neutral2 never started break-off 5,9% 91,3% 2,7% ? Paper/Pencil (PAPI) complete sampling losses, not quality neutral sampling losses, quality neutral2 not known yet refusal

• ther1

break-off 26,2% 39,2% 12,4% 0,9% 3,1% 18,1%

The households not registered for CAWI are switched to PAPI mode (= 100 %)

1 E.g. language problems, no-one reached at housing unit, etc. 2 E.g. housing unit is vacant, not a private household, etc.

TOTAL complete sampling losses, not quality neutral sampling losses, quality neutral2 not known yet refusal

• ther1

break-off 29,9% 35,8% 11,4% 3,5% 2,9% 16,5%

before new letter : 1,7 % before new letter: 1,1 %

www.statistik.at slide 5 | 12 March 2015

Differences in Sample Population

The CAWI Sample, as compared to the PAPI Sample, is made up of: 30% MORE Employees 65% LESS Retirees 25% MORE Households with Children 35% LESS Immigrants 60% MORE People under 45 years 65% LESS People above 60 years 270% MORE People with academic degree 60% LESS People without qualification for university entrance Even when controlled for age, education still makes much difference

www.statistik.at slide 6 | 12 March 2015

Differences in Key-Statistics by mode

Expenditures for… CAWI

Above Median

PAPI

Above Median

CAWI

No Expend.

PAPI

No Expend.

Private Health Insurance 51% 49% 50% 58% Private Life Insurance 58% 42% 56% 68% Dental Health Care 45% 52% 67% 81% Electricity 52% 48% 47% 52%

www.statistik.at slide 7 | 12 March 2015

Selection Effect or Mode Effect?

Private Life Insurance . Expenditures Highest Education CAWI

Above Median

PAPI

Above Median

CAWI

No expend.

PAPI

No expend.

Lower secondary education

• 31%
• 82%

Higher secondary education, without University qualification 53% 49% 39% 61% Higher secondary education, with University qualification 66% 50% 38% 53% Tertiary degree 69% 63% 46% 50%

www.statistik.at slide 8 | 12 March 2015

CAWI: Cost effectiveness and high data quality?

• Does self-administration via CAWI lead to excessive use of „don‘t

know / don‘t want to answer“ and thus a high rate of missing values? HBS contains complex (housing) and possibly sensitive (income) questions

Missing values (“don’t know” / refusal) for… PAPI CAWI Total monthly housing cost (rented flat/house) 4% 5% Monthly rent (rented flat/house) 36% 30% Monthly utility costs (rented flat/house) 31% 26% Electricity cost 5% 9% Household income (either exact or in categories*) 3% 9%

*Income given only as category (of all who gave an income answer) 27% 12%

www.statistik.at slide 9 | 12 March 2015

CAWI: Cost effectiveness and high data quality?

Does self-administration via CAWI promote „sloppyness“ (i.e. speeding through questionnaire)? Answering time for Q. Total housing cost (seconds) Cumulative Percent of CAWI Respondents

www.statistik.at slide 10 | 12 March 2015

Summary & Outlook

• Start of 2nd Mode CAPI (instead of PAPI) by April is already highly

anticipated and will provide further insights

• For implementing CAWI as a new mode, surveys need even

stronger to be methodologically accompanied:

• Further explore, quantify and find ways to correct for the „pure“

mode effects.

• Find useful interpretation of CAWI/CAPI/CATI paradata to create

indicators for data quality.

• Triangulation of methods (additional use of qualitative methods)

needed to fully understand and compensate measurement errors.

www.statistik.at slide 11 | 12 March 2015 Please address queries to: Marc Plate Romana Riegler Contact information: Guglgasse 13, 1110 Vienna phone: +43 (1) 71128-7972 Marc.Plate@statistik.gv.at Romana.Riegler@statistik.gv.at