Designing a Multipurpose Designing a Multipurpose Longitudinal - - PowerPoint PPT Presentation

Designing a Multipurpose Designing a Multipurpose Longitudinal Incentives Experiment for the SIPP

Ashley Westra U.S. Census Bureau* U.S. Census Bureau

DC-AAPOR & WSS: Summer Conference Preview/Review 2015 August 3 2015 August 3, 2015

*Any views expressed are those of the author and not necessarily those

• f the U.S. Census Bureau.

Overview Overview

• Background/Previous SIPP Incentive

Experiments p

• SIPP 2014 Experiment

D i

• Design
• Wave 1 Results
• Wave 2 Goals
• Future Plans

Future Plans

The SIPP Survey The SIPP Survey

Th S f I d P

• The Survey of Income and Program

Participation (SIPP) is a demographic longitudinal survey that collects data and longitudinal survey that collects data and measures change for many topics, including:

• Economic Well-being

Eco o c We be g

• Family Dynamics
• Education
• Assets
• Health Insurance
• Childcare
• Food Security

The SIPP Survey Design The SIPP Survey Design

• Previous Panels (1996, 2001, 2004, 2008)
• 3-5 year panels

y p

• Conducted in waves, each 4 months long
• 4 equally sized rotation groups
• 4 equally sized rotation groups
• 2014 Panel
• 4 year panel
• Conducted in waves, each 1 year long

Conducted in waves, each 1 year long

• No rotation groups

Previous SIPP Incentive Experiments Previous SIPP Incentive Experiments

• Since the 1996 Panel, SIPP has conducted

several incentive tests of different types. yp

• Designed to test the effect of monetary

incentives on overall response rates incentives on overall response rates.

Previous SIPP Incentive Experiments Previous SIPP Incentive Experiments

• Tested both conditional and unconditional
• Tested both conditional and unconditional

incentives.

• Tested both random assignment as well as

discretionary incentives discretionary incentives

• Experimented with the monetary amount

f th i ti ith $10 $20 d $40

• f the incentive, with $10, $20, and $40

being the typical choices.

Results of Previous Experiments Results of Previous Experiments

1996 P l 1996 Panel

• $20 (but not $10) unconditional incentives

$20 (but not $10) unconditional incentives were effective in reducing household nonresponse in Wave 1 and this effect nonresponse in Wave 1, and this effect remained in later waves.

Results of Previous Experiments Results of Previous Experiments

2001 Panel

• For 7 out of 9 waves $40 conditional
• For 7 out of 9 waves, $40 conditional

discretionary incentives increased response rates.

Results of Previous Experiments Results of Previous Experiments

2004 Panel

H h ld th t i $40 di ti

• Households that receive $40 discretionary

incentives are more likely to receive them in later waves.

Results of Previous Experiments Results of Previous Experiments

2008 Panel

• The Wave 1 $20 unconditional incentive

$ effectively improved response rates in Waves 1-3 by 1.1-1.4% compared to the control.

• The discretionary $40 conditional incentive

(in any wave) had an effect in Waves 7-9, i i t b 1 6 3 1% improving response rates by 1.6-3.1% compared to the control.

2014 Panel Experimental Design 2014 Panel – Experimental Design

• Households randomly

Group Wave 1 Wave 2 1 $0 $0

Households randomly put into 1 of 4 equally sized groups (≈ 13,000

1 $0 $0 2 $0 $40

households).

• Conditional incentives

di t ib t d d bit

3 $20 $0 4 $40 (a) $40

are distributed as debit cards by NPC.

• Testing the use of a

4 $40 (a) $40 (b) $0

• Testing the use of a

propensity model to assign incentives in later g waves.

2014 Panel Wave 1 Results 2014 Panel Wave 1 – Results

• Households were

randomly assigned to $0,

Group Wave 1 1 $0

y g $20, or $40 conditional incentives

1 $0 2 $0

incentives.

• $20 increased the response

rate by 1 2%

3 $20 4 $40

rate by 1.2%

• $40 increased the response

b 3 5%

4 $40

rate by 3.5%

2014 Panel Wave 1 Results 2014 Panel Wave 1 – Results

While

Poverty Stratum Response Rates Distribution of Interviewed

While incentives affected

Incentive Group Low Income Non-Low Income Incentive Group Low Income Non-Low Income $0 71% 66% $0 38% 62% $20 73% 67% $20 39% 61% affected

response rates, they did not

$20 73% 67% $20 39% 61% $40 76% 68% $40 39% 61% ALL 72% 67% ALL 39% 61% they did not

affect the distribution of

Urban/Rural Response Rates Distribution of Interviewed Incentive Incentive

distribution of the interviewed.

Incentive Group Urban Rural Incentive Group Urban Rural $0 67% 71% $0 80% 20% $20 68% 72% $20 81% 19% interviewed. $40 70% 74% $40 80% 20% ALL 68% 72% ALL 80% 20%

2014 Panel Wave 2 Model 2014 Panel Wave 2 – Model

• Create a logistic regression model predicting the probability

f i i h h ld h i i

• f response given certain household characteristics
• Census Region
• Age of Householder

Group Wave 1 Wave 2

g

• Gender
• Race
• Hispanic Origin

1 $0 $0 2 $0 $40

p g

• Education
• Marital Status
• Income

3 $20 $0 4 $40 (a) $40 $

Income

• Work Status
• Assign incentives based on the predicted probabilities to

improve coverage

(b) $0

improve coverage.

2014 Panel Wave 2 Tests 2014 Panel Wave 2 – Tests

• Effect of randomly

Effect of randomly assigned incentives on response rates

Group Wave 1 Wave 2 1 $0 $0

response rates

• Does the Wave 1 incentive

ff t t W

2 $0 $40 3 $20 $0

effect carry-over to Wave 2?

3 $20 $0 4 $40 (a) $40

• 4(b) vs. 1
• 3 vs. 1

(b) $0

2014 Panel Wave 2 Tests 2014 Panel Wave 2 – Tests

• Effect of randomly

Effect of randomly assigned incentives on response rates

Group Wave 1 Wave 2 1 $0 $0

response rates

• What is the effect of

d li t i ti ?

2 $0 $40 3 $20 $0

duplicate incentives?

• 4(a) vs. 1

3 $20 $0 4 $40 (a) $40

• 4(a) vs. 4(b)

(b) $0

2014 Panel Wave 2 Tests 2014 Panel Wave 2 – Tests

• Effect of randomly

Effect of randomly assigned incentives on response rates

Group Wave 1 Wave 2 1 $0 $0

response rates

• What is the effect of a later

i ti ?

2 $0 $40 3 $20 $0

incentive?

• 2 vs. 1

3 $20 $0 4 $40 (a) $40 (b) $0

2014 Panel Wave 2 Tests 2014 Panel Wave 2 – Tests

• Effectiveness of the propensity

model in assigning incentives Gro p Wa e 1 Wa e 2 model in assigning incentives, conditional on Wave 1 incentives. Group Wave 1 Wave 2 1 $0 $0

• For a given percentage of

households assigned a model- based incentive compare the 2 $0 $40 3 $20 $0 based incentive, compare the distributions and response rates of: 3 $20 $0 4 $40 (a) $40

• 2 vs. 1
• 4(a) vs. 4(b)

(b) $0

2014 Panel Wave 3 Plans 2014 Panel Wave 3 Plans

• Depending on the results of Wave 2, we

may decide to implement the propensity y p p p y model.

• We are concerned that the group of
• We are concerned that the group of

households that received $40 incentives f i ill for two consecutive waves will expect them again. g

Thank you!

Contacts: Ashley.M.Westra@census.gov Mahdi S Sundukchi@census gov Mahdi.S.Sundukchi@census.gov