SLIDE 1
INCREASING REPRESENTATIVENESS THROUGH THE USE OF PREDICTIVE MODELING AND TARGETED OUTREACH DC-AAPOR/WSS SUMMER CONFERENCE 2019
A.L. DJANGALI, J. JOSEPH-DAVID, & L. TROFIMOVICH
DC-AAPOR/WSS SUMMER CONFERENCE 2019
DC-AAPOR/WSS SUMMER CONFERENCE 2019 INCREASING REPRESENTATIVENESS - - PowerPoint PPT Presentation
DC-AAPOR/WSS SUMMER CONFERENCE 2019 INCREASING REPRESENTATIVENESS - - PowerPoint PPT Presentation
DC-AAPOR/WSS SUMMER CONFERENCE 2019 INCREASING REPRESENTATIVENESS THROUGH THE USE OF PREDICTIVE MODELING AND TARGETED OUTREACH A.L. DJANGALI, J. JOSEPH-DAVID, & L. TROFIMOVICH DC-AAPOR/WSS SUMMER CONFERENCE 2019 STUDY BACKGROUND
SLIDE 2
SLIDE 3
DC-AAPOR/WSS SUMMER CONFERENCE 2019
SURVEY METHOD
▸ Conducted from 2016 to 2018 ▸ Two overlapping censuses in 2016 (≈13,511 each) ▸ Sample in 2017 (n=17,198) ▸ Sample in 2018 (n=18,122) ▸ Mail-CATI sequential mode design ▸ Conducted in English and Spanish
Pre-note letter
- Toll-free RHL
1st Survey
- Separate
English & Spanish materials 2nd Survey
- Outbound
CATI with undeliverable mail cases Outbound CATI
- Transfer to
bilingual staff as needed
SLIDE 4
DC-AAPOR/WSS SUMMER CONFERENCE 2019
2016 RESPONSE ANALYSIS
After 2016 Census:
▸ Logistical regression model to identify which socio-demographic characteristics
would predict response
▸ African-Americans, Hispanics, 18-64 year olds: ▸ Less likely to respond overall ▸ More likely to complete by phone than mail ▸ More likely to complete over phone after making contact ▸ African-Americans and Hispanics disproportionally impacted by ESRD
Hubbard, F.A., Weiss, C., & Djangali, A.L. (2017). Modeling Contact and Cooperation in an Administrative Record Based Mail/CATI Survey. Paper presented at the 2017 American Association for Public Opinion Research (AAPOR) conference, New Orleans, Louisiana.
SLIDE 5
DC-AAPOR/WSS SUMMER CONFERENCE 2019
CHANGES IMPLEMENTED FOR 2017 & 2018
▸ Started CATI follow-up 2 weeks earlier for hard-to-reach (HTR) groups ▸ Used Census tract data to identify sample members more likely to be
Hispanic
▸ Identified previous CATI completers
Djangali, A.L., Joseph-David, J., & Hubbard, F.A. (2018). Methods for increasing response rates for Medicare beneficiaries. Poster presented at the 2018 American Association for Public Opinion Research (AAPOR) conference, Denver, Colorado. DOI: 10.13140/RG.2.2.31912.67845
SLIDE 6
DC-AAPOR/WSS SUMMER CONFERENCE 2019
CHANGES IMPLEMENTED FOR 2017 & 2018
▸ 2016 ▸ 2017
& 2018
Pre-note letter
- Toll-free RHL
1st Survey
- Separate
English & Spanish materials 2nd Survey
- Outbound
CATI with undeliverable mail cases Outbound CATI
- Transfer to
bilingual staff as needed
Pre-note letter
- Toll-free RHL
- Bilingual tickler
message
- Outbound calls
to previous CATI completers 1st Survey
- Bilingual tickler
message
- Double-stuffed
English & Spanish materials 2nd Survey
- Outbound
CATI with undeliverable mail cases
- Bilingual tickler
message
- Double-stuffed
materials
- Early
- utbound
calling for HTR Outbound CATI
- Transfer to
bilingual staff as needed
- Proactively
assign cases to bilingual staff
SLIDE 7
DC-AAPOR/WSS SUMMER CONFERENCE 2019
RESPONSE RATE COMPARISONS†
38% 28% 34% 28% 43% 34% 40% 36% 42% 31% 38% 41%
0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Overall 18-54 Black/Afr. Amer. Hispanic
2016 2017 2018
† AAPOR RR3
SLIDE 8
DC-AAPOR/WSS SUMMER CONFERENCE 2019
SIGNIFICANCE TESTING
▸ 2-sample t-tests of mean scores to control for sample size differences ▸ Significance (p<.05) across all subgroups
Demographics 2016 2017 2018 n Mean SE n Mean SE n Mean SE 18-54 3,588 28.3% 0.75% 4,627 33.9% 0.70% 4,601 30.9% 0.68% Black/Afr. Amer. 6,042 34.3% 0.61% 7,487 39.5% 0.57% 7,380 37.6% 0.56% Hispanic 909 27.9% 1.49% 1,019 35.6% 1.50% 1,079 40.6% 1.49% 18-54 Black/Afr. Amer. Hispanic 2016 vs 2017
* *** ***
2017 vs 2018
*** * ***
2016 vs 2018
* ** ***
* p<.05 ** p<.01 *** p<.001
SLIDE 9
DC-AAPOR/WSS SUMMER CONFERENCE 2019
FINDINGS & RECOMMENDATIONS
▸ Providing preferred language can help to increase representation among
Hispanic respondents
▸ Increasing outreach attempts to traditional HTR respondents helps to
increase response rates for those groups
▸ Small, data-driven changes can make a big impact
SLIDE 10
DC-AAPOR/WSS SUMMER CONFERENCE 2019