Testing Email Invitations in Testing Email Invitations in a - - PowerPoint PPT Presentation

▶

Mar 23, 2023 255 likes •542 views

Testing Email Invitations in Testing Email Invitations in a Nonprobability Panel p y Elizabeth Nichols, Ryan King and Elizabeth Nichols, Ryan King and Jennifer Childs U S C U.S. Census Bureau B This presentation is released to inform

SLIDE 1

Testing Email Invitations in Testing Email Invitations in a Nonprobability Panel p y

Elizabeth Nichols, Ryan King and Elizabeth Nichols, Ryan King and Jennifer Childs U S C B U.S. Census Bureau

This presentation is released to inform interested parties of research and to encourage discussion. The views expressed are those of the authors and not necessarily those of the U S Census Bureau The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau.

SLIDE 2

Why is the U.S. Census Bureau interested in researching email notifications?

SLIDE 3

SLIDE 4

SLIDE 5

SLIDE 6

SLIDE 7

Motivation for nonprobability split-panel testing

Quantitative data:

Quantitative data:

Too many choices to test in large mandatory tests
Qualitative data:
Does opinion data match real response data?

p p

SLIDE 8

Previous research Previous research

S bj li

Subject line
Response deadline (Henderson, 2011 AAPOR)
Topic or no topic (Couper, 2008)
Avoid “survey” (Porter & Whitcomb, 2005)
Shorter the better & Avoid “Help” & “Reminder”

(Genroe – Marketing)

Email content (Klofstad, Boulianne, Basson, 2008)
Informing about reminders
Longer email content

SLIDE 9

Six studies in 2014 Six studies in 2014

h Jan. March May Aug. Oct. Dec. Subject line

x x x

line Email format

x x

Email content

x x

Survey Survey link type

x

Due date

x

Sent time

x x

SLIDE 10

Nonprobability sample Nonprobability sample

Highly

educated 35

>35 yrs
ld
White
White
Employed
Census
Census

SLIDE 11

Response rate (RR2) Response rate (RR2)

21% 20% 25% 14% 13% 15% 20% ( ) 13% 10% January (n=1000) March (n=1500) May (n=1500) 5% y ( ) 0% Overall

SLIDE 12

SLIDE 13

Subject lines Subject lines

J

January
Confidential 2014 Census Study
Respond to the 2014 Census Study

Respond to the 2014 Census Study

March
Confidential 2014 Census Study

y

Help us make the U.S. Census better, answer our survey
A message from the Director of the U.S. Census Bureau
May
Confidential 2014 Census Survey
10 min te U S Cens s S r e to Help o r Comm nit
10-minute U.S. Census Survey to Help your Community
U.S. Census Survey to Help your Community

SLIDE 14

Motivation for January Research team decision

Confidential 2014 Census Study
Short
Emphasizes confidentiality

R d t th 2014 C St d

Respond to the 2014 Census Study
Short
Call to action

SLIDE 15

January open rate January open rate

40% 30% 35% 40% 20% 25% 30% fid i l 10% 15% 0% Confidential Respond 0% 5%

Source: U.S. Census Bureau January Small-scale Nonprobability Panel Research y p y

SLIDE 16

January response rate January response rate

21% 20% 20% 25% 15% 10% 5% 0% Confidential Respond

Source: U.S. Census Bureau January Small-scale Nonprobability Panel Research

SLIDE 17

Survey feedback from January Survey feedback from January

SLIDE 18

Results of opinion question Results of opinion question

38.4% Help us make the U.S. Census better, answer our survey 32.0% 10 minutes to improve the U.S. Census and help your community 31.6% Answer the U.S. Census– Help your community 30.1% Important Census Study 26 7% Improve the U S Census Study 26.7% Improve the U.S. Census Study 13.6% Mandatory Census Study 13.1% Your Civic Duty – Answer the U.S. Census! 6.8% A message from John Thompson, Director of the U.S. Census Bureau 6.3% A message from the Director of the U.S. Census Bureau

SLIDE 19

Motivation for March Motivation for March

Confidential 2014 Census Study
Kept January “winner”

p y

Help us make the U.S. Census better, answer
ur survey
ur survey
Winner of January opinion question
A message from the Director of the U.S.

Census Bureau Census Bureau

Loser of January opinion question

SLIDE 20

March open rate March open rate

45% 35% 40% 45% 20% 25% 30% Confidential 10% 15% 20% Help A message 0% 5%

Source: U.S. Census Bureau March Small-scale Nonprobability Panel Research Source: U.S. Census Bureau March Small scale Nonprobability Panel Research

SLIDE 21

March response rate March response rate

20% 25% 17% 11% 15% 15% 11% 10% 0% 5% 0% Confidential Help A message

Source: U.S. Census Bureau March Small-scale Nonprobability Panel Research

SLIDE 22

Motivation for May Motivation for May

f d l

Confidential 2014 Census Survey
Kept with the March winner
10-minute U.S. Census Survey to Help your

Community

Email provider suggestion that numbers do well
Emphasize U S Census
Emphasize U.S. Census
Data use: Helping a community

U S C S t H l C it

U.S. Census Survey to Help your Community
Like previous panel, but no number

SLIDE 23

May open rate May open rate

35% 40% 45% 25% 30% 35% Confidential 10% 15% 20% Confidential 10-min Help 0% 5% 10% 12-May 13-May 14-May 15-May 16-May 17-May 18-May 19-May 20-May 21-May 22-May 23-May 24-May 25-May 26-May 27-May

Source: U.S. Census Bureau May Small-scale Nonprobability Panel Research

SLIDE 24

May response rate May response rate

20% 25% 12% 15% 15% 12% 11% 10% 5% 0% Confidential 10-minute Help