Recruit itment Messagin ing: From analy lysis to desig ign - - PowerPoint PPT Presentation

▶
recruit itment messagin ing from analy lysis to desig ign
SMART_READER_LITE
LIVE PREVIEW

Recruit itment Messagin ing: From analy lysis to desig ign - - PowerPoint PPT Presentation

Jan 06, 2023 122 likes •586 views

Redesignin ing th the Americ ican Community Survey Recruit itment Messagin ing: From analy lysis to desig ign Jonathan Schreiner American Community Survey Office, U.S. Census Bureau DC-AAPOR/WSS Review-Preview Summer Conference June 12,

slide-1
SLIDE 1

Redesignin ing th the Americ ican Community Survey Recruit itment Messagin ing: From analy lysis to desig ign

Jonathan Schreiner American Community Survey Office, U.S. Census Bureau

DC-AAPOR/WSS Review-Preview Summer Conference June 12, 2019 – Washington, D.C.

This presentation is released to inform interested parties of ongoing research and to encourage discussion

  • f work in progress. Any views expressed on statistical, methodological, operational, or technical issues

are those of the authors and not necessarily those of the U.S. Census Bureau.

slide-2
SLIDE 2

ACS Self-Response Mail Contact Strategy

2

All Mailable Sample Addresses Nonrespondents Nonrespondents

First Mailing Second Mailing Third Mailing Fourth Mailing Fifth Mailing

Letter Instruction Card (internet) FAQ Brochure Multilingual Brochure Reminder Letter Paper Questionnaire Letter Instruction Card (choice) FAQ Brochure Return Envelope Reminder Postcard Reminder Letter

slide-3
SLIDE 3

Holistic Design

  • “Research has been successful in identifying specific design

features that tend to have larger versus smaller effects [on response rates], but generally leaves unanswered how all elements of a design fit together.”

(Dillman, Christian and Smyth 2014)

3

slide-4
SLIDE 4

What if we start from “scratch”?

4

All Mailable Sample Addresses Nonrespondents Nonrespondents

First Mailing Second Mailing Third Mailing Fourth Mailing Fifth Mailing

?

Paper Questionnaire

7 14 4 18

? ? ? ?

slide-5
SLIDE 5

Presentation Outline

Step 1: Develop a “Strategic Framework” for messaging

5

Step 2: Evaluate current materials Step 3: Design new materials Overview of next steps

slide-6
SLIDE 6

6

2 4 3 1

Step 1: Develop a Strategic Framework for messaging

6

slide-7
SLIDE 7

Step 1: Develop a Strategic Framework for messaging

7

slide-8
SLIDE 8

Step 1: : Develop a Strategic Framework for messaging

8

What do we know about the ACS audience? What do the experts think

  • f ACS

messaging? What does the literature tell us about messaging?

STRATEGIC FRAMEWORK

slide-9
SLIDE 9

Four Lessons Learned

1) Limit volume of messaging

9

Oliver, Heimel & Schreiner (2017) “Strategic Framework for Messaging in the ACS Mail Materials”

slide-10
SLIDE 10

Four Lessons Learned

1) Limit volume of messaging 2) Follow Plain Language guidelines

10

Oliver, Heimel & Schreiner (2017) “Strategic Framework for Messaging in the ACS Mail Materials”

slide-11
SLIDE 11

Four Lessons Learned

1) Limit volume of messaging 2) Follow Plain Language guidelines 3) Each mailing needs to have a clear purpose and highlight different reasons to participate

11

Oliver, Heimel & Schreiner (2017) “Strategic Framework for Messaging in the ACS Mail Materials”

slide-12
SLIDE 12

Four Lessons Learned

1) Limit volume of messaging 2) Follow Plain Language guidelines 3) Each mailing needs to have a clear purpose and highlight different reasons to participate 4) Leverage the Census Bureau brand

12

Oliver, Heimel & Schreiner (2017) “Strategic Framework for Messaging in the ACS Mail Materials”

slide-13
SLIDE 13

Step 2: : U Use the Strategic Framework report to Evaluate the Current ACS Materials

13

slide-14
SLIDE 14

Lesson 1: Limit volume of messaging

Mailing 1: 129 messages in these 5 mail pieces

14

slide-15
SLIDE 15

Lesson 2: : Follow Plain Language guidelines

15

Mail Piece FRE Score Interpretation Grade Level Mailing 1 Letter “fairly difficult to read” 10 th -12th Multilingual Brochure “difficult to read” College FAQ Brochure “difficult to read” College

Flesch Reading Ease Scores of ACS Mail Materials

Source: U.S. Census Bureau, American Community Survey Office

Schreiner, Oliver & Poehler (forthcoming)

slide-16
SLIDE 16

16

Lesson 3: : Each mailing should have a clear purpose and highlight different reasons to participate

Repetition across mail materials

1st mailing letter

2nd mailing letter 3rd mailing letter

slide-17
SLIDE 17

4th mailing post card 5th mailing letter

17

Lesson 3: : Each mailing should have a clear purpose and highlight different reasons to participate

Repetition across mail materials

slide-18
SLIDE 18

Lesson 4: : Leverage the Census Bureau brand

18

Graphic Used Envelopes X Instruction Cards X FAQ Brochure X Multilingual Brochure X Letters X X Survey Form X Post Card X

slide-19
SLIDE 19

Four sponsors? 1

1) UNITED STATES DEPARTMENT OF COMMERCE 2) Economics and Statistics

Administration

3) U.S. Census Bureau 4 3 2 4) OFFICE OF THE DIRECTOR

19

Lesson 4: : Leverage the Census Bureau brand

slide-20
SLIDE 20

20

Lesson 4: : Leverage the Census Bureau brand

slide-21
SLIDE 21

Step 3: : Apply Findings to Design New Materials

To address the four lessons learned, ACS is testing two main changes the messaging strategy for our letters.

21

slide-22
SLIDE 22

Change 1: Apply a Strategic Messaging Plan

  • Fewer, more focused messages per mailing
  • Increased comprehension
  • Less repetition across mailings
  • More appeals to respond

22

First Mailing Second Mailing Third Mailing

Trust Benefits Burden Reduction

slide-23
SLIDE 23

23

Change 2: Redesign ACS letters

  • New Design
  • Reduce text
  • Focus messaging
  • Add white space
  • Use plain language
  • Improved branding
slide-24
SLIDE 24

Version 1

  • Clear branding
  • Use color and to draw

attention

  • F-pattern headings to ease

readability, call attention

24

slide-25
SLIDE 25

25

Version 2

  • Use color and icons to draw

attention

  • Tag line to add additional

message and branding

slide-26
SLIDE 26

Version 3

  • Color “Side bar” provides

additional information

  • Text reduced and focused

26

slide-27
SLIDE 27

Consistent design across mailings

27

Letter 1 Letter 2 Letter 3

slide-28
SLIDE 28

Next xt Steps for the Strategic Framework

28

Develop Strategic Framework Develop revisions to ACS mail materials Field Testing Analyze results and make recommendations STEP 1  STEP 2  STEP 3  STEP 5 STEP 6 Inventory and Analyze Current Messaging Qualitative evaluation STEP 4

In progress

slide-29
SLIDE 29

Thank you!

Jon Schreiner jonathan.schreiner@census.gov (301) 763 9286

  • Oliver, Heimel & Schreiner (2017) “Strategic Framework for Messaging in the ACS

Mail Materials”https://www.census.gov/library/working-papers/2017/acs/2017_Oliver_01.html

  • Dillman, Christian & Smyth (2014) “The Tailored Design Method”
  • Schreiner, Oliver & Poehler (forthcoming) “Assessment of Messaging in the 2018

American Community Survey Mail Contact Materials”

29

slide-30
SLIDE 30

30

slide-31
SLIDE 31

Appendices

31

slide-32
SLIDE 32

Appendix A –

  • More complete list of lessons learned by SF area.

32

slide-33
SLIDE 33

What do we know about the ACS audience

  • Some people distrust the government
  • ACS is unknown, but many know the Census Bureau
  • Messages that convey community-level benefits are viewed favorably
  • Messages about the confidentiality do not have high believability

33

This slide summarizes results from the following Decennial Census and ACS mindset and motivation studies: Segmentation and Mindset Study (2007), Census Barriers, Attitudes, and Motivators Surveys I (2008), Census Barriers, Attitudes, and Motivators Surveys II (2010), Collaborative research with Reingold, Inc. (2013-2014), and Collaborative research with Gallup (2014).

slide-34
SLIDE 34

What do the Experts think of f ACS Messaging?

2016 NAS CNSTAT Workshop & Social and Behavioral Sciences Team

  • Each contact needs a clear purpose
  • Messaging between mailings is distinct, but mutually supportive
  • Community-level benefits
  • Emphasize culturally relevant messages of empowerment
  • Attach ACS to the Census Bureau
  • Social norms

34

slide-35
SLIDE 35

What does the Lit iterature tell us about Messaging?

  • Multiple mail contacts should feel like a single, continuous

conversation between the Census Bureau and the ACS recipient

  • Each contact should highlight different reasons to participate
  • Limit the number of messages per contact
  • Establish trust early with messages that are believable
  • Convey the benefits of ACS data to the recipient’s community
  • Reduce respondent’s perceptions of burden
  • Use plain language to communicate

35

This slide summarizes literature from a number of fields, including Communications, Behavioral Sciences, Marketing, Sociology, Survey Methodology, Psychology, and Plain Language.

slide-36
SLIDE 36

Appendix B

  • Current Materials Coding Methodology

36

slide-37
SLIDE 37

Coding Methodology

  • Developed a list of potential messages in ACS materials
  • Strategic Framework & Current Materials
  • 74 codes for messaging in 4 categories:
  • Trust
  • Benefits
  • Reduce Burden
  • Instructions & Information and Required Elements

37

Codebook Example

Code Theme 1.0 Trust: Recipients must believe that the survey is legitimate and that the messages are truthful 1.2 Confidentiality/Data Security 1.2.1  By law, Census must protect your data 1.2.2  Census employees face fines and prison if they violate law

slide-38
SLIDE 38

8

4.5 (Required information) 1.1.1 (Sponsorship) 4.1.1 (Part of a random sample) 2.3.7 (Highlight importance) 2.2.2 (National level benefit) 4.4.1 (Call to action)

A message from the Director, U.S. Census Bureau ... Your household has been randomly selected to complete a very important national survey, the American Community Survey. The U.S. Census Bureau conducts this survey to give our country an up-to-date picture of how we live—

  • ur education, employment, housing, and more. Using the enclosed

instructions , please complete the survey online as soon as possible at:

Example of f a Coded Letter

slide-39
SLIDE 39

Appendix C – Cut slides

39

slide-40
SLIDE 40

Lesson 5: : State data security and confidentiality in simple terms

18

slide-41
SLIDE 41

Paraphrasing

17

Fir irst le letter: “This survey collects critical information used to meet the needs of communities across the United States. For example, results from this survey decide where new schools, hospitals, and fir ire statio ions are needed.” Se Second le letter: “Local communities depend on information from this survey to decide where schools, hig ighways, hospitals, and other important services are needed.”

slide-42
SLIDE 42

Repetition: Data Security & Legal Obligation

Word Count Analysis: Content of each mail piece that communicates data security or legal obligation Letter: 28% MLB: 41% FAQ: 34%

18

slide-43
SLIDE 43

Placement matters

ACS-13L)(2017) (6-2017)

slide-44
SLIDE 44

Next xt Steps for the Strategic Framework

44

Develop Strategic Framework Develop revisions to ACS mail materials Field testing Analyze results and make recommendations STEP 1  STEP 2  STEP 3 STEP 5 STEP 6 Inventory and Analyze Current Messaging Qualitative evaluation STEP 4