Using E Eye-Trac ackin ing t to E Evalu luate N New Americ - - PowerPoint PPT Presentation

▶
using e eye trac ackin ing t to e evalu luate n new
SMART_READER_LITE
LIVE PREVIEW

Using E Eye-Trac ackin ing t to E Evalu luate N New Americ - - PowerPoint PPT Presentation

Jun 24, 2023 317 likes •566 views

Using E Eye-Trac ackin ing t to E Evalu luate N New Americ ican an Community S Survey Mail M ail Material al Desig ign Strateg egies Alfred Dave Tuttle, Jon Schreiner, Elizabeth Nichols, Erica Olmsted-Hawala, and Rodney Terry

slide-1
SLIDE 1

Using E Eye-Trac ackin ing t to E Evalu luate N New Americ ican an Community S Survey Mail M ail Material al Desig ign Strateg egies

Alfred “Dave” Tuttle, Jon Schreiner, Elizabeth Nichols, Erica Olmsted-Hawala, and Rodney Terry

  • U. S. Census Bureau

American Association for Public Opinion Research Conference, May 17, 2019

1

Any views expressed are those of the authorsand not necessarily those of the U.S. Census Bureau.

CBDRB-FY19-CED001-B0006

slide-2
SLIDE 2

Motivation

  • Evaluation of new letters prior to use in a survey experiment
  • How do the visual design strategies affect reading behaviors?
  • Are the new designs counterproductive?
  • Eye-tracking used to try to answer these questions

2

slide-3
SLIDE 3
  • Four letters
  • Each letter shown individually
  • “Please look at it as you would at

home”

  • Four treatments, random

assignment to vary order of presentation

3

Eye-tracking procedures

slide-4
SLIDE 4

Sample

  • n=9
  • Recruiting database
  • Purposive sample
  • Higher vs. lower education
  • Bachelors and beyond
  • Less than bachelors
  • Demographic variety – race/ethnicity, age, gender
  • ~30 minutes; $40 incentive

4

slide-5
SLIDE 5

Test materials

5

slide-6
SLIDE 6

6

slide-7
SLIDE 7

7

slide-8
SLIDE 8

8

slide-9
SLIDE 9

9

slide-10
SLIDE 10

10

slide-11
SLIDE 11

11

slide-12
SLIDE 12

Research questions

  • How do visual elements (color, images, typographic cues) affect

reading behavior?

  • Do they draw attention to important information?
  • Do they distract respondents from seeing important information?
  • How do people read survey letters?
  • Systematic, skimming, non-linear?
  • Pre-attentive processing

12

slide-13
SLIDE 13

Results

13

slide-14
SLIDE 14

Eye-tracking output

  • Videos and static images
  • Gaze plots and heat maps
  • Individual respondents
  • Multiple respondents
  • Not covering:
  • Quantitative metrics
  • Recall test results

14

slide-15
SLIDE 15

15

Typical reading patterns (2X speed)

slide-16
SLIDE 16

16

Atypical (?) reading pattern (2X speed)

slide-17
SLIDE 17

17

All respondents (n=9), normal speed

slide-18
SLIDE 18

18

slide-19
SLIDE 19

19

Pre-attentive processing

slide-20
SLIDE 20

20

Systematic reading

  • “Do I have to

respond?”

  • “…your response is

required by law…”

slide-21
SLIDE 21

21

Systematic reading

  • “…respond now…”
slide-22
SLIDE 22

Takeaways

  • People are noticing important information:
  • Census Bureau, American Community Survey
  • Call to action – Respond now, URL
  • Response is required by law
  • Visual cues were effective at drawing attention
  • Color
  • Bold print
  • FAQ/list format, sidebar
  • Simpler Census Bureau letterhead
  • During pre-attentive processing and during systematic reading
  • Icons, images? More research needed
  • Prevalence of systematic reading
  • Do respondents “act natural” in a lab?
  • Letters are familiar format

22

slide-23
SLIDE 23

Next steps

  • Additional interviews (total of 20)
  • Quantitative analysis of eye-tracking data
  • Cognitive testing of mail packages
  • Clarity of language, impressions
  • How Rs open and process survey materials
  • Survey experiment

23

slide-24
SLIDE 24

Comments, questions? Suggestions for facilitating more “normal” survey letter processing behaviors in lab setting? Dave Tuttle alfred.d.tuttle@census.gov

24