Retrieval of Autobiographical Information Erica Yu and Scott - - PowerPoint PPT Presentation

Retrieval of Autobiographical Information

Erica Yu and Scott Fricker AAPOR May 18, 2014

All views expressed in this presentation are those of the authors and do not necessarily reflect the views or policies of the U.S. Bureau of Labor Statistics

Checked union/professional registers Placed or answered ads Looked at ads …

The Consumer Expenditure Survey

2

Since the first of February, have you or any members of your household purchased for you or your household or for someone outside of your household any sofas?

 Organized in sections of similar items

“Home furnishings and related household items” “Vehicle operating expenses” “Clothing and clothing expenses”

 Interviewers proceed linearly through the

survey in a fixed order

 Is this how respondents store and recall from

memory information about expenses?

 Retrieval is thought to be based on similarity  Probability of recall is proportionate to the

strength of the similarity

Retrieval of information from memory

3

Retrieval of information from memory

 What happens when personal information is

relevant?

4

Retrieval of information from memory

 What is the impact of designing the

survey to meet our data needs rather than respondent needs?

Will the respondent remember that expense later when the question is asked in sequence? Is it more burdensome to have to re-recall information?

Study Aims

 What kinds of associations are being

triggered when we ask respondents to recall past behaviors?

 How can we design survey recall

questions to reduce the burden on the respondent?

 How can we design survey recall

questions to improve data quality?

6

The Task

7

Cue Retrieved Expense 1 Retrieved Expense 2 Retrieved Expense 3 Retrieved Expense 4 Retrieved Expense 5

Connection Connection Connection Connection Connection Categorization Categorization Categorization Categorization Categorization

 Each participant is provided one cue to start recall  Five retrieval cues likely to trigger a range of

retrieval ‘strategies’

 If no more related expenses to report, instructed to

report whatever comes to mind next

The Task Part 1: Recalling expenses

8

The Task Part 2: Describing the associations

9

The Task Part 3: Self-Categorizing Associations

10

Unexpected but reasonable retrievals

11

Shirts Seahawks hoodie NFL Direct Ticket New HD TV Lazy Boy Chair Cooler for beverages

Methods

 n = 887 Amazon mTurk workers

Asked for n = 900 workers Criteria based on location (US), experience, and approval rating Excluded n = 17 (2%) based on apparent misunderstanding of task

 Task duration about 10 minutes  Incentive = $2.20 (effective $13/hr)  879 participants provided all 5 recalls  Participants’ self-categorizations match to

• pen-ended reasons ~ 93%

12

Demographics

 Age

Mean = 34 years SD = 11.3 Median = 31 years

 Household size

Mean = 2.74 people SD = 1.36 Median = 2.5 people

 Education

Median = Associate’s degree

13

What led people to think of the next item?

Cue

Similar items Same purpose Same store Same person

None of the Above

Hospitals Rugs Shirts Sports Fitness clubs

Self-Categorized Associations Across All Retrievals by Condition

14

Cue

Similar items Same purpose Same store Same person

None of the Above

Hospitals

27% 29% 12% 17% 14%

Rugs Shirts Sports Fitness clubs

What led people to think of the next item?

Self-Categorized Associations Across All Retrievals by Condition

15

Cue

Similar items Same purpose Same store Same person

None of the Above

Hospitals

27% 29% 12% 17% 14%

Rugs

29% 23% 15% 14% 20%

Shirts

26% 29% 14% 15% 16%

Sports

26% 32% 11% 16% 14%

Fitness clubs

22% 44% 11% 12% 12%

What led people to think of the next item?

Self-Categorized Associations Across All Retrievals by Condition

Shared ‘purpose’ was the most common reason given for retrieving the next item from memory

16

.0 .5 1.0 1.5 2.0 2.5 3.0 Similar items Same purpose Same store Same person None of the above Number of Switches 1st Retrieval Strategy

How flexible was retrieval?

 Switches between retrieval reasons

Self-categorized reasons provided No significant differences by category cue

 Mean = 2.39 SE=0.04, Median = 2 (out of 4)

*** *

* p < 0.05 *** p < 0.01 Tukey HSD Bars represent standard error 17

Retrieval rarely switches ‘back’ to a previous expenditure category

How flexible was retrieval?

 Switches between expenditure categories

Each recalled expense was coded in to a CE survey section (e.g., “Clothing”) 20% random selection of respondents analyzed

Mean SE Median All switches (out of 5)

1.88 0.09 2

Switches back to a previous category (out of 4)

0.10 0.02

18

Main Takeaway

 Shared purpose seems to create a strong

association for recall of autobiographical information

 Understand that respondents may not think the

way we design surveys

Goals and activities may be key to memory structure and recall (Reiser, Black, & Abelson, 1985)

 Designing for respondent needs rather than data

needs may mean…

More flexibility from us? Reduced interference from other information? Reduced telescoping? Reduced respondent effort?

19

Closing Remarks

 Our theories of response formulation

processes must consider how respondents retrieve information from memory – strategies are not all equal

 Future research can explore the effect of

directed retrieval strategies

 Basic psychological research like this can

apply to many surveys, and complement traditional survey research methods

20

Contact Information

Erica Yu

Research Psychologist Office of Survey Methods Research www.bls.gov/osmr 202-691-7924 yu.erica@bls.gov

Cleaning of the Data

 Excluded 17 people on the basis of

Describing the same expense for each retrieval instead of five different expenses Describing fictional or general expenses instead of actual expenses Reporting no expenses

22

How long did the 1st retrieval strategy last?

23

Was a common retrieval strategy triggered by the question?

Similar items Same purpose Same store Same person Other Hospitals

68 49 8 21 32

Rugs

98 42 7 13 19

Shirts

95 42 14 15 13

Sports

62 66 6 17 24

Fitness clubs

66 81 4 9 16

The question leads participants to think of ‘similar’ items

Self-Categorized Associations for 1st Choice

24

How long did the 1st retrieval strategy last?

Similar items Same purpose Same store Same person Other Similar items 92 128 51 50 68 Same purpose 45 142 30 36 27 Same store 11 24 2 2 Same person 11 21 8 33 2 Other 15 24 15 16 34

Most retrieval strategies led to another item using the same strategy

1st Retrieval Strategy 2nd Retrieval Strategy

25

How flexible is retrieval?

26

How flexible is retrieval?

27

The ‘same store’ strategy typically led to the longest string of associated retrievals

How productive was the first retrieval strategy?

Similar items Same purpose Same store Same person Other Hospitals 1.62 2.06 1.25 2.05

• Rugs

1.40 1.81 2.86 1.54

• Shirts

1.48 2.10 2.29 2.07

• Sports

1.48 1.73 2.00 2.00

• Fitness clubs

1.23 2.37 1.50 1.89

• Number of retrievals using the same self-categorized reason

28