Retrieval of Autobiographical Information Erica Yu and Scott Fricker AAPOR May 18, 2014 All views expressed in this presentation are those of the authors and do not necessarily reflect the views or policies of the U.S. Bureau of Labor Statistics
The Consumer Expenditure Survey Since the first of February, have you or any members of your household purchased for you or your household or for someone outside of your household any sofas? Organized in sections of similar items “Home furnishings and related household items” “Vehicle operating expenses” “Clothing and clothing expenses” Interviewers proceed linearly through the survey in a fixed order Checked union/professional registers Placed or answered ads Is this how respondents store and recall from Looked at ads … memory information about expenses? 2
Retrieval of information from memory Retrieval is thought to be based on similarity Probability of recall is proportionate to the strength of the similarity 3
Retrieval of information from memory What happens when personal information is relevant? 4
Retrieval of information from memory What is the impact of designing the survey to meet our data needs rather than respondent needs? Will the respondent remember that expense later when the question is asked in sequence? Is it more burdensome to have to re-recall information?
Study Aims What kinds of associations are being triggered when we ask respondents to recall past behaviors? How can we design survey recall questions to reduce the burden on the respondent? How can we design survey recall questions to improve data quality? 6
The Task Retrieved Retrieved Retrieved Retrieved Retrieved Cue Expense 1 Expense 2 Expense 3 Expense 4 Expense 5 Connection Connection Connection Connection Connection Categorization Categorization Categorization Categorization Categorization Each participant is provided one cue to start recall Five retrieval cues likely to trigger a range of retrieval ‘strategies’ If no more related expenses to report, instructed to report whatever comes to mind next 7
The Task Part 1: Recalling expenses 8
The Task Part 2: Describing the associations 9
The Task Part 3: Self-Categorizing Associations 10
Unexpected but reasonable retrievals Shirts Seahawks hoodie NFL Direct Ticket New HD TV Lazy Boy Chair Cooler for beverages 11
Methods n = 887 Amazon mTurk workers Asked for n = 900 workers Criteria based on location (US), experience, and approval rating Excluded n = 17 (2%) based on apparent misunderstanding of task Task duration about 10 minutes Incentive = $2.20 (effective $13/hr) 879 participants provided all 5 recalls Participants’ self -categorizations match to open-ended reasons ~ 93% 12
Demographics Age Mean = 34 years SD = 11.3 Median = 31 years Household size Mean = 2.74 people SD = 1.36 Median = 2.5 people Education Median = Associate’s degree 13
What led people to think of the next item? Self-Categorized Associations Across All Retrievals by Condition Similar Same Same Same None of items purpose store person the Above Cue Hospitals Rugs Shirts Sports Fitness clubs 14
What led people to think of the next item? Self-Categorized Associations Across All Retrievals by Condition Similar Same Same Same None of items purpose store person the Above Cue Hospitals 27% 29% 12% 17% 14% Rugs Shirts Sports Fitness clubs 15
What led people to think of the next item? Self-Categorized Associations Across All Retrievals by Condition Similar Same Same Same None of items purpose store person the Above Cue Hospitals 27% 29% 12% 17% 14% Rugs 29% 23% 15% 14% 20% Shirts 26% 29% 14% 15% 16% Sports 26% 32% 11% 16% 14% Fitness clubs 22% 44% 11% 12% 12% Shared ‘purpose’ was the most common reason given for retrieving the next item from memory 16
How flexible was retrieval? Switches between retrieval reasons Self-categorized reasons provided No significant differences by category cue Mean = 2.39 SE=0.04, Median = 2 (out of 4) * 3.0 *** Number of Switches 2.5 2.0 1.5 1.0 .5 .0 Similar Same Same Same None of items purpose store person the above 1 st Retrieval Strategy 17 * p < 0.05 *** p < 0.01 Tukey HSD Bars represent standard error
How flexible was retrieval? Switches between expenditure categories Each recalled expense was coded in to a CE survey section (e.g., “Clothing”) 20% random selection of respondents analyzed Mean SE Median All switches 1.88 0.09 2 (out of 5) Switches back to a previous category 0.10 0.02 0 (out of 4) Retrieval rarely switches ‘back’ to a previous expenditure category 18
Main Takeaway Shared purpose seems to create a strong association for recall of autobiographical information Understand that respondents may not think the way we design surveys Goals and activities may be key to memory structure and recall (Reiser, Black, & Abelson, 1985) Designing for respondent needs rather than data needs may mean… More flexibility from us? Reduced interference from other information? Reduced telescoping? Reduced respondent effort? 19
Closing Remarks Our theories of response formulation processes must consider how respondents retrieve information from memory – strategies are not all equal Future research can explore the effect of directed retrieval strategies Basic psychological research like this can apply to many surveys, and complement traditional survey research methods 20
Contact Information Erica Yu Research Psychologist Office of Survey Methods Research www.bls.gov/osmr 202-691-7924 yu.erica@bls.gov
Cleaning of the Data Excluded 17 people on the basis of Describing the same expense for each retrieval instead of five different expenses Describing fictional or general expenses instead of actual expenses Reporting no expenses 22
How long did the 1 st retrieval strategy last? 23
Was a common retrieval strategy triggered by the question? Self-Categorized Associations for 1 st Choice Similar Same Same Same Other items purpose store person 68 49 8 21 32 Hospitals 98 42 7 13 19 Rugs 95 42 14 15 13 Shirts 62 66 6 17 24 Sports 66 81 4 9 16 Fitness clubs The question leads participants to think of ‘similar’ items 24
How long did the 1 st retrieval strategy last? 2 nd Retrieval Strategy Similar Same Same Same Other items purpose store person Similar items 92 128 51 50 68 1 st Retrieval Strategy Same purpose 45 142 30 36 27 Same store 0 11 24 2 2 Same person 11 21 8 33 2 Other 15 24 15 16 34 Most retrieval strategies led to another item using the same strategy 25
How flexible is retrieval? 26
How flexible is retrieval? 27
How productive was the first retrieval strategy? Number of retrievals using the same self-categorized reason Similar Same Same Same Other items purpose store person Hospitals 1.62 2.06 1.25 2.05 - Rugs 1.40 1.81 2.86 1.54 - Shirts 1.48 2.10 2.29 2.07 - Sports 1.48 1.73 2.00 2.00 - Fitness clubs 1.23 2.37 1.50 1.89 - The ‘same store’ strategy typically led to the longest string of associated retrievals 28
Recommend
More recommend
