National Center for Health Statistics The Role of Respondent Experience in Answering Survey Questions on Opioid-Related Impairment Stephanie Willson Collaborating Center for Question Design and Evaluation Research DC-AAPOR/WSS Summer Preview/Review Conference July 12, 2019
Study Background Cognitive interview evaluation of the plausibility of asking opioid-related questions for household surveys – Use – Impairment – Misuse – Disorder
Study background 180 interviews (40 in Spanish, 140 in English) in summer of 2018 Eight different geographical regions in the United States • DC/Maryland • Massachusetts • Kentucky • Kansas • Alabama • Texas/New Mexico • Washington • Illinois
Impairment (measured as side effects) When taking opioids, do you feel any of the following side effects? [Check all that apply] Drowsiness Lack of concentration Dizziness Blurred vision Confusion Off balance Calm Irrational Carefree Paranoid
Side effects question is important - serves as a filter for other impairment questions During the past 30 days, have you gone to work at a paid job while experiencing those side effects? During the past 30 days, have you driven a car while experiencing those side effects? During the past 30 days, have you had an injury or hurt yourself because of the opioids or the side effects?
Two principle problems with asking about side effects 1. Understanding the intended definition of a side effect 2. Knowing what causes a feeling state
1. Question intent: What are “side effects”? Medical perspective – Side effects are not therapeutic effects (i.e., pain relief) – Side effects are considered secondary effects • Unintended • Sometimes beneficial, but mostly unpleasant or even harmful
Unintended interpretation of the “side effects” question Not all respondents understood the question as asking about side effects in accordance with the medical use of the term Many respondents were thinking about the impact of the therapeutic effect of the opioids
Interpretations were not random but instead often associated with different groups of respondents The likelihood of answering the question as intended was associated with respondents’ experience with opioids, specifically length of and reason for use – Temporary injury or condition/surgery (short-term users) – Chronic pain management (long-term users)
Different interpretations of “side effects” is associated with different experiences with opioids Short-term users (surgery; temporary injuries) – More likely to answer the question in terms of the secondary and unpleasant effects of opioids
Examples: short-term users, unpleasant effects You don’t feel…well. That’s how I have to put it. There’s something wrong, but you don’t know what it is. If I got up and walked around, it would make me really dizzy. [My wife] made me stay in bed [for safety reasons]. Yeah, I couldn’t have a decent conversation. Because I wasn’t thinking right. I’d have people call me on the phone and I’d tell them, ‘Yeah, you know, I’m not up for it. Call me back in a few weeks.’ Because I didn’t know what I was going to say! I tried not to do any business or anything.
Common choices among short-term users Often chose many side effects – To convey the overall unpleasant feeling – Difficult to describe how they felt
Different interpretations of “side effects” is associated with different experiences with opioids Long-term users (chronic pain management) – More likely to answer the question in terms of the therapeutic effect of the opioids
Common choices among long-term users Drowsy Calm Carefree
Meaning of ‘drowsy’, ‘calm’, and ‘carefree’, for long - term users Not negative or unpleasant – ‘Calm’ & ‘carefree’ = absence of pain; allows respondents to function in daily life • Ability to work • Ability to drive – ‘Drowsy’ allows respondents to sleep
Examples: Long-term users, therapeutic effect Drowsy : The pain already wore you out. And when the pain subsides because of the pain medication [exhales loudly], I’m going to take a nap. Because I’ve been struggling through all this discomfort and pain. Calm : You hurt and you take pain reliever – gonna give you that relief, and that makes me calm. Allows you to take a deep breath in. Carefree : It was being able to run with my kids and move again. Because when you are strapped in this much pain all the time, being carefree is not something you can be.
Why does this matter? Question functions mostly as intended for short-term users – Intent of question = inability to function = short term users BUT… Question misses and misunderstands the experiences of long- term users – Unintended meaning = ability to function = long-term users
2. Side effects and determining causality Respondents not always able to ascertain the cause of the feeling states asked about in the question
Other possible causes (besides opioids) Age Personal characteristic or condition Surgery Lack of sleep Pain Multiple medications
Examples: Difficulties in determining causality Age: Once in a while I have trouble hearing and seeing – blurred vision and like this. But I don’t know if that’s from, like, getting old or [the opioid]. I had cataract surgery about five years ago, so I don’t know if that had anything to do with my age or the pills. Personal characteristic or condition: I’m pretty calm in general, so I don’t know if that’s a side effect. Surgery: I assumed those were kind of regular with after-surgery stuff. Or whether it could have been a side effect of that [opioid]? It’s kind of hard to say.
Examples: Determining causality Lack of sleep: I do get drowsy. But whether it’s attributed to this [oxycodone] or not, I don’t know. I’m sleepy a good deal of the time. Now whether it’s this, I don’t know. Pain: I don’t know if I was confused because of the pills or the pain . Multiple medications: I don’t get sleepy. I don’t get dizzy. But my vision is blurry a little bit. [Respondent on 3 medications, all of which list blurred vision as a side effect.]
Different groups = different level of difficulty Short-term users more likely to connect the feeling with the opioid Long-term users less likely to identify causality – Too long to remember – The positive outweighs any negative they associate with opioid use
Why does this matter? Difficult question to answer – Increased respondent burden – Decreased validity of respondent answers (we don’t know whether they are reporting an actual opioid side effect) – Misunderstand long-term users
Conclusion Question design principle: questions that align with respondent experiences capture the most valid data Opioid use: different experiences with opioids affect knowledge, understandings and ability to answer survey questions as intended
Conclusion Opioid impairment: current survey question is more likely to perform as intended among short-term users than among long-term users Opioid use survey questions: should be designed with different groups of users in mind in order to capture the intended information
Thank you! Contact information: swillson@cdc.gov Cognitive interview final reports: https://wwwn.cdc.gov/qbank/
Recommend
More recommend
