Evaluation in primary care: What works, why, and in what - PDF document

4/27/2015 Evaluation in primary care: What works, why, and in what circumstances? EQUIP Conference April 2015 Janet Harris NIHR Fellow in Knowledge Mobilisation University of Sheffield 2 The issues with evaluations in primary health care • Good quality evidence from controlled trials • BUT in primary care we see variations in effectiveness • HOW can we explain why things don’t work as expected? 3 Getting evidence into practice: a linear evaluation design Evidence Implementation Outcome 1

4/27/2015 4 Clinical audit Tells us what works…but why? And in what circumstances? Wikipedia.org Assumptions 5 with clinical audit • The problem can be successfully addressed via clinical audit • Criteria and standards are locally achievable for all • Viewing data will trigger a discussion about challenges in meeting standards ‘ Poor’ performance will be improved 6 A linear evaluation design applied to prescribing statins Evidence of Implementation: Outcomes: effectiveness Assess risk Reduced for statins Prescribe risk What are the assumptions with this design? 2

4/27/2015 7 The importance of context in evaluation Statins context UK 1998 UK health system recommends assessment to identify high risk patients Evidence Implementation: from Assess risk Outcome trials Prescribe Awareness 8 Did context promote awareness? • Practitioners were “generally aware of the evidence relating to the use of statins in secondary prevention of coronary heart disease, but they were less clear about the evidence in primary prevention.” Fairhurst & Huby, 1998 9 How do we decide applicability of evidence? Statins context UK 1998 Evidence Implementation: from Assess risk Outcome trials Prescribe Awareness Applicability 3

4/27/2015 10 How context can affect decisions about applying evidence • Economic context: • Practitioners had reservations about drug costs and workload for assessments • Social context: • Abiding concerns regarding medicalising patients Contextual factors are rarely included when conducting clinical trials 11 How do we decide applicability of evidence? Statins context UK 1998 Evidence Implementation: from Assess risk Outcome trials Prescribe Quality Applicability 12 Approaches to judging quality • Practitioners lacked time and skills to appraise the content of scientific papers critically • Used postgraduate meetings, discussion with colleagues, editorials and comment in journals 4

4/27/2015 13 Credibility needs to be supplemented with trust • “Trustworthiness of evidence was judged on the basis of the perceived motives and interests of its source” 14 Adding components to the intervention Implementation: Evidence Disseminate Assess via opinion Prescribe leaders Credibility Trust 15 Sources of evidence need to align with each other • Research evidence • Interpretation of evidence by credible sources • Results in a ‘groundswell of opinion’ about whether to prescribe 5

4/27/2015 16 Implementation issues Disseminate Implementation: Recommend via opinion Assess Lifestyle leaders Prescribe changes Credibility Medicalisation Trust 17 Developing decision aids http://www.nice.org.uk/guidance/cg181/resources/cg181-lipid-modification- update-patient-decision- aid2 18 Explaining risk and benefit Implementation: Decision aid Outcomes Assess for Prescribe risk/benefits Patient Patients concern reassured s 6

4/27/2015 19 Joint decision making Implementation: Recommend Implementation: Assess Lifestyle Reassess Delay changes Prescribe prescription Patients Patients successful unsuccessful 20 From trial data to practical knowledge Simple Complex Fairhurst & Huby, 1998 21 Evaluating complex interventions • The context for the intervention need to be considered – it may differ by setting • Assumptions about what works can be risky - potential barriers to implementation need to be identified • Possible ways to address barriers need to be added to the intervention 7

4/27/2015 22 Throwing context into the mix • Context at what level? • International, national, local • For what time period? • How important is history, in terms of influence on implementation? • For which stakeholders? • Those who implement? Who require implementation? Others? 23 Another way of looking at variation • When evaluating how something is implemented, instead of controlling for variation, we need to look at it as the phenomenon of interest • Why do you make a change in practice, while I don’t • Realist approaches to evaluation can produce useful information to guide implementation of quality improvement School of Health & Related Research MSc International Health Management and Leadership (Distance Learning) scharr-ihml@sheffield.ac.uk or call (+44) 0114 222 5454. Janet Harris, at janet.harris@sheffield.ac.uk 24 8