Evaluation in primary care: What works, why, and in what - - PDF document

4/27/2015 1

Evaluation in primary care:

What works, why, and in what circumstances?

EQUIP Conference April 2015 Janet Harris

NIHR Fellow in Knowledge Mobilisation University of Sheffield

The issues with evaluations in primary health care

• Good quality evidence from controlled

trials

• BUT in primary care we see variations in

effectiveness

• HOW can we explain why things don’t

work as expected?

2

Getting evidence into practice: a linear evaluation design

3

Evidence Outcome Implementation

4/27/2015 2

4

Clinical audit

Tells us what works…but why? And in what circumstances?

Wikipedia.org

Assumptions with clinical audit

5

• The problem can be

successfully addressed via clinical audit

• Criteria and standards

are locally achievable for all

• Viewing data will trigger a

discussion about challenges in meeting standards

‘Poor’ performance will be improved

A linear evaluation design applied to prescribing statins

6

Evidence of effectiveness for statins Outcomes: Reduced risk Implementation: Assess risk Prescribe

What are the assumptions with this design?

4/27/2015 3

Statins context UK 1998 UK health system recommends assessment to identify high risk patients

The importance of context in evaluation

7

Implementation: Assess risk Prescribe Outcome Awareness Evidence from trials

Did context promote awareness?

• Practitioners were “generally aware of the

evidence relating to the use of statins in secondary prevention of coronary heart disease, but they were less clear about the evidence in primary prevention.”

8 Fairhurst & Huby, 1998

Statins context UK 1998

How do we decide applicability of evidence?

9

Implementation: Assess risk Prescribe Outcome Evidence from trials Applicability Awareness

4/27/2015 4

How context can affect decisions about applying evidence

• Economic context:
• Practitioners had reservations about drug

costs and workload for assessments

• Social context:
• Abiding concerns regarding medicalising

patients

Contextual factors are rarely included when conducting clinical trials

10

Statins context UK 1998

How do we decide applicability of evidence?

11

Implementation: Assess risk Prescribe Outcome Evidence from trials Applicability Quality

Approaches to judging quality

• Practitioners lacked time and skills to

appraise the content of scientific papers critically

• Used postgraduate meetings, discussion

with colleagues, editorials and comment in journals

12

4/27/2015 5

Credibility needs to be supplemented with trust

• “Trustworthiness of evidence was judged
• n the basis of the perceived motives and

interests of its source”

13

Adding components to the intervention

14

Evidence Disseminate via opinion leaders Implementation: Assess Prescribe Trust Credibility

Sources of evidence need to align with each other

• Research evidence
• Interpretation of evidence by credible

sources

• Results in a ‘groundswell of opinion’ about

whether to prescribe

15

4/27/2015 6

Implementation issues

16

Disseminate via opinion leaders Implementation: Assess Prescribe Credibility Trust Medicalisation Recommend Lifestyle changes

Developing decision aids

17 http://www.nice.org.uk/guidance/cg181/resources/cg181-lipid-modification- update-patient-decision- aid2

Explaining risk and benefit

18

Patient concern s Patients reassured Implementation: Assess Prescribe Decision aid for risk/benefits Outcomes

4/27/2015 7

Joint decision making

19

Implementation: Assess Delay prescription Patients unsuccessful Recommend Lifestyle changes Patients successful Implementation: Reassess Prescribe

From trial data to practical knowledge

20

Simple Complex

Fairhurst & Huby, 1998

Evaluating complex interventions

• The context for the intervention need to be

considered – it may differ by setting

• Assumptions about what works can be

risky - potential barriers to implementation need to be identified

• Possible ways to address barriers need to

be added to the intervention

21

4/27/2015 8 Throwing context into the mix

• Context at what level?
• International, national, local
• For what time period?
• How important is history, in terms of influence
• n implementation?
• For which stakeholders?
• Those who implement? Who require

implementation? Others?

22

Another way of looking at variation

• When evaluating how something is

implemented, instead of controlling for variation, we need to look at it as the phenomenon of interest

• Why do you make a change in practice, while I

don’t

• Realist approaches to evaluation can produce

useful information to guide implementation of quality improvement

23

24

MSc International Health Management and Leadership (Distance Learning) scharr-ihml@sheffield.ac.uk or call (+44) 0114 222 5454. Janet Harris, at janet.harris@sheffield.ac.uk

School of Health & Related Research