Brian Haynes McMaster University EBHC Workshop, 2013 The Health - - PowerPoint PPT Presentation

Brian Haynes McMaster University

EBHC Workshop, 2013

The Health Information Research Unit at

McMaster has developed, and produces or supports, several resources that will be discussed (ACP Journal Club, Evidence Updates, DynaMed, MacPLUS FS, ACCESSSS). (IP belongs to McMaster; most services free)

Agenda (negotiate!)

• 1. Intros – and your most frustrating or rewarding

teaching or personal experience in acquiring “current best evidence for clinical practice”

• 2. Favorite sources of EB info?
• 3. Sign up: http://plus.mcmaster.ca/ACCESSSS

(suggest that you subscribe, but can use ID/PW: guestn/guestn)

1.

To define/personalize the nature of the problems in translating evidence into clinical practice

2.

To explore a hierarchy of evidence resources to support clinical care decisions:

3.

To demonstrate and practice how you can use/teach the pyramid to keep up to date (“push services”) and find current best evidence (“pull services”)

• 1. Making decisions consistently

based on current best evidence ?

• 2. Keeping up to date with current

best evidence ?

Clinicians like you usually have more than 5 questions a day that can be answered by current best evidence

• Green. Residents' medical information needs in clinic: are they being met? Am J Med 2000

But…

• 2,000 new articles every day
• including 75 trials
• and 11 systematic reviews

Bastian, Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010

By the year 2020, 90% of clinical decisions will be supported by accurate, timely, and up-to-date clinical information and will reflect the best available evidence. IOM Roundtable on Evidence-Based Medicine

This can’t happen without excellent connections between best evidence and decisions for and by individual patients.

years since graduation r = -0.54 p<0.001

... ...

. ..

. . .... . .... .... ... .. ...

knowledge

• f current

best care

100% 0% 50%

Choudhry, Fletcher and Soumerai, Ann Intern Med 2005;142:260-73 94% of 62 studies found decreasing competence for at least some tasks, with increasing physician age.

McMaster vs U of Toronto

years since graduation knowledge

• f current

best care 100% 0% 50%

From Shin et al, CMAJ 1993

The Slippery Slope

years since graduation

... ...

. ..

. . .... . .... .... ... .. ...

knowledge

• f best

evidence 100% 0% 50% You could be the first generation to stay

• n top and avoid the slippery slope…

…17 to 20 years

1.

To define the nature of the problems in translating evidence into clinical practice

2.

To explore a hierarchy of evidence resources to support clinical care decisions:

3.

To demonstrate how you can use the pyramid to keep up to date (“push services”) and find current best evidence *fast* (“pull services”)

Finding current best evidence is becoming much easier.

Personalized EBM “push” Alerts? eg EvidenceUpdates, ACPJournalWise EBM “pull” Resources? eg UTD, DynaMed, Best Practice, ACP PIER EBM Federated Resources? eg TRIP,

ACCESSSS

Many evidence-based resources help you answer your questions

But no single resource provides all answers or is sufficiently updated

shows results in a pyramid of resources in a few seconds

showing results in a pyramid

• f resources in a few seconds

Summaries Pre-appraised research Non-preappraised research

Summaries

• are E-B clinical textbooks and

E-B guidelines

• integrate best evidence

for individual care topics

• provide actionable

recommendations

But which summary to choose?

Percentage of 60 ICD-10 codes covered by each summary

Error bars: 95% Confidence Interval

Timeliness of updates of 10

• nline evidence-based texts

Average time of updating of 60 topics (randomly selected) as of July 2011

Ranging from 3.5 to 29 months

Summary Timeliness Breadth Quality DynaMed 1 3 2 UpToDate 5 1 2 Micromedex 2 8 2 Best Practice 3 4 7 Essential Evidence Plus 7 7 2 First Consult 9 5 2 Medscape Reference 6 2 9 Clinical Evidence 8 10 1 ACP PIER 4 9 7 PEPID N/A 6 10

Rank order of 10 Online Summaries

No answer?

• r more recent evidence?

Summaries Pre-appraised Research

synopses of systematic reviews systematic reviews synopses of studies

• only a tiny proportion of all research is

“ready for application”

• only a tiny fraction of the “ready”

research is “relevant” for a given clinician

• only a tiny proportion of the “relevant”

research for a given clinician is “interesting” in the sense of being something new, important, and actionable.

Haynes, B. The Best New Evidence for Patient Care. Ann Intern Med. 2008;148(10):JC3-2

~3300 articles per year

Critical Appraisal Filters Clinical Relevance Filters

+ 35,000 articles screened per year ~20 articles per clinician

Up to 99.9 % ‘Noise’ Reduction

Is‘information overload’ no longer a valid excuse for ignorance?

Al-Khatib SM, Thomas L, Wallentin L, et al. Outcomes of apixaban vs. warfarin by type and duration of atrial fibrillation: results from the ARISTOTLE trial. Eur Heart J. 2013 Apr 17.

4 internists 4 neurologists 4 cardiologists 4 hematologists

DISCIPLINE RELEVANCE NEW?

Internal Medicine

6/7 6/7 Neurology 6/7 6/7

Cardiology

5/7 6/7

Hematology

5/7 5/7

Internists Neurologists Cardiologists Hematologists

7/6 6/5 7/7

is published in evidence- based journals and resources

• is continuously sent to

you through MacPLUS, ACCESSSS, ACPJW, EvidenceUPDATES alerts

http://plus.mcmaster.ca/EvidenceUpdates

Methods

• Select 4 leading evidence-based texts
• Select 200 topics that are common to all
• Identify date of most recent update for each topic in each text
• Identify each article in MacPLUS that is more recent than text

update

• Compare conclusions of MacPLUS studies with conclusions in

text

How often does MacPLUS give new and different evidence than Summaries?

Best Practice DynaMed PIER UpToDate

• No. (%) of 200

topics with potential for updates†

119 (60%) 46 (23%) 109 (55%) 104 (52%) *Based on articles in MacPLUS since most recent topic update which have conclusions that differ from topic conclusion(s). † 1-way ANOVA p<0.01 across texts

Summaries Pre-appraised Research Non-preappraised Research

Why search

last?

• Studies are not critically appraised
• Searches yield large outputs with few relevant

studies in the first pages

• Current evidence is diluted and more difficult

to find, but less so if you use filters

Use Clinical Queries filters for questions of

• Prediction Guides
• Prognosis
• Etiology
• Diagnosis
• Therapy

Sensitivity/ Specificity PubMed Equivalent broad filter 99% / 70% ((clinical[Title/Abstract] AND trial[Title/Abstract]) OR clinical trials[MeSH Terms] OR clinical trial[Publication Type] OR random*[Title/Abstract] OR random allocation[MeSH Terms] OR therapeutic use[MeSH Subheading]) narrow filter 93% / 97% (randomized controlled trial[Publication Type] OR (randomized[Title/Abstract] AND controlled[Title/Abstract] AND trial[Title/Abstract]))

E.g. CQ therapy filters

How many search terms to use?

Patients Intervention Comparison Outcomes

Patient Pre-appraised research Works better for Summaries

E.g. COPD mucolytics

+ Intervention and Using Keep it simple!

Works better for unappraised evidence

COPD mucolytics Combining more terms from your PICO elements (exacerbation OR mortality)

looks for you in all resources and displays results in a pyramid

PICO question from the audience

• …

Did I miss any important evidence with my search? Is there any way I could have retrieved less “junk”?

What is the best current evidence?

Alternatives

TRIP Database – EB search, guidelines, patient info, fulltext links, videos, news SUMSearch – DARE at highest level STAT!Ref Google Corporate collections – professional

• rganizations; commercial publishers

Additional slides

Search strategies in PubMed

Search strategy = PICO query AND filters AND additional strategy

Search terms

Previous knowledge Trial and Error MeSH Thesaurus

Boolean operators = OR, AND Combination Initial question P – Patient, Population I – Intervention, Exposure… C – Controls, Comparators Search forward from: Related Articles in Pubmed Screen the bibliography of relevant articles Methodological filters: Ex: Clinical Queries Content filters (topic,specialty) O – Outomes T – Type of question, design

AND AND

Dynamic

Agoritsas & al. Sensitivity and Predictive Value of 15 PubMed Search Strategies to Answer Clinical Questions Rated Against Full Systematic Reviews. JMIR, 2012.

Balance between Sensitivity & Precision

Factors increasing Sensitivity:

• Many search terms for a similar

concept, linked with OR.

• Truncated terms, Wildcards

(e.g. tox*, wom?n)

• Synonyms (pressure sore, decubitus ulcer)
• Variant spelling (tumour, tumor)
• Explosion of MeSH terms
• Proximity search through

«Related articles», Bibliography Factors increasing Precision

• Concepts linked with AND

(P) AND (I) AND (C) AND (O)

• Use of NOT Cochrane
• Limits
• Methodological Filters
• Content Filters

(P1 OR P2 OR P3) AND (I1 OR I2 … ) AND (C1 OR C2 …) AND (O1 OR O2 OR O3 …)

Population Intervention Comparators Outcomes

54

Question Effects of oral mucolytics agents in adults with stable chronic bronchitis or COPD.

P stable chronic bronchitis

COPD (chronic bronchitis) OR COPD

I

Mucolytic agents

• ral mucolytic therapy

mucolytics

C Placebo

placebo

O number of exacerbations

exacerbation First try chronic bronchitis mucolytics

Example of PICO query

Less effective strategy (no filter)

Agoritsas & al. Sensitivity and Predictive Value of 15 PubMed Search Strategies to Answer Clinical Questions Rated Against Full Systematic Reviews. JMIR, 2012.

More effective strategy (with narrow filter)

Agoritsas & al. Sensitivity and Predictive Value of 15 PubMed Search Strategies to Answer Clinical Questions Rated Against Full Systematic Reviews. JMIR, 2012.

McMaster Online Rating of Evidence: >5000 clinicians RELEVANCE