Getting to Know GRADE Eddy Lang Senior Researcher Alberta Health - - PowerPoint PPT Presentation

Getting to Know GRADE

Eddy Lang Senior Researcher Alberta Health Services Associate Professor University of Calgary Peter Morley Associate Professor Director of Medical Education Senior Specialist, Intensive Care Royal Melbourne Hospital University of Melbourne

Disclosures

• Member of GRADE working group

– Projects – Publications – Consultancies

Why another methodology?

• 1. Inconsistency in approaches
• 2. Rigid link between evidence and guidance
• 3. Lack of transparency
• 4. Unstructured approach going from evidence

to recommendations

Where GRADE fits in

Prioritize problems, establish panel Find/appraise or prepare: Systematic review Searches, selection of studies, data collection and analysis (Re-) Assess the relative importance of outcomes Prepare evidence profile: Quality of evidence for each outcome and summary of findings Guidelines: Assess overall quality of evidence Decide direction and strength of recommendation Draft guideline Consult with stakeholders and / or external peer reviewer Disseminate guideline Implement the guideline and evaluate GRADE

Systematic review Guideline development

P I C O

Outcome Outcome Outcome Outcome Critical Important Critical Not Summary of findings & estimate of effect for each outcome Rate

• verall quality of evidence

across outcomes based on lowest quality

• f critical outcomes

RCT start high,

• bs. data start low
• 1. Risk of bias
• 2. Inconsistency
• 3. Indirectness
• 4. Imprecision
• 5. Publication

bias Grade down Grade up

• 1. Large effect
• 2. Dose

response

• 3. Confounders

Very low Low Moderate High Formulate recommendations:

• For or against (direction)
• Strong or weak (strength)

By considering:  Quality of evidence  Balance benefits/harms  Values and preferences Revise if necessary by considering:  Resource use (cost)

• “We recommend using…”
• “We suggest using…”
• “We recommend against using…”
• “We suggest against using…”

I B II V III

GRADE is outcome-centric

Quality: High Quality: Moderate Quality: Low

Older systems

Outcome #1 Outcome #2 Outcome #3

GRADE

How to select outcomes

• Be comprehensive
• Think outside the literature
• Patient / clinician / public input
• Safety considerations (often under-

reported)

• Patient-important

Prioritizing outcomes

• Critical i.e. critical for decision-making
• Important – for decision-making but not

critical

• Less important
• Process should be importance-driven and

not evidence-driven

Why prioritize outcomes

• Focuses search and evidence review /

synthesis

• Defines the elements of the Evidence

Profile and SoF table

• QoE for critical outcomes defines the QoE

for the entire PICO-related evidence base

13

GRADE: Quality of evidence

GRADE defines 4 categories of quality:

– High – Moderate – Low – Very low

The extent to which our confidence in an estimate of the treatment effect is adequate to support a particular recommendation.

14

Conceptualizing quality

       

We are very confident that the true effect lies close to that of the estimate of the effect. High

    

Low Our confidence in the effect is limited: The true effect may be substantially different from the estimate of the effect.

    

Moderate We are moderately confident in the estimate of effect: The true effect is likely to be close to the estimate of effect , but possibility to be substantially different.

      

Very low We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect.

Determinants of quality

What lowers quality of evidence? 5 factors:

Methodological limitations Inconsistency

• f results

Indirectness

• f evidence

Imprecision

• f results

Publication bias

• RCTs start high
• Observational studies start low

Determinants of quality

What raises quality of evidence? 3 factors:

Addressing confounding Dose- response Large effect size

• RCTs start high
• Observational studies start low

Moving from Evidence to Recommendations

www.gradeworkinggroup.org

7 key GRADE elements

• Quality of evidence concep
• Assessment of quality criteria
• Evidence quality – high, mod, low, very

low

• Evidence profiles
• Explicit consideration of desirable / undesir
• Strong and weak
• Support for recommendations

20

Thank You