Getting to Know GRADE Eddy Lang Senior Researcher Alberta Health - PowerPoint PPT Presentation

Getting to Know GRADE Eddy Lang Senior Researcher Alberta Health Services Associate Professor University of Calgary Peter Morley Associate Professor Director of Medical Education Senior Specialist, Intensive Care Royal Melbourne Hospital University of Melbourne

Disclosures • Member of GRADE working group – Projects – Publications – Consultancies

Why another methodology? 1. Inconsistency in approaches 2. Rigid link between evidence and guidance 3. Lack of transparency 4. Unstructured approach going from evidence to recommendations

Where GRADE fits in Prioritize problems, establish panel Find/appraise or prepare: Systematic review Searches, selection of studies, data collection and analysis (Re-) Assess the relative importance of outcomes Prepare evidence profile: GRADE Quality of evidence for each outcome and summary of findings Guidelines: Assess overall quality of evidence Decide direction and strength of recommendation Draft guideline Consult with stakeholders and / or external peer reviewer Disseminate guideline Implement the guideline and evaluate

RCT start high, obs. data start low 1. Risk of bias Grade down Outcome Critical P 2. Inconsistency High 3. Indirectness I Outcome Critical Moderate 4. Imprecision Low C Outcome Important 5. Publication Very low O Outcome Not bias Summary of findings Grade up 1. Large effect & estimate of effect 2. Dose for each outcome response 3. Confounders Systematic review Guideline development Formulate recommendations : Rate • For or against (direction) overall quality of evidence • Strong or weak (strength) across outcomes based on lowest quality By considering:  Quality of evidence of critical outcomes  Balance benefits/harms  Values and preferences • “We recommend using…” • “We suggest using…” Revise if necessary by considering: • “We recommend against using…”  Resource use (cost) • “We suggest against using…”

GRADE is outcome-centric Outcome #1 Quality: High Outcome #2 Quality: Moderate Outcome #3 Quality: Low I B II V III Older GRADE systems

How to select outcomes • Be comprehensive • Think outside the literature • Patient / clinician / public input • Safety considerations (often under- reported) • Patient-important

Prioritizing outcomes • Critical i.e. critical for decision-making • Important – for decision-making but not critical • Less important • Process should be importance-driven and not evidence-driven

Why prioritize outcomes • Focuses search and evidence review / synthesis • Defines the elements of the Evidence Profile and SoF table • QoE for critical outcomes defines the QoE for the entire PICO-related evidence base

GRADE: Quality of evidence The extent to which our confidence in an estimate of the treatment effect is adequate to support a particular recommendation. GRADE defines 4 categories of quality: – High – Moderate – Low – Very low 13

Conceptualizing quality    We are very confident that the true effect lies close to High that of the estimate of the effect.      We are moderately confident in the estimate of effect:      Moderate The true effect is likely to be close to the estimate of effect , but possibility to be substantially different.     Our confidence in the effect is limited: The true effect       Low may be substantially different from the estimate of the effect.    We have very little confidence in the effect estimate:     Very low The true effect is likely to be substantially different from the estimate of effect.  14

Determinants of quality • RCTs start high • Observational studies start low What lowers quality of evidence? 5 factors: Methodological Inconsistency Indirectness Imprecision Publication limitations of results of evidence of results bias

Determinants of quality • RCTs start high • Observational studies start low What raises quality of evidence? 3 factors: Addressing Large effect Dose- confounding size response

Moving from Evidence to Recommendations

www.gradeworkinggroup.org

7 key GRADE elements • Quality of evidence concep • Assessment of quality criteria • Evidence quality – high, mod, low, very low • Evidence profiles • Explicit consideration of desirable / undesir • Strong and weak • Support for recommendations

20

Thank You