The GRADE approach to assessing individual studies Peter Morley and - PowerPoint PPT Presentation

1 1 Vienna 2012 The GRADE approach to assessing individual studies Peter Morley and Eddy Lang

2 Vienna 2012 GRADE expands quality of evidence determinants Individual study Inconsistency Risk of bias of results Failure of blinding Incomplete Methodological reporting Indirectness limitations of evidence Losses to Allocation follow-up concealment Imprecision of results Publication bias Across studies for an outcome 2

3 Vienna 2012 Risk of Bias (RCTs) • Consider studies that provided data for the outcome • Consider overall risk of bias of these studies: – Allocation concealment – Adequate sequence generation – Blinding – Selective outcome reporting – Complete outcome data assessed – other • Is Risk of Bias likely to reduce our confidence in the effect we found?

4 Vienna 2012 Risk of Bias (observational studies)

5 Vienna 2012 Adequate sequence generation • use of a mechanism that ensures that the same sorts of participants receive each intervention – that participants are randomly assigned • results in a balance of baseline characteristics between groups • “randomisation computer generated list” • “randomised by data of birth, when signed up for study”

6 Vienna 2012 Lack of allocation concealment • Those enrolling patients are aware of the group to which the next enrolled patient will be allocated • “sealed opaque envelopes” • ‘transparent envelopes”

7 Vienna 2012 Blinding • Patient, caregivers, those recording outcomes, those adjudicating outcomes, or data analysts are aware of the arm to which patients are allocated • ? “double blinded”

8 Vienna 2012 Selective outcome reporting • Reporting of some outcomes and not others on the basis of the results • With protocol • Without protocol – Heparin (blood thinner) did not measure bleeding as an adverse event – Weight loss diet but did not measure weight!

9 Vienna 2012 Incomplete outcome assessment • Loss to follow-up and failure to adhere to the intention to treat principle when indicated • Reasons for loss to follow up would not affect outcome – moved away – But left trial due to adverse events • Uneven loss to follow up between groups • Rule of thumb: >20% concern

10 Vienna 2012 Other limitations: • stopping early for benefit observed in randomized trials, in particular in the absence of adequate stopping rules • use of un-validated patient-reported outcomes • cross over studies • cluster randomised studies

11 Vienna 2012 C2015 PICO • Population : adult cardiac arrest (prehospital [OHCA], in-hospital [IHCA]) • I ntervention : the use of an ITD • Comparison : compared with no ITD • Outcomes – Neurologically intact survival (critical 9) – Discharge from hospital alive (critical 8) – Return of spontaneous circulation (important 6)

12 Vienna 2012 Inclusion/Exclusion/Articles Found • I ncluded all studies with concurrent controls. • Excluded review articles, studies with historical controls, animal studies, and studies that did not specifically answer the question. Excluded unpublished studies, studies only published in abstract form, unless accepted for publication. • 8 Articles Finally Evaluated

13 Vienna 2012 Risk of Bias in stu tudies table le: ITD+ D+ACDC DCPR vs vs Sta tandard rd CPR

14 Vienna 2012 Risk of Bias in stu tudies ta table: IT ITD+ACDCPR vs vs Sta tandard rd CPR

15 Vienna 2012 Risk of Bias in stu tudies table le: ITD+ D+ACDC DCPR vs vs Sta tandard rd CPR

16 Vienna 2012 Risk of Bias in stu tudies ta table: IT ITD+ACDCPR vs vs Sta tandard rd CPR

17 Vienna 2012 Risk of Bias in stu tudies table le: ITD+ D+ACDC DCPR vs vs Sta tandard rd CPR

18 Vienna 2012 Risk of Bias in stu tudies table le: ITD+ D+ACDC DCPR vs vs Sta tandard rd CPR

19 Vienna 2012 Risk of Bias in stu tudies table le: ITD+ D+ACDC DCPR vs vs Sta tandard rd CPR

20 Vienna 2012 Risk of Bias in stu tudies table le: ITD+ D+ACDC DCPR vs vs Sta tandard rd CPR

21 Vienna 2012 Risk of Bias in stu tudies table le: ITD+ D+ACDC DCPR vs vs Sta tandard rd CPR

22 Vienna 2012 Risk of Bias in stu tudies table le: ITD+ D+ACDC DCPR vs vs Sta tandard rd CPR

23 Vienna 2012 Overall risk of bias for study: Low, Moderate or High • “Low” risk of bias = most or all key criteria listed are met , and any violations are not crucial. • “Moderate” risk of bias if have a crucial limitation in one criterion or some limitations in multiple criteria , sufficient to lower the confidence in the estimate of effect. • “High” risk of bias if have a crucial limitation in one or more criteria , sufficient to substantially lower the confidence in the estimate of effect.

24 Vienna 2012 Risk of Bias in stu tudies table le: ITD+ D+ACDC DCPR vs vs Sta tandard rd CPR

25 Vienna 2012