Register of Clinical Prediction Rules, Methodological Quality - - PowerPoint PPT Presentation

▶
register of clinical prediction rules methodological
SMART_READER_LITE
LIVE PREVIEW

Register of Clinical Prediction Rules, Methodological Quality - - PowerPoint PPT Presentation

Aug 23, 2022 384 likes •786 views

Royal College of Surgeons in Ireland Coliste Roga na Minle in irinn Register of Clinical Prediction Rules, Methodological Quality Assessment and Implementation Strategies Emma Wallace, Claire Keogh, Susan Smith, Borislav D. Dimitrov,

slide-1
SLIDE 1

Division of Population Health Sciences

Royal College of Surgeons in Ireland

Coláiste Ríoga na Máinleá in Éirinn

Register of Clinical Prediction Rules, Methodological Quality Assessment and Implementation Strategies

Emma Wallace, Claire Keogh, Susan Smith, Borislav D. Dimitrov, Tom Fahey

slide-2
SLIDE 2

Division of Population Health Sciences

Outline

  • Section 1
  • Development of register of Clinical Prediction Rules (CPRs)
  • Search strategies to identify CPRs relevant to primary care

from the MEDLINE database

  • Section 2
  • Methodological quality assessment of CPRs
  • Section 3
  • Implementation strategies
slide-3
SLIDE 3

Division of Population Health Sciences

SECTION 1

Development of Register of Clinical Prediction Rules (CPRs)

slide-4
SLIDE 4

Division of Population Health Sciences

Defnitions

  • Clinical Prediction Rule

– Clinical tool that quantifes the contribution of

  • Patient History
  • Physical Examination
  • Diagnostic Tests

– Stratify patients diagnosis

  • Probability of having target disorder.

– Outcome can be in terms of diagnosis, prognosis, referral or treatment

slide-5
SLIDE 5

Division of Population Health Sciences

CPR Examples 1) Ottawa ankle rule

slide-6
SLIDE 6

2) The Centor Score

slide-7
SLIDE 7

Division of Population Health Sciences

Stages of development of CPR

Figure 2. Development of a Clinical Decision Rule

Level of Evidence 4 3 2 1 Step 2. Validation Evidence of reproducible accuracy Narrow Validation Broad Validation Application of a rule in a Application of rule in Similar clinical setting and multiple clinical settings Population as Step 1 and varying prevalence and outcomes of disease Step 3. Impact Analysis Evidence that rule change physician behaviour and improves patient outcomes and/or reduces costs Step 1. Derivation Identification of factors with predictive power

slide-8
SLIDE 8

Division of Population Health Sciences

Ottawa ankle rule: Derivation to Impact Analysis

  • Derived in 2 large ER departments in Canada
  • Validated initially in same ERs with a larger cohort
  • f patients (narrow validation)
  • Validated in many diferent clinical sites with

consistent results (broad validation)

  • Shown to decrease cost by reducing unnecessary

radiographs with no increase in missed fractures (impact analysis)

slide-9
SLIDE 9

Aim

  • Develop a web-based register of CPRs for use in primary care

as part of Cochrane Primary Healthcare Field

slide-10
SLIDE 10

Division of Population Health Sciences

CPR register development

  • Development of an electronic search flter-identify

CPRs relevant to primary care

  • Haynes flter in Clinical Queries
  • Broad and narrow search flters
slide-11
SLIDE 11

Academic Emergency Medicine Family Medicine American Family Physician Family Practice American Journal of Medicine Journal of American Medical Association Annals of Emergency Medicine Journal of the American Board of Family Medicine Annals of Family Medicine Journal of Clinical Epidemiology Annals of Internal Medicine Journal of Family Practice Annals of Medicine Journal of Internal Medicine Annual Review of Medicine Lancet Archives of Internal Medicine Medical Care BMC Family Practice Medical Decision Making British Medical Journal Medicine British Journal of General Practice New England Journal of Medicine Canadian Family Physician Public Library of Science Medicine Canadian Medical Association Journal Primary Care Cochrane Database Systematic Reviews Scandinavian Journal of Primary Health Care

30 journals included on the register

slide-12
SLIDE 12

Division of Population Health Sciences

Methods

  • Manually searched 30 journals relevant to primary care

for the year 2008 (‘reference standard’)

  • 7 individual electronic searches of the 30 journals (each

flter treated as ‘diagnostic tests’)

  • Test accuracy analysis: Sensitivity and specifcity
  • Aim: to maximise sensitivity
slide-13
SLIDE 13

Database Filter name Filter search string PubMed Haynes Broad Filter (HBF) (predict*[tiab] OR predictive value of tests[mh] OR scor*[tiab] OR observ*[tiab] OR

  • bserver variation[mh])

PubMed Haynes Narrow Filter (HNF) (validation[tiab] OR validate[tiab]) EBSCO host McGrath/Murphy Broad Filter (MMBF) ((predict* N3 rule* OR predict* N3 model OR predict* N3 models) OR (decision* N3 rule*) OR (TX validat*)) EBSCO host McGrath/Murphy Narrow Filter (MMNF) ((predict* N3 rule* OR predict* N3 model OR predict* N3 models) OR (decision* N3 rule*)) PubMed Teljeur/Murphy Inclusion Filter 26 item (TMIF-26) "clinical prediction" OR "clinical model*" OR "clinical score*" OR "decision rule*" OR "diagnostic accuracy" OR "diagnostic rule*" OR "diagnostic score*" OR "diagnostic value" OR "predictive outcome*" OR "predictive rule*" OR "predictive score*" OR "predictive value" OR "predictive risk*" OR "prediction outcome*" OR "prediction rule*" OR "prediction score*" OR "prediction value*" OR "prediction risk*" OR "risk assessment" OR "risk score*" OR "validation decision*" OR "validation rule*" OR "validation score*" OR (derivation AND validation) OR (sensitivity AND specificity) OR (symptoms AND signs) PubMed Teljeur/Murphy Inclusion Filter 22 item (TMIF-22) (clinical[tiab] AND predict*[tiab]) OR (clinical[tiab] AND model*[tiab] ) OR (clinical[tiab] AND score*[tiab]) OR (decision [tiab] AND rule*[tiab]) OR (derive*[tiab] AND validat*[tiab]) OR (diagnos*[tiab] AND accura*[tiab]) OR (diagnos*[tiab] AND rule*[tiab]) OR (diagnos*[tiab] AND score*[tiab]) OR (diagnos*[tiab] AND value[tiab]) OR (predict*[tiab] AND outcome*[tiab]) OR (predict*[tiab] AND rule*[tiab] OR (predict*[tiab] AND score*[tiab] ) OR (predict*[tiab] AND validat*[tiab]) OR (predict*[tiab] AND value*[tiab]) OR (risk*[tiab] AND assessment*[tiab]) OR (risk[tiab] AND score*[tiab]) OR (sensitivity[tiab] AND specificity[tiab]) OR (symptoms[tiab] AND signs[tiab]) OR (validat*[tiab] AND decision*[tiab]) OR (validat*[tiab] AND rule*[tiab]) OR (validat*[tiab] AND score*[tiab]) OR (predict*[tiab] AND risk*[tiab]) PubMed Teljeur/Murphy Exclusion Filter (TMEF) (allele OR amino OR animal OR apoptosis OR chromosome OR congenital OR dental OR dna OR endogenous OR endothelial OR epithelial OR mammalian OR mice OR molecule OR molecular OR mouse OR mutate OR mutation OR necrosis OR pathogenesis OR phosphorylation OR polymorphism OR receptor OR signal OR species OR tissue OR tumor OR tumour OR tyrosine OR vitro)

slide-14
SLIDE 14

Results

Filter name N articles retrieved N CPRs retrieved Sensitivity (%) Specificity (%)

Haynes Broad Filter 1251 31 76 81 Haynes Narrow Filter 89 12 29 99 McGrath/Murphy Broad Filter 264 23 56 96 McGrath/Murphy Narrow Filter 63 16 39 99 Teljeur Murphy Inclusion Filter-26 2432 39 95 62 Teljeur/Murphy Inclusion Filter-22 693 34 83 90 Teljeur/Murphy Exclusion Filter 3589 24 59 43

Manual ‘reference standard’ search retrieved 6344 articles, 41 of which were CPRs

slide-15
SLIDE 15

MEDLINE versus the fnal search flter applied to 30 primary care journals (1966 – 2008)

Year of publication

slide-16
SLIDE 16

Creating the International Register

Years Average articles retrieved per year Average CPRs for primary care identifed per year* 2008 – 2000 2622 25 1999-1990 2328 3 1989-1980 2243 2 * Still awaiting inter-library loans

2008-1980 resulted in 252 CPRs relevant to primary care

slide-17
SLIDE 17

Clinical Domains – ICPC2 Percentage articles retrieved for each clinical domain

% %

slide-18
SLIDE 18

Clinical Domains – ICPC2 Respiratory System

Respiratory System N=41 Strep throat 12 Pneumonia 8 Asthma 6 Respiratory symptom/complaint other 3 Chronic obstructive pulmonary disease 2 Cough 2 Respiratory infection other 2 Breathing problems , other 1 Infuenza 1 Respiratory disease other 1 Tonsillitis acute 1 Upper respiratory infection acute 1 Sinusitis acute/chronic 1

slide-19
SLIDE 19

Division of Population Health Sciences

SECTION 2

Methodological Quality assessment of CPR register

slide-20
SLIDE 20

Division of Population Health Sciences

Health Research Reporting

slide-21
SLIDE 21

Division of Population Health Sciences

Health Research Reporting

  • International initiative ‘Equator’ set up in 2008

– Aim to provide resources & education enabling improved reporting and monitor progress – Develop network of reporting guidelines e.g.

  • CONSORT- RCTs
  • STARD- Diagnostic Accuracy studies
  • STROBE- Observational Studies
  • PRISMA- Systematic reviews
slide-22
SLIDE 22

Division of Population Health Sciences

Methodological Quality assessment of CPRs

  • To date Equator have not developed a reporting

guideline for CPR Quality assessment

  • Wasson et al, 1985, Laupacis et al, 1997, Stiell et al

1999,McGinn et al, 2000, Cook et al, 2010

  • Diferent quality markers for derivation and

validation CPR studies

  • Consensus to use McGinn quality checklist due to

content and utility

slide-23
SLIDE 23

Division of Population Health Sciences

Results-preliminary

  • Quality assessment of a small representative sample
  • f CPR register assessed to date
  • 1988
  • 1998
  • 2008
slide-24
SLIDE 24

Division of Population Health Sciences

Results

  • 1988
  • Derivation CPRs=3
  • Validation CPRs=0
slide-25
SLIDE 25

Results CPRs (derivation), 1988, n=3

slide-26
SLIDE 26

Division of Population Health Sciences

Results

  • 1998
  • Derivation CPRs n=3
  • Validation CPRs n=7
slide-27
SLIDE 27

Results CPRs (derivation), 1998, n=3

slide-28
SLIDE 28

Results CPRs (validation), 1998, n=7

slide-29
SLIDE 29

Division of Population Health Sciences

Results

  • 2008
  • Derivation CPRs n=18
  • Validation CPRs n=20
slide-30
SLIDE 30

Results CPRs (derivation), 2008, n=18

slide-31
SLIDE 31

Results CPRs (validation), 2008, n=20

slide-32
SLIDE 32

Division of Population Health Sciences

SECTION 3

Implementation strategies

slide-33
SLIDE 33

Division of Population Health Sciences

Implementation

1.Register of CPRs Cochrane Primary Health Care Field

  • 2. Clinical Decision Support Systems (CDSSs)
slide-34
SLIDE 34

Division of Population Health Sciences

Register of CPRs Cochrane Primary Health Care Field

  • Implement International Register of CPRs that is;

– Publicly available, web based, user friendly – Searchable across clinical domains – Maintained and updated

slide-35
SLIDE 35

Division of Population Health Sciences

Registry

1 2 3 4 5 6

slide-36
SLIDE 36

Division of Population Health Sciences

Implementation

1.Register of CPRs Cochrane Primary Health Care Field

  • 2. Clinical Decision Support Systems (CDSSs)
slide-37
SLIDE 37

Division of Population Health Sciences

Clinical Decision Support Systems

slide-38
SLIDE 38

Division of Population Health Sciences

Clinical Decision Support Systems

  • Clinical Decision Support Systems (CDSSs)

– Systems that are designed to improve clinical decision making

Key points

– Integrated with the electronic patient record – Available at the point of care – Computerised knowledge base – Provide patient-specifc content

slide-39
SLIDE 39

Register

Completing evidence base/CPR register

Systematic reviews