Validation of a tool for summative assessment of endoscopy performance Christen K. Dilly, Zach Morgan, Smitha Marri, Nabil Fayad
Background • Used at the end of each • Construct validity: month of training developed using ASGE recommendations 1 • Completed by 1-2 faculty members per fellow 1. ASGE Training Committee. Principles of training in GI endoscopy. Gastrointestinal Endoscopy. 2012; 75(2):231-235.
Results - Reliability Question Factor Agreement Non- Technical technical Kappa • Internal consistency - 1 – Manipulates scope 1.015 -0.084 0.259** Cronbach’s alpha 2 – Mucosal eval 0.981 -0.031 0.316** 3 – Therapeutic interventions 0.928 0.040 0.254** – Total = 0.957 4 – Procedure indications 0.621 0.336 0.091 – Technical skills = 5 – Recognizes, interprets, 0.967 manages findings 0.815 0.171 0.065 – Non-technical skills 6 – Anticoagulant/ antiplatelet/antibiotic 0.421 0.509 0.048 = 0.928. 7 – Teamwork -0.008 0.860 -0.010 Inter-rater • 8 – Feedback 0.059 0.739 0.044 reliability – Kappa 9 – Quality benchmarks 0.952 -0.003 0.471** 10 – Informed consent/safety -0.015 0.890 0.026 – Categorical 11 – Responsibility 0.020 0.863 -0.014 12 - Documentation 0.064 0.818 -0.010 **= Significant to <0.001
Results - validity Pearson correlations • (relationship of procedure numbers to assessment scores): – Technical skills: 0.606 Nontechnical skills: 0.249 – • Response process – cognitive interviews Conclusions: • Fair to moderate evidence for reliability and validity • Problems with the instrument likely reflect problems with milestones-based assessment • How will this compare to EPAs?
Recommend
More recommend
