A Web-based Module to Facilitate the Direct Observation of Trainees - - PowerPoint PPT Presentation
A Web-based Module to Facilitate the Direct Observation of Trainees Siddharta G. Reddy, MPH Eric S. Holmboe, MD American Board of Internal Medicine Presented at Society of General Internal Medicine 32 nd Annual Meeting 13-16 May 2009
Presented at Society of General Internal Medicine 32nd Annual Meeting 13-16 May 2009
1.7% 7 3rd-year Fellow 1.2% 5 2nd-year Fellow 1.5% 6 1st-year Fellow 0.5% 2 4th-year Resident 18.5% 76 3rd-year Resident 16.6% 68 2nd-year Resident 45.4% 186 1st-year Resident 4.1% 17 4th-year Medical Student 10.5% 43 3rd-year Medical Student % N
49.8% 6.3% 38.3% Hospital Ward (N=128) Longitudinal Outpatient Clinic (N=71) Private Outpatient Office (N=12) Other (N=12) ICU
(N=9) 3.4% 2.2%
Low (N=68) High (N=84) Moderate (N=258) 16.6% 20.5% 62.9%
Counseling (N=96) Data Gathering (N=185) Therapy (N=58) 23.4% 45.1% 17.3% 16.6% Diagnosis (N=71)
0% – 100% 66.3% 2 2.5 Fall Risk Prevention Implemented 30% – 100% 89.9% 10 8.7 Review Of Medications For Adverse Reactions 0% – 100% 63.1% 8 6.6 Review of Unnecessary/Inappropriate Meds 10% – 100% 70.3% 8 6.4 Discharge Planning 0% – 100% 42.2% 3 3.0 Life Sustaining Care/End of Life 0% – 75% 30.1% 10 10.0 Self Care Compliance Eligible Patients Mean Reviews per Faculty: 10 0% – 100% 47.7% 2 2.0 Indications For Urinary Catheter Reviewed Daily 1 – 23.5 days 6.5 days 2 2.0 Average Days Of Catheter Placement N/A N/A 2 2.0 Urinary Catheter Inserted 10% – 100% 61.1% 7 7.1 Pneumococcal Vaccination 20% – 100% 85.0% 10 10.1 Internal Consistency* Range Mean Median Mean Measure
* Internal Consistency, as implemented in the beta test version of the PIM, was a single-item dichotomous measure. A list of criteria and definitions were provided as a guide for faculty. The current implementation, which was revised after the beta test and seen in the screenshot to the left, utilizes those criteria as specific measures of internal consistency. Results from the current implementation are thus not available.
2 – 9 2 – 9 2 – 9 2 – 9 3 – 9 2 – 9 1 – 9 Range 8.3% 6.5 Overall Clinical Competence 8.5% 6.5 Organizational/Efficiency 22.7% 6.4 Medical Interviewing Competencies 33.9% 6.7 Counseling 12.9% 6.5 Clinical Judgment N/A 7.2 Humanistic/Professionalism 34.4% 6.1 Physical Examination Not Observed Mean
73.9% 303 Immediate Period 0.7% 3 Greater than 1 week 3.9% 16 1 week 5.1% 21 24 hours 16.3% 67 Same day % N
5 – 40 hrs 10 hrs 13 hrs Entire module (10 Observation- Audits cycles & Improvement Plan) 5 – 90** min 40 min 43 min Single Observation-Audit- Feedback instance Range Median Mean
** A single outlier of 1500 minutes, deemed an error, was removed.
Module Impact
Evaluation strategy changes made
Feedback changes
Clinical practice change Module Experience
Very good or Excellent overall
Would recommend to colleague
Would use again for MOC
Would use beyond MOC Module Utility (Very Good or Excellent)
Documenting trainee evaluation
Documenting medical record audit
Documenting feedback