Effective Effective Assessment and Feedback Assessment and - PowerPoint PPT Presentation

Effective Effective Assessment and Feedback Assessment and Feedback in Clinical Precepting in Clinical Precepting Jonathan Rohrer, Ph.D., D.Min. Statewide Campus System MSUCOM rohrerjo@msu.edu

Why Competence Assessment? Why Competence Assessment?

Why Competence Assessment? Why Competence Assessment? It is mandated by the AOA

What is Competence What is Competence Assessment? Assessment? Competence assessment enables the medical educator and the student or resident to think systematically about the cognitive and technical skills as well as the informing values that are necessary to practice medicine in the treatment of patients.

What is Competence? What is Competence? The habitual use of knowledge, technical skills, and values in clinical reasoning to effectively treat patients in a practice setting.

Objectives of Presentation Objectives of Presentation � Summarize the elements of appropriate feedback & assessment that enable growth in clinical competence. � Explain four principles of effective assessment and feedback. � Explain key methods for teaching & assessing procedural skills.

Elements of Appropriate Elements of Appropriate Feedback and Assessment: Feedback and Assessment: Summative � Provide accountability and certification � Provide barriers for practice and non- practice

Elements of Appropriate Elements of Appropriate Feedback and Assessment: Feedback and Assessment: Formative and Summative Assessment Formative and Summative Assessment Formative Summative � Promoting reflection � Provide accountability and certification � Improving decision making � Provide barriers � Providing reassurance � Guiding future learning � Providing criteria

Principles for Effective Principles for Effective Feedback Feedback 1. Competency Objectives

Principles for Effective Principles for Effective Feedback Feedback 1. Competency Objectives 2. Conditions: Real World

Principles for Effective Principles for Effective Feedback Feedback 1. Competency Objectives 2. Conditions: Real World 3. Criteria/Cut Off: Level of Competence vs. Non-competence

Principles for Effective Principles for Effective Feedback Feedback 1. Competency Objectives 2. Conditions: Real World 3. Criteria/Cut Off: Level of Competence vs. Non-competence 4. Choice of Assessment-Performance Based with Specific Detailed Behaviors

1. Competency Objectives 1. Competency Objectives What are the competencies enfolded in this outcome? Cognitive-knowledge Behavioral-skills Affective-Attitudes/ Values

1. Competency Objectives: 1. Competency Objectives: Cognitive Domain Cognitive Domain � States principles and practices of industrial hygiene � Defines key terminology of occupational illnesses

1. Competency Objectives: 1. Competency Objectives: Behavioral Domain Behavioral Domain Demonstrates skills necessary to treat occupational illnesses � Demonstrates clinical decision making in the treatment of occupational illnesses � Diagnoses common occupationally-related disorders

1. Competency Objectives: 1. Competency Objectives: Affective Domain Affective Domain Shows commitment to occupational health in the community � Expresses the importance of occupational health issues � Indicates willingness to treat occupational illnesses in practice

2. Conditions 2. Conditions � What is the context of performance assessment? � What is the medium for the performance?

3. Criteria/Cut Off 3. Criteria/Cut Off � How many? ? � How much? � How long?

3. Criteria/ Cut Off: Cognitive 3. Criteria/ Cut Off: Cognitive Must score 80% or better in each of the following domains: � research statistics � principles of industrial hygiene � occupational illness terminology

3. Criteria/ Cut Off: Behavioral 3. Criteria/ Cut Off: Behavioral � Performs a thoracic pump in the treatment of a postoperative laparotomy patient

3. Criteria/ Cut Off: Behavioral 3. Criteria/ Cut Off: Behavioral � Performs a thoracic pump in the treatment of a postoperative laparotomy patient � Inserts a catheter into a male patient who presents at the emergency room with complaints of nausea, extreme pain in the lower back, and an inability to urinate

3. Criteria/ Cut Off: Affective 3. Criteria/ Cut Off: Affective � treats patients with courtesy and respect. � rates willingness to integrate occupational health into practice at 4 or better on a 5-point scale (5 = the most willing)

4. Choice of Assessment: 4. Choice of Assessment: Checklists Checklists � Assess presence or absence of items � Direct attention to observed performance � Easy to construct with clear competency objectives � Require that absence or presence defines competence � Require an inclusive list of behaviors

4. Choice of Assessment: 4. Choice of Assessment: Checklist Example Checklist Example Performs a thoracic pump in the treatment of a postoperative laparotomy patient : Patient is supine with the physician at the head of the table. Hands placed just inferior to clavicles with heels of hands over ribs 2-4 Angle hands laterally with fingers spread Apply gentle rhythmic pumping through flexion-extension motion of elbows (120x minute) Apply treatment for one minute Yes___ No___

4. Choice of Assessment: 4. Choice of Assessment: Rating/ Likert Scales Rating/ Likert Scales � Assess degree to which attitude/ performance is present � Direct attention to observed performance � Easy to construct with clear objectives � Require an inclusive list � Allow judgments for proficiency

4. Choice of Assessment: 4. Choice of Assessment: Example of Rating Scale Example of Rating Scale The physician encourages patient responses. The physician encourages patient responses. Uses directive Uses open- questions; uses ended questions; closed-ended uses echoing questions; uses paraphrasing; discourages uses exploratory questions questions

4. Choice of Assessment: Tips 4. Choice of Assessment: Tips for Checklists and Scales for Checklists and Scales � observe the behavior � write clear instructions � choose descriptors/anchors carefully � train users

4. Choice of Assessment: Likert 4. Choice of Assessment: Likert Scales Scales � items have statements of attitude or opinion (not fact) � items have five to seven points � points are equidistant � points are balanced at either end

Resources Resources � Ronald M. Epstein. Assessment in Medical Education. The New England Journal of Medicine. Volume 356:387-396, January 25. 2007, Number 4. � Norman E. Gronlund. Writing Instructional Objectives for Teaching and Assessment. Seventh Edition, Pearson, Merrill Prentice Hall, 2004 � Johns Hopkins Giving Feedback Online Course http://www.hopkinsmedicine.org/fac_development/vid eo/flash/index.html

Jonathan Rohrer, Ph.D., D.Min D.Min. . Jonathan Rohrer, Ph.D., rohrerjo@msu.edu rohrerjo@msu.edu The materials referenced in this lecture are available on the SCS Website: http://scs.msu.edu/GMESeries2010/