BEYOND THE STANDARD COURSE EVALUATION: EFFECTIVE PEER EVALUATION - - PowerPoint PPT Presentation

BEYOND THE STANDARD COURSE EVALUATION: EFFECTIVE PEER EVALUATION MODELS

9:00 – 9:55am

Please feel free to make comments/ask questions via

pollev.com/carolina anytime during this session.

Moderator: Kelly Hogan Associate Dean of Instructional Innovation, College A&S

WHAT CONSTITUTES EFFECTIVE OR EXCELLENT TEACHING?

WHO WEIGHS IN ON THE EVIDENCE?

credit: Viji Sathy

Students Peers Self- Reflection

Students

• Not linked to student learning

(i.e. no correlation between learning more and highly rated professors)

• Biased (age, race, gender)

Limited but can provide students a voice Solution? “Reduce the weight of student ratings in decision making”

• B. Utl et al., Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and

student learning are not related. Studies in Educational Evaluation, 2017

Self- Reflection

A teaching Statements allows a teacher to reflect on:

• what has worked/not

worked

• growth over time
• their goals for course design

and class time

• evidence they have that

students are learning

• student and peer

evaluations of their teaching

Peers

Peer Review of materials and classroom

• Can evaluate alignment

between curricular goals and teaching practices

• Can evaluate

climate/inclusiveness

• Can evaluate content

expertise and teaching practices

Formative peer review:

• ccurs frequently with

consistency “for the sole purpose of fostering professional growth and improved practice.” Summative peer revew: are for the purpose of reappointment, promotion, or

• tenure. “Summative

evaluations must be based on a clear set of performance standards that are identical to standards used in the ongoing formative process. They must employ a rubric of criterion- referenced assessments.”

Definitions adapted from: nea.org

VS

PEER REVIEW AT SILS

Ron Bergquist Associate Dean for Academic Affairs

The Mission of SILS is to educate the very best information professionals

WHY WE DO PEER REVIEW

The purpose of the peer observation program is to contribute to the continual improvement of the SILS educational environment by helping those who teach develop their teaching abilities and by providing observing faculty members with a broader understanding of the SILS curriculum.

SELECTION OF REVIEWERS

For tenure-track faculty, teams of two faculty members will form visitation groups for the evaluation procedure. At least one member of each team will be knowledgeable in the area being taught and at least one member of each team (preferably both) will be an associate or full professor.

THE PROCESS

Pre-Observation Conference Provide Context Observation Narrative Notes Post-Observation Conference

a balanced picture of the instructor's teaching, specifying areas of particular effectiveness as well as areas that could be improved

Afterwards

Periodicity

Tenured – every third year Non-tenured – three consecutive semesters Term – every third semester Adjuncts – every semester

THE CHALLENGE

The Future?

Maybe some changes needed

Peer Review in Exercise and Sport Science

Alain J. Aguilar

Teaching Associate Professor

Peer Review in Asian Studies

Li-ling Hsiao

Associate Professor, Chinese Language and Literature Associate Dean and Director of First-year Curricula

When?

• When a faculty teaches a new course in the

department

• When the first class observation has not gone

well

• When the faculty is reviewed for reappointment
• r promotion

How?

• The faculty observed must provide the course

syllabus, lesson plan of the class, and the reading assignments to the peer observer in advance.

• After the class is observed, the observer will meet the
• bserved faculty and brainstorm on how to improve

the course.

• The observer will then submit a report to the Chair to

be part of the observed faculty’s dossier.

What?

• A. PREPARATION
• 1. The instructor had a

clear lesson plan.

• 2. The instructor

planned activities to target specific teaching goals.

• 3. The planned activities

are appropriate for the students’ level.

• 4. The plan was geared

toward real/authentic language use.

• 5. The instructor’s plan

includes specific exercises for students to perform and allots time for each segment of the class.

• B. CLASS PRESENTATION AND MANAGEMENT
• 1. Class started on time.
• 2. The lesson was presented effectively.
• 3. The lesson was presented clearly.
• 4. There were smooth transitions between activities.
• 5. The time allotted for activities was appropriate.
• 6. Major points of teaching and activities were sequenced in a logical way.
• 7. The amount of teacher talk and student talk was appropriate.
• 8. The type and amount of teacher feedback was adequate.
• 9. Cultural instruction was integrated into class activities.
• 10. The activities/exercises chosen to achieve the objectives were effective.
• 11. The instructor divided his or her attention among students appropriately.
• 12. Student participation was focused on the task at hand.
• 13. The instructor used the target-language appropriately.
• 14. Audio-visual and other supplementary materials were used appropriately.
• 15. The use of technology (power point, web materials) was appropriate given

the material being presented.

• 16. The instructor displays enthusiasm for teaching the target language.
• C. CLASSROOM

ATMOSPHERE

• 1. Student participation

was active and lively.

• 2. The class atmosphere

was positive, open and accepting.

• 3. The instructor was

sensitive to students' difficulties and abilities.

• 4. The instructor treated

students with respect while not allowing any disruptive behavior. Record comments on the

• bservation and any

suggestions for the instructor here:

Why?

• To assist new faculty develop the teaching skills at the

very beginning stage

• To plan strategies for improvements for new faculty
• To ease the loads of multiple course observations

required during the semester of reappointment or promotional review

• To catch any problem early on instead of unexpected

surprises in the reappointment or promotional review.

BEYOND THE STANDARD COURSE EVALUATION: EFFECTIVE PEER EVALUATION MODELS

Angela Broome, DDS, MS

Associate Professor Department of Diagnostic Sciences, Division of Oral and Maxillofacial Radiology UNC School of Dentistry

9:00 – 9:55am

UNC School of Dentistry Faculty Development Committee

PEER EVALUATION OF CLASSROOM TEACHING

Wilder RS, Guthmiller JM. Empowerment through mentorship and leadership. J Evidence Based Dent Pract 2014;14S:222-226.

Mentoring Program

PEER EVALUATION OF CLASSROOM TEACHING

Newman LR, Roberts DH, Schwartzstein RM. Peer Observation of Teaching Handbook. Shapiro Institute for Education and Research at Harvard Medical School and Beth Israel Deaconess Medical Center. MedEdPortal ID# 9150; 2012. Available from: www.mededportal.org/

Faculty Peer Evaluation

RESOURCES

• UNC Division of Speech and Hearing Sciences
• Peer Review of Classroom Teaching
• University of Texas Dental Branch at Houston
• Peer Evaluation Form for Lectures
• Shapiro Institute for Education and Research at Harvard Medical School and Beth

Israel Deaconess Medical Center

• Peer Observation of Teaching Handbook
• Center for Teaching Vanderbilt University
• Peer Review of Teaching
• Center for Faculty Excellence University of North Carolina
• Faculty Peer Visits Program

GOAL

Provide a positive, learning experience through meaningful feedback Identify strengths and weaknesses as means of formative assessment Develop strategies excellent teaching skills Discover innovative ways of teaching

CURRENT PROCESS

• Once per year by a peer member (three selected by faculty

member, one chosen by mentoring chair)

• Initial meeting of faculty member and peer reviewer for

discussing the objectives of the presentation, teaching strategies, and the evaluation process.

• During presentation, peer reviewer attends presentation and

evaluates

• Within 2 weeks following, faculty member and peer reviewer

meet and discuss evaluation

CHALLENGES

Meaningful Feedback Adherence to Process Review Calibration

OPPORTUNITIES

Existing Form

OPPORTUNITIES

Updated Form

OPPORTUNITIES

Rubric

Category Exceeds Expectations Meets Expectations Does not meet expectations Organization:

1 communicates objectives including how material relates to past/future topics Clearly communicates objectives; relates objectives to course context; relates objectives to expected leraning outcomes; reinforces objectives throughout the session States the objectives in outline, presentation, slide, and/or verbally Does not provide objectives in any format or at any time during the session 2 provides clinical relevance to the topic Explains relevance to other course content, future course, clinic, private practice application; answers the question, "how is this information significant to the student?" and does so throughout the session provides topic relevance during the session does not desribe relevance to course content or future applications 3 presents material in an organized manner presents material in a logical, organized framework and refers back to the framework and learning objectives through verbal or presentation prompts throughout the session presents material in an organized framework but may not provide periodic queues or links to the objectives does not provide material in a logical or recognizable framework; content lacks any recognizable flow 4 presents material within the context of the course demonstrates an undertanding of the course content and how topic relates to previous and future topics references how material may relate to course content does not link topic with previous or future course topics 5 designs session with effective use of instructional media uses any of the following as appropriately for intended, meaningful learning: flipped classroom; think, pair, share; audience polls, developed intereactive peer project/break out sessions with facilitator monitoring; effectively implemented attempts use of a method but not effectively implemented; limited effect or limited facilitator monitoring/interaction during session lacks any clear use of instructional media; misses

• pportunities to use media when it would be effective

for the session

DISCUSSION “RULES”.

• Raise your hand. (Please limit your comments to 60

seconds or less, so that many people have an

• pportunity to join the conversation.)
• Write on a notecard and signal to the moderator

for pick up.

• Use pollev.com/carolina for anonymous

questions/comments.

We invite you to join into the conversation in multiple ways:

QUESTIONS

How is peer review perceived in your department? Peer review involves a collection of methods for assessing faculty performance. What processes have others used for peer evaluation? Should peer review processes be considered formative (improvement) or summative (related to promotion and tenure)? Are the peer reviewers trained? Who are the peer reviewers? How is self-assessment/reflection incorporated into the peer review process?