UTM Fall Faculty Workshop: August 16, 2018 Faculty Evaluation - - PowerPoint PPT Presentation
UTM Fall Faculty Workshop: August 16, 2018 Faculty Evaluation Process Morning Session 8:30am-11:30am, Campus-Wide Two big topics: 1. Faculty Evaluation Process--Overview and Preview of Coming Attractions 2. Bylaws--Especially changes
Morning Session 8:30am-11:30am, Campus-Wide Two big topics: 1. Faculty Evaluation Process--Overview and Preview of Coming Attractions 2. Bylaws--Especially changes mandated by the UT Board of Trustees
OVERVIEW:
systems throughout our state
What was accomplished in 2017-18 (Year One)?
became the College Committee Representatives)
each College Committee to serve on the Campus Wide Committee (cFEC)
from each college) - We have more than that with 10 volunteer departments!
Others involved with cFEC
○ Non-Voting Chair, Year One
○ Non-Voting Coordinator
○ Non-Voting ex-officio member
○ Added by the cFEC Chair, Voting member
○ Non-Voting ex-officio member
This Past Summer’s Fun!
Who are the Beta Groups? Meet the brave ten
Family and Consumer Sciences Health and Human Performance Biological Sciences Computer Science Engineering Math and Statistics Music Accounting, Finance, Economics, and Political Science Management, Marketing, and Information Systems Paul Meek Library
What to expect in 2018-19 (Year Two)
○ System Implementation ○ Determining the roles and future of the college committees ○ Feedback to cFEC and College Committees for edits/changes ○ Negotiations between the departments and their colleges on ranges, requirements, weight ranges… ○ Forms, Scales, Sources for Evaluation, and restrictive ranges for each component will be created ■ Separate forms should be generated for term appointments and adjuncts ○ Determine what changes will be required in department/college Bylaws and the Faculty Handbook ○ Participation in Faculty Evaluation workshop this afternoon and Digital Measures workshop(s) beginning in November ○ Pilot testing with Digital Measures
What to expect in 2018-19 (Year Two)?
○ Digital Measures involvement for this year is still TBD, but most likely the minimum of uploading a c.v. ○ Submit Form Declaring Role Weights at end of this academic year to be used in FEP for upcoming year ○ Will follow current “old” system of evaluation one more year which will be the final year
What will happen in 2019-2020 (Year 3) and beyond?
○ FEP integration with Tenure and Promotion ■ The cFEC does NOT support a system in which tenure and promotion decisions are solely based on the annual composite rating. ■ The cFEC DOES support a process by which the yearly annual composite rating informs decisions on tenure and promotion. ○ Determine how FEP should impact Merit Pay ○ Continually adjust and improve
community and consensus
Overview of Beta Process for Upcoming Year 1.Establish the Faculty Role Model 2.Determine Role Parameter Values 3.Define Each Role’s Components 4.Establish the Role Component Weights 5.Determine Appropriate Sources 6.Establish Source Weights 7.Determine How to Gather Information 8.Design or Select Appropriate Forms *Conduct both new and old process.Compare/Feedback/Edit FEP.
Institutional Requirements v/s College/Department Decisions
Institutional Requirements - Set Items Campus-Wide
Weights of Components of Teaching Ranges for three roles
Teaching (50% to 90%)
Scholarship (5% to 40%)
Service (5% to 40%)
College/Department Decisions
the yearly evaluation process
enact smaller ranges within the set campus ranges for their units
submission of Declaration of Role Weights Form
for their departments to conduct face to face meetings with faculty for annual review.
○ Fixed Amounts or Faculty Flexibility? ○ How much value or weight should be placed on the information provided by selected sources?
weights)
○ Forms, Checklist, Questionnaire, or Other? ○ Build Data Gathering Matrix for each role
MUST DO at either College or Department Level:
Other Items to Consider:
○ TWO outcomes ■ A numerical score between 1.00 and 4.00 ■ One of four Board of Trustees categories (similar to the current system) ○ At this time, there is no fixed translation between the numerical score and the Board of Trustees category. The cFEC will collect and analyze data from this year’s beta testing (and perhaps next year’s process results) before making any recommendations regarding how to interpret the numerical score as a categorical rating.
departments on existing student rating instruments
institutions nationally)
For more information, see https://www.ets.org/sir_ii/about
○ Faculty Evaluation Tab on Assessment Website
■ FEP OVERVIEW ■ GUIDEBOOK ■ FAQs ■ MEETINGS and DEADLINES CALENDAR ■ PROJECT TIMELINE in progress ■ COMMITTEES and BETA GROUPS - Who is doing what? ■ RESOURCES: FORMS/TEMPLATES/FACULTY EVALUATION WORKSHOP SLIDES (today’s presentation)
Digital Measures
What is it? Software to track Faculty Productivity (teaching, scholarship, and service) What does it do? Creates faculty vitas, annual planning and assessment reports, etc. Who uses it? The College of Business and Global Affairs already uses it. We will be using it campus-wide. Is it hard to learn or use? No--we will have a guidebook and training sessions. We are planning multiple opportunities to meet in small-group settings and work. When do I have to start using it? Spring 2019. We’ll start small (what have you done in 2018-19) then work backwards to fill in earlier years. Some departments are planning to have student workers help with inputting information. Stay tuned for more info....
16 August 2018 caldwell@utm.edu
It didn’t, but you forgot to volunteer.
Bias disclaimer: I always seek to
bylaws with our colleagues, interspersed with large group discussion. The first items will take the most time, later ones are quick and easy.
Planned order: 1, 2, 3, 4, 5, (6 & 7), 8, 9, 12, 11, 10
[BOT Resolution for BT0006, 18 Mar 2018] 3. (b) … Departments shall submit the required procedures [for Appendix A: Tenure] for approval no later than November 30, 2018 and shall be effective beginning with the next tenure review cycle following approval.
“… having a set of bylaws to address promotion and tenure within the university framework instructs outside evaluators what the department considers important among the criteria for T&P. It was that specific issue that led to us developing bylaws back when I was dean. The performing arts people needed it the most as did several of our departments in A&S.” “Bylaws also codify rules for selecting department chairs and for committee structures. There are good reasons departments and colleges should have their
CC 3
(Don’t get defensive yet, please wait until the next slide.)
Suggestion: Simplify Using Hierarchy When Possible
Faculty Handbook BOT Policy Federal and State Law Campus Procedures College Bylaws Departmental Bylaws
(< 5 min)
add reference or page)
If so, place a checkmark in the query box.
List 1
(Uniqueness? Protection?)
What are (or should be) the differences between the standards for tenure and for promotion to associate professor?
(In the white 25 page June BOT Excerpt)
Approved New Handbook Text:
Enhanced Tenure Track Review (ETTR)
PPC meets August 30, Senate votes September 25, …, BOT votes November 2.
All tenure-track faculty members shall submit a dossier to be reviewed in either the third or fourth year of the faculty member’s probationary period (to be determined by the department chair) for an Enhanced Tenure Track Review (ETTR) to be conducted by department faculty and the department chair following the same committee structure, voting procedures as tenure with the exception that external reviews are not required. The department faculty will record their vote and write a recommendation including suggestions for furthering the progression towards tenure, and a minority and majority report if required using the form at the Academic Affairs website. The department chair will meet with the faculty member to share the report of the department faculty, as well as his or her own written assessment, then discuss plans enhancing the faculty member’s progression toward tenure. The committee report and the chair’s assessment will be shared with the college dean.
for PPPT
site
15
if FEP becomes sufficient that could be adjusted.
adjustment is. Merit pay? Annual raises? Raises with promotion?
[Faculty Handbook 3.4.1] The criteria for evaluating non-tenure-track faculty for purposes of hiring and retention must be approved by the departmental faculty and published in the departmental bylaws.
Music Example: Proposed amendments to these bylaws must be brought to the full faculty by the Advisory Committee, which will accept suggestions from any faculty member. Proposed changes must be distributed electronically to the full faculty at least 10 business days prior to a vote. The bylaws may be amended by a simple majority vote at any departmental meeting; faculty members who are unable to attend may submit their vote on any amendment(s) to the Chair in advance of the meeting. Amendments must be approved by the Dean of the College of Humanities and Fine Arts and the Vice Chancellor for Academic Affairs. Math Example: These Bylaws may be amended at any regular or special Departmental faculty meeting for which particular notice has been given. A simple majority vote is required to amend any provision. Any amendments will be forwarded to the Dean and the VCAA. Ed Studies: These by-laws may be amended at any regular or special Departmental faculty meeting for which particular notice has been given. Amendment of any provision requires a simple majority vote.
https://www.utm.edu/departments/music/ https://www.utm.edu/departments/educate/documents.php
Music Example: 2.4 Voting. Full-time tenured and tenure-track faculty are eligible to vote on all motions and matters brought before the department. Full-time lecturers may vote on all issues with the exception of those dealing with personnel. Part-time, adjunct faculty, and faculty with visiting appointments are not eligible to vote. If a faculty member cannot attend a meeting, they may submit their vote
Proposals must be vetted and approved by the appropriate committee prior to consideration by the faculty. The committee chair must distribute proposals … A simple majority vote with a quorum will decide all issues. … Math/Stat Example: Membership and Voting: All faculty members who are serving as full-time faculty or have budgeted part-time teaching responsibilities within the Department pursuant to an appropriate appointment by the Vice Chancellor for Academic Affairs (VCAA) are entitled to vote, subject to University guidelines, on all matters for which faculty input is sought. Administrative staff who have faculty rank in the department are considered voting members of the faculty during those semesters in which they are teaching. Ed Studies Example: Membership and Voting: All term, tenured, or tenure-track faculty with 50% or more teaching responsibility who are serving as full-time faculty within the Department of Educational Studies pursuant to an appropriate appointment by the Vice Chancellor for Academic Affairs (VCAA) are entitled to vote, subject to University guidelines, on all matters for which faculty input is
in all Departmental responsibilities.
[BT0006 App B. 2 p. 23] The department head shall direct the tenured departmental faculty to consider the faculty member’s performance in teaching, research, and service and, by an anonymously cast vote taken in accordance with applicable department or college bylaws, to make a recommendation on the question of whether the faculty member’s performance constitutes Adequate Cause for termination. [Question: Does this imply bylaws must proscribe the method?]
“What important questions remain unanswered?”
Remainder of Morning Session (if any)
LUNCH 11:30am-1:30pm
Beta Groups, Deans
Chris K Caldwell 16 Aug 2018
CFEC (GUIDEBOOK PAGE 8)
57
The Campus-wide Faculty Evaluation Committee (cFEC) is the faculty group that will continue to oversee and adjust this evaluation process. They will: 1. Listen 2. Provide Guidance 3. Collect Data 4. Adjust
They will continue to work with the Senate’s PPC.
Professor Emeritus The University of Tennessee Health Science Center
58
59
60
REMINDER FROM THE GUIDEBOOK (PAGE 1)
61
“Until we have used this process once or twice, there will be no fixed translation between these two types of outcomes (numeric and categorical).”
FOUR CATEGORIES (GUIDEBOOK PAGES 13-14)
62
FOUR CATEGORIES (GUIDEBOOK PAGE 14)
Professional Level for Rank “Applies to faculty who, during this rating period, consistently met the campus, college and departments’ standards for professional
valued professionals on whom UT Martin relies for the continued success of our programs, missions, and goals; and should include the clear majority of our faculty.”
FOUR CATEGORIES (GUIDEBOOK PAGE 13)
Exemplary Performance for Rank “Applies to faculty who, during this single rating period, consistently and substantially exceeded the campus standards of professional
rare faculty members who are true models of peak academic performance and who make significant contributions to their departments, university, academic field, and society.”
PURPOSES OF FACULTY EVALUATION (1 OF 2)
65
PURPOSES OF FACULTY EVALUATION (2 OF 2)
66
REMINDER FROM THE GUIDEBOOK (PAGE 6) 1.2. Tenure and Promotion “The results from this annual faculty evaluation process will inform decisions about tenure and promotion in the same way the results from the old evaluation processes did, but they will not directly change any of the current tenure and promotion requirements or processes.”
FROM ARREOLA’S TRAINING
68
69
1. Establish the Faculty Role Model 2. Determine Role Parameter Values 3. Define Each Role’s Components 4. Establish the Role Component Weights 5. Determine Appropriate Sources 6. Establish Source Weights 7. Determine How to Gather Information 8. Design or Select Appropriate Forms
70
STEP 1: ESTABLISH THE FACULTY ROLE MODEL
71
Faculty Role Model
Teaching
Scholarship
Service
STEP 2: DETERMINE ROLE PARAMETER VALUES
72
minimum weight
Faculty Role Model
maximum weight
50% Teaching 90% 5%
Scholarship
40% 5% Service 40%
EXAMPLE: MATH/STAT ROLE PARAMETERS
73
minimum weight
Faculty Role Model
maximum weight
50% Teaching 90% 15%
Scholarship
40% 15% Service 40%
REMINDER FROM THE GUIDEBOOK (PAGE 12)
74
2.3. Setting the Weights of Components “It is important to note that these weights are measures of value and are not the same as
independent of the amount of time it took. It is entirely possible that some faculty teaching only
their evaluation based on the quality of their
will be considered in the weight negotiation.”
STEP 2: ACTIVITY ONE (5 MINUTES)
75
Locate and Verify
versions?
STEP 3: DEFINE EACH ROLE’S COMPONENTS
76
STEP 4: ESTABLISH ROLE COMPONENT WEIGHTS
77
STEP 4: COMPONENT WEIGHTS FOR TEACHING
78
Total Source Impact Weights
SOURCES
Rol e Components
Instructional Design Instructional Delivery Instructional Assessment Course Management
Component Weights
fixed
STEP 4: MATH/STAT WEIGHTS FOR SCHOLARSHIP
79
100%
Total Source Impact Weights
SOURCES
Role Components
Proficiency Discovery / Creativity Dissemination Translation
Component Weights
20 – 60% 20 – 60% 20 – 60% – 40%
STEP 4: POLITICAL SCIENCE WEIGHTS FOR SCHOLARSHIP
80
100%
Total Source Impact Weights
SOURCES
Role Components
Proficiency Discovery / Creativity Translation
Component Weights
20% 0% 80% 0%
Dissemination
STEP 4: OTHER BSBA WEIGHTS FOR SCHOLARSHIP
81
100%
Total Source Impact Weights
SOURCES
Role Components
Proficiency Discovery / Creativity Disseminatio n Translation
Componen t Weigh ts
20 - 40% 20 - 40% 30 - 50% 0 - 5%
82
This component of scholarship is based on the general principle of utilizing research results within a discipline, not necessarily from your
the development of new products, services, performances, or artistic expressions of value
social audience.
STEP 4: AG WEIGHTS FOR SERVICE
83
100%
Total Source Impact Weights
SOURCES
Role Components
To the Institution To the Profession To the Community To the Students
Component Weights
40 - 75% 20 - 50% 10 - 40% 0 - 70%
STEP 4: BIOLOGICAL SC. WEIGHTS FOR SERVICE
84
100%
Total Source Impact Weights
SOURCES
Role Components
To the Institution To the Community To the Students
Component Weights
15 - 60% 0 - 45% 0 - 45% 15 - 60%
To the Profession
STEP 4: ACTIVITY TWO (10 MINUTES)
85
86
Exciting features!
87
Potential features:
88
HOW NOT TO DETERMINE APPROPRIATE SOURCES
89
STEP 5: DETERMINE APPROPRIATE SOURCES
90
STEP 5: DETERMINE APPROPRIATE SOURCES
91
STEP 5: SOURCE IDENTIFICATION MATRIX FOR TEACHING
92
Role Components
Students Peers Chair Instructional Delivery Instructional Design Instructional Assessment Course Management
STEP 5: ACTIVITY THREE (10 MINUTES)
Lightly (in pencil) place “yes” for sources you will use and “no” for those you will not.
REACH CONSENSUS!
(If you group is done early, start thinking about Step 7: what will be used to collect info from these sources: rubric? form? survey?)
93
STEP 5: ACTIVITY FOLLOW-UP
94
agreeing on these?
students for instructional delivery?
Quick Questions:
Determine how much value or weight should be placed on the information provided by the various selected sources
95
STEP 6: DETERMINE SOURCE WEIGHTS
Determine the impact the information from the various sources will have on the overall evaluation of each role. (“source impact weights”)
96
STEP 6: SOURCE WEIGHTS FOR TEACHING
97
Total Source Impact Weights
Peers
Role Components
Instructional Delivery Instructional Assessment Course Management
Component Weights
100 25 75 100 80 20
Students Chair
YES NO NO YES YES NO NO YES YES NO NO YES
Instructional Design
STEP 6: SOURCE IMPACT MATRIX FOR TEACHING
98
Total Source Impact Weights
Peers
Role Components
Instructional Delivery Instructional Assessment Course Management
Component Weights
100 25 75 100 80 2
Students Chair
Instructional Design
STEP 6: ACTIVITY FOUR (15 MIN)
Agree on source weights for your source matrices
99
step)
reasonable for your unit?
Determine what type of form, questionnaire, checklist, or other data gathering method will be used to obtain the specified information from each source.
100
STEP 7: DATA GATHERING MATRIX FOR TEACHING
101
Student s
Chair Peers
Role Components Sources
Instructional Delivery Instructional Design Instructional Assessment Course Management
STEP 7: DATA GATHERING MATRIX FOR TEACHING
102
Students
Chair Peer s
Role Components Sources
Instructional Delivery Instructional Design Instructional Assessment Course Management
Rating Form
Peer Review
Interview
Checklist, Grade Report
Rating Form
Peer Analysis
STEP 7: DATA GATHERING MATRIX
103
Again, consider how much you are biting off.
you are already doing?
processes at start?
chair at first?
in the room may be able to address?
STEP 7: ACTIVITY FIVE (10 MIN)
104
In the appropriate cells on your source matrix identify the method you plan to use to collect data from that source.
We will work 10 minutes, then pause to discuss & pool
we need.
105
STEP 8: DESIGN AND/OR SELECT FORMS
106
STEP 8: DESIGN AND/OR SELECT FORMS
107
STEP 8: DESIGN AND/OR SELECT FORMS
108
DRAKE’S PROFESSIONAL ACTIVITIES RECORD (PAR)
109
STEP 8: OUR CHOSEN RATING FORM
110
Teaching components
Organization and Planning and Section E. Supplementary Instructional Methods
and Section C. Faculty/Student Interaction
nationally)
STEP 8: ACTIVITY SIX (10 MIN)
STEP 8: DESIGN / SELECT APPROPRIATE FORMS
112
STEP 8: WHAT IF NOT FROM 1 TO 4?
TRANSLATIONS DO NOT NEED TO BE LINEAR
114
rating description 4.0 Rare or exceptionally high student ratings in every section taught 3.6 For more than half of the sections, scores are exceptionally high 3.2 For all class sections, student ratings are at least satisfactory 2.9 For the majority of class sections, student ratings are satisfactory 2.0 For the majority of class sections, student ratings are unsatisfactory 1.5 Clear evidence of consistently unfavorable student ratings 1.0 Rare or exceptionally low student ratings in all sections; or fails to complete any student evaluations
CREATING SCALES (GUIDEBOOK PP. 17-18)
115
“Departmental and college rating scales will be used to rate faculty in each component of each role in the faculty evaluation system, so it is essential that these scales be developed with the proper input of the faculty who will be held accountable to the expectations reflected in the rating scales.”
CREATING SCALES (GUIDEBOOK PAGE 18)
116
CREATING SCALES (GUIDEBOOK PAGE 18)
117
TABLES ARE USEFUL FOR THE SUBJECTIVE
118
For example, suppose you need to evaluate one of
publications
USING TABLES (A SIMPLE RUBRIC)
119
Level 3.0 Achievement of at least one of the following results:
by peers and departmental chairs/directors
etc., at an acceptable academic, professional, or pedagogical conference or meeting
having a significant impact based on quality or publication outlet
book
publication.
From https://advance.washington.edu/resources/docs/rubrics.pdf
INSTRUCTIONAL DESIGN (PAGE 20)
120
Required documentation:
standards
appropriate student learning outcomes
department course descriptions and standards have been taught
3.2.4 COURSE MANAGEMENT (PAGE 22)
121
Required documentation:
deadlines (including Banner grade submission reports)
communication with class members regarding alternative assignments/expectations
materials and resources are available for the course, ensuring that students receive appropriate feedback, making appropriate arrangements for field trips, etc.)
INSTRUCTIONAL DELIVERY / ASSESSMENT
122
3.2.2. Instructional delivery (page 21) Required documentation:
delivery 3.2.3. Instructional assessment (page 22) Required documentation:
STEP 8: ACTIVITY SEVEN (15 MIN)
123
REMEMBER THE GOAL (GUIDEBOOK PAGE 18)
124
TEACHING EXAMPLE FROM GUIDEBOOK (PAGE 20)
125
See also:
STEP 8: ACTIVITY EIGHT A (10 MIN)
126
Service respectively
BETA COLLABORATION AT EVERY STEP
127
STEP 8: ACTIVITY EIGHT B
128
12 9
130
REMINDER FROM THE GUIDEBOOK (PAGE 17)
During the beta year, units will be experimenting with ways of valuing evidence and will have no track record with which to standardize scoring. It may be that “professional” faculty members score below 3 (or an “unacceptable” faculty member scores above 3) because of an error or unexpected consequence of the draft system. For this reason cFEC recommends that chairs of beta units assign the categorical level “as usual.” For the beta units that use both the old and new systems during the beta year, this will likely mean using the old system for the categorical rating and the new for a numerical rating.
USING THE COMPLETED SYSTEM: RATINGS
✓ The evaluation of a faculty member within a role is called the Composite Role Rating (CRR).
The overall evaluation of a faculty member combining all role evaluations into one is called the Overall Composite Rating (OCR)
132
EXAMPLE: COMPUTING THE COMPONENT RATINGS
133
100%
25% 75% 20% 100%
Instructional Delivery Instructional Design Instructional Assessment Course Management Students Peers Chair
80%
+ +
EXAMPLE: COMPUTING THE OCR
E = EXEMPLARY ► 4ish P = PROFESSIONAL ► 3ish I = NEEDS IMPROVEMENT ► 2ish U = UNACCEPTABLE ► 1ish
UT Martin’s Chosen Scale
135
Recommended Overall System Evaluation Scale
Warning math rant:
E, P, I, U are categorical data. They are ordinal, but not interval…. The numbers 4, 3, 2, 1 are numerical data that are both interval and
consequences when we equate the two (think reverse).
KEEP THE CONVERSATION GOING
✓ More info from cFEC coming soon ✓ Share info among beta groups ✓ Provide feedback:
Look for updates on Office 365 Beta Team & join the conversation!