Welcome! We cannot solve our problems with the same thinking we - - PDF document

4/1/2016 1

2016 Program Evaluator Candidate Face-to-Face Training

Spring, 2016

2

We cannot solve our problems with the same thinking we used when we created them. ~ Albert Einstein

• Facilitator Introductions
• List of Participants – on each table
• Introduce Yourself – first time you speak

Welcome!

4/1/2016 2

3

ABET Headquarters Representatives

ABET Welcome

4

• Provide you with knowledge, skills, and

abilities to conduct ABET accreditation evaluation.

• Learn how to provide guidance to programs

for improvement.

• Our goal: To improve consistency of

program evaluation.

Why Are We Here? PEVC Course Purpose

4/1/2016 3

5

• PEVC Face-to-Face (F-2-F) Training is

experiential workshop that simulates real visit as closely as possible.

• Works in conjunction with online Pre-Work

modules, designed for your independent and convenient learning.

Course Purpose (cont’d)

6

At completion of PEVC F-2-F Training, you will be able to:

• Plan effective, cordial, efficient campus visit.
• Interact with your team during visit.
• Given specific situation or evidence (e.g., transcripts,

student work):

 Determine which criteria are relevant to determining accreditation action.  Determine if program is in compliance with criterion. If not, determine level of compliance and accreditation action.

Why Are We Here? Learning Outcomes

4/1/2016 4

7

At completion of training, you will be able to:

• Conduct interviews of faculty, staff, and students.
• Write exit statement to institution.
• Conduct informal debrief with program head and

exit meeting with leaders of institution.

• Complete Program Evaluator forms.
• Demonstrate “in practice” effective use of PEV

competencies in PEV Competency Model.

Learning Outcomes (cont’d)

8

Agenda – Day 1, Morning

Module Name Timing

1) Welcome and Introduction 8:00‐8:15 a.m. 2) Connection to Pre‐Work and Assessment Fundamentals 8:15‐9:30 a.m. Break 9:30‐9:45 a.m. 3) Display Materials and Facilities 9:45‐11:00 a.m. 4) Sunday Team Meeting 11:00‐12:30 p.m. Lunch 12:30‐1:15 p.m.

4/1/2016 5

9

Agenda – Day 1, Afternoon

Module Name Timing 5) Interviews 1:15‐2:05 p.m. 6) Monday Night Meeting 2:05‐3:35 p.m. Break 3:35‐3:50 p.m. 7) Writing Exit Statements 3:50‐4:55 p.m. 8) Homework 4:55‐5:15 p.m. Wrap‐Up Day 1 5:15‐5:25 p.m.

10

Agenda – Day 2, Morning

Module Name Timing Day 2 Overview 8:00‐8:10 a.m. 9) Context for Judgment 8:10‐9:00 a.m. 10) Informal Program Debrief and Exit Meeting 9:00‐10:05 a.m. Break 10:05‐10:20 a.m. 11) Exit Statements 10:20‐11:00 a.m. 12) Connection to PEV Competencies 11:00‐11:20 a.m. 13) Post‐Visit Work 11:20‐11:29 a.m. 14) Post‐Training 11:29‐11:38 a.m. 15) Closing Exercise 11:38‐11:45 a.m.

4/1/2016 6

11

• Speak with intention.
• Listen with attention.
• Know that wisdom is in room.
• Ensure representation from all pertinent voices.
• Model PEV competencies in all facets of training.
• Turn off/silence electronic devices (cell phones,

laptops, etc.).

• Start/stop on time.

Agreements

12

• Notebook

Slide Presentation

• Reference Notebook

General Accreditation Materials Commission-Specific Materials

• Exercise Handouts
• Table Supplies
• Issue Bins

Training Materials

4/1/2016 7

13

Module 2: Connection to Pre-Work

14

• 1. Overview
• 2. The Accreditation Process
• 3. The Role of the Program Evaluator
• 4. Continuous Improvement of Student

Learning

• 5. Applying the Criteria
• 6. The PEV Appraisal Process

Pre-Work Modules

4/1/2016 8

15

Pre-Requisite Work Modules:

• Overview
• The Accreditation Process
• The Role of the Program

Evaluator

• CI of Student Learning
• Applying the Criteria
• The PEV Appraisal Process

Completed:

• Proficiency Assessments (3)
• Upper State University Self-

Study and Transcript Analysis (PEV Report and PEV Worksheet as appropriate for Commission)

Completed Pre-Work

16

In your table groups:

• Use refresher cards to test your knowledge of Pre-Work topics:
• Each take turn asking, answering, and recording the result.
• If you get incorrect response, another group member may “help” out.
• Your Support Facilitator has answers and is available for assistance.
• Record answers your team gets correctly first time, second time,

and third time AND number of cards you have gone through.

• YOU HAVE 50 MINUTES.

Handout: Refresher Cards Answer Key : Handout 2-2

Pre-Work Refresher Cards: Exercise

4/1/2016 9

17

• Pre-Work analysis provides starting point for

evaluation of Upper State University.

• F-2-F training simulates campus visit to Upper

State University.

 “On-site” activities reveal additional insight and information about program.

• Therefore:

 Keep open mind.  Stay in information-gathering mode.  Do not look through lens of “reinforcing pre-drawn conclusions.”

Upper State University Pre-Work Analysis

18

At end of F-2-F training you’ll leave with:

• Table team decision whether program meets

criteria:

 Team consensus.  My interpretation and your interpretation of like circumstances must be same.

Upper State University Site Visit

4/1/2016 10

19

• Accept responsibility for decisions you make...
• Serve only in areas of competence.
• Act as agent of ABET and avoid conflicts of interest.
• Keep decisions confidential.
• Be objective and truthful.
• Behave honorably, ethically and lawfully.
• Report concerns with respect to internal financial matters.
• Treat all persons fairly.
• Assist colleagues… in following the Code of Conduct.
• Support prompt and fair adjudication of alleged violations...

ABET Code of Conduct

(Rules of Procedure Section EIGHT A & B)

20

• Current or past employment as faculty, staff, or

consultant by institution or program

• Current or past discussion or negotiation of employment

with institution or program

• Attendance as student at institution
• Receipt of honorary degree from institution
• Institution or program where close family relative is, or

was, student or employee within past 10 years

• Unpaid official relationship within past 10 years with

institution

Conflict of Interest

(Rules of Procedure Section SEVEN A & B and APPM II.C)

4/1/2016 11

21

• Day 0: Business Casual

Usually a Sunday afternoon campus visit. Time used to review course materials, assessment information, and tour laboratories.

Dress Code for Visits

22

• Day 1: Business Professional

Meetings with Faculty, Students, Support Areas

• Day 2: Business Professional

Exit Meeting with CEO of Institution

Dress Code for Visits

4/1/2016 12

23 24

Module 3 Display Materials and Facilities

4/1/2016 13

25

Interview with Laboratory Manager

26

Labs/Design Studios

• Sufficient number and size of labs.
• Appropriate coverage across breadth of

specializations within program.

• Appropriate equipment, in good repair.
• Appropriate student access (including evening

and weekend access).

• Appropriate technician support and instructional

support in lab.

What to Look For: Facilities

4/1/2016 14

27

Labs/Design Studios (cont’d)

• Safe physical arrangement and appropriate

safety practices followed.

• Comments from students about lab experience.
• Is there some type of “Laboratory Plan” for

maintaining and upgrading instructional laboratories? This is not required by ABET, but, in general, formal or informal plan of some sort should exist.

What to Look For: Facilities (cont’d)

28

Classrooms

• Appropriate physical arrangement, support for computer projection,

etc.

• Not overcrowded.

Support facilities

• Sufficient computer access, with appropriate off-hours access.
• Appropriate spaces for students to gather (not explicit criterion but

relates to several criteria).

• Appropriate shop with parts, repair facilities, etc.

Faculty offices

• Sufficient size, privacy.

What to Look For: Facilities (cont’d)

4/1/2016 15

29

Panel Discussion

30

Display Materials

4/1/2016 16

31

Display Materials

32

• Course materials including course syllabi,

textbooks, example assignments and exams, and examples of student work, typically ranging from excellent through poor.

• Evidence program educational objectives are

based on needs of stated program constituencies.

• Evidence of assessment, evaluation, and

attainment of student outcomes.

• Evidence of actions taken to improve program.

Display Materials (APPM II.G.6.b)

4/1/2016 17

33

Descriptions of materials for:

• Curriculum/Course Work
• Senior Design Projects, Capstone or Integrating

Experience (if required)

• Demonstration of Student Outcomes
• Assessment Materials

Handouts 3‐1 & 3‐2 Small Group Sessions with USU Display Materials

34

• Courses appear appropriate to accomplish Program

Educational Objectives and Student Outcomes.

• Student work indicates active engagement,

demonstration of learning, and reasonable grading standards.

• Evidence with respect to specific Student Outcomes

(including those in criteria) as appropriate to assessment plan.

• Few missing items, or even entire missing courses, are

not necessarily systemic problems. Pursue any major gap to see if represents serious problem.

What to Look For: Course Materials

4/1/2016 18

35

• Evidence (lab reports) of appropriate

student learning (not just cookbook).

• Evidence to support Student Outcomes.

What to Look For: Laboratory Reports

36

• Evidence all students complete major

design, capstone or integrating experience (if criteria require) that draws on previous courses and incorporates standards and realistic constraints.

• Student work should demonstrate via a

complete project report or some other mechanism.

What to Look For:

Major Design or Integrating Experience

4/1/2016 19

37

• Examples of instruments used and actual

data collected.

• Summaries of the data with results

reported in usable form.

• Recommendations for program

improvement based on data.

• Implementation and results.

What to Look For: Assessment Materials

38

• Institutional catalogs and similar publications must clearly indicate

the programs accredited by the commissions of ABET as separate and distinct from any other programs or kinds of accreditation.

• Each accredited program must be specifically identified as

“accredited by the _________ Accreditation Commission of ABET, http://www.abet.org.”

• Each ABET-accredited program must publicly state the program’s

educational objectives and student outcomes.

• Each ABET-accredited program must publicly post annual student

enrollment and graduation data per program.

Handout: 3-4

Documentation (APPM II.A.6)

4/1/2016 20

39 40

Module 4: Sunday Team Meeting

4/1/2016 21

41

• Agenda Review
• Visit Schedule
• Program Evaluator Pre-Visit Briefings

 Program Strengths  Potential Program Shortcomings  Recommended Action

• Display Material and Facility Findings
• Agree on Common Findings and Plans to

Investigate

• Debrief

Handout 4-1

Sunday Team Meeting Agenda

42

• Usually involves review of multiple programs

with one PEV per program.

• Discussions will focus on common shortcomings

across programs and what team needs to do to ensure consistent evaluation and recommended action.

• Usually two meetings:

 Before review of display materials and facility tours.  After review of display materials and facility tours.

Actual Visit Sunday Meeting

4/1/2016 22

43 44

Module 5: Interviews

4/1/2016 23

45

• Direct interface with members of program:

Facilitates understanding of program. Provides an opportunity to clarify items in Self-Study and to probe for further details. Effective way to identify problems not addressed in Self-Study.

Why Interview?

46

Panel Discussion

4/1/2016 24

47

Advisor Interview

48

In your table groups:

• Draw card from scenario card deck.
• Share ideas for dealing with situation:

 What would you say?

• Facilitator: Capture ideas on sticky-notes.
• Post ideas/responses for each scenario on

appropriate flipchart in room.

Handout: Interview Cards

Managing Problem Interviews: Exercise

4/1/2016 25

49

• All interview questions should be

DESIGNED to elicit detailed response:

Determine and clarify objectives BEFORE interview. Prepare questions.

The Art of Asking Questions

50

• Begin with:

Why? How? What? When? Where?

• Avoid leading questions that assume an

answer.

• Prompt for specific information:

“Show me”

The Art of Asking Questions (cont’d)

4/1/2016 26

51

• Turn interviewee into teacher:

“Can you explain that to me?”

• Ask an obvious question to establish

interviewee as expert.

• Periodically verify what you heard:

“Do I understand you to say…?”

• Remember, silence is golden.

The Art of Asking Questions (cont’d)

52

• Suggest interview of junior/senior level class (~20

minutes without instructor present)

• Briefly describe ABET and Comprehensive Review
• Possible questions:

 How well are laboratory facilities and equipment working?  How satisfactory and available are computer laboratories?  How helpful are enrollment and career advising?  How do courses from Chem, Phys, and Math support engineering courses?

Handout 5-1

Student Interviews

4/1/2016 27

53 54

Module 6: Monday Night Meeting

4/1/2016 28

55

• Each PEV Candidate in turn brief team on

potential finding including:

 Criterion to be cited  Evidence gathered  Compliance level

• Discuss evidence, criteria, and compliance

as team and develop consensus for finding.

• You have maximum of 60 minutes.

Monday Night Meeting

56

• Similarities with simulation
• Differences with simulation

Actual Monday Team Meeting

4/1/2016 29

57 58

Module 7 Writing Exit Statements

4/1/2016 30

59

Handout 7-1

Types of Statements

60

• Statement will generally include one or more

types of findings:

 Strength  Deficiency  Weakness  Concern  Observation

• Format will vary depending on type of visit:

 General Review  Interim Visit

Types of Findings

4/1/2016 31

61

General Review

62

• Introduction
• Program Strengths
• Program Shortcomings
• Deficiencies (in order of Criteria)
• Weaknesses (in order of Criteria)
• Concerns (in order of Criteria)
• Observations

NOTE: ETAC does not require an Exit Statement. The PAF serves as the Exit Statement.

Handout 7-2

Outline of Exit Statement

4/1/2016 32

63

• Include information such as:

Type or special characteristics of program. Emphasis area(s). Number of enrolled students. Size of most recent graduating class. Number of faculty members. Other information that could be helpful to next team.

Introduction Statement

64

“The industrial engineering BS program prepares students for careers in system design,

• ptimization, and ergonomics. The program has

150 students, and 14 full-time faculty members support the program. The program had 39 graduates in the 2014-2015 academic year. All courses in the major are available by distance education, but students must come to campus or have facilities available through their employers for laboratory experiences in two courses.”

Introduction: Example

4/1/2016 33

65

• Contents:

Criterion/policy citation What was observed Effect on program

Program Shortcoming Statements

66

• Criteria/Policy Citation

 State which criterion or policy applies  Quote excerpts as needed

Criteria/Policy Citation

4/1/2016 34

67

• What was observed:

 Describe what was observed (evidence).  Describe how your finding points to lack of compliance with criterion or policy or points to lack of strength of compliance.

• Effect:

 Describe negative effect it has on the program (D or W) with respect to criterion.  Describe potential future effect on program (C) with respect to criterion.

What Was Observed & Effect

68

Students Criterion

Criterion 1 requires that the program must have and enforce procedures to ensure and document that students who graduate meet all graduation requirements.

Deficiency: Example (Part 1)

4/1/2016 35

69

While most students who graduated successfully completed all requirements, there were a few students who did not pass all of the required courses designated by the program.

Students can graduate from program without passing all the required courses. Thus, program is not in compliance with Student criterion.

Deficiency: Example (Part 2)

70

Program Educational Objectives Criterion

Criterion 2 requires that there must be a documented, systematically utilized, and effective process, involving program constituencies, for the periodic review of these PEOs that ensures they remain consistent with the institutional mission, the program’s constituents’ needs, and these criteria.

Weakness: Example (Part 1)

4/1/2016 36

71

The program has relied on a review of the PEOs every three years by the faculty only. The other constituencies (alumni, employers, students) were not involved in this review.

Without involvement of all constituencies, program can not ensure that PEOs meet needs of all

• constituencies. Thus strength of compliance with

criterion is lacking.

Weakness: Example (Part 2)

72

Institutional Support Criterion

Criterion 8 requires that the resources available to the program must be sufficient to attract, retain, and provide for the continued professional development

• f a qualified faculty. The resources available to the

program must be sufficient to acquire, maintain, and

• perate infrastructures, facilities, and equipment

appropriate for the program, and to provide an environment in which student outcomes can be attained.

Concern: Example (Part 1)

4/1/2016 37

73

At present, it appears that resources are adequate to support the program. However, there have recently been large reductions in the operating budget of the program.

If these budgetary reductions continue, future compliance with criterion may be jeopardized.

Concern: Example (Part 2)

74

Handouts 7-3 & 7-4

Statement Critiquing: Exercise

4/1/2016 38

75

• Must be based on conclusions from evidence

found by team to be credible and significant.

• Names of individuals or titles that identify

individuals must not be used.

• Avoid attributing opinions, conclusions or

recommendations to other individuals or groups

• f individuals besides Commission.

 Poor: The team discovered that…  Better: Evidence indicated that…

Additional Guidelines for Writing Statements

76

• Provide enough detail that program and

institution will know precisely what is inadequate.

• Provide enough detail to Commission to justify

type of finding.

• Provide enough detail that team making the

next general or interim review will be able to determine amount of progress that has been made since original finding.

Additional Guidelines for Writing Statements (cont’d)

4/1/2016 39

77

• If wording does not add to or clarify

finding, do not include it.

• Avoid use of acronyms or other

abbreviations.

• Avoid combining findings on two criteria

into one statement.

Additional Guidelines for Writing Statements (cont’d)

78

• ONLY document left with institution at end
• f the visit.
• MUST be consistent with Exit Statement.
• Level of compliance.
• Wording to describe what was observed

and effect on program. Handout 7-5

Program Audit Form [PAF]

4/1/2016 40

79

Program Audit Form [PAF] (cont’d)

80

Program Audit Form [PAF] (cont’d)

4/1/2016 41

81

• Introduction
• Program Strengths
• Program Shortcomings

Deficiencies (in order of Criteria) Weaknesses (in order of Criteria) Concerns (in order of Criteria)

• Observations

Format of Exit Statement

82

• Use statement rubric to carefully review

your exit statement before turning in to Team Chair.

• Ask another PEV to review statement for

clarity and consistency with PAF.

• Read statement out loud to your fellow

team members.

Reviewing Your Statement

4/1/2016 42

83

Table Discussion

Statement Writing Summary

Handout 7‐6

84

4/1/2016 43

85

Module 8 Homework

86

• Read two Judgment Scenarios.
• Update your PEV Worksheet to reflect new

information from today’s activities:

 Interviews, display materials, information from Dean, and team discussions.

• Draft your Exit Statement following statement

format and writing guidelines.

 Introduction, and shortcomings based on your team consensus for Upper State University.

• Complete the Program Audit Form.

Handouts 8-1 & 9-1

Homework Assignment

4/1/2016 44

87

• You will email completed Exit Statement and

Program Audit Form to ABET Staff person whose email address will be provided.

• PAF for your commission is available online in

Module 7.

• Homework will be evaluated by your table

Support Facilitator using statement rubric.

• We will discuss Judgment Scenarios Sunday

morning. Handouts 8-1 & 9-1

Homework Assignment (cont’d)

88

• Day One once-around the room

What do you leave with today? What’s “top of mind” about the day?

Thank you for your time, energy, and attention today!

Day 1, Closing

4/1/2016 45

89

Welcome Back!

Day 2 PEV F-2-F Training

Do the right thing. It will gratify some people and astonish the rest. ~ Mark Twain

90

Agenda – Day 2, Morning

Module Name Timing

Day 2 Overview 8:00‐8:10 a.m. 9) Context for Judgment 8:10‐9:00 a.m. 10) Informal Program Debrief and Exit Meeting 9:00‐10:05 a.m. Break 10:05‐10:20 a.m. 11) Writing Exit Statements 10:20‐11:00 a.m. 12) Connection to PEV Competencies 11:00‐11:20 a.m. 13) Post‐Visit Work 11:20‐11:29 a.m. 14) Post‐Training 11:29‐11:38 a.m. 15) Closing Exercise 11:38‐11:45 a.m.

4/1/2016 46

91

Module 9 The Context for Judgment

92

• Evaluation of program for accreditation

requires PEV judgment on extent to which each criterion is satisfied.

• PEV judgment is based on evidence
• btained during evaluation process.
• Judgment was demonstrated in exit

statements written last night for Upper State University as discussed yesterday.

Context for Judgment

4/1/2016 47

93

Throughout case study:

• Where did you use evidence as basis for

judgment?

• How and when did you see evidence-based

judgment applied by others?

• What did you struggle with?
• Why is evidence-based judgment

important?

Context for Judgment (cont’d)

94

• Accreditation has moved away from

rigid “bean counting” process to one that is based on continuous quality improvement processes.

• ABET Criteria are based on principles
• f Continuous Quality Improvement

(CQI). Context for Judgment (cont’d)

4/1/2016 48

95

An educational program CQI process should reflect clear understanding of:

• Mission
• Constituents
• Objectives
• Outcomes
• Processes (internal practice to achieve outcomes)
• Facts (data collection)
• Evaluation (interpretation of facts)
• Action (feedback to support decision making and

improve processes)

Context for Judgment (cont’d)

96

Handout 9-2

Judgment Scenario: Exercise

4/1/2016 49

97

• Many questions of judgment center around

assessment and continuous improvement.

• When such issues arise, ask:

 Are assessment processes adequate to determine achievement of each of outcomes?  Are assessment processes robust enough to identify shortcomings and achievements?  Are assessment processes sustainable?  Will assessment processes lead to program improvements if appropriate?

Context for Judgment (cont’d)

98

• Definition:

 Program educational objectives are broad statements that describe what graduates are expected to attain within a few years of graduation. Program educational

• bjectives are based on the needs of the program’s

constituencies.

• Key Words:

 Broad Statements  Expected to Attain  Needs of Constituencies

Program Educational Objectives

4/1/2016 50

99

• Should be Forward Looking
• Should use Expected to Attain language

and not Preparing Graduates to Achieve

• Must periodically review PEOs
• Must ensure PEOs remain consistent with

institutional mission, program’s constituents’ needs, and these criteria

• Can include language from Student

Outcomes

PEOs: Important Concepts

100

• Judgment centers on observed evidence

that supports achievement of each criterion.

• When issues arise about specific

components of criteria, ask:

Does evidence support my decision? What additional evidence could I seek?

Summary

4/1/2016 51

101 102

Module 10 Informal Program Debrief & Exit Meeting

4/1/2016 52

103

• Must have complete Exit Statement and

Program Audit Form at this stage.

• Must be able to elaborate on statement

and explain conclusions without falling into debate with program head.

• Must be able to relate every shortcoming

to Criteria.

Informal Debrief Preparation

104

• Must understand what not to say, particularly

recommended accreditation action.

• May make comments intended to be helpful

while making it clear that these comments (which may appear in the report as Observations) have no relation to findings with respect to Criteria.

• Must remain completely consistent in all

statements to every person.

Informal Debrief Preparation (cont’d)

4/1/2016 53

105

Note: This is NOT Upper State University Handout 10-1

Informal Debrief Demonstration

106

Simulating Informal Debrief: Exercise

4/1/2016 54

107

• Must have complete Exit Statement and

Program Audit Form that follow all ABET guidelines.

• Statement is read verbatim.
• Must be no deviation from written Exit

Statement.

• If questions are asked, answers must

essentially re-state Exit Statement without introducing new material.

Exit Meeting

108

• Always consult with Team Chair if doubt

about appropriate response to question.

• Always be courteous and respectful.
• Thank institution for campus hospitality.

Exit Meeting (cont’d)

4/1/2016 55

109

• Do not deviate from Exit Statement.
• Do not make comments regarding aspect of program or

institution other than those in Exit Statement.

• Do not compare visited institution to your own institution.
• Do not make disparaging comments about students or

anyone else.

• Do not disagree with other team members in any way.
• Do not make any social conversation that could possibly be

misinterpreted.

• Do not arrive late or leave early.

What NOT TO DO at Exit Meeting

110

4/1/2016 56

111

Module 11 Writing Exit Statements & Other Items

112

• Observations and Statement Format
• Interim Actions
• Evaluation of Online/Hybrid and Multi-

Site Programs Writing Exit Statements & Other Items (cont’d)

4/1/2016 57

113

Handout 11-1

Types of Statements

114

Strengths & Observations

4/1/2016 58

115

• Strength is exceptionally strong, effective practice
• r condition that stands above norm, and has

positive effect on program.

• For each Strength, state:

 What was observed  What makes it stand above norm  Positive effect it has on program ETAC Note: Strengths are considered observations.

Strength Statements

116

“The program judiciously integrates laboratory experience with classroom work in the courses. Offering hardware and software based laboratories and project experiences help students better appreciate the course material, promotes excellence in student education, provides hands-on experience, enhances teamwork, and better prepare students for the real world. The positive feedback from students attests to utility of their laboratory and project experiences.”

Strength: Example

4/1/2016 59

117

Definition: “Comment or suggestion which does not relate directly to the accreditation action but is offered to assist the institution in its continuing efforts to improve its programs”

• ABET Accreditation Policy & Procedures Manual

Section II.G.9.a.(3)

Statement of Observation

118

“Use of CSE 2221, Computing in Engineering & Science, as a prerequisite to senior-level courses could be considered as a way to enhance the computational experience of the students.”

Statement of Observation: Example

4/1/2016 60

119

• Take 5 minutes and read Strength

statements on page 1 of handout.

• Determine whether each statement is

appropriate Strength statement or not.

• Discuss as table.
• Repeat for Observation statements on

page 2 of handout.

• You have 20 Minutes. Handout 11-2

Strength and Observation: Exercise

120

• If program undergoing General Review Visit

and receives one or more Weaknesses or Deficiencies, then subject to Interim Action:

Weakness  Interim Report or Interim Visit Deficiency  Show Cause Report or Show Cause Visit

Interim Actions

4/1/2016 61

121

• Interim Report or Show Cause Report

Team Chair writes Draft Statement

• Interim Visit or Show Cause Visit

PEVs and Team Chair writes Exit Statement, PAF, and Draft Statement For Interim Statement, experienced PEVs involved – some differences in format from General Review

Interim Actions (cont’d)

122

• More common that programs are offered
• nline/hybrid/multi-site.
• Criteria is same regardless of delivery

method.

• For program offered using multiple modalities,

program must be able to demonstrate program is equivalent in all modalities. If offered at multiple sites, program must be able to demonstrate program is equivalent at all sites.

Online / Hybrid / Multi-Site Programs

4/1/2016 62

123

• Online/hybrid/multi-site program may require

greater time commitment in preparation and evaluation than normal for single-site program delivered face-to-face.

• “Weakest link” concept applies to program
• evaluation. If issue is found within one delivery

modality or at specific site, finding and resulting accreditation action (if impacted by finding) will apply to program in its entirety, regardless of delivery method.

Online / Hybrid / Multi-Site Programs

(cont’d)

124

Online / Hybrid / Multi-Site Programs

(cont’d)

• If program or one or more required

courses supporting program are delivered by more than one method or at different physical sites, evaluators expect to see separate course/assessment materials for each delivery method/location.

4/1/2016 63

125

• Most online programs are delivered through a

learning management system (LMS). If program is dependent on LMS, PEV will be provided with necessary access to course materials, discussion boards, chat rooms, and graded student work, preferably well before physical

• visit. Video conferencing or other electronic

means may be used to facilitate student, faculty and constituents interviews.

Online / Hybrid / Multi-Site Programs

(cont’d)

126

4/1/2016 64

127

Module 12 Connection to Competencies

128

• Technically Current
• Effective Communicator
• Interpersonally Skilled
• Team-Oriented
• Professional
• Organized

Handout 12-1

ABET PEV Competency Model

4/1/2016 65

129

At your table, discuss:

• Which competencies were effectively

demonstrated within group? How?

• Which competencies can we individually

and collectively improve?

PEV Competency Model: Exercise

130

• Competencies form basis of performance

appraisal:

Evaluation of PEV Candidates:

• Evaluated by your table Support Facilitator and

PEVC peers at end of F-2-F training.

Evaluation of PEVs:

• Each PEV will be evaluated by institution, Team

Chair, and PEV peers after each visit.

Competency-Based Performance Appraisal Process

4/1/2016 66

131

• Serves as a vehicle for professional

development.

• Tool for continuous improvement.
• Clearly communicates what is expected.
• Everyone gets evaluated every time.
• Evaluate against Competency Model.

Competency-Based Performance Appraisal Process (cont’d)

132

• 360-Degree Process:

 Team Chair  Team Members  Institution

• “Met Expectations” (rating of 3) describes how

competency demonstrated pre-visit and onsite. This is benchmark.

• Actionable comments for items < 3 rating.

Competency-Based Performance Appraisal Process (cont’d)

4/1/2016 67

133

Performance Appraisal Process

134

4/1/2016 68

135

Module 13 Post-Visit Work

136

The Accreditation Timeline

January Institution requests accreditation for programs February - May Institution prepares self-evaluation (Program Self-Study Report) March - June Team chairs assigned, dates set, team members chosen September - December Visits take place, draft statements written and finalized following 7-day response period December - February Draft statements edited and sent to institutions February - April Institutions respond to draft statement and return to ABET May - June Necessary changes to statement, if any, are made July Commission meets to take final action August Institutions notified

• f this action

Year 1 Year 2

4/1/2016 69

137

• 7-Day Response from institution.
• Draft Statement prepared, edited, and sent to

institution.

• 30-Day Due Process Response from institution.
• Final Statement prepared and edited.
• Commission takes final accreditation action.
• ABET sends Final Statement and accreditation

letter to institution.

Post-Visit Process

138

• Within one (1) week after visit is

completed, the PEV must:

Submit visit expenses using online expense system; include original itemized receipts.

• And, PEV should:

Complete Team Chair and Peer PEV Performance Appraisal Forms online.

PEV Responsibilities – Post-Visit

4/1/2016 70

139

• PEV should not communicate with

program concerning ABET matters after visit has ended.

• Team Chair may ask PEV to help evaluate

Due Process Response from program.

PEV Responsibilities – Post-Visit (cont’d)

140

• Always use latest ABET Travel Policy and

Procedures Manual.

• Make reservations through ABET’s travel agent.
• Submit expenses online using ABET online

expense Web tool.

• Scan original receipts for ALL lodging, meals,

and transportation (tolls, parking, taxi, bus, rail)

• r submit original receipts by mail.

ABET Travel Policy

4/1/2016 71

141

Module 14 Post-Training

142

• Leave behind completed Exit Statement and

Program Audit Form (PAF).

• Evaluation of Exit Statement and PAF by

Facilitator.

• Evaluation by Facilitator and peers using

modified PEV performance appraisal.

• Member society notified of completion of training

and availability of evaluation results.

• Update your bio on ABET secure website with

dates available for 2016 Fall visit.

What Happens Next?

4/1/2016 72

143

• Training Mentor there to support you!
• Debrief with Training Mentor soon.

Focus on key learnings from F-2-F training. Ask any follow up questions. Share any other feedback on training (Pre-Work and F-2-F).

Contact Your Training Mentor

144

• Member society makes assignments based on

institutions’ program needs and PEV pool.

• Some societies will require Observer visit first.
• Institutions check team list for any conflict of

interest.

• Team Chair approves your participation and
• btains approval from institution then contacts

you.

Getting Assigned

4/1/2016 73

145

Questions on what happens after the training? Handout 14-1

Questions and Answers

146

Module 15 Closing

4/1/2016 74

147

Compared to yesterday morning, how prepared do you now believe you are to go out on an ABET accreditation visit?

What Have We Accomplished?

A Lot!

148

• Complete evaluations and leave in

evaluation envelope at your table.

Support Facilitator PEV Candidate Peers Just-in-Time Evaluations Overall Training Evaluation

Evaluations

4/1/2016 75

149

GOODBYE

150

Thank you for your time, energy, and attention.

Safe travels!