Faculty Unplugged? Empowering Faculty in Todays Assessment Culture - - PowerPoint PPT Presentation

ACCSC Faculty Development Workshop

February 11, 2015 Arlington, Texas

Faculty Unplugged? Empowering Faculty in Today’s Assessment Culture

Presented by:

Julie Basler, PhD

(Vice President of Academic Affairs-Platt College)

Juanita L. Gurubatham, PhD

(Director of Institutional Review and Development-ACCSC)

• Defining the Loop: Defining measureable

course objectives, program objectives, and how to assess it all both internally and externally

• Creating the Loop: Tying course objectives to

program objectives

• Enforcing the Loop: Using valid, clear course

activities to achieve objectives

• Closing the Loop: Reaching program outcomes

at the end of a program

T

• da y’s Ag e nda

De fining the L

• op

• Course objective: Statements of expectation

written in measurable terms that express what a student will know, do, or think at the end of a learning experience.

• Course objectives are measureable learner-
• riented abilities that are consistent with

standards of a professional practice.

Br e aking it Down in T e r ms

• Program Objectives: Two types: Internal

Measurements and External Measurements

• Internal Measurements: Performance

indicators that reflect the extent to which the purposes of the program are achieved and by which program effectiveness is documented.

• External Measurements: Consumer-Oriented

indexes designed to evaluate the degree to which the program is achieving its mission and goals.

Br e aking it Down in T e r ms

• Competencies: Measurable behaviors,

knowledge, actions, and skills essential to the practice of a profession.

• Institutional Effectiveness: Written document

based on measurable standards that reflect the process of ongoing comprehensive assessment

• f all program components, including program
• bjectives, etc.

Br e aking it Down in T e r ms

• A fun and meaningful way to teach all of

these terms is to use a map.

• A map provides major interstates as the Program

Outcomes

• By-passes and highways as the competencies

and so on

• Choose a map of the area of each person is from

and let that person explain using his/her own home town area

T a king it Ba c k Home

• Course Objective
• Competency
• Program Outcome(s)
• Faculty the “key” to this involvement

How is faculty the key and the starting point to this whole process?

How Doe s this T ie T

• ge the r

?

• Students can see how material is related to their

educational goals

• Exams correspond to the stated learning
• bjectives (Once faculty have written learning
• bjectives, they have defined their assessment

materials.)

• Students know what they are expected to be

able to do after the instruction is complete daily, quarterly, etc.

Why ar e Cour se Obje c tive s Impor tant?

• Course is organized (With objectives, the topics

fit together and have direction.)

• In short, course objectives communicate what

the instructor is trying to teach; what the students are to be expected to be able to do; how their achievement will be measured; and what will be accepted as evidence that they have achieved the goals.

Why ar e Cour se Obje c tive s Impor tant?

• Step One: Determine where the course lies in the

curriculum.

• Step Two: Determine what knowledge or skills your

students have from previous courses.

• Step Three: Establish or Determine a Course Goal
• Step Four: Break down the course goal into

subcomponents. Ask yourself what the specific knowledge, skills, and attitudes you expect the students to have at the end of the course, based on the course goal and where the course appears in the curriculum.

Cre a ting a Course Obje c tive

• Step Five: Rewrite these subcomponents into measurable units (aka

learning objectives). These measurable units should contain the following criteria:

• A = Audience. Who will be completing the subcomponent? In the

case of your class, it is always the student, therefore most people leave audience out of the learning objective.

• B = Behavior. What knowledge, skill or attitude do you expect the

student to have? You can use Bloom’s taxonomy as a guide to determine the level of behavior.

• C = Condition. Any special circumstances needed to complete the
• bjective.

Cre a ting a Course Obje c tive

Course goal: Students will be able to correctly identify anatomical structures in the human body. Specific learning objectives: At the completion

• f the course, students will be able to:
• Recall the attachments of the muscles of the

arm.

• Identify the muscles of the arm on a model,

picture, diagram, drawing, or specimen.

E xa mple Course Obje c tive

Course goal: Students will learn common theories of learning and apply them to teaching preschool students. Specific learning objectives: At the completion

• f the course, students will be able to:
• Compare and contrast the learning theories of

Vygotsky and Piaget.

• Design an appropriate playtime activity for three year
• lds using Piaget’s theory

From the VCU Center for Teaching Excellence

E xa mple Course Obje c tive

Don’t ask for people to write course

• bjectives on the spot:

It causes faculty to freak out Instead, have them evaluate ones that you pull and them have them submit revised ones privately, not publicly Think of ways to praise faculty for well written course objectives: free things exist too: Car wash tokens Lottery scratch offs

T a king it Ba c k Home

• Using Bloom’s Taxonomy-

Student Learning Outcomes

A committee of colleges, led by Benjamin Bloom (1956), identified three domains of educational activities:

Cognitive: mental skills (Knowledge) Psychomotor: manual or physical skills (Skills) Affective: growth in feelings or emotional areas (Attitude)

Using Course Obje c tive s

• Writing Student Learning Outcomes Using

the Domains

• 1. Remembering
• 2. Understanding
• 3. Applying
• 4. Analyzing
• 5. Evaluating
• 6. Creating

Cate gor ie s of Bloom’s T axonomy

• Have Faculty Members Draw an

Illustration in Stick People for Each of These!

• 1. Remembering
• 2. Understanding
• 3. Applying
• 4. Analyzing
• 5. Evaluating
• 6. Creating

T a king It Ba c k Home

Remembering: Recall previous learned information. Examples: Recite the safety rules in administering

• insulin. Knows the steps of wrapping a sprained

ankle. Key Words: defines, describes, identifies, knows, labels, lists, matches, names, outlines, recalls, recognizes, reproduces, selects, states.

Cate gor ie s of Bloom’s T axonomy

Understanding: Comprehending the meaning, translation, and interpretation of instructions and

• problems. State a problem in one's own words.

Examples: Explain in one's own words the steps for creating an excel spreadsheet. Key Words: comprehends, converts, defends, distinguishes, estimates, explains, extends, generalizes, gives an example, infers, interprets, paraphrases, predicts, rewrites, summarizes, translates.

Cate gor ie s of Bloom’s T axonomy

Applying: Use a concept in a new situation or unprompted use of an abstraction. Applies what was learned in the classroom into novel situations in the work place. Examples: Apply laws of statistics to evaluate the reliability of a written test. Key Words: applies, changes, computes, constructs, demonstrates, discovers, manipulates, modifies, operates, predicts, prepares, produces, relates, shows, solves, uses.

Cate gor ie s of Bloom’s T axonomy

Analyzing: Separates material or concepts into component parts so that its organizational structure may be understood. Distinguishes between facts and inferences. Examples: Troubleshoot a piece of equipment by using logical deduction. Key Words: analyzes, breaks down, compares, contrasts, diagrams, deconstructs, differentiates, discriminates, distinguishes, identifies, illustrates, infers, outlines, relates, selects, separates.

Cate gor ie s of Bloom’s T axonomy

Evaluating: Make judgments about the value of ideas or materials. Examples: Select the most effective solution. Explain and justify a new budget. Key Words: appraises, compares, concludes, contrasts, criticizes, critiques, defends, describes, discriminates, evaluates, explains, interprets, justifies, relates, summarizes, supports.

Cate gor ie s of Bloom’s T axonomy

Creating: Builds a structure or pattern from diverse

• elements. Put parts together to form a whole, with

emphasis on creating a new meaning or structure. Examples: Write a company operations manual. Integrates training from several sources to solve a problem. Key Words: categorizes, combines, compiles, composes, creates, devises, designs, explains, generates, modifies,

• rganizes, plans, rearranges, reconstructs, relates,

reorganizes, revises, rewrites, summarizes, tells, writes.

Cate gor ie s of Bloom’s T axonomy

• Measureable behaviors, knowledge, actions,

and skills essential to the practice of any profession. Competency Types:

• Knowledge Competencies - practical or

theoretical understanding of subjects.

• Skill and Ability Competencies - natural or

learned capacities to perform acts.

• Behavioral Competencies - patterns of action
• r conduct.

Wha t a re Compe te nc ie s?

• Achievement of the Program Outcomes for the

Bachelor of Science in Nursing will be measured using the following five competencies based on the national standards of the Essentials of Baccalaureate Education and the Prelicensure knowledge, skills, and attitudes delineated by QSEN: (1) Nursing Process, (2) Nursing Informatics, (3) Evidence-Based Practice

Sa mple Compe te nc ie s

Lunch

More importantly -- Dessert

Cre a ting the L

• op

• What do I want my graduate to look

like when s/he graduates?

• What broad headers do I want them

to fall under?

• Internal vs External-What’s the

Difference?

Cour se Obje c tive T

• Pr
• gr

am Obje c tive s

• What do I want my graduate to look

like when s/he graduates?

• Have people draw this out
• Crayons, paper, etc.

T a king It Ba c k Home

• The end result of what the graduate

looks like at the end of the entire program on graduation day

Prog ra m Outc ome s

• What the graduate looks like-Internal
• Also, the program indices play a part

in the external measurement (i.e. pass rates, placement rates, etc.)

Prog ra m Outc ome s

A program outcome must flow directly from, and support, the school and program/department mission. The connection between the mission and the outcome should be clear. A program

• utcome must be directly related to the discipline of the program.

Consider these examples:

• General: Graduates of the Criminal Justice program will be critical

thinkers.

• Program-specific: Graduates of the Criminal Justice program will

analyze a current issue in criminal justice, evaluate evidence, and construct an argument.

Prog ra m Outc ome s

E nforc ing the L

• op

ASSE SSME NT F

OR L

E ARNING ASSE SSME NT OF L E ARNING

USING CLASSROOM ASSESSMENT TO ENGAGE STUDENTS

Asse ssme nt F

OR L

e ar ning

• T

urns the c la ssro o m a sse ssme nt pro c e ss a nd re sults into a n instruc tio na l inte rve ntio n

• Stude nts a nd instruc to rs wo rk a s a te a m
• Pro vide stude nts with a c le a r visio n o f the le a rning

ta rg e t

• Pro vide stude nts with de sc riptive fe e db a c k-sho w

the m ho w to impro ve .

• Ac c ura c y (o f I

nfo rma tio n) + de sc riptive fe e db a c k + stude nt invo lve me nt = a c hie ve me nt g a ins

Stiggins, R. “Assessment FOR Learning Defined” and Classroom Assessment for Student Learning

7 Str ate gie s of Asse ssme nt F

• r

L e ar ning

Where am I going? 1. Provide a clear and understandable vision of the learning target. 2. Use examples of models of strong and weak work. Where am I now? 3. Offer regular descriptive feedback. 4. Teach students to self-assess and set goals. How can I close the gap? 5. Design lessons to focus on one aspect of quality at a time. 6. Teach students focused revision. 7. Engage students in self-reflection, and let them keep track of and share their learning.

T hr e e Basic Que stions for E valuating Cour se s

Instr ume nts of E va lua tion

• Are the learning objectives of the

course being met? Are students being inspired and motivated to think analytically and creatively and to develop habits of mind appropriate to the discipline?

• Measures of student learning based
• n students’ in-course papers,

projects, or exams evaluated by the faculty member’s explicit standards and criteria

• Students’ performance on

standardized tests

• Students’ performance in

subsequent courses

• Are the course material, concepts,

and activities rigorous, current, relevant for students’ needs, and consonant with the announced course description?

• Colleague examination of course

syllabus, exams, and other material

• Do students perceive themselves to

be well taught?

• Student course evaluations,

surveys, focus groups

• Alumni surveys

Walvo o rd, B.E ., Asse ssme nt Cle ar and Simple

• Using a “MAP” at the Course Level.
• Remind faculty that course objectives/outcomes

should not be a secret.

• A map provides major interstates as the Course

Objectives

• By-passes and highways as the competencies

and so on

• Encourage faculty to set the “destination” in the

first class period and continue to evaluate progress

T a king it Ba c k Home

T

• ols: Gra de s
• Exams and assignments must actually

measure learning goals

• Establish criteria for assessment
• Develop systematic ways of to communicate

student weaknesses and strengths to decision makers

T

• ols: Rubric s
• Allows for consist evaluation
• Provides various criteria and standards that

may be used to evaluate student work

• May be used at course level and program level

Sample Rubr ic

Trait 14: Sentence Construction Level

5 Clear and concise sentences vary, with the degree of complexity reflecting the audience and purpose. 4 Sentences vary, with the degree of complexity reflecting the audience and purpose. 3 Sentence variety is limited but attempts complex structure. 2 Complex structure is attempted without success and/or sentence structure is simplistic, but not throughout the text 1 Sentences are simple and repetitive.

Cognitive Level and Quality of Writing Assessment : CLAQWA Online, Cross-Disciplinary. Retrieved from http://claqwa.com

T

• ols: E
• Portfolio
• Portfolios contain authentic evidence of learning

that students produce in the everyday course of their studies, both in and out of the classroom (e.g., in internships, independent research projects, and other “real-world” experiences).

• Enables students to collect and reflect on

evidence of what they are learning.

• May be used at course level or program level

Banta, T., et al, Three Promising Alternatives for Assessing College Students’ Knowledge and Skills

T

• ols: E
• Por

tfolio

These basic questions need to be answered when undertaking an ePortfolio project. They include the following:

• When and from where will student work be

collected?

• Are there specific assignments that correspond

to specific outcomes?

• Is there a grading rubric?
• Who will evaluate the student ePortfolios (if

Programmatic Assessment)

• Break faculty into groups by

subject area

• Discuss/identify best tools for

assessing competencies

T a king it Ba c k Home

Asse ssme nt OF L e ar ning

• Assessments that happen after learning
• ccurred to determine if it did.
• Primary users include policy makers, program

planners, faculty, etc.

• Used to “certify” student competence

T

• ols: Standar

dize d E xams Asse ssme nt OF L e ar ning

• Useful in specific content areas
• Quantitative measure with established

benchmarks

• Has limitations
• Licensure and Certification Exams

Pr

• gr

am Asse ssme nt

• Now you have a program curriculum—is it

right?

• Assessment is an ongoing process
• Develop a plan

– Keep it simple

• Collect and evaluate your data
• Remember to close the loop

– Use results to improve curriculum

• Course-level assessments of student

learning should map to program assessment

• Program faculty should think collectively

about how course assessments contribute to the program review.

• Keep it Simple
• Consider preset dates for annual program review
• Limit the number of objectives reviewed each year
• Put someone in “charge”

T a king it Ba c k Home

Building Bloc ks for Pr

• gr

am Asse ssme nt

• Program Mission

– Statement of purpose and intentions – Should connect to the institutional mission – Provides foundation for development of goals and

• bjectives

Building Bloc ks for Pr

• gr

am Asse ssme nt

• Goals and Objectives

– What do you want a graduate to “look like” – Must be measureable – More is not always better

Building Bloc ks for Pr

• gr

am Asse ssme nt

• Evaluation

– Determine what you will assess – Evaluation tools

• Direct Assessment
• Indirect Assessment

– Who – When – Close the loop—documentation!

Goals/Outcomes Curricular Design Pedagogy Assessment Mission

AAC& U (2004), Taking Responsibility for the Quality of the Baccalaureate Degree

• Planning your Day
• Incentive for participating
• Don’t overwhelm participants
• Keep it simple

T a king it Ba c k Home