Andrea M. Karkowski, Jody S. Fournier, Richard M. Ashbrook, Terry D. - - PowerPoint PPT Presentation

Andrea M. Karkowski, Jody S. Fournier, Richard M. Ashbrook, Terry D. Lahm

Higher Learning Commission Annual Conference ● Chicago, IL ● April, 2013

 Identify Diverse Views on Assessment and Learning  Develop Student Learning Goals Programs/Majors  Align Student Learning Goals to IDEA Objectives  Map IDEA Objectives Across Required Courses  Help Students Improve their Learning  Document the Results  Use Results to Improve Courses and Curricula

Photo by A. M. Karkowski of Dr. Terry Lahm, September 17, 2012, Capital University

Photo by A. M. Karkowski of Lanee Williams, Jordan Aires, Jessica Heck, Andre Cousin, Capital University, September 17, 2012

Janet Leigh in Psycho, Directed by Alfred Hitchcock, Universal Pictures

 Direct Measures

• What students learned or can actually do
• More difficult assessment to do and more highly valued
• Example: Capstone Project

 Indirect Measures

• What students and faculty think students learned
• Easier assessment to do and undervalued
• Example: NSSE and LSSE

How can the IDEA survey be used as an indirect assessment measure?

OBJECTIVE 1: Gaining factual knowledge (terminology, classification, methods, trends) OBJECTIVE 2: Learning fundamental principles, generalizations, or theories OBJECTIVE 3: Learning to apply course material (to improve thinking, problem solving, and decisions) OBJECTIVE 4: Developing specific skills, competencies, and points of view needed by professionals in the field… OBJECTIVE 5: Acquiring skills in working with others as a member of a team OBJECTIVE 6: Developing creative capacities (writing, inventing, designing, performing in art, music, drama) OBJECTIVE 7: Gaining a broader understanding and appreciation of intellectual/cultural activity… OBJECTIVE 8: Developing skill in expressing oneself orally or in writing OBJECTIVE 9: Learning how to find resources for answering questions or solving problems OBJECTIVE 10: Developing a clearer understanding of, and commitment to, personal values OBJECTIVE 11: Learning to analyze and critically evaluate ideas, arguments, and points of view OBJECTIVE 12: Acquiring an interest in learning more by asking questions and seeking answers

How do we develop student learning goals for programs and majors?

Develop Learning Goals for Programs and Majors

University Learning Goals Department Student Learning Goals APA Recommendations (Halonen et al., 2006) Develop a sense of values that guides personal decision making Sensitivity to ethical issues Weigh evidence, tolerate ambiguity, act ethically, and reflect other values that are the underpinnings of psychology Develop intercultural competencies Recognition of ethnic, cultural, social and gender diversity Recognize, understand, and respect the complexity of sociocultural and international diversity

Students will develop…



A broad and deep knowledge base incorporating both social and natural science aspects of psychology



Understanding of different theoretical traditions



Methodological competence in statistics, research design and assessment methods including the use of technology for these purposes



Practical research, laboratory, or field experience to generalize a scientific style of thought to the realm of application



Effective communication skills



Sensitivity to ethical issues



A recognition of ethnic, cultural, social and gender diversity

What is the connection between program or major student learning goals and IDEA Objectives?

DEPARTMENT LEARNING GOAL

 A broad and deep knowledge

base incorporating both social and natural science aspects of psychology

 Understanding of different

theoretical traditions

IDEA OBJECTIVE

 Objective 1: Gaining factual

knowledge (terminology, classification, methods, trends)

 Objective 2: Learning

fundamental principles, generalizations, or theories

(HANDOUT)

DEPARTMENT LEARNING GOAL

 Methodological competence

in statistics, research design and assessment methods including the use of technology for these purposes

IDEA OBJECTIVE

 Objective 4: Developing

specific skills, competencies, and points of view needed by professionals in the field most closely related to this course

 Objective 9: Learning how to

find resources for answering questions or solving problems

(HANDOUT)

DEPARTMENT LEARNING GOAL

 Practical research,

laboratory, or field experience to generalize a scientific style of thought to the realm of application

IDEA OBJECTIVE

 Objective 3: Learning to

apply course material (to improve thinking, problem solving, and decisions)

 Objective 11: Learning to

analyze and critically evaluate ideas, arguments, and points of view

(HANDOUT)

DEPARTMENT LEARNING GOAL

 Effective communication

skills

IDEA OBJECTIVE

 Objective 5: Acquiring

skills in working with

• thers as a member of a

team

 Objective 8: Developing

skill in expressing oneself

• rally or in writing

(HANDOUT)

DEPARTMENT LEARNING GOAL

 Sensitivity to ethical issues  A recognition of ethnic,

cultural, social and gender diversity

IDEA OBJECTIVE

 Objective 10: Developing a

clearer understanding of, and commitment to, personal values

 -- NONE --

(HANDOUT)

Three IDEA Objectives Do Not Align with Department Learning Goals

 Objective 6: Developing creative capacities (writing, inventing,

designing, performing in art, music, drama)

 Objective 7: Gaining a broader understanding and appreciation of

intellectual/cultural activity (music, science, literature, etc.)

 Objective 12: Acquiring an interest in learning more by asking

questions and seeking answers

How do we know which IDEA

• bjectives are addressed in

which courses for a program or major?

IDEA Objective Required Courses for Psychology Major 100 110 200 210 220 300 310 320 330 340 350 370 410 1. E E I E E E E I I 2. E I E I E E E E 3. I I E E E I I E I I E E 4. I E E E I E I E I 5. I I 6.

This IDEA Objective is not aligned with department learning goals and thus is not assessed.

7.

This IDEA Objective is not aligned with department learning goals and thus is not assessed.

8. E I E E I 9. E I 10. I I I 11. I E I I I E 12.

This IDEA Objective is not aligned with department learning goals and thus is not assessed.

(HANDOUT)

Photo by A. M. Karkowski of M. Torello, S. Sagaria, J. Fournier, J. Loopstra,

• S. Saleem, K. Bell, September 17, 2012

How do we ensure that students understand the connections between course activities and learning goals and

• bjectives?

 Dr. L. Dee Fink  Capital Alumnus  Creating Significant Learning

Experiences

 Emphasized making explicit the

link between learning goals and course activities

Department Goal IDEA Objective Learning Objective Specific to Course Course Activity How the Learning Objective is Assessed Students demonstrate methodological competence in statistics, research design and assessment methods including the use of technology for these purposes IDEA Objective 4: Developing specific skills, competencies, and points of view needed by professionals in the field most closely related to this course Design, conduct, and analyze empirical research studies to test a hypothesis

• Develop

Research Studies

• Participation
• APA Style Paper

(Method)

• Collect and

Analyze Data

• APA Style Paper

(Results) Students demonstrate sensitivity to ethical issues IDEA Objective 10: Developing a clearer understanding of, and commitment to, personal values Apply ethical reasoning in a research context

• Complete IRB

CITI Training

• Research Ethics

Discussion

• Research Ethics

Class Activity

• IRB Protocol

Assignment

(HANDOUT)

Metacognition



Understanding Learning Goals  Linking New and Prior Learning



Building Knowledge Structures that Facilitate Future Learning

Student View of Metacognition



Requires Work

Benefits of Metacognition



Metacognition  Academic Success (Gourgey, 1998; Nietfeld et al., 2005)

Teach Students



About Course and Department Learning Goals and Objectives



How Learning Goals are Themes Across the Curriculum

Syllabi for Required Psychology Courses



Include a Map the Department Learning Goals for Required Courses



Seemingly Different Courses Can Achieve the Same Learning Goal

Identify when Course Activities Target Particular Learning Goal



Understand that Learning Goals Are Achieved by Accumulation of Many Different Learning Activities

When students have built and can use the metacognitive structures, ratings of the IDEA Objectives then provide one way to assess whether students perceive that learning goals are being met.

How do we document?

Assumptions

 Our primary concern is whether or not

students are achieving the course objectives

 A single data point (i.e., a data from a single

class) is not useful without the larger context

Student Ratings of Meeting Course Objectives (page 2 of the IDEA Feedback Form)

Excel Spreadsheet

 http://www.capital.edu/psychology-resources/  Can tailor to individual classes  Can graph classes over time

Directions Three Tabs

(HANDOUT)

IDEA Objectives Course Number

(HANDOUT)

SEMESTER F'08 S'09 MEANS 70 82 PSYCH 350 Semester 1 Semester 2

• 21. Gaining factual knowledge…

70 80

• 22. Learning fundamental principles…

80 90

• 23. Learning to apply course material…

70 80

• 24. Developing specific skills, competencies, and points of view…

60 70

• 25. Acquiring skills in working with others as a member of a team
• 26. Developing creative capacities…
• 27. Gaining… appreciation of intellectual/ cultural activity…
• 28. Developing skill in expressing myself orally or in writing

70 90

• 29. Learning how to…use resources for answering questions…
• 30. Developing… personal values
• 31. Learning to… evaluate ideas, arguments, and points of view

70 80

• 32. Acquiring an interest in learning…

10 20 30 40 50 60 70 80 90 100

F'08 S'09 F'09 S'10 F'10 S'11 F'11 S'12 F'12 S'13 F'13 S'14 % Students Agreeing

Course Met Course Objectives for PSYCH 350

(HANDOUT)

How do you get faculty to do this?

 Requested in Annual Report

• Includes reflection and future plans

 Requested in Promotion and Tenure Process

But, this is assessment, so don’t we need to “close the loop” and make changes based on the assessment?

Discussion of Course Objectives

 Courses Taught by Multiple Instructors  Examine Key Learning Objectives  Forced to Choose Only 3-5 Objectives  Identify how Course Activities Support Objectives

 Faculty: I really think “Developing skill in

expressing myself orally or in writing” should be listed as one of the objectives for PSYCH 110.

 Chair: What do you do in 110 that requires

students to speak and write so that they can develop those skills?

 Faculty: Well, nothing, really.

 Are course objectives being met?  Do enough courses target particular

• bjectives?

Develop Student Learning Goals Align Department Student Learning Goals with IDEA Objectives Map IDEA Objectives across the Curriculum Help Students Develop Metacognition Document IDEA Results Use IDEA Results & Discussions to Improve Courses & Curricula

Jody S. Fournier

jfournie@capital.edu

Terry D. Lahm

tlahm@capital.edu

Richard M. Ashbrook

rashbrook@capital.edu