From Outcomes and Maps to Developing A Plan to Assess Student - - PDF document
10/2/2011 From Outcomes and Maps to Developing A Plan to Assess Student Learning Presented at UW-Stevens Point March, 2011 Peggy Maki Education Consultant Assessment Editor and Writer pmaki86@gmail.com 1 Foci Visualizing a Cycle of
1
2
Visualizing a Cycle of Inquiry to Develop
Assessing for Enduring Learning Determining What You Want to Learn
Identifying When You Want to Learn Identifying or Designing Aligned
3
Developing Criteria and Standards of
Analyzing, Interpreting, and Acting on
Implementing Proposed
Re-entering the Assessment Cycle Appendices
4
Mission/Purposes Learning Outcomes How well do students achieve our
What Do We Want to Learn?
Gather Evidence And Data
Enhance teaching/ learning; inform decision- making, planning, budgeting Analyze and Interpret Evidence and Data
5
6
7
Integrate Transfer Apply or re-apply Re-use Synthesize Analyze Create Re-position their understanding
8
What misconceptions do students
What mental models or
9
Why do students have difficulty
What’s the relationship between
What patterns of weakness in
10
How do time restrictions or demands
What forms of animation or non-
11
What kinds of representational
How effective are hypermedia
12
Open-ended; not closed ended: You
Collaboratively developed based on
13
Baseline—at the beginning?
Formative—along the way?
Summative—at the end?
14
Product-focused: What and how
Process-focused: How students
15
16
Surface Learning Deep Learning
17
Tasks that require students to
Tasks that require students to
18
Focus on how students represent or
Align with students’ learning experiences
Align with curricular design verified
19
Invite collaboration in design (faculty
20
Psychometric approach—values
History of validity and reliability Quick and easy adoption and efficient
One possible source of evidence of
21
Evidence of strategies, processes, ways of
Evidence of complex and diverse ways in
Highly useful results that relate to
22
Focus on integrated learning Directly align with students’ learning
Provide opportunity for students to
Provide opportunity for students to
23
Immediate reliability and validity
Usually do not provide easy scoring
24
E-portfolios Capstone projects (mid-point and end
Performances, productions, creations Visual representations (mind mapping,
25
Case studies with Analysis/Self-Reflection Disciplinary or professional practices, such
Agreed upon embedded assignments that
Writing, to speaking, to visual
26
Team-based or collaborative projects Internships or Practica or Service Projects Internally or externally juried review of
Oral examinations or defenses or
27
Simulations/virtual simulations Computer-generated scenarios Performance on a national exam or locally
Learning logs or journals (online) Data Mining projects (webquests) Think Aloud Protocol
28
Students’ documentation of their learning
29
SALG (Open-ended questions in Student
30
Students’ perception of their learning, such
SGID (small group instructional diagnosis) Focus group (representative of the
Interviews (representative of the
31
Syllabi Audits (where and how often
Grades over Time Course-taking Patterns Other data at UW-Stevens Point?
32
Identify both direct and indirect
Based on each method, identify the
33
34
35
Are useful to students, assisting them
Raters use them to derive patterns of
36
Criteria descriptors (ways of
Creativity Self-reflection Originality Integration Analysis Disciplinary logic
37
Criteria descriptors (traits of the
Coherence Accuracy or precision Clarity Structure
38
Performance descriptors (describe
39
Conceptual understanding apparent Consistent notation, with only an
Logical formulation Complete or near complete
40
Emerging work in professional and
Research on learning (from novice to expert) Student work itself—derive traits and levels
41
Interviews with students or
Observations based on previous
42
Apply to student work to assure you
Schedule inter-rater reliability times:
43
Collect data from different sources to
Collect data you believe will be useful to
Organize reports around issues, not solely
44
Establish soft times and neutral zones to
Seek patterns against criteria and cohorts Tell the story that explains the results based
45
Interpret your data so that results of that
46
Analysis and Presentation of results during a
47
Comparison of Scores on Chronological Exams for Students Enrolled in The Traditional or Online B.S. Program in Health Sciences Beginning in 2006 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 2006 2007 2008 2009
Exam Years Online Traditional
48
61% 20% 11% 8% 0% 10% 20% 30% 40% 50% 60% 70% Emerging Developing Proficient Exemplary
49 50
51
Establish soft times and neutral zones to engage in
Seek patterns against criteria and cohorts Tell the story that explains the results—
52
Aggregate and disaggregate data to guide focused
Interpret your data so that your interpretation
Determine what you wish to change, revise,
53