SLIDE 1
STUDENT-LED ASSESSMENT:
RUBRIC DEVELOPMENT AND PEER GRADING IN WRITING ASSIGNMENTS
Presented by Brice Particelli, PhD English Department, New York City 2017 Faculty Institute, Pace University
STUDENT-LED ASSESSMENT: RUBRIC DEVELOPMENT AND PEER GRADING IN - - PowerPoint PPT Presentation
STUDENT-LED ASSESSMENT: RUBRIC DEVELOPMENT AND PEER GRADING IN - - PowerPoint PPT Presentation
STUDENT-LED ASSESSMENT: RUBRIC DEVELOPMENT AND PEER GRADING IN WRITING ASSIGNMENTS Presented by Brice Particelli, PhD English Department, New York City 2017 Faculty Institute, Pace University LIST What do you expect from a good Faculty
SLIDE 2
SLIDE 3
A small study on student responses to a writing assignment where student-led assessment was a central element, and a look into pedagogical implications. Students developed the rubric for the assignment, and they graded each other based on that rubric.
STUDENT-LED ASSESSMENT:
RUBRIC DEVELOPMENT AND PEER GRADING IN WRITING ASSIGNMENTS
SLIDE 4
PAIRS
What strategies do you use to make expectations explicit for your writing assignments? Do you use rubrics to communicate this? (Perhaps use a recent writing assignment as example.)
SLIDE 5
GUIDING QUESTIONS FOR THE DAY
How can we make more aspects of assessment more formative than summative? (assignment sheet, rubrics, peer review, grades, etc.) How do we make expectations more transparent (for the benefit of long-term student learning)? What are the pedagogical implications of letting students develop the assignment’s rubric? What are the pedagogical implications of letting students grade each other?
SLIDE 6
ENG 201: WRITING IN THE DISCIPLINES
- “…writing effective essays and research papers in disciplinary modes
and in students' field of interest. It may include interviews, analysis of journal articles, and appropriate documentation style formats…”
- Focus on writing about their disciplines:
- first a profile of an individual within their discipline (6 weeks)
- then a genre study of a genre within their discipline (6 weeks)
- and a presentation of how a genre works in their discipline (2 wks)
- Goals:
- think critically about their field
- think critically about how their discipline communicates
- write more effectively within the language of their field
- write more effectively across disciplines by better understanding
why and how each communicates differently
SLIDE 7
PROFILE ESSAY
A “public” genre vs. “occluded” genre Familiar to anyone who reads magazines, blogs, or websites that feature individuals Scaffolded learning: from comfortable to
- complex. Genre study of the public, moving to
self-guided genre study of the occluded Read 20+ broad exemplar texts: from Sports Illustrated to Business Insider to New York Magazine to Foreign Policy to Elle Six-week assignment Genre Study: “Genre as a typified social action that responds to a recurring situation,” as Irene Clark writes, and one that can better be “understood as originating from suitability and appropriateness, rather than from arbitrary formal conventions” (2005).
SLIDE 8
DEVELOPING CLASS CULTURE
- Readings that established metacognitive questions: How do writers learn to write
with authority? How do genres function within a discourse community? (i.e. Bartholomae’s “Inventing the University”). Goal: expose “authority,” help students access it.
- Read exemplar texts: Without a formal assignment sheet, we began reading profile
essays (across fields, styles, and subjects)
- Write to question: Weekly Blackboard writing focused on genre study - comparative
and focused analyses on audience, structure, intent, etc
- Write for emulation/creation: In-class writing to test the waters for their own profile
subject, constantly questioning why writers make the choices they do (depending on audience, subject matter, experience, time, etc)
- Surveys (3) to broaden discussion: i.e. Survey 1: faith in grades despite stated lack of
transparency; reliance on peers to understand assignment; benefits of peer review
- Discussions on grading: Questioned how writing is assessed and who determines the
goals, professionally and in class. (Asked: Most frequently awarded grade at Pace?)
SLIDE 9
Rojstaczer & Healy - research on the average grades of hundreds of universities Of all letter grades given in 4-year colleges, 45% are As; 34% are Bs; 14% are Cs.
SLIDE 10
ASSESSMENT
More problem with grades…
- Often cited study by Brookhart (1991) that concluded that instructors offer a
“hodgepodge grade of attitude, effort, and achievement” Why use a rubric? (To make assessment more meaningful and helpful)
- In a 2010 review of rubric use in higher education, Reddy and Andrade found that
“One striking difference between students’ and instructors’ perceptions of rubric use is related to their perceptions of the purposes of rubrics. Students frequently referred to them as serving the purposes of learning and achievement, while instructors focused almost exclusively on the role of a rubric in quickly, objectively and accurately assigning grades” (439). This is a problem because research shows “higher achievement and deeper learning by students who have rubrics to guide their work” (439).
- Similarly, Margaret Heritage wrote, “When used by students as part of a formative
assessment of their works in progress, rubrics can teach as well as evaluate. Used as part of a student-centered approach to assessment, rubrics have the potential to help students understand the targets for their learning and the standards of quality for a particular assignment, as well as make dependable judgments about their own work” (p. 437, 2010).
SLIDE 11
ASSESSMENT
Why get students involved in assessment?
- Benefits to writer: Peer-review is frequently used: more eyes on their work,
- ffering more feedback. Students read each other with an assessment goal in
mind (i.e. “look at the evidence used, specifically focused on xxxx.”)
- Benefits to reviewer: pushes reviewer to better understand their own
writing through the critique of others (Lundstrom & Baker 2009) Why get students involved in the grading piece of assessment?
- It’s reliable: In a 2006 study of 708 online students across 16 different
courses at 4 universities, Cho, et al, found that when at least four peers grade a piece of writing, the aggregate grade is as reliable and valid as a single
- instructor. And in less that one-third of the courses, did students offer low
ratings of reliability and validity for the peer assessment process (2006).
- It empowers students: It offers the same benefits as peer-review, but with
higher stakes and greater responsibility. Students must embrace becoming experts (with authority) on our version of the genre.
SLIDE 12
TIC TOC
- Rubric Development (3wks out): “popcorn” to create consensus;
assignment sheet and rubric sent out after that process (yes…late)
- “Rater Training” (1-2 wks out): Applied our rubric as a class to an
anonymous student-written profile; Applied it as part of peer review to a peer’s profile; Applied it to their own
- Grading Day: Students brought in two printed copies. Each graded
two, offering written feedback as well as the filled-out numeric rubric.
- Rewrites Reminder: To push them toward formative-focused
responses (and remove social pressures) I reminded them that I allow for rewrites. Only 3 took me up on rewrites (few rarely do), but it helped ease the worry.
- Control: I graded independently, putting my grades in a spreadsheet
before reviewing the peer-graded copies.
SLIDE 13
RESULTS
Average grade offered by students: 87.4% Average grade offered by instructor: 87.9% I sided with the student average in all but three cases, where a peer-grader was particularly harsh, offering a D when the other peer-grader (and I) offered a B- or C+. That resulted in an uptick to a final 88.3% average. Inflated? They’d been informally graded by a peer, informally graded themselves, done two guided peer-review sessions, and gotten direct feedback from me. Plus they’d picked the subject. If they didn’t get a B, they’d probably missed multiple classes (and were lost)…
SLIDE 14
RESULTS
Survey 2: one week before due date (and peer grading):
- 89% Agreed or Strongly Agreed that “I understand the
basic elements of this genre,” while 11% answered Neutral/No Opinion. No one Disagreed nor Strong Disagreed.
- The results for “I understand the requirements of this
writing assignment” were even more positive, at 97% Agreed or Strongly Agreed, and 3% Neutral/No Opinion
SLIDE 15
Survey Survey Question Strongly Disagree Disagree Neutral/No Opinion Agree Strongly Agree Initial: It is possible for students to grade each other's written assignments fairly and competently 2 9 18 8 1 Pre-Grading: I am confident my peers will grade my work fairly and competently 6 17 11 5 Post-Grading: My peers graded my work fairly and competently 1 3 8 15 3 Initial Survey Pre-Grading Survey Post-Grading Survey
Surveys over time: Students' confidence in their peers' ability to grade their work fairly and competently.
2 6 1 9 17 3 18 11 8 8 5 15 1 3
2 4 6 8 10 12 14 16 18 20
Strongly Disagree Disagree Neutral/No Opinion Agree Strongly Agree
RESULTS
SLIDE 16
Student responses to the Post-grading survey statement "Grading my peers' writing, I learned (check all that apply)"
15 14 13 12 11 9 9 3 How to better evaluate peers thoughtfully and effectively A better understanding
- f how others
evaluate my writing More about my
- wn approach
to this writing assignment About the subject-matter within the essay Styles and techniques that I can incorporate into future writing How to read similar texts more critically It reinforced things I already know about writing and reading I did not learn anything of value 2 4 6 8 10 12 14 16
RESULTS
SLIDE 17
A PROGRESSIVE JOURNEY
As Stadler writes, instructor expertise does not means that a “unidirectional practice of teacher-as-assessor is itself always justified or best. A strong case can be made that students should be taught how to change their pattern of thinking so that they know not only how to respond to and solve (externally sourced) problems but also how to frame problems themselves. They need this partly to guide their learning in between, or to prepare for, teacher assessments, but equally as part of their progressive journey into self assessment, and at more advanced levels, as a key skill for professional life” (2006)
SLIDE 18