SLIDE 1 Page 1 of 13
ASSESSING EFFECTIVENESS OF PEER ASSISTANCE PROGRAMMING
Sally Lipsky, Ph.D., Peer Assistance Coordinator (sal@iup.edu) Center for Learning Enhancement, Developmental Studies Department Indiana University of Pennsylvania, Indiana, PA 15705 U.S.A. www.iup.edu/devstudies At Indiana University of Pennsylvania peer assistance consists of Supplemental Instruction and Walk-In
- Tutoring. Both target high-risk freshman-level courses, that is, courses with “D/F/W” rates of 30% and higher.
Effectiveness of these academic support models is measured accordingly:
( undergraduate students using the programs ). Outcomes are reported for participation rates, final course grades, success rates as measured by % of D/F/W’s, as well as annual satisfaction surveys sent via e-mail.
( undergraduate, paraprofessionals leading the sessions ). Outcomes are measured according to objectives set forth in the 1-credit training course, which all must successfully complete before starting their paid positions. Also, outcomes are reported for end-of-term surveys and focus groups, which produce qualitative information regarding the academic, personal/social, and career effect of peer educators’ work experiences.
( Indiana University of Pennsylvania ). Outcomes are reported according to student persistence and revenue saved by the peer assistance programs. Since students’ participation results in an approximate 10% higher persistence rate, peer assistance is quite cost-effective for the institution.
Lipsky, S. A. Assessing Efgectiveness of Peer Assistance Programming. 20th International Conference on The First-Year Experience, Kona, Hawaii, July 2007.
SLIDE 2 Page 2 of 13
Student Participants Student Participants: Annual Satisfaction Survey
SUPPLEMENTAL INSTRUCTION SUPPLEMENTAL INSTRUCTION
* E-mailed to 347 students attending 3 or more times; 109 responded — an overall response rate of 31%. My participation helped me learn course content.
75%
3%
- B. Somewhat 22%
- D. Not sure0%
My participation helped me to develop efgective learning strategies & study skills.
46%
6%
- B. Somewhat 48%
- D. Not sure1%
The leader was prepared and organized.
84%
3%
- B. Somewhat 13%
- D. Not sure0%
The leader incorporated active participation by all students.
79%
3%
- B. Somewhat 17%
- D. Not sure 1%
Most valuable features of SI? – Worksheets, practice exams, study guides. – Increased understanding of course material. Suggestions for improving? – More time, sessions, individual help.
WALK-IN PEER ASSISTANCE WALK-IN PEER ASSISTANCE
* E-mailed to 181 students attending 2 or more times; 27 responded — an overall response rate
My participation helped me complete assignments & learn course content.
85%
4%
- B. Somewhat 11%
- D. Not sure0%
My participation helped me develop more efgective learning strategies & study skills.
70%
4%
- B. Somewhat 26%
- D. Not sure0%
The leader responded to my questions & needs.
96%
0%
- B. Somewhat 4%
- D. Not sure 0%
Most valuable features? – Helpful peer educators. – Flexible, able to come and go as needed. Suggestions do you have for improving? – More tutors, more times. * Results for fall term 2006.
Lipsky, S. A. Assessing Efgectiveness of Peer Assistance Programming. 20th International Conference on The First-Year Experience, Kona, Hawaii, July 2007.
SLIDE 3 Student Participants Student Participants: Participation Rates, Final Course Grades, and Success Rates
Supplemental Instruction Supplemental Instruction
Ofgered in 29 high-risk* natural science course sections for 2006-7.
- Mean fjnal course grade for participating
students (1 or more times) was 2.08 vs. 1.55 for non-participants.
- This .53 mean grade difgerence exceeds the
national mean of .29 for 4-yr. public universities.
- Participants averaged 7 sessions per term.
- Participants’ mean D/F/W rate* was 32% vs.
48% for non-participants, a difgerence of 16%.
- Total contact hours = 6,093.
- Average contacts per week = 218.
Results reported for academic year 2006- 7.
Walk-In Assistance Walk-In Assistance
Ofgered for lower-level, high-risk* courses with many sections.
- Approximately 16% of students in targeted
courses participated.
- Participants’ mean fjnal course grade was
1.96 vs. 1.69 for non-participants, a difgerence of .27.
- Participants averaged 3 sessions per term.
- Participants’ mean D/F/W rate* was 34% vs.
47% for non-participants, a difgerence of 13%.
- Total contact hours = 1,887. (1,265 fall;
622 spring)
- Average contacts per week = 93 fall; 48
spring.
SLIDE 4 Reporting Outcomes for Student Participants Reporting Outcomes for Student Participants
P PARTICIPATION
ARTICIPATION R
RATES
ATES
- Participation rate for students in courses with peer assistance. (Contrast with national data for 2-yr./ 4-yr.
institutions.)
- Mean number of sessions participants attended (refer to results of participation study).
- Total student contacts for peer assistance at your institution. Can you chart growth rates?
M MEAN
EAN F
FINAL
INAL G
GRADES
RADES
- Difference in mean final course grades: participants vs. non-participants.
- Are participants’ mean final course grades at or above the 2.0 cut-off for good academic standing?
- Note criteria for identifying participants: students attending 1 or more times during the term, 3 or more times, or
- ther.
- Contrast your mean to national mean for 2-yr./ 4-yr. private or public institutions.
D/F/W R D/F/W RATES
ATES
* As an academic retention model, peer assistance targets high-risk high-risk, introductory-level courses— courses in which 30% or more students earn D’s, F’s, or withdraw. High D/F/W rates increase costs for students and the institution since students repeat courses (filling valuable seats), become increasingly frustrated and less satisfied, and are more likely to leave. As a result, the institution expends more money to recruit new students as
- replacements. Peer assistance intervenes in this costly cycle via peer-led sessions focusing on how to learn course
- content. Thus, a measure of success of peer assistance is a reduction in D/F/W’s for historically difficult courses.
- D/F/W rate for participants vs. non-participants.
- Overall rate for peer assistance courses. Has it fallen below the 30% standard for high-risk courses?
SLIDE 5
Student Participants Student Participants: Incoming Characteristics
One common assumption is that students who choose to participate in SI are academically stronger than students choosing not to participate. However, previous research has demonstrated that this is not the case—both SI participants and non-participants tend to possess similar incoming characteristics. For the 2006 admit year , I examined mean SAT scores for beginning freshmen in the same major and college—Nursing/Allied Health—at IUP . Hypothesis: There is no difgerence in academic ability, as measured by SAT total scores, between freshmen choosing to participate in SI and freshmen choosing not to participate in SI for Biol 105: Cell Biology. Results: No statistical difgerence in SAT scores was revealed between the two groups. However, mean SAT scores for freshmen choosing not to participate in SI was 24 points higher than for the participants. FALL 2006 Freshman Nursing & Allied Health majors NO, did not attend SI for Biol 105 YES, attended SI * for Biol 105 n = 83 n = 64 Mean SAT total score 998 974 difgerence mean SAT scores 24 favoring non-participants t-value (2-tailed) .43 – not signifjcant at .05 level Mean fjnal grade Biol 105 1.43 1.88 difgerence mean fjnal grades .45 favoring SI participants *Averaged 8.6 sessions during fall term.
SLIDE 6
Note: The freshmen attending SI had lower entering SAT scores yet earned higher fjnal grades in Cell Biology, underscoring the positive afgect of peer-led SI sessions, especially when students participate regularly as did these freshman participants.
SLIDE 7 Peer Educators Peer Educators: : Outcomes measured according to Objectives for 1-credit Training Course
R ROLE
OLE
OF
OF
THE
THE P
PEER
EER E
EDUCATOR
DUCATOR: T
: THEORY
HEORY, P
, PRACTICE
RACTICE, & A
, & ASSESSMENT
SSESSMENT
COURSE OBJECTIVES COURSE OBJECTIVES: Students will be able to:
- Describe purposes & impacts of peer educator roles.
- Observe & summarize characteristics of effective peer-led sessions.
- Understand elements of facilitative helping & collaborative learning.
- Demonstrate effective problem solving skills.
- Demonstrate knowledge of appropriate resources for referring peers.
- Demonstrate understanding of how to apply college learning strategies.
- Assess performances of peer educators.
- Relate effective practices to student success outcomes.
EVALUATION METHODS EVALUATION METHODS
Actively participate in class activities (discussions, simulated sessions, role-playing) & complete all
- assignments. 25% of fjnal grade
Observe & critique four peer-led sessions. 50% of grade Lead & critique a trial session. 25% of grade
SLIDE 8 Peer Educators Peer Educators: : Results of End-of-Year Surveys
Peer educators report benefits of their work experience:
Academic Growth Academic Growth
- 100% reported the position infmuenced their desire to learn & decision-making skills.
- 91% reported improved ability to think through complex issues & solve problems.
Social Growth Social Growth
- 91% reported improved ability to work efgectively & cooperatively with others.
- 83% developed relationships with individuals of difgering backgrounds.
- 75% indicated that their work helped them accept the uniqueness of others.
“I have a special way to present the topic of aerobic respiration to students. I can see the light bulbs switching on, and it amazes me that I can have that kind of power and efgect!”
Personal/Career Growth Personal/Career Growth
- 75% indicated that their work helped them to defjne a career path for themselves.
“I have decided to get my masters degree in biology so that I can teach on the college
- level. My experiences as a peer educator have led me to this decision.”
- 81% rated their role as having a substantial impact in their satisfaction with IUP
. “I love the SI program. My job has helped me to become a part of the campus community, and I learn from the students I work with. I can’t imagine life without it!”
SLIDE 9 Peer Educators Peer Educators: Qualitative Study
T TITLE
ITLE:
: Effects of Work Experiences for Undergraduate Peer Educators
Effects of Work Experiences for Undergraduate Peer Educators
P PURPOSE
URPOSE:
: Assess the effects of the work experience for peer educators. M METHOD
ETHOD:
:
- Thirteen peer educators participated; all worked at least two terms.
- Each participated in 1 of 3 focus group sessions.
- A graduate student facilitator presented open-ended questions about work
experiences & provided prompts, as needed, to spark discussion.
- Each discussion was taped and transcribed.
- Two investigators collated and categorized responses independently.
P PRELIMINARY
RELIMINARY R
RESUL
TS ESUL TS:
: Subjects consistently reported benefits of their work experience in terms of increased increased:
Academic skills. .
- Communication & relationship-building skills
Communication & relationship-building skills. .
- Personal & professional skills
Personal & professional skills. . T TENTATIVE
ENTATIVE C
CONCLUSION
ONCLUSION:
: Peer educators regard their positions as more than just another campus job. Besides monetary compensation, they gain cognitive, social, personal, and professional growth and rewards.
SLIDE 10 Institution Institution: Measuring Cost-effectiveness of Peer Assistance for Freshman Freshman Students Students
Approximately 10% more students persist until graduation than do non-participating students with similar characteristics.*
Cost-Effectiveness per Term Cost-Effectiveness per Term
- 1. Approximately 200 freshmen attend at least 1 SI session for a targeted course during a term at
Indiana University of PA (IUP).
- 2. If 10% more persist,* approx. 20 more freshmen remain at IUP as a result of SI participation.
- 3. A full-time undergraduate student residing in Pennsylvania contributes approx. $2,900 a term for
tuition & fees.
- 4. The state contributes about $2,500 per student per term.
- 5. Total revenue generated per student is about $5,400 per term.
- 6. If 20 more students remain at IUP due to participation in SI, the approx. retained revenue
per term is $108,000 – & every term after until each of the 20 students graduates. * Congos, Dennis (2001). How SI Retains Revenue for Colleges and Universities. University of Central Florida.
SLIDE 11 Institution Institution: Measuring Cost-Effectiveness over Four Years
Cost-Effectiveness over Four Years Cost-Effectiveness over Four Years
- 1. The next term another 200 freshmen attend at least one SI session, resulting in another 20
students remaining at the institution.
- 2. Another $108,000 in tuition & fees is retained the next term, or approx. $216,000 is retained over
two terms.
- 3. Each term SI generates a new cadre of freshman participants.
- 4. For each group of freshmen attending SI, the institution retains approximately $756,000 in
revenue through graduation of the students. These fjgures do not include:
Costs to recruit students to replace 10% otherwise lost to attrition. Added revenue from upper-class students attending SI. Costs of peer assistance programming (personnel, materials, facilities).
SLIDE 12 Based on an average of 200 FRESHMAN participants per term, resulting in 20 students persisting each term until graduation. Costs spent
- n recruiting students to replace the 10% of students otherwise lost to attrition and revenue from participating upper-class students are not
- included. Costs of peer assistance programming (personnel, materials, facilities) are not deducted.
YEAR 1: End of FALL TERM YEAR 1: End of SPRING TERM YEAR 2: End of FALL TERM YEAR 2: End of SPRING TERM YEAR 3: End of FALL TERM YEAR 3: End of SPRING TERM YEAR 4: End of FALL TERM YEAR 4: End
TERM TOTAL RETAINED REVENUE Group A 20 Freshme n saved 0. Group A 20 students saved $108,000. Group A 20 students saved an added $108,000. Group A 20 students saved an added $108,000. Group A 20 students saved an added $108,000. Group A 20 students saved an added $108,000. Group A 20 students saved an added $108,000. Group A 20 students saved an added $108,000. Group A 7 terms of persistenc e saved $756,000. Group B 20 Freshmen saved 0. Group B 20 students saved $108,000. Group B 20 students saved an added $108,000. Group B 20 students saved an added $108,000. Group B 20 students saved an added $108,000. Group B 20 students saved an added $108,000. Group B 20 students saved an added $108,000. Group B 6 terms of persistenc e saved $648,000. Group C 20 Freshmen saved 0. Group C 20 students saved $108,000. Group C 20 stu den ts sav ed an add ed $10 8,0 00. Group C 20 stu den ts sav ed an add ed $10 8,0 00. Group C 20 students saved an added $108,000. Group C 20 students saved an added $108,000. Group C 5 terms of persistenc e saved $540,000. Group D 20 Freshmen saved 0. Group D 20 students saved $108,000. Group D 20 students saved an added $108,000. Group D 20 students saved an added $108,000. Group D 20 students saved an added $108,000. Group D 4 terms of persistenc e saved $432,000.
SLIDE 13
Group E 20 Freshmen saved 0. Group E 20 students saved $108,000. Group E 20 students saved an added $108,000. Group E 20 students saved an added $108,000. Group E 3 terms of persistenc e saved $324,000. Group F 20 Freshmen saved 0. Group F 20 students saved $108,000. Group F 20 students saved an added $108,000. Group F 2 terms of persistenc e saved $216,000. Group G 20 Freshmen saved 0. Group G 20 students saved $108,000. Group G 1 term of persistenc e saved $108,000. Group H 20 students saved 0. Group H 0 terms of saved $0. $0 retained Fall YEAR 1 $108,000 retained Spring YEAR 1 $216,000 retained Fall YEAR 2 $324,000 retained Spring YEAR 2 $432,000 retained Fall YEAR 3 $540,000 retained Spring YEAR 3 $648,000 retained Fall YEAR 4 $756,000 retained Spring YEAR 4 $3,024,000 Total retained after 4 years
