Influential Interventions: Improving STEM Learning Outcomes for - - PowerPoint PPT Presentation

Influential Interventions: Improving STEM Learning Outcomes for Underrepresented Students

James Diedrick, Associate Dean of the College Drew Homa, Academic Assessment Coordinator

25th International Conference on The First-Year Experience Vancouver, B. C. • July 17, 2012

Agnes Scott College Atlanta, Georgia

Student Body as of Fall 2011:

• 883 students, representing 41

states and territories and 29 countries

• 89% of traditional students live on

campus

• 11% are international students
• More than 40% are students of

color.

• About 40% will study abroad

before they graduate

• Historically and presently, Agnes

Scott students have earned academia’s most prestigious scholarships including the Marshall, Rhodes, Fulbright, Goldwater, the Pickering Fellowship and the Gates Millennium Scholarship. Students in the Generating Excellence in Math & Science Summer Scholars Program (GEMS), Summer 2011

Our Commitment to STEM Education

• Majors: Biology, Biochemistry and Molecular

Biology, Chemistry, Mathematics, Mathematics- Physics, Mathematics-Economics, Neuroscience, Public Health, Physics & Astronomy

• Dual degree programs in Engineering,

Computer Science and Nursing with Emory University & Georgia Tech

• State-of-the art facilities in new science

building, observatory on campus

• Emphasis on supporting & encouraging women

in pursuit of STEM fields

Increasing Ethnic Diversity 1992-2011

74.2% 3.9% 4.0% 17.9%

White Non-resident International 1992

57.9% 5.2% 4.5% 32.5%

White 2002

40.4% 10.8% 4.7% 44.1%

2011

Increasing Ethnic Diversity 1992-2011

1992 2002 2011 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% American Indian/Alaska Native Unknown Hispanic Two or more races Asian/Pacific Islander Non-resident International African American White

Diversity on Campus: Total Undergraduate Population

1992 2002 2011 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% American Indian/Alaska Native Unknown Hispanic Two or more races Asian/Pacific Islander Non-resident International African American White

Diversity on Campus: Total First-Year Population

Achievement Gap for URM STEM Students

Percentage of STEM Majors Having a GPA ≥3.33 (2010)

• Analysis of performance in introductory STEM

courses clearly demonstrated a trend that faculty had long noticed (data from 2004-2009)

• Achievement gap persists even for students who

choose to major in a STEM discipline

GEMS Program

Generating Excellence in Math & Science

• Began 2007
• Year-round program aimed at supporting the

success of students pursuing majors in mathematics and/or the natural and physical sciences

• GEMS Summer Scholar Program, 2009-2011
• Activities include bi-monthly meetings, excursions,

service projects and collaboration with ASC faculty

• Students encouraged to take advantage of

academic support centers.

• Living-Learning component initiated for 2012-13

year.

GEMS Summer Scholars on field trip to Georgia Aquarium, 2011

Factors influencing success & persistence in STEM

Family Background Skills & Abilities Prior Schooling Classroom Climate Stereotype Threat Feedback on Learning Inquiry-Based Learning Exposure to Real- World Applications & Careers Extracurricular Activities Peer Group Interactions Academic Integration Social Integration Re-evaluate commitments & goals Decision to enroll in STEM class/major Decision to persist or depart from STEM Potential for peer-led Supplemental Instruction in gateway math & science courses to have a positive influence University of Michigan Center for Research and Learning

Teagle Foundation Support for Interventions to Promote STEM Student Success

• Two-year grant received in collaboration with Davidson College to develop

academic support initiatives that particularly benefit first generation and URM students

• Initiative runs from August 2011-May 2013
• Goal includes use of student learning outcome data to continue the levels
• f academic support beyond the funding period
• Initiative includes joint meetings of two institutions, workshops on campus

climate issues, stereotype threat, STEM pedagogy

Project Objectives

• Increase student success and persistence in STEM disciplines by adding

peer-led supplemental instruction (SI) sessions to gateway math and science courses

• All students are encouraged to attend SI sessions
• Specific impact to URM and first generation students determined through

assessment

• SI has been associated with more dramatic gains among URM students than

among their peers*

• Addressing/improving campus/classroom climate issues (Diverse

Learning Environments Survey (DLE) administered Spring 2012, comparative data to be released September 2012)

* Rath, Kenneth, et. al. “Supplemental Instruction in Introductory Biology I: Enhancing the Performance and Retention of URM Students” (CBE – Life Science Education 6 [2007]:203-216).

New forms of academic support

Coordinator for Resource Center for Math & Science (RCMS)

• Assisting in facilitating the implementation and assessment of the SI program
• Supporting & coordinating the work of all LA’s and peer tutors (individual peer tutors have

been in place for two decades)

• Increasing the level & variety of academic support available to our students in math and the

sciences

Peer Learning Assistants (LA’s)

• Juniors or seniors selected by faculty
• Training: 2-day session in August, ½ day session in January, and on-going training at regular

meetings throughout each semester

• Responsibilities: attending course lectures, leading SI sessions, developing workshop content

(to varying extents), holding 1:1 tutoring hours & attending regular staff meetings

• Grant support allowed for hiring students not eligible for traditional work-study

Math & Science Learning Center Coordinator

• Dr. Molly Smith, Math & Science Learning

Center Coordinator

• Ph.D. in Chemical Engineering from University
• f Michigan
• 6 years experience working in a university

environment as a graduate student, including voluntary service roles in coordination, leadership and planning of education programs and events

• 5 years corporate experience managing

projects and programs involving team members in multiple locations and partners at

• ther companies and universities; included

teaching intensive mini-courses on biology and chemistry fundamentals for employees

• Dr. Molly Smith

Ethnic & Racial Diversity of LA’s

Total of 11 LA’s for 2011-12 and 12 LA’s for 2012-13

55% 36% 9%

White 2011-12

50% 25% 8% 17%

White 2012-13

SI Implementation: Overview

FALL 2011 Biology 191: Cell and Animal Biology Chemistry 101: Fundamental Concepts of Matter and Reactions Math 118: Calculus I

Implemented SI in 6 gateway math & science courses (13 total sections, 194 individual students) during 2011-2012

SPRING 2012 Biology 192: Molecular Biology and Genetics Chemistry 102: Periodicity and Chemical Reactions Math 118: Calculus I Math 119: Calculus II

COMMON

• All students encouraged to

attend SI

• SI sessions offered weekly, in the

evenings

• SI sessions led by peer Learning

Assistants (LA’s)

• LA’s attend course lectures as
• ften as possible

VARIED

• Method of encouraging

attendance

• Number of sessions per week
• Level of guidance professors

provided LA’s for SI content

Some aspects of implementation were common across courses, while others varied by course/professor

SI Implementation: Overview

Supplemental Instruction Rubric

Session Date & Day of Week: __________________ SI Leader: ______________ Course: ____________________ Course Instructor: _______________________ Objective: What are the two most difficult concepts the students need to work

• n today?

_________________________________________________________________ _________________________________________________________________ Beginning reminders: Arrange seats in a circle Hand out Participation Log Set agenda with group Remember to relax and be flexible!

Possible Processes: Informal Quiz, Matrix, Reciprocal Questioning, Paired Problem-Solving, Turn to Use Your Partner, Note Processing, Problem-Solving Rubric, Formal Definition (or ID’s), Text Review (Divide and Conquer), Pictorial Representations, Sequencing Closure options: Predict next lecture, summarize session, informal quiz, 1-minute writing

• -Rubric adapted from Deanna, Martin C. and David R. Arendale, Supplemental Instruction:

Improving First-Year Student Success in High-Risk Courses. (National Resource Center for the Freshman Year Experience, University of South Carolina, 1992).

Content to cover Processes to use*

Supplemental Instruction Rubric - continued

SI Implementation: Details Fall 2011

Definitions for level of guidance provided to LA’s

• Low: LA’s develop nearly all content, based on their observations during lecture & recollections from when they

took the course

• Mid: Profs & LA’s meet weekly to discuss possible SI topics and profs provide some guidance on specific

exercises/problems to do

• High: Profs provide detailed worksheets and meet regularly with LA’s

Course Enrolled # Sections & Profs LA’s SI/week Attendance Incentive Guidance to LAs* BIO 191 95 3 sections 3 profs 4 4 Encourage mid CHEM 101 76 3 sections 2 profs 2 4 Require high Math 118 29 1 section 1 prof 2 2 Exam Bonus Points (starting½ way) high

* Each CHEM 102 professor used a different approach to encourage attendance Course Enrolled # Sections & Profs LA’s SI/Week Attendance Incentive Guidance to LAs BIO 192 63 2 sections 2 profs 3 3 Encourage Low CHEM 102 60 2 sections 2 profs 2 4 Encourage* Require* Mid* High* MATH 118 30 1 section 1 prof 2 2 Exam bonus points High Math 119 29 1 section 1 prof 1 2 Exam bonus points High

SI Implementation: Details Spring 2012

Assessment Methodologies

Quantitative Data

• SI session atuendance logs
• Mid-term and Final grades
• Descriptjve student

variables (class year, URM, FGEN, etc.) Qualitative Data

• Focus group interviews with

First-Year students in STEM gateway courses

• Meetjngs and focus groups

with Learning Assistants

• Workshops and discussions

with STEM faculty using LA’s and SI

Session Attendance Trends

• 194 Individual students participated
• Of those, 135 (70%) were “Target” students, either FGEN or URM
• Since some students took more than one course, we have a total
• f 346 class-students who we studied.
• Of those 346 students, 254 (73%) of them participated in at least 1

workshop

• The ratio of Target students who participated in workshops is the

same as those attending the class (108 of 156 = 70%.)

SI Attendees Perform Better Overall

191 192 101 102 118 FA 118 SP 119 1 2 3 4

SI Group Non-SI Group Average Final Grade

BIO CHEM MATH

SI Group: atuended 2 or more sessions Non-SI Group: atuended 0 or 1 session(s)

Times Atuended Semester Class # students 1-2 3-5 6+ 11/FA BIO-191 28 46% 11% 32% 11% 11/FA BIO-191 41 22% 24% 20% 34% 11/FA BIO-191 17 41% 29% 6% 24% 11/FA CHE-101 24 8% 8% 33% 50% 11/FA CHE-101 27 37% 15% 30% 19% 11/FA CHE-101 23 17% 22% 22% 39% 11/FA MAT-118 26 31% 42% 19% 8% 12/SP BIO-192 21 33% 19% 14% 33% 12/SP BIO-192 33 12% 15% 30% 42% 12/SP CHE-102 23 30% 13% 17% 35% 12/SP CHE-102 37 41% 19% 16% 22% 12/SP MAT-118 25 24% 16% 16% 44% 12/SP MAT-119 24 4% 25% 33% 38%

Varied Levels of Session Attendance

Grade Target Students Non-Target Students Improved or was an A at mid-term (40%) 95 (55%) 55 No Change (22%) 53 (43%) 43 Worsened (38%) 89 (16%) 16

From Mid-term Grade to Final Grade... ...Based on Number of Sessions Attended

Positive Impact of Regular SI Attendance

Improved or A at mid-term

0.1 0.2 0.3 0.4 0.5

Percentage of Students

# tj tjmes atu tuended

Improved or A at at mid-term

0.1 0.2 0.3 0.4 0.5

Percentage of Students

# tj tjmes atu tuended

From Mid-term Grade to Final Grade From Fall 2011 to Spring 2012

Of the 102 students who took an SI class in both semesters:

More improvement in Spring than Fall Same improvement in Spring and Fall Less improvement in Spring than Fall Earned A's at Mid-term and Final, both Fall and Spring More sessions attended in Spring than Fall 19 6 14 2 Same sessions attended in Spring as in Fall 4 5 2 2 Fewer sessions attended in Spring than Fall 10 3 21 3 No sessions attended Fall

• r Spring

11

Comparison of Fall and Spring Performance

Samples of Student Feedback—first years

[My LA] helps me to restore my faith in chemistry and want to study

• more. She has been very helpful, kind, patient, and honest. She

explains topics in real world scenarios by making references to reality TV shows, elementary schools, and everyday life situations. At times when the textbook and word problems become to wordy she breaks them down into smaller simpler pieces that help me to piece things together and think through the question in a more useful, sensible way instead of getting stressed out and giving up. She teaches us with authority while showing humility and compassion, which makes it easier to relate to her and ask her questions that might seem trivial. I do not know what I would have done this semester without her help. The [LAs] were complete life savers- always so friendly and able to

• help. I absolutely loved them and was privileged to work with them.

“What were some of your favorite things about being a Teagle Learning Assistant?”

• Getting to work with students and feeling a sense of

accomplishment knowing I helped them learn something.

• Students trusted and respected me and I had an active role in

helping them succeed.

• Helping others understand concepts and seeing them apply them

to all areas.

• Having a close bonding with first year calculus students
• Having the experience of teaching college material.
• Having fun creating new forms of teaching and activities
• Sharing the excitement of every “aha” moment; and of course

introducing the world’s coolest subject to a new generation

Samples of LA Feedback

Next Steps/Future Plans…

• SI Rubric used by all faculty
• Move from encourage to require attendance at SI sessions
• Facebook page to facilitate contact between students and LAs
• Sharing positive impacts we’ve seen with students
• Analyzing DLE data and comparing ASC with national data
• Adding additional classes to SI/LA initiative (Physics)
• Working with Davidson College to compare findings and best

practices

Selected References

Brown, Marie Kendall, et al. “Teaching for Retention in Science, Engineering and Math Disciplines: A Guide for Faculty” University of Michigan Center for Research and Learning Occasional Paper No. 25, 2009. Davis, Jeff. The First-Generation Student Experience: Who They Are, Their Characteristics, and Strategies for Improving Their Persistence and Success. Stirling, Virginia: Stylus, 2010. Deanna, Martin C. and David R. Arendale. Supplemental Instruction: Improving First-Year Student Success in High-Risk Courses. The Freshman Year Experience: Monograph Series Number 7. 3rd Ed. Columbia, South Carolina: National Resource Center for the Freshman Year Experience, 1992. Eagan, Kevin & Jessica Sharkness. “A Strong Start in the Sciences: Factors Influencing Minority Students’ Academic and Social Engagement.” Conference Presentation, 28th Annual Conference on the First Year Experience, Orlando, FL February 2009. Freeman, Scott, et. al. “Prescribed Active Learning Increases Performance in Introductory Biology. CBE— Life Sciences Education, 6 (2007): 132–139. Jones Taylor, Valerie and Gregory M. Walton. “Stereotype Threat Undermines Academic Learning (Personality and Social Psychology Bulletin 37 (2011): 1055–1067. Rath, Kenneth A., et. al. “Supplemental Instruction in Introductory Biology I: Enhancing the Performance and Retention of Underrepresented Minority Students.” CBE—Life Sciences Education 6 (2007): 203– 216. Steele, Claude. “A Threat in the Air: How Stereotypes Shape Intellectual Identity and Performance” American Psychologist 52 (1997): 613-629.

Thanks to:

• Dr. Molly Smith, Math & Science Learning Center Coordinator
• Dr. Lerita Coleman Brown, Director of the Science Center for

Women

• Jennifer Cannady, Assistant

Dean of the College, Director of Academic Advising & Student Success

• Agnes Scott students, staff

& STEM faculty

• Agnes Scott ITS Staff

Bradley Observatory

• Dr. Laura Palucki-Blake, Assistant

Director of the Cooperative Institutional Research Program (CIRP) at the Higher Education Research Institute, UCLA

• Dr. Valerie Jones Taylor, Assistant

Professor of Psychology at Spelman College

• The National Center for

the First-Year Experience and Students in Transition

• The Teagle Foundation
• Dr. Laura Palucki Blake

And also thanks to: