Research Tools and Methods: Investigating Student Success in - - PowerPoint PPT Presentation

Research Tools and Methods: Investigating Student Success in Calculus-based Physics I

KATHY KOENIG AND ALEX MARIES, PHYSICS DEPT UNIVERSITY OF CINCINNATI

Higher education is “now reaching a major tipping point,” Coleman

• added. “We cannot condone poor teaching of introductory STEM

courses because we are trying to weed out the weaker students in the class or simply because a professor, department and/or institution fails to recognize and accept that there are, in fact, more effective ways to teach.” AAU Five-year Status Report 10-2-17 In 2011, it launched its Undergraduate STEM Initiative to encourage systemic reforms to science education to improve teaching and learning, especially in first- and second-year courses.

Failing to adopt evidence-based teaching practices “must be viewed as irresponsible, an abrogation of fulfilling our collective mission to ensure that all students …..are provided with the maximum opportunity to succeed”. Department-wide innovations are becoming institutional priorities, teaching and learning centers are being redesigned, and data and analytics are being used to monitor and improve student learning.

Investigating Student Success in Calculus-based Physics I

Setting:

• Spring Semester 2017
• 7 sections of Calculus-based physics (~130 students each)
• 6 faculty taught across these sections (various teaching styles)

Data Collection:

• Common exam scores and final course grades (DFW rates)
• Force Concept Inventory (FCI) – pre-test (and post-test)
• ACT Math Scores
• Student engagement survey
• Conversations with faculty to better understand teaching style

Teaching Styles

2 Flipped Classrooms

• “Flipped” and “Flipped (New)”
• videos assigned outside of class (selected from internet)
• essentially no lecture
• in-class problem solving (Powerpoint slides)
• frequent use of clickers

Teaching Styles (cont’d)

“Active” for Active Learning

• assigned textbook reading (no videos assigned)
• essentially ~30%-40% lecture
• in-class problem solving (similar problems to Flipped)
• frequent use of clickers

Teaching Styles (cont’d)

“Lec-to-Active”

• initial instructor went on medical leave after 1st Exam (~week 6)
• assigned textbook reading
• used clickers and group problems solving (similar to others)

“Lec-Based 1”, “Lec-Based 2”

• Lec-Based 1 assigned textbook reading
• essentially ~75% lecture (reported by students)
• some in-class problem solving (similar problems to others)
• one section included use of clickers

0% 5% 10% 15% 20% 25%

Active Flipped 1 Flipped (New) Lec-to-Active Lec-Based 1 Lec-Based 2

DFW Rate DFW Rates Spring 2017

What goe t goes i into t

• the Cal

alc-Ba Base sed P Physi sics g grade?

50% common exams 20% quizzes and other points up to the instructor 15% recitation attendance and homework 15% WebAssign Online Homework Red font represents common elements across courses.

26.5 27.0 27.5 28.0 28.5 29.0 29.5 30.0 Active Flipped Flipped (New) Lec-to-Active Lec-based 1 Lec-based 2

Mean ACT Math (36 max)

Mean ACT Math Scores

65% is mastery

5 10 15 20 25 30

Active Flipped Flipped (New) Lec-to-Active Lec-based 1 Lec-based 2

Pre-FCI Mean (of 30) Force Concept Inventory Pre-test

50 55 60 65 70 75 80 85

Active Flipped Flipped (New) Lec-to-Active Lec-based 1 Lec-based 2

Exam Mean (%)

All Exams

(Avg of 3 Exams Plus Final)

45 50 55 60 65 70 75 80 85 90 95

1 2 3 4 Mean of All Exams (%) Quartile

Mean Exam Score by Quartiles

Active Flipped Flipped (New) Lec-to-Active Lec-based 1 Lec-based 2

45 50 55 60 65 70 75 80 85 90 95

1 2 3 4 Mean of All Exams (%) Quartile

Mean Exam Score by Quartiles

Active Flipped Flipped (New) Lec-to-Active Lec-based 1 Lec-based 2 13% spread 12% spread 10% spread 7% spread

How might these patterns be explained?

Measuring Classroom Climate

The 4 Properties of Powerful Teachers Personality - approachable, humor, good-natured Presence – “own” the pedagogy, connect with content and students Preparation - the obvious plus reassess b/n each class Passion

– Rob Jenkins, Chronicle of Higher Education, March 16, 2015

Student Engagement Survey

• Survey given to all students last two weeks of the term
• Survey was anonymous but tagged to instructor
• Extra credit tied to survey completion (91% completed it!)
• Objectives:
• student background info not available through

Institutional Research

• how students engaged in course components
• how useful students thought those components were
• how each faculty conducted class and set up classroom

climate

How often did you read the text? Active Flipped (both*) Lec-based (all 3**) Read text >75% of the time 6% 25% 2%-8% Read text ~50% of the time 8% 14% 1%-9% Read text ~25% of the time 36% 33% 24%-40% Read none of the text 51% 28% 33%-66%

Of those who read the text: ~10% indicated text extremely useful all the time 18-33% indicated text never useful

*responses similar **Lec-based 2 did not assign textbook reading according to students

How often did you do the

• nline homework?

Active Flipped (both) Lec-based (all 3) Most of the time 86% 78%-86% 79%-86% 75% of the time 12% 11%-12% 8%-17% 50% of the time 0% 1%-8% 2%-4% 25% or less of the time 2% 1%-3% 1%-3% Was the HW useful? Active FL New Lec-based Very useful 38% 24% 19% 18%-23% Somewhat useful 58% 62% 55% 52%-62% Not at all useful 4% 14% 26% 17%-24%

Instructor of Active class assigned HW for all sections to complete.

Attendance (self-reported)

• 80% attended every class (another 10% attended 3/4 of classes)

What motivated to attend? Choose ALL. Active Flip Flip (New) Lec- Active Lec 1 Lec 2 Instructor took attendance. 72% 37% 39% 65% 90% 4% Clickers-participation grade. 80% 83% 73% 36% 2% 7% Lecture was useful. 73% 34% 8% 65% 30% 70% Instructor worked problems. 74% 63% 9% 55% 32% 63% Students worked problems. 56% 58% 11% 37% 23% 32% None of above motivated me. 0% 1% 9% 6% 5% 13%

When didn’t understand something or couldn’t do a HW problem, where did you generally go for help? (check ALL)

Choice (select ALL that apply) Response

Textbook Khan Academy or online lecture source Other students SI Leader Learning Assistance Center (tutoring, etc) Office Hours with instructor Office hours with LA or graduate TA Google/internet I didn’t seek out help Other

When didn’t understand something or couldn’t do a HW problem, where did you generally go for help? (check ALL)

Choice (select ALL that apply) Response

Google/internet 87% Other students 77% Khan Academy or online lecture source 51% Textbook 45% SI Leader 17% Other 10% Learning Assistance Center (tutoring, etc) 7% Office Hours with instructor 5% Office hours with LA or graduate TA 3% I didn't seek out help 0%

Ranged from 0%-11%

When didn’t understand something or couldn’t do a HW problem, where did you generally go for help? (check ONE)

Choice (select ALL that apply) Response

Google/internet 51% Other students 23% Khan Academy or online lecture source 10% Textbook 9% SI Leader 1% Other 1% Learning Assistance Center (tutoring, etc) 1% Office Hours with instructor 1% Office hours with LA or graduate TA 1% I didn't seek out help 0%

27% 38% Flip (New)

Of the time you spent on physics each week, how much time was spent working with other students?

• a. I never worked with other students outside of class.**
• b. I occasionally worked with other students outside of class.
• c. I often worked with other students but it was informal without

regular meeting times or necessarily the same students.

• d. I often worked with the same set of students but we never set

specific meeting times.

• e. I often worked with the same set of students and we set regular

study group meeting times.

Measuring Classroom Climate

My physics instructor made me feel comfortable in asking questions either during or after class.

Active Flip Flip (New) Lec- Active Lec 1 Lec 2

Strongly Disagree/Disagree 7% 18% 34% 9% 17% 8% Disagree Little/Agree Little 17% 41% 52% 41% 47% 37% Agree/Strongly Agree 76% 41% 14% 50% 36% 55%

Attended Office Hours: 11% 4% 0% 6% 2% 3%

Measuring Classroom Climate

My physics instructor encouraged students to seek help if needed (i.e. office hours, Physics Learning Center, MASS, SI, etc.).

Active Flip Flip (New) Lec- Active Lec 1 Lec 2

Strongly Disagree/Disagree 3% 5% 24% 2% 10% 7% Disagree Little/Agree Little 12% 17% 38% 27% 41% 34% Agree/Strongly Agree 85% 77% 40% 71% 50% 59%

Measuring Classroom Climate

My physics instructor made me feel like I could succeed in this course.

Active Flip Flip (New) Lec- Active Lec 1 Lec 2 Strongly Disagree/Disagree 3% 10% 38% 7% 15% 10% Disagree Little/Agree Little 9% 29% 49% 50% 53% 37% Agree/Strongly Agree 89% 59% 14% 42% 22% 53%

Measuring Classroom Climate

My instructor was clear and specific about what he/she expected students to learn.

Active Flip Flip (New) Lec- Active Lec 1 Lec 2

Strongly Disagree/Disagree 4% 9% 39% 3% 20% 9% Disagree Little/Agree Little 7% 22% 36% 34% 46% 28% Agree/Strongly Agree 89% 68% 25% 62% 63% 63%

“Active Learning” instructor posts a Weekly Pacing Guide (“Roadmap”) on Bb the week before so students know what to read and when HW is due, etc.

If you were to take this course again, list three things you would like to see kept the same. If you were to take this course again, list three things you would like to see done differently.

Preferred Learning Environment in Physics Class

*

Choice Response Listen to instructor lecture/solve problems ~50%* Blend of lecture, examples, group/individual work ~50% All group work with content via video/read text 2%-6% Listen to instructor lecture/NO solve problems 1%-5% *74% for Flipped (New)

How might this be useful?

• Data-driven decision making
• Reflective teaching (improve personal teaching style)
• Know student behavior (revise assignments, due dates)
• Know student needs (revise topics covered, time spent)
• Should OneStop designate AL/Flipped versus LEC?
• Should students with certain attributes be

automatically placed with certain instructors?

Questions?

Kathy Koenig kathy.koenig@uc.edu

5 10 15 20 25

FCI Mean Scores (30 max)

Pre- and Post-FCI Mean Scores

Pre-FCI Post-FCI

65% is considered mastery

Other questions to consider asking…

Indicate the percent of class time your instructor typically spent on each activity (the total should add to 100%). My instructor tried hard to make sure students understood the physics concepts discussed in class. My physics instructor provided explanations of course content that were understandable. My physics instructor seemed to genuinely care about whether or not I learned the material.

7 Learning Principles

#1-- All learning involves connecting up with and building on prior thinking and knowledge base. #2 Motivation to learn is essential. Is necessary element of teaching. #3– Engagement - People must think hard about a subject to learn it. #4 Need effective feedback--Guidance that shapes thinking and learning. #5 Expert thinking involves more than knowing information, is also how information is organized, applied, and learned. #6 Part of memory that remembers and process ideas on short times scales extremely limited capacity! More loaded down, less well it processes, less learned. #7 Requirements for retention well established.