Assessment Day ’18 - Social Science Your Learning Outcomes Leaders (LOLs) Osceola West East Doreen Watson Jovan Trpovsky Jamie Shipley Eric Model Melissa Sierra Ellen Pastorino Adrienne Mathews* Derek Schorsch* Scott Creamer Tarteashia Harris Heather Bryson Mike Szalma Tyler Branz Bonnie Oliver Lake Nona Poinciana Winter Park Jorge Vallardes Susan Dunn Debra Hollister
Today’s Plan • 1 – Overview/ Recent history of our efforts • 2 – Critical Thinking • Overview of results/plan for future • 3 – Ethical Responsibility • Discussion of instrument/ plan for rollout/intervention • 4 – Open q uestions and discussion • 5 – At Risk Groups • 6 – Top 20 Classes Success/DFW rates • 7 – Info Lit./Written Comm. For those of us w/ Gordon Rule classes (History, Political Science)
1 – Overview/ History of Assessment • Why are we doing this? – SACS – Eventually, to inform our teaching
1 – Overview/ History of Assessment
1 – Overview/History of Assessment
1 – Overview/History of Assessment • 2015/ Spring 2016 – – Created Ethical Responsibility Instrument – November - Rolled out CT instrument to 8877 students – Assessment Day – Reviewed CT data, -- tweaked Eth. Resp. instrument, made plans for Fall ’16 rollout • Fall 2016/Spring 2017 – Made tweaks to both instruments (CT and Eth. Resp.), analysis of data, Writing and Inf. Lit pushed to 16/17 - October - Rolled out Eth. Resp. instrument to 9800 students - Refined instrument/process for assessment of Information Lit., Writing - History and Poli Sci Profs gathered/ analyzed data
1 – Overview/History of Assessment • Assessment Day ‘17 – • Reviewed Ethical Responsibility Data • Broken down by Class • Broken down by Online/Hybrid/Face to Face – Made plan for intervention • Addition of Student Code of Conduct module to put into Blackboard/Canvas • Plan to reassess in Fall 2018 • Critical Thinking – Reviewed Data – Planned to eliminate some questions/ revise some others
2 – Critical Thinking • Fall 2017 • Made revisions to CT instrument • Cut pool of questions from 30 q. down to 23 q. • Made revisions to several others • Rolled out CT instrument via Atlas/Qualtrix to all students Compare this to: • 27.3% response rate in Fall ‘16 • 31% response rate in Fall ‘15
2 – Critical Thinking • 2 other interesting #’s for response rates…
2 – Critical Thinking • Fall ‘15 sample (n= 2857) : 6.14/10 • Fall ‘17 sample (n= 1940) : 6.42/10 Social Science CT Results 2017 379 400 329 329 350 300 252 231 250 Frequency 200 152 150 113 100 69 59 26 50 0 0 1 2 3 4 5 6 7 8 9 10 More
2- Critical Thinking • What does that mean? FT and PT 7 6.8 6.6 6.48 6.4 6.32 6.2 6 5.8 5.6 5.4 5.2 5 Full Time Students Part Time Students
2- Critical Thinking First Gen. 7 6.8 6.6 6.5 6.4 6.2 6.2 6 5.8 5.6 5.4 5.2 5 First Gen. Student Non First Gen.
2- Critical Thinking Modality 7.00 6.78 6.80 6.60 6.38 6.40 6.24 6.20 6.00 5.80 5.60 5.40 5.20 5.00 Hybrid Online F2F
2- Critical Thinking Sex 7 6.8 6.58 6.6 6.34 6.4 6.3 6.2 6 5.8 5.6 5.4 5.2 5 Student= Female Student= Male Student= Blank • Female N=1279 Male=620 Blank = 41
2- Critical Thinking Race and Ethnicity 7.01 7 6.74 6.7 6.64 6.59 6.57 6.52 6.5 6.44 6.5 6.27 6.18 6.15 6.11 6.01 5.97 5.96 5.95 6 5.8 5.5 5 Student = Af. Af. Student= Asian Asian Student = Cauc. Cauc. Student= Student= Hisp. Hisp. Student= Student= Multi Multi Student= Af. American American Asian Male Female Cauc. Male Female Hawaiian Hispanic Male Female Indian Multi Male Female Unknown American Male Female • 82 African American males completed the CT assessment. While their score of 6.01/10 was lower than Asian (6.27, n=33), Caucasian (6.74, n=180), Multi-Racial (6.59, n=17), and Hispanic (6.52, n=196) males, they outperformed African American females (5.95, n=187) and Hispanic Females (5.96, n= 456).
2- Critical Thinking Major/ Pre Major 7 6.8 6.71 6.62 6.6 6.42 6.38 6.4 6.27 6.27 6.2 6 5.8 5.6 5.4 5.2 5 Major= AA Major= AS Major= General Major= Undeclared Business PreMajors Psych PreMajors Interest
2- Critical Thinking Top 3 High Schools Bottom 3 #1 Legacy 7.62 #32 Foreign High Scho 5.97 #2 Apopka 7.5 #33 Ocoee 5.95 #3 Olympia 7.29 #34 Liberty 5.87 CT Score and % disadvantaged in 28 high schools 8 7.5 7 6.5 6 r= -.30 5.5 5 0 20 40 60 80 100 120
2- Critical Thinking GPA 7.15 7 6.69 6.47 6.5 6.3 6.11 6.01 6 5.89 5.5 5 0.00 - 2.0 2.01 - 2.4 2.41 - 2.8 2.81 - 3.2 3.21 - 3.6 3.61-3.99 4 • r= .16
2- Critical Thinking Scatterplot Score and GPA 10 8 6 4 2 0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
2- Critical Thinking Gen Ed. Class 6.98 7 6.86 6.8 6.7 6.63 6.6 6.47 6.4 6.27 6.2 6 5.8 5.6 5.4 5.2 5 POS ECO AMH PSY SYG ANT • N= 296 214 156 1165 225 54
2- Critical Thinking POS Data N=296 7 6.81 6.77 6.8 6.65 6.6 6.45 6.43 6.4 6.28 6.27 6.27 6.22 6.2 6.04 6 5.8 5.6 5.4 5.2 5 POS Hybrid POS Online POS F2F POS East POS LN POS Osc POS West POS WinterP POS Full Time POS Part Time Profs Profs
2- Critical Thinking ECO Data N=214 7.2 7 6.86 6.78 6.76 6.72 6.68 6.65 6.63 6.5 5.96 6 5.5 5 ECO Hybrid ECO Online ECO F2F ECO East ECO LN ECO Osc ECO West ECO Full Time ECO Part Time Profs Profs
2- Critical Thinking AMH Data N=156 7.5 7.3 7.08 7 6.87 6.82 6.72 6.72 6.7 6.5 6 5.5 5 AMH Online AMH F2F AMH East AMH Osc AMH West AMH Full Time Profs AMH Part Time Profs
2- Critical Thinking PSY Data N=1165 7.6 7.5 7 6.65 6.62 6.5 6.29 6.27 6.24 6.24 6.21 6.07 6.03 6 5.59 5.5 5 PSY Hybrid PSY Online PSY F2F PSY East PSY LN PSY Osc PSY Poinciana PSY West PSY WinterP PSY Full Time PSY Part Time Profs Profs
2- Critical Thinking SYG Data N=225 8 7.47 7.41 7.5 7 6.82 6.65 6.62 6.55 6.52 6.49 6.43 6.5 6 5.5 5 SYG Hybrid SYG Online SYG F2F SYG East SYG LN SYG Osc SYG West SYG Full Time SYG Part Time Profs Profs
2- Critical Thinking ANT Data N=54 8 7.76 7.68 7.5 7.2 7 6.77 6.68 6.61 6.57 6.5 6 5.5 5 ANT Hybrid ANT Online ANT F2F ANT East ANT West ANT Full Time ANT Part Time Profs Profs
2- Critical Thinking Score and Seconds (between 2 and 10 minutes) 10 9 8 7 6 5 4 3 2 1 0 100 150 200 250 300 350 400 450 500 550 600 • r= .33
2 – Critical Thinking • Thoughts? What is of value there? • What can we say about our students? About our teaching? • Can we use any of this to say that we’ve taught them to think critically?
2 – Critical Thinking • The challenge for Social Science: • We are disparate. • The challenge for measuring critical thinking: • Well, lots. • A solution?
2 – Critical Thinking • What do we want our students to be able to do? • Read articles? Think critically about them?
2 – Critical Thinking • What do we want our students to be able to do? • Watch videos? Think critically about them?
2 – Critical Thinking - Proposed • What do we want our students to be able to do? • Watch videos? Think critically about them? • https://www.youtube.com/watch?v=vG6-UaBECN4
2 – Critical Thinking - Proposed • Methodology • Switch to a pre/post • Keep Qualtrics/Atlas method • Keep it as multiple choice
3 – Ethical Responsibility • Quick Review. • 10 questions in 2 parts • First 5 – Vignettes tied to Student Code of Conduct • Second 5 – 5 pt Likert scale on Ethical/Social Responsibility adapted from APA learning outcomes
Q1 Q1- Academic Di Dishonesty ty – At the beginning of the semester, the course syllabus indicates a midterm exam. This midterm will take place the week after spring break. While on spring break, one student emails the professor and explains that their uncle has died and they will miss the exam. In the email, they ask if they could take the midterm exam late. The professor agrees but asks for verification of the story upon the students’ return. The student returns, and takes the test, but fails to provide any evidence of the story.
Q2 - Ch Cheatin ing – Subhas and Sally are taking a multiple choice test. After 45 minutes of the test, Sally sneezes accidently and her scantron sheet flies off the desk. Subhas, as a nice gesture, picks up the scantron for Sally. However, prior to giving it back, Subhas looks at the scantron very briefly. After returning to his own test, Subhas then proceeds to change several of his answers to what he saw on Sally’s sheet. Rate Subhas’ behavior:
Indicate the extent to which you agree or disagree with the following: SA, A, N, D, SD 6 – My __________ course challenged me to think more broadly about poverty, health, human rights, and/or other issues of prejudice and discrimination.
Indicate the extent to which you agree or disagree with the following: SA, A, N, D, SD 7 – My ___________ course enhanced my ability to see the world from someone else’s perspective.
Recommend
More recommend
