Assessment Day 18 - Social Science Your Learning Outcomes Leaders - - PowerPoint PPT Presentation

▶
assessment day 18 social science
SMART_READER_LITE
LIVE PREVIEW

Assessment Day 18 - Social Science Your Learning Outcomes Leaders - - PowerPoint PPT Presentation

Sep 29, 2023 24 likes •606 views

Assessment Day 18 - Social Science Your Learning Outcomes Leaders (LOLs) Osceola West East Doreen Watson Jovan Trpovsky Jamie Shipley Eric Model Melissa Sierra Ellen Pastorino Adrienne Mathews* Derek Schorsch* Scott Creamer

slide-1
SLIDE 1

Assessment Day ’18 - Social Science

Your Learning Outcomes Leaders (LOLs)

Osceola

Jamie Shipley Ellen Pastorino Scott Creamer Mike Szalma

Lake Nona

Debra Hollister

West

Doreen Watson Eric Model Adrienne Mathews* Tarteashia Harris Tyler Branz Poinciana Jorge Vallardes

East

Jovan Trpovsky Melissa Sierra Derek Schorsch* Heather Bryson Bonnie Oliver Winter Park Susan Dunn

slide-2
SLIDE 2

Today’s Plan

  • 1 – Overview/ Recent history of our efforts
  • 2 – Critical Thinking
  • Overview of results/plan for future
  • 3 – Ethical Responsibility
  • Discussion of instrument/ plan for rollout/intervention
  • 4 – Open questions and discussion
  • 5 – At Risk Groups
  • 6 – Top 20 Classes Success/DFW rates
  • 7 – Info Lit./Written Comm. For those of us w/

Gordon Rule classes (History, Political Science)

slide-3
SLIDE 3

1 – Overview/ History of Assessment

  • Why are we doing

this?

– SACS – Eventually, to inform our teaching

slide-4
SLIDE 4

1 – Overview/ History of Assessment

slide-5
SLIDE 5

1 – Overview/History of Assessment

slide-6
SLIDE 6

1 – Overview/History of Assessment

  • 2015/ Spring 2016 –

– Created Ethical Responsibility Instrument – November - Rolled out CT instrument to 8877 students – Assessment Day

– Reviewed CT data,

  • - tweaked Eth. Resp. instrument, made plans for Fall ’16 rollout
  • Fall 2016/Spring 2017

– Made tweaks to both instruments (CT and Eth. Resp.), analysis of data, Writing and Inf. Lit pushed to 16/17

  • October - Rolled out Eth. Resp. instrument to 9800 students
  • Refined instrument/process for assessment of Information Lit., Writing
  • History and Poli Sci Profs gathered/ analyzed data
slide-7
SLIDE 7

1 – Overview/History of Assessment

  • Assessment Day ‘17 –
  • Reviewed Ethical Responsibility Data
  • Broken down by Class
  • Broken down by Online/Hybrid/Face to Face

– Made plan for intervention

  • Addition of Student Code of Conduct module to put into Blackboard/Canvas
  • Plan to reassess in Fall 2018
  • Critical Thinking

– Reviewed Data – Planned to eliminate some questions/ revise some others

slide-8
SLIDE 8

2 – Critical Thinking

  • Fall 2017
  • Made revisions to CT instrument
  • Cut pool of questions from 30 q.

down to 23 q.

  • Made revisions to several others
  • Rolled out CT instrument via

Atlas/Qualtrix to all students Compare this to:

  • 27.3% response rate in Fall ‘16
  • 31% response rate in Fall ‘15
slide-9
SLIDE 9

2 – Critical Thinking

  • 2 other interesting #’s for response rates…
slide-10
SLIDE 10

2 – Critical Thinking

  • Fall ‘15 sample (n= 2857) : 6.14/10
  • Fall ‘17 sample (n= 1940) : 6.42/10

26 59 113 152 231 329 379 329 252 69 50 100 150 200 250 300 350 400 1 2 3 4 5 6 7 8 9 10 More Frequency

Social Science CT Results 2017

slide-11
SLIDE 11

2- Critical Thinking

  • What does that mean?

6.48 6.32 5 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7 Full Time Students Part Time Students

FT and PT

slide-12
SLIDE 12

2- Critical Thinking

6.2 6.5 5 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7 First Gen. Student Non First Gen.

First Gen.

slide-13
SLIDE 13

2- Critical Thinking

6.24 6.78 6.38 5.00 5.20 5.40 5.60 5.80 6.00 6.20 6.40 6.60 6.80 7.00 Hybrid Online F2F

Modality

slide-14
SLIDE 14

2- Critical Thinking

  • Female N=1279 Male=620 Blank = 41

6.34 6.58 6.3 5 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7 Student= Female Student= Male Student= Blank

Sex

slide-15
SLIDE 15

2- Critical Thinking

  • 82 African American males completed the CT assessment. While

their score of 6.01/10 was lower than Asian (6.27, n=33), Caucasian (6.74, n=180), Multi-Racial (6.59, n=17), and Hispanic (6.52, n=196) males, they outperformed African American females (5.95, n=187) and Hispanic Females (5.96, n= 456).

5.97 6.01 5.95 6.18 6.27 6.15 6.64 6.74 6.57 5.8 6.11 6.52 5.96 6.5 6.59 6.44 6.7 7.01 5 5.5 6 6.5 7 Student = Af. American Af. American Male Af. American Female Student= Asian Asian Male Asian Female Student = Cauc. Cauc. Male Cauc. Female Student= Hawaiian Student= Hispanic Hisp. Male Hisp. Female Student= Indian Student= Multi Multi Male Multi Female Student= Unknown

Race and Ethnicity

slide-16
SLIDE 16

2- Critical Thinking

6.27 6.38 6.62 6.42 6.27 6.71 5 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7 Major= AA Major= AS Major= General Interest Major= Undeclared Business PreMajors Psych PreMajors

Major/ Pre Major

slide-17
SLIDE 17

2- Critical Thinking

Top 3 High Schools Bottom 3 #1 Legacy 7.62 #32 Foreign High Scho 5.97 #2 Apopka 7.5 #33 Ocoee 5.95 #3 Olympia 7.29 #34 Liberty 5.87

5 5.5 6 6.5 7 7.5 8 20 40 60 80 100 120

CT Score and % disadvantaged in 28 high schools

r= -.30

slide-18
SLIDE 18

2- Critical Thinking

  • r= .16

5.89 6.11 6.01 6.3 6.47 6.69 7.15 5 5.5 6 6.5 7 0.00 - 2.0 2.01 - 2.4 2.41 - 2.8 2.81 - 3.2 3.21 - 3.6 3.61-3.99 4

GPA

slide-19
SLIDE 19

2- Critical Thinking

2 4 6 8 10 0.5 1 1.5 2 2.5 3 3.5 4 4.5

Scatterplot Score and GPA

slide-20
SLIDE 20

2- Critical Thinking

  • N= 296 214 156 1165 225 54

6.47 6.7 6.86 6.27 6.63 6.98 5 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7 POS ECO AMH PSY SYG ANT

Gen Ed. Class

slide-21
SLIDE 21

2- Critical Thinking

6.04 6.77 6.43 6.81 6.27 6.27 6.22 6.45 6.65 6.28 5 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7 POS Hybrid POS Online POS F2F POS East POS LN POS Osc POS West POS WinterP POS Full Time Profs POS Part Time Profs

POS Data N=296

slide-22
SLIDE 22

2- Critical Thinking

5.96 6.72 6.86 6.65 7.2 6.78 6.63 6.68 6.76 5 5.5 6 6.5 7 ECO Hybrid ECO Online ECO F2F ECO East ECO LN ECO Osc ECO West ECO Full Time Profs ECO Part Time Profs

ECO Data N=214

slide-23
SLIDE 23

2- Critical Thinking

7.3 6.72 6.7 7.08 6.72 6.87 6.82 5 5.5 6 6.5 7 7.5 AMH Online AMH F2F AMH East AMH Osc AMH West AMH Full Time Profs AMH Part Time Profs

AMH Data N=156

slide-24
SLIDE 24

2- Critical Thinking

6.03 6.65 6.21 6.27 6.24 6.07 5.59 6.62 7.6 6.24 6.29 5 5.5 6 6.5 7 7.5 PSY Hybrid PSY Online PSY F2F PSY East PSY LN PSY Osc PSY Poinciana PSY West PSY WinterP PSY Full Time Profs PSY Part Time Profs

PSY Data N=1165

slide-25
SLIDE 25

2- Critical Thinking

7.47 6.62 6.55 6.49 6.65 7.41 6.43 6.82 6.52 5 5.5 6 6.5 7 7.5 8 SYG Hybrid SYG Online SYG F2F SYG East SYG LN SYG Osc SYG West SYG Full Time Profs SYG Part Time Profs

SYG Data N=225

slide-26
SLIDE 26

2- Critical Thinking

6.77 7.76 6.57 6.61 7.2 7.68 6.68 5 5.5 6 6.5 7 7.5 8 ANT Hybrid ANT Online ANT F2F ANT East ANT West ANT Full Time Profs ANT Part Time Profs

ANT Data N=54

slide-27
SLIDE 27

2- Critical Thinking

  • r= .33

1 2 3 4 5 6 7 8 9 10 100 150 200 250 300 350 400 450 500 550 600

Score and Seconds (between 2 and 10 minutes)

slide-28
SLIDE 28

2 – Critical Thinking

  • Thoughts? What is of value there?
  • What can we say about our students? About our

teaching?

  • Can we use any of this to say that we’ve taught

them to think critically?

slide-29
SLIDE 29

2 – Critical Thinking

  • The challenge for Social Science:
  • We are disparate.
  • The challenge for measuring critical thinking:
  • Well, lots.
  • A solution?
slide-30
SLIDE 30

2 – Critical Thinking

  • What do we want our students to be able to do?
  • Read articles? Think critically about them?
slide-31
SLIDE 31

2 – Critical Thinking

  • What do we want our students to be able to do?
  • Watch videos? Think critically about them?
slide-32
SLIDE 32

2 – Critical Thinking - Proposed

  • What do we want our students to be able to do?
  • Watch videos? Think critically about them?
  • https://www.youtube.com/watch?v=vG6-UaBECN4
slide-33
SLIDE 33

2 – Critical Thinking - Proposed

  • Methodology
  • Switch to a pre/post
  • Keep Qualtrics/Atlas method
  • Keep it as multiple choice
slide-34
SLIDE 34

3 – Ethical Responsibility

  • Quick Review.
  • 10 questions in 2 parts
  • First 5 – Vignettes tied to Student Code of Conduct
  • Second 5 – 5 pt Likert scale on Ethical/Social

Responsibility adapted from APA learning outcomes

slide-35
SLIDE 35

Q1 Q1- Academic Di Dishonesty ty – At the beginning of the semester, the course syllabus indicates a midterm exam. This midterm will take place the week after spring break. While on spring break, one student emails the professor and explains that their uncle has died and they will miss the exam. In the email, they ask if they could take the midterm exam late. The professor agrees but asks for verification of the story upon the students’ return. The student returns, and takes the test, but fails to provide any evidence of the story.

slide-36
SLIDE 36

Q2 - Ch Cheatin ing– Subhas and Sally are taking a multiple choice test. After 45 minutes

  • f the test, Sally sneezes accidently and her scantron sheet flies off the desk.

Subhas, as a nice gesture, picks up the scantron for Sally. However, prior to giving it back, Subhas looks at the scantron very briefly. After returning to his own test, Subhas then proceeds to change several of his answers to what he saw on Sally’s sheet. Rate Subhas’ behavior:

slide-37
SLIDE 37

Indicate the extent to which you agree or disagree with the following: SA, A, N, D, SD 6 – My __________ course challenged me to think more broadly about poverty, health, human rights, and/or other issues of prejudice and discrimination.

slide-38
SLIDE 38

Indicate the extent to which you agree or disagree with the following: SA, A, N, D, SD 7 – My ___________ course enhanced my ability to see the world from someone else’s perspective.

slide-39
SLIDE 39

Indicate the extent to which you agree or disagree with the following: SA, A, N, D, SD 9 – My ___________ course influenced my openness to having my views challenged.

slide-40
SLIDE 40

National Institute for Learning Outcomes Assessment http://www.learningoutcomeassessment.org/TFcomponents.htm

NILOA Transparency Framework based on research of

  • ver 1,000 institution

websites.

How do we close the loop?

slide-41
SLIDE 41

3 – Ethical Responsibility – Next Steps

We want this to be simple, straightforward, meaningful, impactful

  • Derek –
  • Put Student Code of Conduct module in online front door

classes

  • Incentivize student to read/utilize this module
  • Perhaps as part of a “syllabus quiz” at the start of the term.
  • In Fall 2018, compare to Fall 2016 data
  • By year (2016 v. 2018)
  • By modality (online vs. onsite)
  • Questions: Just online? Just full time?
slide-42
SLIDE 42

3 – Ethical Responsibility – Next Steps

We want this to be simple, straightforward, meaningful, impactful

  • Derek –
  • Put Student Code of Conduct module in online front door

classes

  • Incentivize student to read/utilize this module
  • Perhaps as part of a “syllabus quiz” at the start of the term.
  • In Fall 2018, compare to Fall 2016 data
  • By year (2016 v. 2018)
  • By modality (online vs. onsite)
slide-43
SLIDE 43

4 – Open Questions and Discussion

slide-44
SLIDE 44

5 – At Risk Groups

slide-45
SLIDE 45

5 – At Risk Groups

slide-46
SLIDE 46

From Achieving the Dream (AtD) to the Academic Initiative Review (AIR) at Valencia

Four Student Populations Identified to be at Risk:

  • 1. students who are not successful in any of their first five

courses, as it is more likely they will not graduate.

  • 2. new students who are attending college for the first time as

“student success is largely determined by a student’s experience during his or her first-year of college” (Upcraft, et al., 2005).

slide-47
SLIDE 47
  • 3. students taking mathematics who have been identified as

being in need of support (developmental). The 2015 Strategic Indicators Report found that 46% of students were successful in developmental mathematics from fall 2011 to summer 2013.

  • 4. students who are among those that have shown declining

rates of persistence, such as African American males - of the number of students who persist and graduate, African American males have the lowest rate of graduation.

slide-48
SLIDE 48

13.2% 19.4% 25.9% 36.6% 23.… 32.2% 35.4% 38.8%

0% 20% 40% 60% 80%

Fall 2003 Fall 2004 Fall 2005 Fall 2006 Fall 2007 Fall 2008 Fall 2009 Fall 2010 Fall 2011

5 Year Graduation Rate

FTIC Degree-seeking Students

African American Caucasian Hispanic Other

Graduation Rates: By Race and Gender See the Gaps

slide-49
SLIDE 49

5 – At Risk Populations

  • 82 African American males completed the CT assessment. While

their score of 6.01/10 was lower than Asian (6.27, n=33), Caucasian (6.74, n=180), Multi-Racial (6.59, n=17), and Hispanic (6.52, n=196) males, they outperformed African American females (5.95, n=187) and Hispanic Females (5.96, n= 456).

5.97 6.01 5.95 6.18 6.27 6.15 6.64 6.74 6.57 5.8 6.11 6.52 5.96 6.5 6.59 6.44 6.7 7.01 5 5.5 6 6.5 7 Student = Af. American Af. American Male Af. American Female Student= Asian Asian Male Asian Female Student = Cauc. Cauc. Male Cauc. Female Student= Hawaiian Student= Hispanic Hisp. Male Hisp. Female Student= Indian Student= Multi Multi Male Multi Female Student= Unknown

Race and Ethnicity

slide-50
SLIDE 50

6 – Top 20 classes #6 PSY 2012

slide-51
SLIDE 51

6 – Top 20 classes #6 PSY 2012

slide-52
SLIDE 52

6 – Top 20 classes #10 POS 2041

slide-53
SLIDE 53

6 – Top 20 classes #10 POS 2041

slide-54
SLIDE 54

6 – Top 20 classes #14 ECO 2013

slide-55
SLIDE 55

6 – Top 20 classes #14 ECO 2013

slide-56
SLIDE 56

6 – Top 20 classes #20 SYG 2000

slide-57
SLIDE 57

6 – Top 20 classes #20 SYG 2000

slide-58
SLIDE 58

7 – Inf. Lit./Writing

  • For Poli Sci/ History Profs
  • Will collect in Fall, send to Jovan Trpovski in Jan.
  • Will code and return them by Feb.
  • Grade in March.
  • Have data for Assessment

Day in May ‘19.