Understanding the First-Year Experience Four Ways to Generate Data- - - PowerPoint PPT Presentation

Understanding the First-Year Experience

Four Ways to Generate Data- Driven, Actionable Insights

Steve Wygant, Danny Olsen, Bryce Bunting

Brigham Young University

26th International Conference on the First-Year Experience

Waikoloa, Hawaii, June 2013

Introduction: Brigham Young University

 Private, faith-based institution

in Provo Utah, USA

 30,000+ undergraduates  Academically competitive:

 Mean HS GPA (2012): 3.81 (high=4.0)  Mean ACT/SAT (2012): 28/1270

2

Academics

 Strong programs in accounting, business, engineering,

animation, music, etc.

 Ranked #9 in Baccalaureate-origins of U.S. Doctorate

Recipients

 Ranked #68 in US News & World Report 2013 Best

Colleges

3

International focus

4

 60 languages taught  75% of students have a second

language

 33% enroll in a foreign language

course each semester

 7% go on Study Abroad  6% are International Students  Outreach efforts around the

world

2012 First Year Class

 2,455 Women  2,975 Men  5,430 Total  55% of

applicants are admitted

 80% of admitted

enroll

5

Retention Graduation

89% retained

(full-time first-time, Fall 2011- Fall 2012)

4 year - 31% 6 year - 78% 8 year - 86%

(2004 cohort as of 2012)

6

Athletics

7

A Tale of Two Students

8

Guiding questions

 Why do intelligent, capable students flounder?

 Academic warning/probation  Fail to achieve goals, live up to potential

 Can we identify students who will flounder?

 Using data obtained before students arrive on campus?  Using data obtained during the course of the school year?  How could and should such data be used?

 Individually, and as a campus community, how do

we help students flourish?

 Can data help us identify student motivations, strengths and

challenges in order to connect them with resources they need?

 How would such data be used?

9

As we get going …

Focus on processes, not specific findings Questions you might ask yourselves

 “How would processes like this work with our campus and

• ur students?”

 “What data do we have that could help us predict student

success?”

 “What other data can we gather that might help us predict

student success?”

Session as conversation

 Ask questions  Share your practice

10

Survey instruments used

 Pre-arrival Survey

 High school experiences & practices  Expectations for college  Self-descriptions

End-of-year Survey

 Engagement: work ethic, participation  Attitudes, practices  Self-descriptions  Outcomes, influences

11

Getting and using good data

12

Four ways of looking at data

• 1. Before-after
• How do attitudes & behaviors change? Expectations vs. experience?
• 2. Flourish-flounder
• How are students who are thriving by the end of the first year

different from those who are not?

• 3. Over-achieving vs. underachieving
• How are students who exceed academic expectations different from

those who perform below expectations?

• 4. Cluster analysis
• How do sub-groups of students who begin the first year with

different attitudes and motivations differ in behavior and performance?

13

Before-after analysis

14

PA EOY

Using internet/phone (not school work) Extra-curricular activities Working for pay Academic work outside of class Attending classes

Very low Low Moderate High Very high

Expected vs. actual hours spent per weekday

(Expected hours from Pre-arrival survey [PA], Actual hours from End-of-year survey [EOY])

Before-after analysis

15

think about problems in

• riginal ways

evaluate the credibility of information sources explore a variety of ways to solve a problem evaluate your own assumptions, conclusions and reasoning

Percent of students who rated themselves as "Very competent" or "Quite competent"

PA EOY Pre EOY Post

Flourish-flounder analysis

16

Flourishing

strongly somewhat

Getting by

somewhat

Floundering

badly Very low Low Moderate High Very high

Flourish - flounder

(End-of-year survey)

Agree (low) - Disagree (high)

(Pre-arrival survey)

… satisfied with my life … like the kind of person I am … able to make friends easily …. know & understand myself

Flourish-flounder analysis

17

Agree (low) - Disagree (high)

(on Pre-arrival survey)

strongly somewhat somewhat badly BYU was the right choice I will be able to make friends at BYU I will do well academically at BYU Very low Low Moderate High Very high

Flourishing-floundering

(End of school year)

Flourishing Floundering

Getting by

Flourish-flounder analysis

18

strongly somewhat somewhat badly Very low Low Moderate High Very high

Flourishing-floundering

(End of school year) Frequency during first year Adapt study habits & strategies Persist in doing class work Procrastinate doing class work

Flourishing

Getting by

Floundering

Underachieving vs overachieving

Comparing students whose first-year GPA was higher than expected to those whose GPA was lower than

• expected. (Expected GPA [EGPA] is based on ACT/SAT and HSGPA.)

 Badly underachieve: GPA 1/2 point or more lower than EGPA

(Example: EGPA=3.75, GPA<=3.25)

 Somewhat underachieve: GPA ¼ to ½ point below EGPA  Matched: GPA within ¼ grade of EGPA  Somewhat overachieve: GPA ¼ to ½ point higher than EGPA  Strongly overachieve: GPA more than ½ point above EGPA

19

Underachieving vs overachieving

20

badly

Underachieved

somewhat

Matched

somewhat

Overachieved

strongly Flourished (1) or floundered (5) Hours/week studied last year Did readings, assignments before class Academic success: Ability (1) v. effort (5) Procrastinated doing class work Very low Low Moderate High Very high

High school experience

Underachieving (GPA < expected) vs overachieving (GPA > expected)

Underachieving vs overachieving

21

Underachieving (GPA < expected) vs overachieving (GPA > expected)

Hours per typical weekday in first year

badly somewhat Matched somewhat strongly academic work outside of class socializing in person

• n-line (not school work)/gaming

working for pay Very low Low Moderate High Very high

Underachieved Overachieved

Underachieving vs overachieving

22

Underachieving (GPA < expected) vs overachieving (GPA > expected)

badly somewhat

Matched

somewhat strongly

Complete readings & assignments pre-class Get feedback before submitting a paper Work harder than you thought you could Seek help when you did not understand Adapt your study habits and strategies Persist in doing the work required Procrastinate doing the work needed

Very low Low Moderate High Very high

Underachieved

Overachieved

Cluster analysis

 Questions:

 How do sub-groups among first-year students differ in

ability, attitude, motivation?

 How do those differences affect behavior and

performance?

 Identified subgroups using cluster analysis  Process

 Factor analysis of Pre-arrival survey questions  Create scales using multiple items from factor analysis  Use scales to look for clusters

 Art and science

23

Sample items that form scales

24

High school academic engagement (How frequently did you …) Adapt your study habits and strategies to help you do better in your classes Persist in doing the work required to succeed in your classes, even when it was difficult Seek help when you did not understand important ideas or concepts Spiritual/social reasons for choosing BYU (How important was … in your decision to choose BYU?) Getting a college education in a spiritually enriching environment Opportunity to associate with peers who have values and beliefs similar to yours Opportunity to develop a greater capacity to do good for others Social/academic engagement (How important will ... be during your first year at BYU?) Talking with faculty members outside of class Participating in study or discussion groups Discussing ideas from your reading or classes with others outside class

Sample scales that distinguish between clusters

25

Scale AF AU Dis Bal High school academic engagement Very high Low Low High Spiritual/social reasons for choosing BYU Moderate Moderate Low High Importance of social/ academic engagement in FY Moderate Moderate Low High Cluster

AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced

Clusters defined

1.

Academically focused (AF)



Very high on academic experience/motivation scale



Average on other scales

2.

Academically undeveloped (AU)



Low on academic experience/motivation scale



Average on other scales

3.

Disengaged/reticent (Dis)



Low on every scale

4.

Balanced/well adjusted (Bal)



High on every scale

26

27

Importance of college goals AF AU Dis Bal Socialize Learn new ideas and perspectives Enhance your personal spirituality Grow from being challenged intellectually

Very low Low Moderate High Very high

Clusters

AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced

Cluster analysis

28

AF AU Dis Bal

Very low Low Moderate High Very high

Work harder than you thought you could Write multiple drafts of a paper

Clusters

Complete readings and assignments First-year academic participation Discuss course-related ideas outside class Get feedback before submitting work

AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced

Cluster analysis

29

AF AU Dis Bal

Very low Low Moderate High Very high

Weekly hours spent, first year

Clusters

Academic work outside of class Working for pay (on or off campus) Socializing in person Using the internet, other than school

AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced

Cluster analysis

30

AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced

AF AU Dis Bal Academic effort during the year Satisfaction with academic effort Academic success: Ability (1) vs effort (5)

Very low Low Moderate High Very high

Clusters

Cluster analysis

Effort and related outcomes

How do clusters look at the end

• f the first year?

31

Goal accomplishment AF AU Dis Bal

Very low Low Moderate High Very high

Clusters

Socialize Learn new ideas and perspectives Enhance your personal spirituality Grow from being challenged intellectually

AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced

How do clusters look at the end

• f the first year?

32

Evaluations AF AU Dis Bal I feel like I belong at BYU Made the right choice to attend BYU Flourishing Would choose BYU again

Very low Low Moderate High Very high

Clusters

AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced

Four ways of looking at first-year data

• 1. Compare student self-descriptions at the

beginning vs end of year

• 2. Compare students who describe themselves as

flourishing or floundering at the end of the year

• 3. Compare overachieving vs. underachieving

students

• 4. Compare pre-arrival clusters of students over the

course of the year

33

Academically Focused Academically Undeveloped Disengaged Balanced Admissions Orientation Seminars Learning Communities Residence Life Advisement Curriculum Assessment

34

To ponder:

Implications for Intentional Action

Thank you for coming!

Contact:

• steve_wygant@byu.edu
• danny_olsen@byu.edu
• bryce_bunting@byu.edu

35

36