THE VALUE OF STUDENT ENGAGEMENT STRATEGIC ENROLMENT MANAGEMENT - - PowerPoint PPT Presentation

STRATEGIC ENROLMENT MANAGEMENT SUMMIT 2013 CALGARY, ALBERTA TUESDAY, MAY 7, 2013

MEASURING AND ASSESSING THE VALUE OF STUDENT ENGAGEMENT

Informing the Future of Higher Education

Richard Wiggers, Executive Director, Research & Programs & Valerie Lopes, Research Consultant

WHAT IS HEQCO?

Higher Education Quality Council

• f Ontario
• HEQCO’s creation was recommended

in the Rae Review (2005)

• It is an independent agency with a

mandate to conduct research and give policy advice to the Minister of Training, Colleges and Universities (MTCU) on all aspects of PSE

• The primary focus is to employ

evidence to influence policy

3

Informing the Future of Higher Education

HEQCO has completed or launched

• ver 250 research projects

4

Informing the Future of Higher Education

Published Completed Underway Total Projects

• 1. Accessibility/Participation

32 12 16 60

• 2. Accountability/System Design

18 2 2 22

• 3. Learning Quality

34 28 68 130 Internal Research (Various) 32 2 6 40 Manuscripts 3 3 Opportunities to Innovate Fund (OIF) 1 1 8 10

Total 120 45 100 265

Many current projects focus on teaching, learning and student outcomes

Student Services Large Classes Technology- Assisted Learning Professional Development Graduate Studies NSSE Collegiate Learning Assessment AHELO Tuning Work Integrated Learning

Informing the Future of Higher Education

5

Course–Based Interventions Student Services Teaching

Student success is impacted by many different factors

6

Informing the Future of Higher Education

STUDENT SUCCESS

More recently HEQCO developed and released a national “how to” Guide

Informing the Future of Higher Education

7

• Based on a similar document

developed by McMaster University

• Endorsed by the Society for

Teaching and Learning in Higher Education (STLHE) and the Canadian Association of College and University Student Services (CACUSS)

• September 18, 2012 release

Researching Teaching and Student Outcomes in Postsecondary Education: A Guide

Susan Elgie

STUDENT ENGAGEMENT

University and non-university attainment for main OECD countries, percentage of population aged 25-64, 2006

23 24

5 10 15 20 25 30 35 Tertiary-type B education (Non-university) Tertiary-type A education (University)

Canada already leads the OECD in PSE attainment

Informing the Future of Higher Education

9

Ontario is achieving the goal of “mass education” in PSE

0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 25-64 55-64 45-54 35-44 25-34

Highest educational attainment

Trades cert. & registered apprenticeship Total College or CEGEP Total University

67.5 61.4 Ontario Goal

Informing the Future of Higher Education

10

The postsecondary student population is now much more varied

Deep Strategic Surface Entitlement % sample Tentative Label Group 1

2.62 2.67 3.49 3.15

7.4% Memorizer with expectations Group 2

2.85 2.71 2.99 2.30

15.1% Just puttin’ in time Group 3

2.90 3.50 2.80 2.80

14.5% Strategist Group 4

3.22 3.28 2.41 1.88

14.5% Relaxed student Group 5

3.43 3.03 3.05 2.86

12.4% Non-strategic worker bee Group 6

3.49 3.52 3.31 3.65

6.9% Driven to succeed Group 7

3.70 3.49 2.58 2.50

17.3% Inquiring student Group 8

4.07 3.86 1.99 2.06

11.8% Student scholar

Source: Schoner, V., Andrey, J., Jayasundera, R., Joakim, E. & Hambly, D. Students’ Learning Syles and Sense of Entitlement (2011).

Informing the Future of Higher Education

11

Engaged students are curious and have a passion for learning

12

Informing the Future of Higher Education

Source: metapreneurship.net/wp-content/uploads/2012/02/CollegeProfessorsMeme.jpg

I found this course fascinating and I’m thrilled I had the

• pportunity to take it.

I would love to have had this class all year and I would recommend it to anyone!. This has been one of the best classes I’ve taken in four years.

Disengaged students don’t appreciate high academic standards

13

Informing the Future of Higher Education

Source: metapreneurship.net/wp-content/uploads/2012/02/CollegeProfessorsMeme.jpg

Really needs a life! Out to fail you to compensate for whatever he’s lacking. He is literally ruining my university career! He is a total ASS.

Millennials have a different view of the value of education

Informing the Future of Higher Education

14

Source: Arthur Levine, Diane R. Dean, Generation on a Tightrope: A Portrait of Today’s College Student (2012).

1969 1976 1993 2009 The chief benefit is that it increases one’s earning power

• 44

57 67 Essential goal: to get training and skills for an occupation 59 67 70 73 1969 1976 1993 2009 Essential goal: to formulate life values and goals for my life 71 62 52 50 Essential to learn to get along with people 76 66 47 38

One HEQCO study looked at the gap between expectations and reality

Source: M. Mancuso et al., Disappointment, Misunderstanding and Expectations: A Gap Analysis of NSSE, BCSSE and FSSE (HEQCO: 2010).

• 30
• 20
• 10

10 20 30 40 50

Gains in General Education Gains in Practical Skills Student Success Support Diversity Information Technology Course Interaction Out-of-Class Interaction Collaborative Learning Active Learning Higher-Order Thinking Skills Writing Course Challenge

Disappointment Gap BCSSE-NSSE Misunderstanding Gap NSSE-FSSE

Informing the Future of Higher Education

15

16

Informing the Future of Higher Education

Helicopter parents are raising more “bubble wrap kids”

Source: R. Finnie, S. Childs and T. Qiu, Patterns of Persistence in PSE (2011); Academica, From Postsecondary Application to the Labour Market (2010); Government of NB, High School Follow-Up Survey (2007).

17

Informing the Future of Higher Education

Reasons for Leaving PSE Ontario Rest of Canada Didn't like it/Not for me 35.7% 30.3% To change schools or programs 35.2% 33.7% Health, personal or other reason 17.3% 20.5% Not enough money 3.1% 6.7% Wanted to work 3.1% 5.0% Marks too low 5.6% 3.9%

More PSE students are uncertain about their direction

Quite a few students arrive in PSE unprepared academically

Informing the Future of Higher Education

18 Ontario College “A” 2006 2007 2008 2009 2010

Below College English 58.7% 59.3% 59.8% 59.3% 55.0%

College English 39.4% 38.4% 38.2% 36.5% 41.0% Total Responses (Count) 6,150 6,409 6,380 6,834 7,179

PSE isn’t even the equivalent of full- time work for most students

14.0 15.2 15.9 17.0 27.3 28.4 29.3 31.7 10 20 30 40 below 70 70 to 79 80 to 89 90+ Hours per Week Overall University Grade

Total Time at School work Total Time Studying

Source: Lori McElroy and Alex Usher, HESA, Measuring Differences in Academic Challenges and Grades at Canadian Universities (PowerPoint presentation for CIRPA 2010).

Informing the Future of Higher Education

19

Being “networked” today means being connected/disconnected

20

Informing the Future of Higher Education

Source: W. Drexler (2008)

The Paradox of Choice

21

Informing the Future of Higher Education

HOW DO WE MEASURE IT?

What does student “engagement” look like?

23

Informing the Future of Higher Education

• Volunteering
• Hours on campus outside of class
• Curious
• Enthusiastic
• Showing up for things
• Intentional
• Actively participating
• GPA

What does student “engagement” look like?

24

Informing the Future of Higher Education

• Reflective
• Doing school work
• Empowered
• Clarity in institutional pledge to students
• Actively seeking learning
• Confident
• Active alumni
• Internally motivated

What does student “engagement” look like?

25

Informing the Future of Higher Education

• Institutional pride
• Depends on the student
• Classes are full throughout
• Interest
• Asking questions
• Mental and physical health
• Time and effort students devote to their studies and

related activities….

What variables and factors encourage student engagement?

26

Informing the Future of Higher Education

• Passionate, engaged and well prepared faculty and staff
• Mentorship
• Activities that challenge
• Timeliness
• Student involvement
• Instruction that is relevant to where students are at
• Same level of commitment, etc. that we expect of

students

What variables and factors encourage student engagement?

27

Informing the Future of Higher Education

• Assuring that there is a fit when students come in
• Physical space
• Students have clear goals
• Outreach to K-12 about what it is all about
• Availability of learning resources (physcial space, etc.)
• Relevant ways to deliver services
• Mandatory voluntarism in the community

What are the barriers to student engagement?

28

Informing the Future of Higher Education

• Time
• Money
• Collective agreements
• Tired and uninspired faculty and staff
• Conflicting commitments
• Tired and uninspired students

What measures of student engagement are you currently using on your campus?

29

Informing the Future of Higher Education

• Attendance
• NSSE
• NCHA
• Participation in clubs and student leadership
• Required to withdraw rates
• Program satisfaction
• Retention

What measures of student engagement are you currently using on your campus?

30

Informing the Future of Higher Education

• Course evaluations
• Student focus groups
• Feedback forms
• Graduation rates
• Undergraduate survey

What do these measures of student engagement actually measure?

31

Informing the Future of Higher Education

• Satisfaction
• Academic performance
• Activity
• Extracurricular activities
• Impact of student supports
• Persistence rates
• Number of students engaged with peers

What should we be measuring in terms of student engagement?

32

Informing the Future of Higher Education

• Why are they here and what do they want to study?
• What are their expectations?
• Satisfaction in exit surveys
• Expectations of program, faculty, etc.
• Perception of their own engagement
• Measure of student engagement and learning

(transformation, desire to continue to learn?)

• How many resources are going to already engaged

students

What should we be measuring in terms of student engagement?

33

Informing the Future of Higher Education

Students are more satisfied with their PSE experience

Informing the Future of Higher Education

34

Source: Arthur Levine, Diane R. Dean, Generation on a Tightrope: A Portrait of Today’s College Student (2012).

1969 1976 1993 2009 I am satisfied overall 62 71 79 79 I would rather be going to university than anything else 69 69 75 83 I am satisfied with teaching at my institution 67 72 81 87 1969 1976 1993 2009 My grade point average is A- or higher 7 19 26 41 My grade point average is C or less 25 13 9 5 I have taken a basic skills or remedial course

• 29

32 45 I work hard at my studies

• 86

89 93

Some innovations increase satisfaction but not academic performance

35

Informing the Future of Higher Education

Source: J. Waldman, Measuring the Impact of Hybrid College Programs (Presentation to ISSOTL, 2012)

Student Success (A, B or C) GPA Student Satisfaction Withdrawal Rates University of Central Florida

High None to slightly positive effect No effect Not measured

Sheridan

(FOB, FT Faculty, PY Control) High No effect No effect Possibly small negative effect

Sheridan

(College- wide, many controls) Not measured Small negative effect No effect Small negative effect

Course evaluations are a standard way to measure student “satisfaction”

Source: P. Gravestock and E. Gregor-Greenleaf, University of Toronto, Student Course Evaluations: Research, Models and Trends (HEQCO: 2008).

Course evaluations typically include questions to students about the following: Common uses for course evaluations include the following:

Course content Teaching improvement (faculty) Instructor’s communication skills Personnel decisions (PSE institutions) Student-Teacher interaction Course selection (students) Course difficulty and workload Teaching awards Assessment practices in the course Student self-assessment

Informing the Future of Higher Education

36

NSSE has limitations as a means to measure student engagement

37

Informing the Future of Higher Education

• Too blunt for institutional comparisons, and too broad

for course-based assessments

• It can be useful for program level and sub-group

analyses.

• “most student subgroups are not universally engaged
• r disengaged, but rather show more or less

engagement across items within and across benchmarks in a given program, and more or less engagement for particular items across programs”

Source: C. Conway, H. Zhao and S. Montgomery, The NSSE National Data Project Report (HEQCO: 2011; C. Conway and H. Zhao, The NSSE National Data Project: Phase Two Report (HEQCO: 2012).

According to the CLA, students are studying less, and learning less

Source: Richard Arum and Josipa Roksa, Academically Adrift: Limited Learning on College Campuses (2011)).

Informing the Future of Higher Education

38

• Measures critical thinking, problem solving, writing, etc.
• 45% of students demonstrated no significant

improvement in learning during the first 2 years of PSE

• 36% of students demonstrated no significant

improvement over 4 years of PSE

• For students who show improvement, it is modest only
• Main culprit is lack of rigor in terms of reading

requirements and study time (12-13 hours/week)

WHAT CAN PSE INSTITUTIONS DO?

Source: SSRC, 2012 from Learning to Earning, 2012, presentation by Brent Herbert-Copley, SSHRC.

Unemployment By Student Performance (Sample Of Recent US Bachelor’s Graduates) 2 4 6 8 10 12 CLA scores Academic Engagement/Growth Low Medium High

Informing the Future of Higher Education

40

Students must be reminded that jobs require more than a credential

There are a number of different ways to introduce experiential learning

1 2 3 4 *Organize class visits to local business, government, or community workplaces *Arrange job shadowing opportunities for students *Invite business, government, or community guest speakers into the classroom *Assign projects that require students to interact with local business, government or community organizations *Invite students to share career goals with class *Provide class time for students to reflect on their work experiences *Design academic course content (readings, discussions) to integrate with students’ work experiences *Use authentic assessment strategies to assess students’ ability to perform real-world tasks *Use business/community/workplace case studies for student assignments *Invite students to share their work experiences with class *Talk to students individually about their work experiences *Use business/community/workplace examples to illustrate concepts in class *Talk to students individually about their career goals/concerns

University College

Never 1-5 times 6-10 times 11-20 times

41

Source: Peters, J., Faculty Experiences with and Perceptions of Work-Integrated Learning (WIL) in the Ontario Postsecondary Sector (Toronto: HEQCO, 2012).

Informing the Future of Higher Education

Ontario students indicated what made a teacher effective

Factor % of Students Selected

Clearly outlines expectations at the beginning of the semester 22.4% Students often do well in his/her course 17.2% A prominent researcher at the cutting edge of his/her field 13.2% Integrates technology well into the learning environment 6.9%

Source: Ontario Undergraduate Student Alliance (OUSA), Ontario Results of the Canadian Student Survey (PowerPoint presentation, 2010).

Informing the Future of Higher Education

42

Factor % of Students Selected

Delivers interesting, well prepared and organized lectures 83.7% Enthusiastic, entertaining or motivating in the classroom 74.6% Able to communicate the subject in multiple ways 52.4% Able to meet regularly with students 25.8% Clearly outlines expectations at the beginning of the semester 22.4% Students often do well in his/her course 17.2% A prominent researcher at the cutting edge of his/her field 13.2% Integrates technology well into the learning environment 6.9%

Some ideas and advice for our faculty and programs

Informing the Future of Higher Education

43

Integrate technology where appropriate Find ways to involve students in their own learning Set and maintain reasonable but high standards Articulate clear learning outcomes Utilize transparent assessment practices (rubrics, etc.) Encourage students to utilize student services and supports Remind students of the value and relevance of learning

Thank You!

Informing the Future of Higher Education

Richard Dominic Wiggers

Executive Director, Research and Programs HEQCO (416) 212-3881 rwiggers@heqco.ca

Valerie Lopes

valerie.lopes@senecacollege.ca

Research Consultant