How NSSE Focus Group Outcomes Shape Our Understanding of the ESU - - PowerPoint PPT Presentation

How NSSE Focus Group Outcomes Shape Our Understanding of the ESU Experience

The Role of Qualitative Data from Residents and Commuters

1

Today’s Objectives

• Review the methods used to conduct the NSSE Student

Services Focus Group Pilot Study

• Describe the composition of the four FG sessions’ participants
• Review the highlights of the FG’s exploratory findings
• Discuss the next steps leading to a confirmatory survey of

ESU students

2

Background – The NSSE

• OAA administers the National Survey of Student Engagement

(NSSE) in the spring semester to all eligible freshmen and seniors who have been at ESU at least since the preceding fall.

• The online survey asks students about their experiences at ESU

in a variety of areas in an effort to gauge their engagement in the university campus culture and academic life.

• Many questions focus on students’ attitudes toward various

services, offices, events, and people that are under the auspices

• f ESU’s Student Affairs division.
• A brief analysis of the results of this survey are typically

presented to Student Affairs individuals at the annual SA retreat in January. This includes data from the last 3 administrations.

• Focus groups have been informally discussed as a potential

follow-up to the NSSE since at least summer 2014.

3

Methodology

• July 2015: Drs. Tobin and Moses approached Dr. Ziner,

OAA Director, for help in conducting a focus group of ESU students to build on data gathered during the last NSSE administration.

• Through a series of meetings that included representatives

from the Student Enrollment Center as well as Housing and Residence Life, the group decided to focus on students’ different experiences in attending ESU based on their residency status – whether they lived on campus in a dorm

• r commuted to ESU.
• Four focus groups were to be conducted in the fall 2015

semester, once the drop-add period had ended: Two for residents (split by underclassmen and upperclassmen) and two for commuters (also split by class).

4

• To have the greatest chance of meeting a 12-person

attendance target, Dr. Ziner asked Kizzy Morris (Student Enrollment Center) to draw a random selection of 200 students who fit into each of the four categories for a total of 800 students who were asked to participate.

• Students were emailed at their ESU addresses asking if they

wanted to commit one hour of their time for a “NSSE Student Services Focus Group.”

• They were given only one time slot, depending on their

residency and class status, and were asked to contact Bob Moses’ office to confirm their participation.

• Students were offered a $20 gift card to their choice of either

Wawa or Walmart, and all participants were told snacks and beverages would be available.

5

Methodology - Continued

• Students were given approximately a week’s notice before

focus groups were scheduled to be conducted, and they were sent reminders regularly to ensure higher participation rates.

• Focus group sessions were moderated by Dr. Ziner on

Tuesday, November 3 and Thursday, November 5 at 2:00pm and 3:30pm in the Housing and Residence Life conference room located in Hemlock Suites.

• Each student who attended a focus group session was

provided with an informed consent form that documented their rights and responsibilities as far as this research project entailed, while Bob Moses confirmed their gift card choice before each session.

• Gift cards were awarded approximately one week after the

focus groups concluded. A student had to attend the entire group session to be eligible for the gift card.

6

Methodology - Demographics

Methodology - Demographics

• In total, 53 students volunteered to participate in one of the

four focus groups, out of a maximum number of 56 spaces allotted (14 for each group).

• 42 students actually participated in the four focus group

sessions, with group sizes ranging from 8 to 13.

• Overall, 23 students (54%) were observed to be White, while

13 (31%) were observed to be Black.

• 11 students (26%) were male, while 31 (74%) were female.
• Students’ majors were from a wide range representing each
• f the five colleges.
• The demographics associated with each focus group are as

follows:

7

Residents, Freshmen and Sophomores November 3, 2015 – 2:00-3:00pm

• Total Attendance: 13
• 6 males, 7 females
• White: 7 (54%)
• Black: 4 (31%)
• Hispanic: 1 (8%)
• Other: 1 (8%)

Commuters, Freshmen and Sophomores November 5, 2015 – 2:00-3:00pm

• Total Attendance: 8
• 2 males, 6 females
• White: 5 (63%)
• Black: 3 (37%)

Residents, Juniors and Seniors November 3, 2015 – 3:30-4:30pm

• Total Attendance: 9
• 1 male, 8 females
• White: 3 (33%)
• Black: 3 (33%)
• Hispanic: 1 (11%)
• Asian: 2 (22%)

Commuters, Juniors and Seniors November 5, 2015 – 3:30-4:30pm

• Total Attendance: 12
• 2 males, 10 females
• White: 8 (67%)
• Black: 3 (25%)
• Hispanic: 1 (8%)

8

Methodology - Demographics

Déjà Vu All Over Again

9

At Last Year’s Retreat, I Pointed Out …

ü FGs are useful as a type of exploratory pilot study ü Provide rich data often unobtainable by other methods ü Can be conducted quickly and generally are low-cost to run ü Not useful for making predictions or supporting hypotheses ü FGs raise serious questions of accuracy & reliability, if used as a single study method to shape or inform policy ü Should be used to shape the scope and direction of follow- up survey-based (quantitative) research, when considering the implementation of FG outcomes/findings as policy

On Familiarity with Student Services

In this first area, FG sessions’ objectives included:

• 1. Do students know what is a ‘student service”? (“What comes to

mind?”)

• 2. Identifying which student services they have used at ESU (Using a

list of services provided on an index card)

• 3. For those services cited as being used, to determine whether the

experience was positive or negative, whether anything stood out, and whether a specific person contributed to that experience

• 4. Identifying whether there was a student service area/office that did

not meet their expectations

10

On Familiarity with Student Services

11

We found:

• 1. Some differences in response patterns between residents and

commuters:

• Commuters provided a much larger array of positive and negative

stand outs compared to their resident peers

• Residents provided a larger array of student service areas that did

not meet their expectations compared to their commuter peers

• 2. The possibility that a response bias may be in play among session

participants, where FG attendance may mask their candor compared to the anonymity of a survey

On Experiences Living on Campus at ESU

In this second area, FG sessions’ objectives among residents included:

• 1. Identifying what they liked about living on campus in residence halls,

including providing at least one positive and one negative experience

• 2. Identifying whether or not their campus experience living in a residence hall

could be improved

• 3. Determine the types of campus activities to which they are engaged

12

We found:

• 1. Some differences in response patterns between F/So and Jr/Sr class groupings

were identified (e.g., Jr/Sr participants reported more engagement in activities)

• 2. F/So participants reported more negative experiences than their Jr/Sr peers (We

hypothesize that Jr/Sr participants may be used to many of the negatives reported by F/So (e.g., people up late, thin walls, fire drills)

On Experiences Living off Campus

In this third area, FG sessions’ objectives among commuters included:

• 1. Identifying what they liked about living off campus, including providing at

least one positive and one negative experience

• 2. Identifying whether or not their experience living off campus effects their

level of engagement in campus activities, including the types of activities in which they are engaged

13

We found:

• 1. No clear differences in response patterns were identified among F/So and

Jr/Sr commuters, i.e., they shared similar positive and negative experiences

• 2. One area of departure was in their respective engagements in campus activities,

where Jr/Sr commuters reported higher levels than their F/So peers

On How Students Spend Their Time

In this fourth area, FG sessions’ objectives among residents and commuters included:

• 1. Identifying how students spend their time when they are not in class
• 2. Determining whether students believe the university offers enough
• pportunities to be involved socially on campus, including identifying the

ways students became aware of these opportunities and how effective they are

14

We found:

• 1. Among residents, F/So participants were more likely than their Jr/Sr peers to

(1) report ESU did not provide enough opportunities to be involved socially on campus and (2) be less aware of the ways to become involved socially on campus

• 2. Commuters report they are at a disadvantage living off campus when it comes to

being aware of opportunities for involvement socially on campus

On Students’ Overall Experience at ESU

In this last area, FG sessions’ objectives among residents and commuters included:

• 1. Identifying if students would enroll at ESU if they could do it over

again, including the reasons why or why not

• 2. Determining if students would recommend ESU to prospective

students, including the rationale for their decision

• 3. Determining if their opinion of ESU has changed since they began

their studies and, if so, how and why

15

On Students’ Overall Experience at ESU

16

We found:

• 1. Among residents and commuters, majorities reported they would enroll at

ESU again, pointing to “smaller campus size,” “club experiences,” “specific program” and “people I’ve met.” Negatives include “bad advisor” and “need to try new experiences”

• 2. Both groups are split on whether they’d recommend ESU to prospective

students (“depends on what they’re looking for,” added one student)

• 3. Among residents and commuters, nearly all reported their opinion of ESU

has changed since they began their studies:

• Among “For the worse,” reasons include “dining – feel like gagging,”

”plague outbreak,” “financial aid problems” and “verification process is harder”

• Among “For the better,” reasons include “relationships with professors,”

“small classes,” “diversity,’ “like the area” and “passion of professors”

Moving From FG Studies to Survey Research

• The next step is to recast this non-generalizable qualitative study,

based on non-probability samples, as a quantitative survey designed to represent all ESU students in the study’s sample and outcomes

• We’ll use results obtained from the FG sessions to frame the response

set (choices to survey items) available to respondents

• Survey results/outcomes will provide reliable data that, in turn, can be

used to shape policy on campus – what FG research cannot do

17