Basic Skills: How Can Institutional Researchers Help? Dr. Kristen - - PowerPoint PPT Presentation

Basic Skills: How Can Institutional Researchers Help?

• Dr. Kristen Corbell, Director of Program Evaluation,

Research and Performance Management, NCCCS

8/25/2015 2

Ever felt like this when hearing about Basic Skills?

Where we want to be!

8/25/2015 3

Goals

• Give update on the Basic Skills Data Initiative

Focus Area Team and what has been completed.

• Understand what is available in Colleague for

Basic Skills data and how institutional researchers can help.

• Understand the Federal Measures Basic Skills

Departments are responsible for under NRS.

• Understand the State Performance Measure

for Basic Skills and the data associated with it.

8/25/2015 4

Basic Skills Simplified

• Students work to improve literacy and numeracy skills to

prepare for the workforce, college, and/or personal

• goals. All students must be functioning at or below the

12th grade level.

• Students grouped into Educational Functioning Levels

(EFL) based upon their competencies in literacy and numeracy.

• Help prepare students to get their High School

Equivalency, Adult High School Diploma, enter college and/or workforce.

• Student progress measured by improvements in pretests

and post-tests. These improvements are called “level gains”.

8/25/2015 5

Students – They are the reason we do what we do

8/25/2015 6 Nate DeGrandpre is a plant coordinator at Timken. DeGrandpre got earned his GED at Isothermal. Degrandpre worked at Timken while taking classes at the college for two years. He said the support he received from the company and his family was the encouragement he needed to continue working toward his degree. . DeGrandpre said that his education at ICC is what helped him to become successful and achieve his dreams. Contributed Photo Mike Gavin

8/25/2015 7

Basic Skills Focus Area Team

• Began March 30, 2012
• Oh, the data problems….. Placement, AHS,

NRS requirements

• But we fixed it!
• One last thing to do is a web based

attendance to assist with accountability.

8/25/2015 8

8/25/2015 9

How we were feeling in 2012…..

Basic Skills Focus Area Team

• Began work in March 2012
• Issues that were present that had to be addressed

were:

– NRS changes for 2012-13 needed to be implemented in Colleague – Data validations were not in Colleague and thus data quality was not good across colleges – Definitions of data elements in Colleague were not fully understood by colleges – Adult High School data were inconsistent across colleges and graduate information was not accessible in Colleague

8/25/2015 10

And More Issues

– Automatic Placement was not correct in Colleague, thus pulling placement for a student would not yield the same result that was submitted for accurate state and federal reports. – Reports for placement were not correct in Colleague – Web Attendance was not working for all of Basic Skills needs – Training was needed on each change made in Basic Skills data

8/25/2015 11

What the Basic Skills Focus Area Team Learned:

8/25/2015 12

The Answer: Being Data Detectives

8/25/2015 13

Federal Reporting Changes

• Data needed for the 2012-13 program year were

put in Colleague in 2012.

• Calculated student age correctly and implemented

error checks to be consistent with federal laws.

• Data that were no longer that had error checks

were removed

• Definitions for each element added were put into

Colleague.

• Educational Level and Labor Force at Entry were

put on literacy only screens to make sure cohorts weren’t changed by other departments for federal reporting (XLETES).

8/25/2015 14

Adult High School Data

• Adult High School data are used for:

– State funding of AHS graduates – Level completion for some ASEL students per federal guidelines – Computing federal cohorts for follow-up measures. – Other research requests

• XLGA was completely changed to allow for data to

be entered on a regular basis without changing what was originally entered and to get consistent data across 58 colleges.

8/25/2015 15

AHS Student Story – “Don’t Give Up”

8/25/2015 16

AHS Student Dropped out of high school with only a few credits left “If I could tell people one thing it would be ‘Go back and finish. Don’t give up and push through,'” Greene said. “You think you will be okay without it (a diploma) but you may get to a point in your career that you want more but you can’t have more because you don’t have a diploma.”

AHS Data continued

• Data needed for all reporting was put into Colleague

with specific definitions.

• AHS Graduates are now graduated through the
• fficial graduation process, thus allowing for state

funding to be handled from an official LEIS file submitted and not through spreadsheets.

• Validations at data entry were put into the fields to

ensure data quality at time of entry.

• Reports put into Colleague to show who the Adult

High School students are and who are the Adult High School Graduates for the program year.

8/25/2015 17

XLGA

8/25/2015 18

XLEAHS report

• Shows all AHS students for a program year:

Based upon having a first class attempt date in the program year.

8/25/2015 19

XLEGRA report

• AHS Graduate Report for a program year.

Must be officially graduated.

8/25/2015 20

Examples of Validations

• AHS data cannot be entered until a student has an

active AHS program (AHS or AHSMINOR).

• Required credits have to be >=16 and <= 35.
• Credits transferred in must be no more than 1 less than

required credits and must be >= 0.

• AHS Credits earned date cannot be a future date, cannot

be duplicated with another entry, must be after the birth date, and when it is entered, the Credits earned filled must be completed for the program year.

• 1st Class complete date and 1st class attempted dates

must only have one entry per program year, cannot be a future date, must be after birthdate, must be on or after the AHS academic program date.

8/25/2015 21

Graduated AHS student on XLGA

8/25/2015 22

Informer Reports that would help

• Pull all AHS Graduates and make sure that

all have the following:

– a 1st class attempt date in the program year – a 1st class complete date in the program year – sum of transferred in credits and credits earned at CC is greater than or equal to the number of required credits for graduation (all data on XLGA)

8/25/2015 23

How can IR help?

• Each college has a different way of naming

Basic Skills classes. Some have classes specific to AHS and an informer report of students taking AHS classes and not having a first class attempt date in the program year would be a good flag that there are missing data.

• What happens to AHS graduates? Are they

enrolling in Curriculum or Occupational Training courses?

8/25/2015 24

NRS Approved tests

8/25/2015 25

ABE: TABE, CASAS, GAIN ASE: TABE, CASAS, GAIN ESL: CASAS, TABE CLAS_E, Best Literacy, Best Plus

Automatic placement

• Biggest change of the LEIS system to-date.
• Basic Skills love the new screens, the

correctness of the automatic placement.

• Basic Skills Directors and LEIS Coordinators

are paying attention to placement, and calling

• r emailing when they don’t understand a

placement.

• Reports have been added to Colleague to

assist schools and are being used. The reports are not as powerful as Informer, so IR staff could be a great help in this area.

8/25/2015 26

XLETES

8/25/2015 27

Basic Skills Only: Will automatically update employment and educational level if empty on screen used by other users.

Will only have NRS approved tests from April 2, 2014 forward. Historical tests still on XLTE, but are read only.

Indicates date a student first enrolled with Basic Skills

• r after a 90

day stop out

IR can help with Informer

• Many colleges do not have Informer in Basic

Skills or those entering data don’t know how to use it.

• IR can help them with Informer programs

that they can run to ensure data are entered correctly.

• We have tried putting reports into

Colleague, but there are limitations in what a Colleague report can do vs. Informer.

8/25/2015 28

Intake Dates

• Colleges are to enter Intake Dates of when

students first come to Basic Skills or after they have not attended any classes for 90 days straight.

• Many students attend multiple classes and

some across multiple locations, thus intake dates can be difficult without knowing class enrollment

• Students may stop coming to a class before

the official end date.

• If intake dates are put in that are not correct it

can influence placement.

8/25/2015 29

Last Date of Attendance

• Colleges enter Last date of attendance on FGRN for

all students who stop coming before the end of a class, and we encourage putting it in for all students to assist with intake date.

• Last date of attendance is used in determining

separation when the file comes up to the system

• ffice.
• If students come through the end of the class, then

some colleges leave this blank especially if they don’t have informer.

• We are asking this to be updated automatically

when web attendance comes.

8/25/2015 30

Informer Suggestion from a College

• Informer statements that look at all Basic Skills sections

that have a last date of attendance on FGRN between 4/2/15 and 6/30/16 for the 15-16 program year.

• Give section name, last date of attendance, ID and return

the maximum last date of attendance. Include all students who do not have a current status of “X”. ]

• Make sure to map files from STUDENT

DENT.C .COU OURS RSE.SEC E.SEC to STUDENT DENT to X XLE.S E.STUDE UDENT NT

• Live excel is best, but a data file works as well.
• Colleges can then take this list and make sure intake

dates aren’t entered if a student attended classes less than 90 days ago.

• Will only work if the Basic Skills staff enter a last date of

attendance for everyone.

8/25/2015 31

Total Contact Hours

• Basic Skills operates on total attendance hours

across all classes for the program year (July 1 - June 30).

• All classes are to begin no earlier than July 1

and end no later than June 30.

• Basic Skills departments could benefit from

assistance with getting total contact hours for all students in a program year in some kind of report they could match with the students’ placement information.

8/25/2015 32

How Basic Skills felt when problems were confirmed in placement.

8/25/2015 33

XLETES

8/25/2015 34

Will only have NRS approved tests from April 2, 2014 forward. Historical tests still on XLTE, but are read only. If Intake Date is between July 1 and June 30 of program year, no tests 90 days back will be pulled forward.

Validations in Placement

• Placement validations are done primarily at the

entry screen level.

• Only approved NRS tests for the program year can

be entered and only those approved will be considered for placement.

• Test scores must be in range as specified by the

publisher

• Forms/levels must be on an approved list for the

program year

• Dates cannot be entered in the future
• Warning displays when a form/level is given twice in

a row.

8/25/2015 35

Archive Screen

• An Archive Screen is produced for all

students each program year they attend which has the placement for that program year including pre-tests, post-tests, whether they made a level gain, intake dates during that program year, and AHS information that leads to level gains for students.

• Accessed from XLETES

8/25/2015 36

XLETES

8/25/2015 37

Drill down to Archive Screen Did he complete a level? Placement info

XLEARC

8/25/2015 38

Tests used in Placement All NRS Tests

XLEARC

Initial Placement Highest EFL All Tests in the Placement Subject All NRS Tests One Archive file per year per student

XLEARC

Reports in Colleague

• Reports are available in Colleague for:

– Placements for all students (XLEPLA) – Students without an initial placement (XLENIP) – Students without a post-test (XLENPO) – Students with a post-test (XLEPTS) – Students who completed a level (XLECLV) – Federal NRS Tables 4 and 4B (XLET4)

8/25/2015 41

Report Cautions

• You cannot look at a teacher’s section and say

she is responsible for all student level gains in that class

– Must match section subject taught to placement

• Total contact hours is not necessarily the

number of contact hours between a pre-test and a post-test. So, 60 hours doesn’t necessarily mean a student has had 60 hours

• f instruction since the pre-test or the only 60

hours of instruction has occurred since pre- test.

8/25/2015 42

XLEPLA

8/25/2015 43

XLEPLA Section Report

8/25/2015 44

IR Help

• The section reports are good if you want to

look up all sections.

• An Informer to pull just one specific section

would be very helpful to Basic Skills

8/25/2015 45

Other Placement Reports

• XLENIP – students without an initial
• placement. All students with 12 or more hours

should have an initial placement.

• XLENPO – all students without a post-test. We

should have 65% of our students post-testing. Sometimes students aren’t post-tested and sometimes they don’t stay long enough to post- test.

• XLEPTS – All students who have post-tested
• XLECLV – All students who have completed a

level

8/25/2015 46

Federal NRS Measures

• Level Gain (all students with 12 or more

total contact hours)

• Follow-up outcomes

– Entered Employment – Retained Employment – Completion of Secondary Diploma – Post-secondary enrollment _ current year – Post-secondary enrollment _ prior year

8/25/2015 47

Targets

• Each EFL except Adult Secondary Education

has a target and programs should meet these targets.

• Each follow-up outcome has targets
• Targets negotiated with USDOE each year
• 2015-16 targets for some are significantly

higher as each target had to move up under WIOA.

8/25/2015 48

2015-2016 Targets

8/25/2015 49

MEASU SURE RE

2015-2016 6 Targets ts

ABE Beginning Literacy

31%

ABE Beginning

40% 40%

ABE Low Intermediate

41%

ABE High Intermediate

39% 39%

ASE Low

34% 34%

ESL Beginning Literacy

40% 40%

ESL Low Beginning

47%

ESL High Beginning

46% 46%

ESL Low Intermediate

37%

ESL High Intermediate

34% 34%

ESL Advanced

22% 22%

Secondary Diploma Completion

90% 90%

Entered Postsecondary Ed / Training

36% 36%

Entered Employment

41%

Retained Employment

63% 63%

XLET4 – Table 4

8/25/2015 50

This is what should be at or above the targets List of students who did complete a level is in the XLECLV NRS Reportable list. From XLEPLA report you have to match up to 12 or more total contact hours and take out those without an initial placement. ASEH will always be 0% because there isn’t level completion.

Employment Measures

• Will change with the enactment of WIOA.
• Still awaiting final regulations on this.
• For 2015-16 it looks at percentage of

unemployed seeking or employed at time of entry students who exit and enter employment

• ne quarter after exit.
• Retained employment is percentage of

employed students at entry and those who were unemployed at entry but employed 1 quarter after exit who retained employment 3 quarters after exit.

8/25/2015 51

Secondary Diploma Completion

• Will change under WIOA
• Is used in 2015-16 for sure

– Includes those who take all HSE tests or are an AHS student functioning at the ASEH level and exit during the program year – % who obtain their HSE Diploma or AHS Diploma

8/25/2015 52

Post Secondary Enrollment

• Will have after WIOA, regulations still being

defined

• For 2015-16 program year:

– HSE passers, AHS Graduates, or enrolled in a basic skills class designed for transitioning (i.e. Basic Skills Plus Students) who exit during the program year – Percent who enroll after exit in post-secondary education or occupational training – Track the current program year cohort as well as the past year’s cohort.

8/25/2015 53

State Performance Measures

• Revised State Performance measures were

released in 2013, and these measures were approved for three years.

• Every three years, a committee will review

the measures and make suggestions for changes.

• Baseline and Goal remain the same for 3

years.

8/25/2015 54

Basic Skills Measures

• Changed the cohort for the Basic Skills

Progress Measure

• Removed the GED Passing Rate as a measure

for 3 years

– 2014-15 data didn’t have large enough numbers that met the cohort definition to yield reliable results for all colleges. – Unknown start time for two additional HSE’s – With the changes in HSE and additional tests, more time was needed to be able to get at least two years of consistent data before having it as a measure

8/25/2015 55

Basic Skills Progress Measure

• Still looks at level completion as the

measure of success, but changes the cohort

• f students in the measure
• Has a closer relationship to NRS than the

previous measure

8/25/2015 56

Basic Skills Students

Attribut ributes es Lit iteracy eracy St Students dents NR NRS St Students dents 2016 St State e Perfo formance rmance Measure sure Cohor

• rt

Contact hours Greater than 0 12 12 Enrolled in BSP class in program year Yes Yes Yes Literacy Data Required Yes Yes Yes Post Test required No No Yes 2013-14 students 106,971 81,910 45,358

8/25/2015 57

Basic Skills Progress Measure

8/25/2015 58

2013 Version sion 2016 Version sion Total Contact Hour minimum 60 12 EFL’s included All students except those at ASEH. This includes those who didn’t have a placement All four ABE EFL’s ASEL All 6 ESL EFL’s Those without an initial placement are not included since these students wouldn’t have post-tested Post Test required No Yes Program years included 2011-12; 2012-13; 2013-14 2014-15; 2015-16; 2016-17 Same as NRS No NRS Table 4B, but success will be an overall percentage and breakout by EFL in the appendix.

Contact Info

• Dr. Kristen Corbell

919-807-6981 corbellk@nccommunitycolleges.edu

8/25/2015 59