Higher Education Data Workshop 28 July 2011 Agenda Data and coding - - PowerPoint PPT Presentation

Higher Education Data Workshop 28 July 2011

Agenda

– Data and coding issues – Audit outcomes, timetable and future of audit – Using HESA data for end of year monitoring – The HESA IRIS system – Data quality analysis – For our Future indicators – The inclusion of FEIs in Unistats and the NSS – Timetable – Key Information Sets (KIS) – Monitoring of the maximum student number – Redistribution – Changes to HESES – 2012/13 funding consultation

Data and Coding Issues – HESA student data

• Persistent issues with return of module

data (MODSTAT and MODOUT)

• JACS coding
• Fundability status
• Issues found during HESA/EYM analysis
• LANGPCNT

Changes proposed for HESA and LLWR

• HESA have requested proposed changes from

statutory customers for 2012/13 and 2013/14 instead

• f a major review of the student record for 2013/14

– HEFCW to respond with changes:

• for 2012/13 by 22 August 2011
• for 2013/14 by January 2012

– Institutions informed of changes:

• for 2012/13 in January 2012
• for 2013/14 in June 2012
• Changes to the LLWR record will be discussed at the

September meeting with FEIs

Changes proposed for the HESA record 2012/13

• HEFCW will propose inclusion of:

– A flag indicating is the student is eligible under the new fee regime – A domicile field for students under the new fee regime (as assessed for fees purposes) – The fees charged to the student under the new fee regime – An APEL field on the student on module entity to indicate if the module was taken through APEL

• These changes will be reflected in the LLWR

Audit Outcomes – External Audits (1)

Findings of the audits:

• Completion status of modules and students not

recorded correctly on the system

• Credit values counted twice on EYM
• Systems not capable of running historic reports
• Issues with recording of qualifications on entry
• Students counted in the WA premium not having

an eligible postcode

• Cross checks with franchise partners not made
• Hard copy enrolments forms not found
• .

Audit Outcomes – External Audits (2)

• Income returned on RAS and HEBCIS not reconcilable to

the audited accounts HEBCIS:

• Copies of contracts returned not available
• No evidence to support the number of CPD days
• Income counted in more than one category
• Expenditure and not income returned for some projects
• Number of consultancy contracts understated
• Lack of evidence for inclusion of some spin outs and

graduate start ups, and staff start ups also included in spin out count

Audit Outcomes – Internal Audits (1)

Recommendations included:

– HESA/LLWR student record » The final version of the HESA return should be kept » FUNDCOMP and MODOUT should be correctly assigned » Validation reports for non-standard academic years and students taking more than one course should be produced » The procedure for calculating the proportion through Welsh should be documented » Anomalies in the recording of DSA should be investigated » The highest qualification on entry should be appropriately identified and recorded » Evidence of the LLWR vs EYM check should be documented and retained

Audit Outcomes – Internal Audits (2)

– HESES/EYM » The data returns procedure should be updated to reflect any new HEFCW guidelines » Sufficient evidence should be retained to support assignments to JACS codes » The process to calculate estimates should be documented » Credit values in each column should not be aggregated before completion of the return » Modules should be reviewed to determine if ASC allocations are correct » It should not be assumed that all students eligible to resit do so – HESA staff record » Final returns should be scrutinised by an appropriate member

• f staff

» Casual workers should be included

Audit Outcomes – Internal Audits (3)

– HEBCIS » A control should be put in place for the approval of the electronic return » The calculation of estimates should be documented » Training should be provided to an additional officer » The reasons for data amendments should be recorded – System » Written evidence of withdrawal should be available » The mainstream funding activity should be entered into the system at the same time as the student details – Other » The risk of relying on a key individual should be considered for inclusion on the risk register » Procedures should be documented

Audit Timetable

• August 2011

– Feedback from internal and external audits – Official notification of external audits for 2011

• Autumn 2011

– External audits take place

• Future

– 2011 is the last year of the current 5-year cycle – Will be retendering for external auditors next year – Future audits may cover data used in FoF, fee plans

• r specific provision or initiatives, not just funding

data

Using HESA Data for End of Year Monitoring

– Technical group – last meeting September 2009 – 2007/08, 2008/09 and 2009/10 extractions carried out using algorithm – Timetable revised last year as data quality not good enough to replace EYM survey data – Data collected on EYM survey likely to change

Timetable

2011

August Return of EYM 2010/11 data Summer/Autumn Extracted data from the 2010/11 HESA record made available through the IRIS system Late October HESES 2011/12 published November Start consultation with sector about requirements of EYM survey December Consultation outcomes published and decisions made about the future of the EYM survey

2012

Spring Extraction of end of year data from the 2010/11 record (clawback

• utcomes compared to 2010/11 EYM clawback outcomes)

August EYM survey published but no detailed EYM credit value data collected directly from HEIs (subject to the outcome of the consultation) 2013 Spring Extraction of end of year data for HEIs from the 2011/12 HESA record used for calculating adjustment to grant for 2011/12 for HEIs

Using HESA data for 09/10 EYM

Student registrations – HE sector totals by residential status and eligibility for HEFCW funding

Status EYM HESA

% difference

Home and EU fundable 102,777 102,287

• 0.5%

Home and EU non-fundable 9,812 7,090

• 27.7%

Island and Overseas 14,443 13,791

• 4.5%

Total 127,032 123,168

• 3.0%

Using HESA data for 09/10 EYM

Student registrations – HE sector totals for Home and EU fundable students

EYM HESA

% difference

Full-time/sandwich undergraduate 60,502 59,841

• 1.1%

Full-time/sandwich postgraduate 6,100 6,583 7.9% Part-time undergraduate 27,165 27,101

• 0.2%

Part-time postgraduate 9,012 8,763

• 2.8%

Total 102,777 102,287

• 0.5%

Using HESA data for 09/10 EYM

Other student registration tables

ITT Medicine and Dentistry (compared against HESES) ERASMUS (compared against HESES) Art, Design and Performing Arts – performance element (compared against HESES)

Using HESA data for 09/10 EYM

Sector wide issues on registration tables

ITT students

• ITTPHSC

(Primary = 15-19, 53, 61-65. Secondary = 24-26, 55-57)

• Secondary phase split of subject of study

(SBJCA and SBJPCNT) Students on a sandwich year out

• SPECFEE = 1

ERASMUS

• SPECFEE = 3

Using HESA data for 09/10 EYM

Sector wide issues on registration tables (cont…)

Mapping of subject codes - JACS to ASC

• ASC 2, 3 and ASC 8

Medicine and Dentistry

• Pre-clinical and clinical split using SBJCA
• Intercalated year

(COURSEAIM = H24) NLIAH funded students coded as HEFCW fundable

Using HESA data for 09/10 EYM

Sector wide issues on registration tables (cont…)

Franchised out students

• PCOLAB = 100 for all modules
• TINST = 4001, 4002, 4003, 4004

Using HESA data for 09/10 EYM

HE sector totals for Home and EU fundable credit values

Mode / Level EYM HESA

% difference

Full-time/sandwich undergraduate degree 6,640,019 6,616,256

• 0.4%

Full-time/sandwich undergraduate non-degree 396,222 377,607

• 4.7%

Full-time/sandwich postgraduate taught 622,019 584,066

• 6.1%

Part-time undergraduate (non franchised out) 914,973 928,677 1.5% Part-time undergraduate franchised out 243,896 244,934 0.4% Part-time postgraduate taught 441,614 401,555

• 9.1%

Total 9,258,743 9,153,096

• 1.1%

Using HESA data for 09/10 EYM

Sector wide issues on credit value tables

Module outcomes

• Unknown (MODOUT=6)
• Some institutions with no modules returned as non-

completion (MODOUT=4) Module status

• Institutions having all their modules contained in

reporting year (MODSTAT=2)

Using HESA data for 09/10 EYM

Sector wide issues on credit value tables (cont…)

More credits on EYM than on HESA More franchised out credits on HESA than EYM Effects of module coding issues on estimation methods

Using LLWR data for 09/10 and 10/11 EYM

Extra fields introduced for 2010/11 to HE dataset

Mode of attendance Qualification Aim Subject of Study and associated percentages Qualifications awarded and grade

Month 12 data recently received from Welsh Government Both years analysis and issues to be presented and discussed at meeting in HEFCW on 28th September 2011

HESA IRIS system

– For Welsh HEIs from 2010/11 – For 2010/11 will include:

• EYM extraction (credit values and registrations)
• Premium, per capita and PGR summaries and data
• Data used to monitoring the maximum student

number – At TEST COMMIT and COMMIT stages of submission, a file created by HEFCW will be available for download

HESA Data quality analysis 2009/10

HESA Data Quality Analysis 2009/10 – 1

Purpose

– HEIs to compare their data with the sector – HEFCW to draw attention to particular fields

Tables

– Tables 4 & 16–21 now have trend data for 3 years – The other tables are similar to 2007/08 with 5 years’ trend data where possible

HESA Data Quality Analysis 2009/10 – 2

Table 1 – Postcodes

– 3% postcodes missing (16% at one HEI)

Table 2 – Ethnicity

– 1% ethnicity not known (4% at one HEIs) – 2% ethnicity refused

HESA Data Quality Analysis 2009/10 – 3

Table 3 – Welsh subject provision

– Q500 Celtic studies used instead of Q560 Welsh or Q561 Welsh literature

Table 4 – Proportion of module taught through Welsh

– Split by fundable/not fundable – Majority of not fundable are Q560/Q561 or FUNDCODE ne 1

HESA Data Quality Analysis 2009/10 – 4

Table 5 – Coding of non-standard academic year

– 23% of enrolments starting after 1 January and leaving after 31 July coded as standard academic year – 17% of full-time masters lasting more than 40 weeks coded as standard academic year

Table 6 – Average FTE/credits per enrolment

– Maximum, minimum and standard deviation added – High maximum with high standard deviation may warrant investigation

HESA Data Quality Analysis 2009/10 – 5

Data linked to previous academic year Table 7 – by HUSID or HUSID and instance

– Differing commencement dates between years

Table 8 – using HEFCE linking

– 6% of students with different HUSID in previous year – Majority of students with different HUSID have changed HEI

Table 9 –Starters also present in previous academic year

– 63 starters present in previous year (most of these at one HEI)

HESA Data Quality Analysis 2009/10 – 6

Table 10 – Non-completion of current year

– 60% left the institution (9% to 99% at individual HEIs) – 16% suspended studies or were dormant (71% at one HEI)

Table 11 – Non-completion reasons for leaving

– 39% left due to personal or other reasons – 23% were written off after lapse of time – 14% successful completion of course – 6% unknown reason for leaving

Table 12 –Welsh fluency

– 87% with known Welsh speaker status (83% in 2008/09)

HESA Data Quality Analysis 2009/10 – 7

Table 13 – Disability

– 95% have known disability status (45% for one HEI)

Table 14 – Disabled Students’ Allowance (DSA)

– 13% of disabled student enrolments with information on DSA not known (between 0% and 39% at individual HEIs)

Table 15 –Level of subject coding

– 28% of enrolments coded to three or more places (between 9% and 47% at individual HEIs)

HESA Data Quality Analysis 2009/10 – 8

Table 16 – Outcome of module

– 79% completed, gaining full credit (between 42% and 92% at individual HEIs) – 9% module outcome not yet known (37% at two HEIs) – Partial completions recorded at two HEIs – Modules taken on a not-for-credit basis recorded at two HEIs

Table 17 – Module outcome not yet known

– 21% continuing into next reporting year (between 0% and 95% at individual HEIs) – 5 HEIs where100% NOT continuing into next reporting year

• Note. four institutions have enrolments on modules coded as taken in previous

year (outcome only) (i.e. MODSTAT=4 ). In 2007/08 there were none.

HESA Data Quality Analysis 2009/10 – 9

Table 18 – Term-time postcode

– 11% with term-time postcode missing (between 0% and 51% at individual HEIs)

Table 19 – Term-time & home postcode of students in halls

– 0% with term-time & home postcode the same – Nearly a third had an invalid or missing term-time or home postcode or were non-UK domiciled

HESA Data Quality Analysis 2009/10 – 10

HESA Data Quality Analysis 2009/10 – 11

Table 20 – Socio-economic classification (SEC)

– 24% SEC not classified (between 16% and 31% at individual HEIs) – 7% SEC not recorded (between 0% and 37% at individual HEIs)

Table 21 – Highest qualification on entry

– 5% highest qualification on entry not known (between 0% and 19% at individual HEIs)

HESA Data Quality Analysis 2009/10 – 12

Contact details Trine Karran 029 2068 2241 trine.karran@hefcw.ac.uk

For our Future Indicators

Annex A of HEFCW corporate strategy 2010-11 to 2012-13 www.hefcw.ac.uk/publications/corporate_documents/corpor ate_strategy.aspx W11/05HE: Higher Education Data Requirements 2011 www.hefcw.ac.uk/publications/circulars/circulars_all.aspx W11/25HE: Request for strategic planning information, forecasts and Annual Monitoring Statements 2011 www.hefcw.ac.uk/publications/circulars/circulars_all.aspx

For our Future Indicators

Forecasts required for: Widening access Module completion rates National student survey overall satisfaction Welsh medium Employability Spin-out

For our Future Indicators

29th July submit forecasts (six) to HEFCW notify HEFCW of amendments to 2008/09 or 2009/10 actual data October progress at sector level published in HEFCW annual report progress at institutional level published on HEFCW website

For our Future Indicators

2009/10 progress – met or likely to be met Widening access Module completion rate National student survey overall satisfaction Overseas students Employment Spin-out Governance

For our Future Indicators

2009/10 progress – not met or not likely to be met Welsh medium Part-time Research council income Income To be updated Participation

For our Future Indicators

Widening access

increased from 15.6% in 2008/09 to 16.2% in 2009/10 target likely to be exceeded in 2012/13 proportion has fallen at two HEIs

Module completion rate

increased from 87.6% in 2008/09 to 89.5% in 2009/10 target likely to be exceeded in 2012/13 proportion has fallen at four HEIs

For our Future Indicators

National student survey

average overall satisfaction score for Wales has fallen from 83.3% in 2009, to 83.0% in 2010 target met in both 2008/09 and 2009/10, as higher than UK score in both years five HEIs have average scores higher than the UK and five have average scores lower than the UK

Welsh medium

number in 2009/10 almost the same as in 2008/09 the number has fallen at five HEIs

For our Future Indicators

Overseas students

annual growth rose from 15.1% between 2007/08 and 2008/09 to 26.7% between 2008/09 and 2009/10 target met in both 2008/09 and 2009/10, as higher than UK annual growth in both years annual growth lower than UK at two HEIs in 2009/10

Employment

risen from 91.2% in 2007/08 to 91.9% in 2009/10 target met in 2007/08, 2008/09 and 2009/10, as higher than UK in all years proportion lower than UK proportion at four HEIs in 2009/10

For our Future Indicators

Part-time

the number has fallen by 4,471 from 54,714 in 2008/09 to 50,243 in 2009/10 fallen by more than the number the target is required to increase (increase by 4,286 by 2012/13) drops at eight HEIs

Spin-out increased from 252 in 2008/09 to 301 in 2009/10 target exceeded in 2009/10

For our Future Indicators

Research council income

annual growth in income fell from 17.4% between 2007/08 and 2008/09 to 3.0% between 2008/09 and 2009/10 target was met in 2008/09 but not met in 2009/10, although difference in 2009/10 between Wales and UK figure was only 0.1% income fell at seven HEIs between 2008/09 and 2009/10

For our Future Indicators

Income in 2008/09, 4 (36%) HEIs had income in excess of the UK median and in 2009/10 there were 5 (45%) in 2008/09 and 2009/10 there were 4 HEIs with income below the UK lower quartile Governance no HEI was classified as ‘high risk’ in 2008/09 and 2009/10

For our Future Indicators

29th July submit forecasts (six) to HEFCW notify HEFCW of amendments to 2008/09 or 2009/10 actual data brief explanation of drop in part-time numbers

FECs & National Student Survey

Directly funded FECs to be included in National Student Survey 2012 W10/35HE: Plans for the National Student Survey 2011 www.hefcw.ac.uk/publications/circulars/circulars_2010_a ll.aspx Additional fields added to LLWR in 2010/11 HEFCW to extract list of students to be surveyed 28th September HE LLWR data meeting

Timetable for 2011/12 (1)

22 July Funding consultation document issued 28 July Workshop for institutions 5 August EYM return date 23 August EYM verifications sent out Early August Redistribution circular issued 5 September Funding consultation return date 13 September EYM verifications return date Sept/October Audit of data (selected institutions)

Timetable for 2011/12 (2)

Mid September Data submitted to HESA Mid Sept/Oct HESA data quality checking Mid September Redistribution return date End October Outcomes of funding consultation published End October Further guidance on control of student numbers published End October HESES survey published November HESES returned

Monitoring of the maximum student number 2011/12 (1)

• Monitored for the first time using HESA/LLWR

data in early 2013 for 2011/12

• Circular to be published once the outcomes of the

funding consultation known, will include

• Coding for data extraction
• Guidance about particular groups of students
• Details of penalty arrangements

Monitoring of the maximum student number 2011/12 (2)

• New student instances starting in 2011/12 will be

counted, that are:

– FT UG or PGCE – Completions (or estimated completions)

• HEIs should follow HESA guidance about when to

start a new student instance

Monitoring of the maximum student number 2011/12 (3)

• Students studying for a top up to a degree after

finishing an HND/foundation degree:

– At the same institution, we would expect these to keep the same instance on HESA – On the LLWR, need to be able to link back to a learning activity starting before 1 August 2011 – Would be counted as new entrants if moved to a different institution (unless moving from a franchisee to a franchisor and keeping the same instance)

• Students resuming studies won’t be counted as

new entrants unless they are returned with a new instance

Monitoring of the maximum student number 2011/12 (4)

• UHOVI – if the number of FT UG UHOVI

students exceeds the contracted number of UHOVI students, the difference, converted to new entrants, will be included in the monitoring number (pro rata to UHOVI recruitment at each institution)

• Numbers may be amended for 2011/12

redistribution:

– Redistribution of reward credits – Redistribution between institutions

Monitoring of the maximum student number 2011/12 (5)

• Verification of data 2011/12

– Numbers used for monitoring sent to institutions with details of students included – Amendments can be made

• Details of amendments and reasons for amendments

provided to HEFCW

• Need approval by HEFCW
• Numbers fitting the criteria in 2010/11 will be

reported back to institutions through HESA IRIS and by HEFCW for information

Redistribution

• Delayed so that institutions are aware of

proposals in funding consultation

• Circular will be issued in August for return in

September

• Redistribution:
• Of reward credits
• To reflect recruitment patterns
• Between institutions
• Likely that only PT and PGT funded credits will be

taken forward to the funding model

HESES (1)

Delayed this year due to the funding consultation

– Usually publish mid October – will now be late October – Same amount of time for completion – Verification period may be shorter

Changes will depend on outcome of funding consultation

– Whether credit based funding used for PT and PGT only – Whether premium based funding used for FT UG and PGCE

HESES (2)

Likely to revise tables:

• to collect information about students fitting the

eligibility criteria for the new fee regime

– New entrants – FT UG or PGCE – Welsh or EU domiciled – Without a degree already (with some exceptions)

• to see the proportion of students at the institution in

2011/12 who fit the eligibility criteria for the fee regime

• to collect less information about credit values for full-

time undergraduates and PGCE students