DATA WORKSHOP 27 June 2006 Higher Education Data Workshop Outline - - PowerPoint PPT Presentation

HIGHER EDUCATION DATA WORKSHOP 27 June 2006

Higher Education Data Workshop

Outline

• Changes to data use and requirements
• Timetable
• Data quality analysis
• HESA/EYM comparisons
• Audit
• Redistribution
• Comparative funding
• WAG presentations

Changes to Data Use and Requirements

• Welsh medium modules

– Welsh medium costs study – Seminar in September – Discussion of data issues

Changes to Data Use and Requirements – Widening Access Premium

• Claritas Super Profiles out of date
• New method needed to determine

low affluence

Changes to Data Use and Requirements – HESA student record review

• Second consultation in March 2006
• Responses from seven Welsh HEIs
• New structure and fields introduced for

2007/08

• Some changes to module fields

Changes to Data Use and Requirements – Cell weights

• Unit of funding now used instead of unit
• f resource
• Relativities between ASCs have

changed

• New cell weights for calculation of

2007/08 funding

2006/07 Timetable

27 June Workshop for institutions 2 August EYM return date 21 August EYM verifications sent out 31 August Return date for redistribution exercise 7 September EYM verifications return date Sept/October Audit of data Mid September Data submitted to HESA Mid Sept/Oct HESA data quality checking Mid October HESES survey issued Mid November HESES return date

COMPARISONS BETWEEN HESA AND EYM DATA

Total number of assumed completed credit values extracted from HESA compared to verified EYM data In 2003/04 HESA 4% lower than EYM data In 2004/05 HESA 2% higher than EYM data

COMPARISONS BETWEEN HESA AND EYM DATA

% difference (HESA to EYM) by institution for total assumed completed credit values range from: FT –4% to +9% (sector +0.3%) SW -17% to +75% (sector +0.1%) (small numbers) PT -11% to +60% (sector +11%) Total -7% to +7% (sector +2%)

COMPARISONS BETWEEN HESA AND EYM DATA

% difference (HESA to EYM) for total assumed completed registrations: FT +0.6% SW -10% (small numbers) PT -5% Total -2%

COMPARISONS BETWEEN HESA AND EYM DATA

Observations:

Spread of credits across ASCs differ. Higher number of regs/creds in ASC11b at EYM than HESA. Perhaps due to Y000 JACS coding? Subjects coded with wrong JACSCODES. Students recorded with stuload=0 but were not NSAY and undertook modules during year. Column 1/ column 2 differences.

COMPARISONS BETWEEN HESA AND EYM DATA

Observations (2):

Some students not coded correctly as full-time sandwich year out on HESA. Significantly higher number of PGT credits at HESA than EYM. Higher number of undergraduate non-degree credit values on HESA than EYM.

Some instances of fundcode seem to have been coded wrong.

COMPARISONS BETWEEN HESA AND EYM DATA

Observations (3):

Franchised Out Some values of tinst1 wrongly coded? Fundable/non-fundable students. Some individual college information incorrectly recorded.

COMPARISONS BETWEEN HESA AND EYM DATA

In conclusion: HEFCW will continue to make comparisons

• f the HESA data against EYM returns with a

view to reviewing the data collected at EYM.

If anyone has any comments on the comparisons or would like us to look at specific elements of their data in more detail please email hestats@hefcw.ac.uk

HESA Data Quality Analysis 1

• Fields regularly used in analysis, funding

and monitoring

• Looked at missing values, inconsistencies

and changes between years

• Individual institution level and sector level

HESA Data Quality Analysis 2

• Table 1 – Postcodes

– Proportion of missing/invalid postcodes improved from 2003/04 (3% to 1%) – Still some large proportions invalid/missing at some institutions for certain modes/levels

• Table 2 – Ethnicity

– Used in monitoring – Still some large proportions of unknowns but small overall (3%)

HESA Data quality analysis 3

• Table 3 – Welsh subject provision

– Q500 used instead of Q560, Q561 – Difficult to analyse Welsh provision

• Table 4 – Proportion taught through Welsh

– Some small proportions

HESA Data quality analysis 4

• Table 5 – Non-standard academic years

– Academic years that are potentially non-standard coded as standard (e.g. courses starting in January 2005, with expected length of year greater than 40 weeks)

• Table 6 – Average FTE/credit values per

enrolment

HESA Data quality analysis 5

• Table 7 – Data linked by HUSID/instance to

2003/04

– Some differences between birth date and ethnicity – Greater numbers have different commencement date, domicile and/or postcode

• Table 8 – Data linked by HEFCE linking method

to 2003/04

– Linked using birth date, name etc as well as HUSID – Overall 5% of HUSIDs differ between the two years

HESA Data quality analysis 6

• Table 9 – Students starting in 2004/05

– Some also present in 2003/04 with same instance (1% overall) – Proportions at individual institutions vary from 0% to 5% overall, some large proportions for certain mode/level combinations

Audit 1

• Process for this year the same as last

year

• Process has been reviewed internally

– Letter to Heads of Institutions going out after this workshop detailing proposals

Audit 2

• Proposals include:

– Reliance placed on institutions’ internal audit reports – External audit required where an increased risk of error has been identified – External audit required at least once every five years regardless of risks identified

REDISTRIBUTION

• Circular issued: 07/06/06
• Response deadline: 31/08/06
• Changes to rules:

None But…….

• Remember paragraph 8

Institution Concerns

• Partial Completions

THE FUNDING GAP

Frances Good Head of Statistics and Information Services HEFCW

www.hefcw.ac.uk/Finance_and_Assurance/statistical_data

OVERVIEW

• The purpose of the analysis
• The methods used
• The main findings
• Data issues

THE PURPOSE OF THE ANALYSIS

• The WAG asked HEFCW to provide a

comparison of funding in Wales with the rest of the UK

• The analysis needed to take into account

differences in types of HEI and subject mix

• Historical as well as current figures were

requested

• To be repeated this summer

APPROACH

• Two separate methods
• Analysis uses:
• 1. Published accounts from funding councils +

HESES or EYM fundable enrolments

• 2. Data from HESA
• Both used to calculate Grant per Fundable

FTE

• 1. TOP-DOWN APPROACH

Uses:

• Funding allocated in each country from

published accounts with adjustments for Access funds, SRIF and TDA funding; converted to academic year

• Enrolment data from HESES (England) or

final end of year figures (Wales and Scotland)

• 2. BOTTOM-UP APPROACH

Uses:

• Funding received by each institution as

recorded in HESA Finance Record

• Teaching grant, grant + fees and total

income also taken from HESA Finance Record

• Fundable FTEs taken from HESA Student

Record

DRAWBACKS OF BOTTOM- UP APPROACH

• Excludes HE in FEIs and top-sliced grant
• Treatment of capital in HEIs’ accounts

(shown as release of deferred capital grants)

• Inaccurate recording of fundable FTEs in

HESA data

• Errors in estimates at individual HEI level

where data quality is poor

CALCULATION OF THE FUNDING GAP

• Funding Gap calculated as:

(Funding per Fundable FTE in Wales – Funding per Fundable FTE in England) x Number of Fundable FTEs in Wales

• Needed to add around £9m to compensate

for ‘deferred capital grant’ to bottom-up estimates

MAIN FINDINGS – TOP DOWN

• Welsh HEIs received less grant than

English or Scottish HEIs

• Initially similar levels in Wales and

England (ie 2000/01 and 2001/02)

• Gap has become larger since 2002/03
• Funding gap with England is around £26m
• Funding gap with Scotland around £75m

COMPARISON OF GRANT-IN- AID PER FUNDABLE FTE

2000/01 2001/02 2002/03 2003/04 HEFCW £4,307 £4,649 £4,687 £4,942 HEFCE +TDA £4,334 £4,609 £4,917 £5,296 SHEFC £5,087 £5,401 £5,643 £5,946

MAIN FINDINGS – BOTTOM-UP

• Lower grant +fees and overall income in

Wales than in England or Scotland

• Similar grant for teaching in Wales and

England

• Funding gap with England in range £16m -

£24m

Grant and Income per FTE 2003/04

HE E Grant ant per r fundab dable e HE FTE HE E Teac aching hing Grant ant per er fundab dable e taught ght HE FTE E Al All Grant ant and d Fees es per r FTE Al All Incom

• me

e per r FTE

Wales £4,671 £3,367 £6,028 £9,262 England £4,796 £3,385 £6,345 £10,101 Scotl tlan and £5,919 £4,374 £7,119 £11,595

GROUPS OF HEIs

• Low Research (<10% of grant)
• Moderate Research (10% - 29% of grant)
• High Research (30% - 49% of grant)
• Very High Research (50% + of grant)
• Conservatoires
• Open University

DIFFERENCES IN STRUCTURE OF SECTORS

• No Very High Research HEIs in Wales
• Fewer Low Research HEIs (45% of FTEs in

Wales; 52% England)

• More Moderate Research HEIs (34% of

FTEs in Wales; 18% England)

• Similar High Research HEIs (20% of FTEs

in Wales; 19% England) Therefore need to compare within groups

COMPARISONS OF SIMILAR HEIs

• Compared within groups in terms of %SET,

%grant from research, %UG, grant per FTE

– Higher % UG in Wales – Low research HEIs had higher %SET in Wales

• Compared each Welsh HEI with English

comparators with similar characteristics

FUNDING GAP WITHIN GROUPS

Grant excluding specialist institutions (from HESA data)

Difference in Grant per FTE Number of FTEs Funding shortfall in Wales

Low research - small

• 142

9,183

• 1,303,986

Low research - medium 6 12,464 74,784 Low research -large

• 4

11,196

• 44,784

Moderate research 228 24,298 5,539,944 High research 433 14,649 6,343,017 Capital grant not included in HESA Finance Record 9,273435 Total 71,790 19,882,410

DATA ISSUES

• Main problem was errors in FTEs

1% error in FTEs => £50 error in grant per FTE (73,000 FTEs) £3.65 million for the sector

• Possible errors in EYM data
• Possible errors in Finance data

HIGHER FTEs IN HESA

Higher FTEs in HESA than EYM due to:

• Student, not modules, being defined as fundable –

some non-fundable modules counted as fundable in HESA

• Some HEIs not reducing HESA FTE to reflect
• nly activity actually undertaken by early leavers
• Students not completing one semester counting in

HESA but not EYM Expect HESA to be about 3% above EYM

OTHER SOURCES OF DIFFERENCES IN FTEs

Other possible sources of error include:

• Wrong percentage for FTEs recorded in

HESA

• Method of converting credit values to FTEs
• Errors in recording students as fundable
• Different treatment of non-standard years

DIFFERENCES BETWEEN HESA AND EYM FTEs

Institution EYM fundable FTE HESA fundable FTE % difference Institution A 11,151 11,196 0.40% Institution B 6,891 7,088 2.90% Institution C 5,840 6,014 3.00% Institution D 14,180 14,649 3.30% Institution M 1,737 1,909 9.90% Institution E 7,809 8,171 4.60% Institution L 1,166 1,115

• 4.40%

Institution F 6,917 6,662

• 3.70%

Institution G 5,229 5,801 10.90% Institution H 3,507 3,504

• 0.10%

Institution I 3,848 4,294 11.60% Institution J 1,472 1,385

• 5.90%

Institution K 515 510

• 0.90%

Total 70,261 72,299 2.90%

HIGHER EDUCATION INSTITUTIONS DATA WORKSHOP 27 JUNE 2006 Andrea Melvin Funding Analysis Manager

Data collection for FE learners at HEIs

CHANGES FOR 2005/06: Not required: Post-16 Early Learner Record data (PELR) End of Year Further Education Student Statistics (EYFESS) Forms B & C Required: End of Year Individualised learner data for all Welsh Assembly Government (WAG) funded learners enrolled during the 2005/06 AY.

End of Year Individualised Learner Data

– Why is this data required?

• To inform future funding allocations
• Calculate any reclaim adjustments
• To inform planning decisions

End of Year Individualised Learner Data

– Individualised data return consists of 20 data items, of which:

• 12 collected previously via EYFESS return
• 7 collected via HESA return
• 1 (Expected End Date) not collected via

HESA or historical EYFESS data returns

Submission of Individualised Learner Data

1. Individualised data to be recorded on a spreadsheet template provided and sent to WAG; 2. WAG to validate received data; 3. reports showing any validation errors issued to HEIs;

Steps 1-3 will continue until data clean of errors or submission deadline exceeded. Then:

1. data to be sent to auditors; and 2. audited data to be sent to HEIs for verification.

End of Year Individualised Learner Data : WfA

– From 2005/06, institutions are required to record the following titles for each level of the WfA framework (ref: 2005/06 Conditions of Funding):

Mynediad/Entry Sylfaen/Foundation Canolradd/Intermediate Uwch/Higher Hyfedredd/Proficiency Entry 1 2 3 4

End of Year Individualised Learner Data : WfA (cont)

Changes to draft guidance:

– Learning Aim Title (field 6)

Field must not be blank when a generic learning aim reference (i.e. beginning ‘L’) has been entered at field 5

• r where the learning activity is WfA (identified where

field 7=‘FN.357’). ‘S’: indicates a missing title where a generic learning aim reference has been entered at field 5 or the learning activity is WfA

End of Year Individualised Learner Data : WfA (cont)

Changes to draft guidance:

– Notional G/NVQ level (field 8)

An entry of 1,2,3,4,E,0,9 where a generic learning aim reference has been entered at field 5 or the learning activity is Welsh for Adults

HESA data

Why can’t HESA data be used?

• Timing:

data not available until the December after the end of the Academic Year (AY)

• Rigidity:

any proposals for new fields/amendments are required by HESA 12-18 months prior to the AY start

HESA data (cont)

– HESA data will be used to inform:

• 1. Attainment funding;
• 2. attainment rate statistics

(vital that HUSID is correctly recorded on HESA and end of year return); and

• 3. other statistics e.g WfA progression.

Changes for 2006/07

FE individualised learner data return:

Format expected to remain the same as 2005/06 but will include 2 extra data items:

– Postcode at site of learning – Third Party Indicator

TIMETABLE

July Issue of End of Year Individualised Data request 10 Aug Deadline for initial return of individualised end of year data 8 Sept Deadline for submission of clean (‘error free’) end of year data 15 Sept End of Year data sent to nominated external auditors

Statistical Directorate

HE Data Workshop Welsh Assembly Government use of HESA data

Glyn Jones and Susan Full Higher Education & Lifelong Learning Statistics Statistical Directorate Welsh Assembly Government

Statistical Directorate

Outline  Who are we  Use of HESA data  Some data quality concerns  Other issues

Statistical Directorate

Who are we?  Part of the Welsh Assembly’s Statistical Directorate  Team of 7  Very close links with the Assembly’s Higher Education and Student Finance Divisions; HEFCW; HESA  Also cover Student Finance (liaison with Student Loans company etc)  Head of section/statistician: Glyn Jones  Email post16ed.stats@wales.gsi.gov.uk

Statistical Directorate

Our role in relation to HESA data  Statutory Customer of HESA (along with HEFCW, DfES etc.)  Represented on most HESA groups - e.g. Student Record Review, Statutory Customers technical group, DLHE Longitudinal Steering Group  Represent the Assembly’s views and requirements  Interest in data on Welsh domiciles, Welsh institutions and UK comparisons

Statistical Directorate

What use do we make of HESA data?  National Statistics Publications  Policy development and monitoring  Internal briefing  Assembly Questions  Ad-hoc requests

Statistical Directorate

National Statistics publications

• Series of First Releases and Statistical Bulletins

intending to provide easy access to Welsh data, and to inform policy, for key stakeholders and the public in general:

– Students in Higher Education Institutions, Wales – Staff and Finance of HEIs in Wales – Destination of Leavers from HE – Initial Teacher Training in Wales (& ITT destinations) – HE Performance Indicators – Student Flows in Wales, 2001/02 and 2002/03 – Welsh in Higher Education, 2004/05 (new this year)

Statistical Directorate

National Statistics publications

• Focus is on national trends of Welsh domiciles or

students at Welsh institutions

• Some institutional data are published
• Sit alongside the HEFCW annual volume
• Can be found on www.wales.gov.uk/statistics
• Interactive tables available on www.statswales.wales.gov.uk

Statistical Directorate

StatsWales

Statistical Directorate

Policy development and monitoring

• First Rees Review - Assembly Learning Grants
• Second Rees Review - variable fees options
• Graham Review into part-time study
• Student projections
• Cost of student loans
• Welsh medium; Reaching Higher targets
• Widening access

Statistical Directorate

Example: Student projections model

 Updated annually and used to develop budget forecasts of the cost of student finance for Welsh domiciles  Makes intensive use of HESA data

 Overall numbers  New entrants  Domicile, location of study  Year of study  Length of course

 Data quality on new entrants caused particular issues this year and led to revisions to the model

Statistical Directorate

Example: Graham Review into part-time study

 Provided vast amount of HESA data to researchers to build up evidence base  Calculated ‘distance travelled’ for PT students  Provided evidence on the make-up of part-time provision (intensity of study, qualification aims, age, gender etc)  Produced example costs of policy options

 e.g cost of extending Disabled Students Allowance to part-time students on less than 50% FTE. Used HESA data on DSA take-up by mode of study  e.g cost of various tuition fee options outlined in final report.

 Review group frustrated by availability of data on tuition fees, employer funding and socio-economic class

Statistical Directorate

Example: Cost of student loans

 Model used to forecast the likely repayment of student loans by graduates  Intensive use of HESA Student Record and Destination data

 Overall numbers  New entrants  Length of course  Continuation  Salaries from DLHE  Employment status from DLHE

 Longitudinal DLHE will provide important evidence

Statistical Directorate

Assembly Questions and Ad-hoc requests

 Assembly Questions - submitted by Assembly members for oral

• r written answer by the ELL Minister

 Subject matter depends on Member’s interest or what topics are ‘hot’ at any point in time  Ad-hoc requests - can come from anywhere e.g

 Internal policy colleagues  Students  Researchers  LEAs, other govt departments

 We answer questions relating to Wales and in general send UK- wide requests to HESA information service

Statistical Directorate

Popular topics for AQs and ad-hoc requests

• Trends in student numbers
• Where Welsh students study
• Students for specific Unitary Authorities, HEIs, Wards, Constituencies
• Non-completion rates: individual HEIs and sub-groups of students
• Welsh medium - study through Welsh; staff teaching through Welsh;

ability in Welsh

• Equality issues - ethnicity, disability, gender
• Focus often on specific subjects - trends; profiles of students;

destinations of graduates

Statistical Directorate

Data quality issues

 Concern less on funding fields compared with HEFCW’s interest  Examples of recent concerns:

 Year of study data  Postcodes  Subject coding  NS-SEC  Employer funding of part-time provision  Welsh speaking ability  Modules taught through Welsh  Staff teaching through Welsh

Statistical Directorate

Some final thoughts for discussion

 Some specific Assembly requirements

 e.g. Welsh medium ability, Welsh Bacc, additional consultation question on NS-SEC for part-time students

 Would welcome further direct liaison with sector and HEFCW  Aware of managing burden on institutions  Data collection issues  Question bank To sum up…  Data collected are invaluable, not just for funding but for policy development, monitoring and the wider public interest.  We never know which HESA data items we will be using next…….

Statistical Directorate

Questions and discussion