Excellence Framework Follow us on Twitter at REF consultation - - PowerPoint PPT Presentation

▶
excellence framework
SMART_READER_LITE
LIVE PREVIEW

Excellence Framework Follow us on Twitter at REF consultation - - PowerPoint PPT Presentation

Dec 26, 2023 434 likes •697 views

Consultation on the second Research Excellence Framework Follow us on Twitter at REF consultation events #REF2021 David Sweeney Director of Research, Knowledge Exchange and Education at HEFCE What is the REF? UK-wide framework for

slide-1
SLIDE 1

Consultation on the second Research Excellence Framework

REF consultation events

David Sweeney Director of Research, Knowledge Exchange and Education at HEFCE Follow us on Twitter at #REF2021

slide-2
SLIDE 2

What is the REF?

UK-wide framework for assessing research in all disciplines

It is a a pee peer revie iew process. The purpose of the REF is to:

  • Inform the allocation of research funding by the four UK funding bodies
  • Provide accountability for public funding and making the case for future

investment

  • Provide benchmarks and reputational yardsticks

Additional purposes as recognised by the Stern Review

  • It provides a rich evidence base to inform strategic decisions about national

priorities across science, social science, engineering, medicine and arts and humanities research.

  • It can create a strong performance incentive for universities and for individual

academics.

  • It can be used by universities and other bodies to inform decisions on resource

allocation.

slide-3
SLIDE 3

REF Evidence

slide-4
SLIDE 4

Lord Stern’s independent review

  • Spending Review 2015 – Review announced
  • December 2015 – Review to be chaired by

Lord Stern

  • January 2016 – Call for evidence published
  • July 2016 – Independent review published
  • Government response to independent

review: ‘I have asked HEFCE to work together with the other HE funding bodies to develop detailed proposals which take forward the recommendations from the Review, including any further testing and piloting.’

slide-5
SLIDE 5

Impact of Lord Stern’s recommendations

Key principles

  • Lower burden
  • Less game-playing
  • Less personalisation, more institutionally focused
  • Recognition for investment
  • Making space for long-term research
  • More rounded view of research activity
  • Interdisciplinary emphasis
  • Broaden impact
slide-6
SLIDE 6

Consultation on the second Research Excellence Framework

REF consultation events

Kim Hackett REF Manager Follow us on Twitter at #REF2021

slide-7
SLIDE 7

Approach to the consultation

REF 2021 will be based on peer review (informed by metrics, where appropriate), and assessing research

  • utputs, impact and environment

Consultation proposals are based on:

  • The evaluation of REF 2014 and
  • Implementation of the recommendations in Lord Stern’s Independent

review:

  • 9 recommendations on the next REF
  • 3 recommendations on the wider context of research assessment

(not directly addressed in the consultation)

slide-8
SLIDE 8

Features of the REF consultation

We are seeking views on the following features of the next REF

  • Overall approach
  • Unit of Assessment structure
  • Expert panels
  • Staff
  • Collaboration
  • Outputs
  • Impact
  • Environment
  • Institutional-level assessment
  • Outcomes and weighting
  • Timetable
slide-9
SLIDE 9

Features of the REF consultation

Overall approach

  • Broad continuity with REF 2014

Unit of Assessment structure

  • ~36 UOAs
  • 4 Main panels
  • Continuity between exercises
  • Consistency and streamlining across panels
  • Reduced visibility
  • High volumes of submissions

Expert panels

  • Development of submissions guidance and panel criteria
  • Representativeness of expert panels
slide-10
SLIDE 10

Features of the REF consultation

Staff and outputs

  • All research-active staff to be submitted
  • HESA definition of ‘research active’ – consistent & auditable
  • Staff mapping to REF UOAs – HESA cost centres
  • De-coupling staff and outputs, overall multiplier of two
  • Flexibility on number of outputs per staff member – 0/1 to 6 outputs
  • No individual staff circumstances but code of practice
  • Portability of outputs – recognition of HEI investment
  • Definition of ‘demonstrably generated’
  • Particular impacts on certain groups
slide-11
SLIDE 11

Summary of key issues

Challenges raised with proposed use of HESA

  • Defining research-active staff
  • Contracts (as recorded through HESA) don’t necessarily reflect activity
  • Including staff on T&R according to standard practice – not all

research-active

  • R contracts in some cases reflecting KE / innovation activity
  • No clear-cut way to identify who are independent researchers

(particularly those on R-only contracts)

  • Mapping to UOA by cost centre
  • Allocation of staff to cost centre reflects teaching purpose – not REF

purpose

  • May result in some very small (e.g. 1 staff member) submissions
  • Panels would receive material for assessment not within remit
  • Would not allow HEI to submit cross-departmental units
  • Would present challenges for presenting coherent environment
slide-12
SLIDE 12

Features of the REF consultation

  • Identify research-active staff in cost

centres

  • Map to REF UOAs
  • Calculate average FTE over set period
  • Multiply by 2
  • 1. Number
  • f outputs

per UOA

  • Identify eligible outputs in REF period

(demonstrably generated at HEI)

  • Select outputs up to number required
  • Allocate each output to at least one

research-active staff member (max 6

  • utputs per person)
  • 2. Selected
  • utputs for

submission

Illustration of proposals

slide-13
SLIDE 13

Features of the REF consultation

Staff (continued)

  • Staff identifier
  • Categories of staff eligibility
  • Remove Category C staff
  • Clarity of definition for research assistants
  • Fractional appointments

Collaboration

  • Responding to the Dowling Review
  • Output flexibility
  • Research environment
slide-14
SLIDE 14

Features of the REF consultation

Outputs

  • Implementation of Open Access policy
  • Reserve outputs for late publications
  • Interdisciplinary Research
  • Interdisciplinary ‘champions’
  • Mandatory identifier
  • Environment template
  • Interdisciplinary research advisory panel (IDAP)
  • Metrics
  • Not a replacement for peer review
  • Quantitative metrics appropriate to the discipline
  • Forum for Responsible Metrics
slide-15
SLIDE 15

Features of the REF consultation

Impact

  • Broadening and deepening the definition of impact and guidance on certain

types of impact

  • Align definition of impact with Research Councils UK
  • Move the impact template to the assessment of environment
  • Number of case studies to be submitted, minimum of one
  • Mandatory and optional fields within case study template
  • Impact arising from research, research activity or a ‘body of work’
  • Two-star threshold for underpinning research & standards of rigour
  • Impact evidence for audit and for assessment
  • REF2014 – REF2021
  • (Institutional-level assessment)
slide-16
SLIDE 16

Features of the REF consultation

Environment

  • More structured environment template
  • More quantitative data
  • Common Indicators
  • Equality and Diversity
  • Open Access and Open Data
  • Impact strategy
  • Forum for Responsible Metrics
  • (Institutional-level assessment)
slide-17
SLIDE 17

Features of the REF consultation

Institutional-level assessment

  • Impact
  • Impact case studies arising from multi and interdisciplinary and

collaborative work

  • 10-20 per cent of total impact case studies
  • Minimum of one
  • Environment
  • Environment statement to avoid duplication of content across UOAs
  • Engagement and impact strategies
  • Progress made against previous plans
  • Promoting IDR, supporting collaboration
  • Supporting ECRs
  • Promoting equality and diversity practices
slide-18
SLIDE 18

Features of the REF consultation

Outcomes and weightings

  • Weighting for outputs maintained at 65 per cent
  • Total weighting for impact no less than 20 per cent
slide-19
SLIDE 19

Features of the REF consultation

Timetable

1 1 Aug August 20 2013 13 Start of period for income and impacts 1 1 Jan anuary 20 2014 14 Start of period for outputs 17 17 Mar arch 20 2017 17 Consultation deadline Mid id-2017 Publish initial decisions on the next REF Mid id-2017 17 Appoint panel chairs 2018 2018 Publish guidance on submissions and panel criteria 2019 2019 Invite HEIs to make submissions 31 31 July 20 2020 20 End of assessment period (for research impacts, the research environment and related data) November 20 2020 20 Closing date for submissions 31 31 Dec December 20 2020 20 End of publication period for publication of research

  • utputs and outputs underpinning impact case

studies 2021 2021 Assessment year Dec December 20 2021 21 Publication of outcomes Spri Spring g 20 2022 22 Publication of submissions and reports

slide-20
SLIDE 20

Breakout discussions

Follow us on Twitter at #REF2021

slide-21
SLIDE 21

Next steps

Closing plenary

Follow us on Twitter at #REF2021

slide-22
SLIDE 22

Next steps

  • 14 week consultation
  • Deadline: midday on Frid

Friday 17 17 March 201 2017

  • Analysis of responses in spring/summer 2017
  • Initial decisions on next REF to be published in Summer 2017
  • Advisory panels:
  • Equality and Diversity Advisory Panel (EDAP)
  • Interdisciplinary Advisory Panel (IDAP)
  • Forum for Responsible Metrics
  • Pilot activity through 2017
slide-23
SLIDE 23

REF2021 Timetable

Pilot work 2010 Criteria phase 2011-2012 Submissions phase 2012-13 Assessment phase 2014 Impact pilot Guidance on submissions (July 2011) REF submissions system launched Panel assessment Recruit main panels Panel criteria (Jan 2012) Submissions deadline: 29 November 2013 Outcomes published: December 2014 Pilot work 2017 Criteria phase 2018-2019 Submissions phase 2019-20 Assessment phase 2021 Institutional level pilot Develop and publish guidance

  • n submissions

and panel criteria Launch REF submissions system Panel assessment Recruit main panels Submissions deadline Outcomes published: December 2021

slide-24
SLIDE 24

Thank you for listening

researchpolicy@hefce.ac.uk

Follow us on Twitter at #REF2021