Output Quality for REF (unofficial) Steve Williamson FREng IC - PowerPoint PPT Presentation

Output Quality for REF (unofficial) Steve Williamson FREng

• IC Graduate 1970, PhD 1973 • Lecturer, Aberdeen University 1973-81 • Senior Lecturer/Reader, Imperial College 1981-89 • Professor, University of Cambridge 1989-97 • Technical Director Brook Hansen 1997-2000 • Professor/Head of School, UMIST/Manchester University 2000-2009 • Deputy Vice-Chancellor for Research, University of Surrey 2009-2013 • RAE2008 and REF2014 Panel Chair • RAE2014 and RAE2021 Hong Kong Panel Chair • Gentleman of leisure 2013 -

Outputs • Originality • Rigour • Significance

REF2014 Panel Criteria and Working Methods • Part 2B : Main Panel B criteria • Section B2: Assessment criteria : outputs • Criteria and level definitions • Interpretation of generic criteria • Paragraph 66 ‘Where appropriate to the output type, sub -panels may consider editorial and refereeing standards as part of the indication of rigour’

My Interpretation • Outputs published in the top journals have a better chance of a high score • Thompson Reuters WoS – Engineering, Electrical and Electronic • 257 Titles • 59 in Q1 to choose from – Materials Science, Multidisciplinary • 271 Titles • 68 in Q1 to choose from • Not all IEEE Transactions are Q1 !

REF2014 Panel Criteria and Working Methods • Part 2B : Main Panel B criteria • Section B2: Assessment criteria : outputs • Output types • Paragraph 34 ‘ . . sub -panels will accept the submission of review articles only where they contain a significant component of unpublished research or new insights’

Significance I • Part 2B : Main Panel B criteria • Section B2: Assessment criteria : outputs • Criteria and level definitions • Interpretation of generic criteria • Paragraph 65 ‘ Significance will be understood as the extent to which the work has exerted, or is likely to exert, and influence on an academic field or practical applications’

Significance II • Part 2B : Main Panel B criteria • Section B2: Assessment criteria : outputs • Additional information on outputs • Factual information about significance • Paragraph 55 ‘Sub -panels 11,12,13,14, and 15 consider that the nature of their disciplines is such that the significance of the output may not be evident within the output itself. They therefore invite factual information to be provided (max 100 words) that could include, for example, additional evidence about how an output has gained recognition, led to further developments, or has been applied. They would welcome the inclusion of relevant and verifiable information for all outputs wherever available’

FACT! • The same output, submitted by authors from two different universities, did not always achieve the same score. • The only difference was in the 100 word statement • Conclusion: Time spent on the 100-word statement is time well-spent.

Fact • In the School of EEE, for RAE2008 we reviewed and discussed every single 100-word statement in the presence of the author and the relevant research group members. • EEE achieved it’s highest -ever ranking in RAE2008