I nternational research assessm ent revisited - A com parison betw - - PowerPoint PPT Presentation

1

I nternational research assessm ent revisited - A com parison betw een the research perform ance of KTH and selected UK universities

• Folke Snickars, Faculty Board, KTH
• Björn Marklund, Rector’s Office, KTH
• Tom Letcher, Evidence Ltd, UK

2

KTH Research Performance Assessment Project

• Evidence Ltd has experience from UK RAE
• Interesting method to bench learn in research assessment
• Access to data in UK Higher Education Research Yearbook
• KTH has asked Evidence to create test KTH footprints
• Comparison is with 10 selected UK universities as peers
• Aim is to be proactive concerning research assessment
• Bibliometrics foreseen as part of national funding system
• Methods and indicators for an internal allocation system
• Project may serve as model for School dialogues

3

Performance Indicators

• Research degrees Licentiate
• Research degrees PhD
• Research funding

–Government grants –Research Council grants –Government Agency grants –Foundation grants –Private sector grants –EU

• Staff

–Professors –Associate professors –Research associates –Senior and junior lecturers –Research staff –Funded doctorate students

• Bibliometrics

–Publications –Rebased impact

4

Subjects for Assessment

• KTH
• Mathematics
• Chemistry
• Information Technology
• Engineering Physics
• Electrical Engineering,

Electronics and Photonics

• Chemical Engineering
• Biotechnology
• Engineering Mechanics
• Material Science
• Civil Engineering and

Architecture

• Engineering and Business

management

• Environmental Engineering
• Urban Studies
• Engineering Science (other)
• UK HEA
• Mathematics
• Chemistry
• Computer Science
• Physics
• Electrical and Electronic

Engineering

• Chemical Engineering
• Biological Science
• Mechanical Engineering
• Metallurgy and materials
• Civil Engineering
• Built environment
• Business and management

Studies

• Environmental Science
• Town and Country Planning
• General Engineering

5

Background Data

• KTH Annual Report
• Data from Statistics Sweden (SCB)
• ISI Thomson University Publication Database 1996-2005
• UK RAE 2001-2004

– Imperial College, Sheffield, Leeds, Southhampton, Birmingham, Manchester, Surrey, Edinburgh, Strathclyde, College London

6

Three Perspectives

Publishers Financiers Universities

7

KTH

1 2 3 4 5 6 7 8

Tot al Per FTE

Physics vs Engineering Physics

Univ Leeds

1 2 3 4 5 6 7 8

Imperial Coll London

1 2 3 4 5 6 7 8

1 Research active staff 2 Research fellows 3 Research assistans 4 Research degrees (PhD) 5 Studentships 6 RGCI 7 Articles in ISI journals 8 Rebased Impact

8

Civil Engineering

KTH

1 2 3 4 5 6 7 8

Tot al Per FTE

Imperial Coll London

1 2 3 4 5 6 7 8

Univ Leeds

1 2 3 4 5 6 7 8

1 Research active staff 2 Research fellows 3 Research assistans 4 Research degrees (PhD) 5 Studentships 6 RGCI 7 Articles in ISI journals 8 Rebased Impact

9

Mechanical, Aeronautical and Manufacturing Engineering

KTH

1 2 3 4 5 6 7 8

Tot al Per FTE

Imperial Coll London

1 2 3 4 5 6 7 8

Univ Leeds

1 2 3 4 5 6 7 8

1 Research active staff 2 Research fellows 3 Research assistans 4 Research degrees (PhD) 5 Studentships 6 RGCI 7 Articles in ISI journals 8 Rebased Impact

10

Metallurgy and Materials

KTH

1 2 3 4 5 6 7 8

Tot al Per FTE

Imperial Coll London

1 2 3 4 5 6 7 8

Univ Leeds

1 2 3 4 5 6 7 8

1 Research active staff 2 Research fellows 3 Research assistans 4 Research degrees (PhD) 5 Studentships 6 RGCI 7 Articles in ISI journals 8 Rebased Impact

11

Physics

50 100 150 200 250 300 KTH Birmingham Edinburgh Imperial Leeds Manchester Sheffield Southampton Strathclyde Surrey London AcademicStaff ResearchActiveStaff

12

Physics

0,5 1 1,5 2 2,5 3 3,5 4 KTH Birmingham Edinburgh Imperial Leeds Manchester Sheffield Southampton Strathclyde Surrey London Yearly publications 2001-2005/R&D active staff 2004

13

Physics

0,00 0,50 1,00 1,50 2,00 2,50 3,00 KTH Birmingham Edinburgh Imperial Leeds Manchester Sheffield Southampton Strathclyde Surrey London AverageRebasedImpact2001_2005 AverageRebasedImpact1996_2001

14

Physics

1000 2000 3000 4000 5000 6000 7000 8000 KTH Birmingham Edinburgh Imperial Leeds Manchester Sheffield Southampton Strathclyde Surrey London Articles2001_2005 Articles1996_2001

15

• Number of applicants for academic positions
• Number of teachers with doctoral degree
• Number of female teachers
• Number of applicants to undergraduate programs
• Student’s average grading of courses
• Throughput of undergraduate students after three semesters
• Inbound and outbound undergraduate exchange students
• Number of graduates holding a position after six months
• Study time for completing PhD degree
• Average impact value of publications
• Number of citations
• Research income from competitive funding agencies
• Income from commissioned research and education

KTH School Indicators

16

Some Lessons Learned

• Possible to make comparison of publication work
• Comaprable data on staff and funding with difficulty
• Shows the need to be careful with information systems
• Creates a background for internal work on indicators
• Shows the need to be knowledgeable in the subject areas
• KTH present in broad range of subjects
• KTH shows particular strength in core engineering fields
• KTH publication performance varies over time
• KTH exhibits same level of productivity as UK peers
• Productivity varies substantially across universities in field