Evaluation of Impact of e-Infrastructure Investments : a UK view - - PowerPoint PPT Presentation

evaluation of impact of e infrastructure investments a uk
SMART_READER_LITE
LIVE PREVIEW

Evaluation of Impact of e-Infrastructure Investments : a UK view - - PowerPoint PPT Presentation

Evaluation of Impact of e-Infrastructure Investments : a UK view point Jane Nicholson, Head of Research Infrastructure and International, Engineering and Physical Sciences Research Council, UK Over view What is EPSRC and how do we define


slide-1
SLIDE 1

Evaluation of Impact

  • f

e-Infrastructure Investments : a UK view point

Jane Nicholson, Head of Research Infrastructure and International, Engineering and Physical Sciences Research Council, UK

slide-2
SLIDE 2

Over view

  • What is EPSRC and how do we define impact
  • Approaches to evaluating the impact of investments in research
  • Examples of impact evidence
  • Challenges for measuring impact in e-Infrastructures
  • Conclusions
slide-3
SLIDE 3

What is EPSRC

One of the seven UK Research Councils that work together as RCUK EPSRC is the main UK government agency for funding research and training in engineering and the physical sciences EPSRC invests around £850m a year in world class research and training to promote future economic development and improved quality

  • f life

EPSRC is key to tackling challenges such as energy security, climate change, our ageing population, crime and economic resilience EPSRC is generating the fundamental knowledge and skilled people essential to business, government and other research organisations

slide-4
SLIDE 4

How we see the future

Our strategy has 3 clear goals Delivering impact Shaping capability Developing leaders

slide-5
SLIDE 5

Delivering impact

We will ensure excellent research and talented people deliver maximum impact for the health, prosperity and sustainability of the UK We will build strong partnerships with

  • rganisations that can capitalise on our

research and inform our direction We will promote excellence and impact, and ensure it is visible to all

slide-6
SLIDE 6

Definitions

What is impact ?

slide-7
SLIDE 7
slide-8
SLIDE 8

WE SUPPORT AROUND

8,000

ACADEMIC RESEARCHERS OUR £4BN PORTFOLIO HAS LEVERAGED AN ADDITIONAL

£700m

IN BUSINESS AND COLLABORATOR CONTRIBUTIONS WE HAVE SUPPORTED

12,300

HIGHLY SKILLED POSTGRADUATES TO INDUSTRY AND THE PUBLIC SECTOR OVER THE PAST DECADE AROUND

43%

OF OUR PORTFOLIO IS MULTIDISCIPLINARY SECTORS DEPENDENT ON ENGINEERING AND THE PHYSICAL SCIENCES ACCOUNT FOR

30%

OF GDP AND

88%

OF MANUFACTURING EXPORTS WE COLLABORATE WITH MORE THAN

2,300

COMPANIES AND MORE THAN

100

PUBLIC BODIES INCLUDING 12 GOVERNMENT DEPARTMENTS

The facts and figures

slide-9
SLIDE 9

e-Infrastructures

slide-10
SLIDE 10

EPSRC’S role in support of e Infrastructure

RCUK Managing agent for provision of shared national HPC systems Ran core programme of RCUK e Science programme Have previously joint funded with JISC the development of the National Grid Service (UK NGI in EGI) Fund software development activities in Engineering and Physical Sciences Fund research programmes of users of e-infrastructures

slide-11
SLIDE 11

Support for e-Infrastrcuture : Hardware example UK National Supercomputing Service

HECToR – the UK National HPC service. Based on a 366 TF Cray XT6 system plus 114TF Cray XT4 System. Operated by EPCC, University

  • f Edinburgh with science support

provided by NAG Ltd. Funded by EPSRC,NERC and BBSRC

slide-12
SLIDE 12

Support for e-Infrastructure :Software Example Software Sustainability Institute

EPSRC funds the The Software Sustainability Institute (SSI). Run by researchers from the University of Edinburgh, Manchester and Southampton , it works in partnership with research communities to identify key software that needs to be sustained. SSI’s vision is to make software useful for future generations of researchers by improving usability, maintainability and quality. This increases research productivity by enabling more people to use software, delivers added value by leveraging our expertise and connections, and ensures researchers can continue to use their chosen software as a cornerstone of their research. SSI works with groups to facilitate the self-sustainability of research software within the communities that develop and use it, drawing on a skilled team based at the Universities of Edinburgh, Manchester and Southampton. This team includes talented individuals with a breadth of experience in software development, project and programme management, publicity and dissemination, and community engagement.

slide-13
SLIDE 13

Approaches to Evaluation of Impact

  • Bibliometrics
  • Collection of Statistics
  • Feedback from Collaborators
  • Case Studies
  • Peer review e.g. Theme days, International Reviews
slide-14
SLIDE 14

Bibliometrics : Supporting excellence

EPSRC sponsored researchers achieve a higher citation rate of 1.6 compared to the UK average

  • f 1.4 and a world average
  • f 1.0

Figures from 2009/10 Annual Report

slide-15
SLIDE 15

Approaches to Evaluation of Impact

  • Bibliometrics
  • Collection of Statistics
  • Impact on Collaborators
  • Case Studies
  • Peer review e.g. Theme days, International Reviews
slide-16
SLIDE 16

Support of next generation skills

Around 12,300 of the highly-skilled postgraduates supported by EPSRC have moved on into careers in industry and the public sector in the past decade Career Path studies - comparision of average salaries vs. PhD salaries First Destination statistics for PhD students – working with learned societies to track career paths Number of research students who have used facilities – e.g, use of National Grid Service

Figures from 2009/10 Annual Report

slide-17
SLIDE 17

Impact from excellence

Nearly 40% of the research and training we fund is directly involved in collaboration with industry To date, 132 follow-on projects worth £11.5 million have been supported Our Partnership with the Technology Strategy Board has led to over 270 research projects being sponsored and a joint portfolio of over £250 million Our 28 Strategic Partners bring in £77 million of additional funding

Figures from 2009/10 Annual Report The prototype magnetometer.

slide-18
SLIDE 18

Statistics on use of e- Infrastrcuture

HECToR has 1400 UK academic registered users :40% of the users from the Engineering and Physical sciences area have industrial collaborators Over 800 attendees on HECToR training courses to date , OMII has delivered 111 training events to 2000+ researchers The Taverna workflow tool supported by the e-Science programme has had 65,000 downloads and has been used by 350 organisations and 23 companies in 35 countries. The EPSRC funded OMII has supported software deployment on NGS sites plus on Lyceum, Legion,ECDF and Iridis campus infrastructures plus on TeraGrid, NAREGI, D-Grid and EGEE

slide-19
SLIDE 19

Approaches to Evaluation of Impact

  • Bibliometrics
  • Collection of Statistics
  • Impact on Collaborators
  • Case Studies
  • Peer review e.g. Theme days, International Reviews
slide-20
SLIDE 20

Collaboration Example with regional impact

Funding to the University of Newcastle by the e-Science Programme led to substantial regional impact through collaboration with Arjuna

  • Technologies. Seven PhDs and thirty MScs from Newcastle University

have worked for Arjuna, one PhD, many MScs and a Business Development Manager have returned from Arjuna to Newcastle University, and the institutionalised network of knowledge transfer to local companies has led to “cutting-edge technologies:” e.g. web services, grids and clouds and the regional involvement of Red Hat, Amazon and Microsoft. Newcastle University estimates that Arjuna/RedHat have contributed ~£16M GVA to the regional economy.

slide-21
SLIDE 21

Collaboration example with spin out

Nationally, the Distributed Aircraft Maintenance Environment Project (DAME) partnered with Rolls-Royce, Data Systems and Solutions and Cybula Ltd to use e-Science to reduce engine maintenance times and to improve the interoperation of the maintenance team. The technologies developed are now used on Rolls-Royce Trent engines and the result was a spin-off company: Oxford Bio-Signals (OBS).

slide-22
SLIDE 22

Approaches to Evaluation of Impact

  • Bibliometrics
  • Collection of Statistics
  • Impact on Collaborators
  • Case Studies
  • Peer review e.g. Theme days, International Reviews
slide-23
SLIDE 23

Case Studies of Impact – Example 1

Two EPSRC funded EngD students from the University of Southampton developed the technology used in the building of Amy William’s gold medal winning skeleton sled

Credit: Sarah Winterflood/UK Sport

slide-24
SLIDE 24

Case Study Example 2: Reducing Noise Pollution in Aircraft

Aircraft pressure relief valves are used to protect the fuel tanks of wide-body civil aircraft from over-pressurization. The relief valve

  • utlet is typically in the shape of a cylindrical hole, or cavity, cut in the

underside side of the wing skin. At typical approach speeds of around Mach 0.3, the flow past the cylindrical cavity can become unsteady. This produces an unwanted tonal contribution to the airframe landing

  • noise. A numerical investigation of this phenomenon was performed

in consultation with Airbus France, as part of the EU programme AeroTraNet. Dept of Engineering, University of Leicester, UK

slide-25
SLIDE 25

Case Study Example 3 : Improving the modelling of Cardiac Arrythmia

Medical science is increasingly turning to computational models to study the possible effects of drugs and surgical interventions, before moving on to patient trials. One active area of research is in heart modelling. Researchers at the Oxford e-Research Centre (OeRC) and EPCC (University of Edinburgh), will be using HECToR to optimise heart-modelling

  • software. If successful, this work will enable much greater

integration of computer simulation with the operating theatre and could, ultimately, lead to personalised medicine.

slide-26
SLIDE 26

Case study Example 4 : Software development benefits for basic science

NAG HPC experts, working under Research Council Funded NAG’s Computational Science and Engineering (CSE) support service for HECToR, the UK’s national academic supercomputing facility, have

  • ptimised a Quantum Monte‐Carlo application for multicore

architectures, resulting in a performance increase of a factor of four, potentially saving £760k in computing resources on HECToR for a single one year research project, with several million pounds of savings when applied to future research on HECToR and other supercomputers used to run CASINO.

slide-27
SLIDE 27

Case Study Example 5 : Software development benefits for climate research

Dr Andrew Coward, who is the manager of the Global Ocean Modelling Consortium on HECToR estimated that their group used around 6M AUs running NEMO last year. Reducing the wall clock time of NEMO by up to 25% will result in a saving in notional cost of AUs by as much as £95,000 per year (up to £400,000 for the remainder of the service), for

  • nly six months of person effort.

Other consortia using NEMO on HECToR have used around 40M AUs

  • ver the same period. If the code modifications are accepted into the

main code base, then these other users could benefit too, leading to the possibility of multi‐million pound savings overall.

slide-28
SLIDE 28

Approaches to Evaluation of Impact

  • Bibliometrics
  • Collection of Statistics
  • Feedback from Collaborators
  • Case Studies
  • Peer review e.g. Theme days, International Reviews
slide-29
SLIDE 29

Peer Review Approach Example

slide-30
SLIDE 30

An example : International Review of UK e Science programme

UK e Science Programme funding for projects ran from 2001- 2007 Total of £250+M investment made by six UK Research Councils. Review planned for after close of programme to be able to evaluate impact and longevity of actions. Review took place December 2009. International review approach chosen given breadth and range of research supported plus desire for independent view of progress

slide-31
SLIDE 31

Approach

International Panel formed of 16 academics and industrialists Data and evidence of activity collected to produce a comprehensive data summary. Panel spent a week in UK in December 2009 – interacting with researchers, stakeholders and produced a report of findings and recommendations. RCUK grateful to Prof. Dan Atkins ( panel chair) and the Panel for the time and energy they put into the review and report preparation.

slide-32
SLIDE 32

Terms of Reference for e-Science International Review Panel

The role of the review panel was to: assess the impact of the programme on research areas nationally and internationally, on broader wealth creation and quality of life, and on e- Science itself; assess and compare the quality of the UK research base in e-Science with the rest of the world via triangulation of data, panel and community perception; comment on the added value of this programme; and present findings and recommendations about the strength, weakness and opportunities for the future to the Research community and Councils.

slide-33
SLIDE 33

Specific question on impact posed to panel ?

What has been the impact (accomplished and potential) of the UK e- Science Programme? To what extent has the research undertaken through the e-Science Programme benefited the UK economy and our global competitiveness? To what extent did the research undertaken through the UK e-Science Programme address key technological/societal challenges? What evidence is there to show that the UK e-Science Programme supported the development of a creative and adventurous research base and portfolio?

slide-34
SLIDE 34

2009 International Review of e-Science

slide-35
SLIDE 35

Findings of review

The Panel identified three ways in which projects had a substantial economic impact : Direct involvement of industry from the inception of the project University led linkages with industry during and after development of new technologies Entreprennurial investigators The panel also assessed science quality, outcomes for human capital and made recommendation for the future.

slide-36
SLIDE 36

Outcome of Review

  • Peer review identification of direct impacts :

Programme attracted £20 million in industrial collaboration and £7.1 M cash and in kind industry transfer Programme resulted in 138 stakeholder collaborations 30 licenses or patents 14 spin off companies 103 key results taken up by industry

  • Secondary impacts

Improvements in prediction of extratropical storms by numerical weather prediction using storm identification and tracking software led to a reduction of in the flood and wind related damage due to storms Comment in other areas e.g.healthcare still to early to identify the full effect. “

slide-37
SLIDE 37

Challenges for assessing impact of e- infrastructures

Knowing who your “users” are. Measuring the importance of the contribution of the use of e-infrastructure to the final science output. Identifying the added value of the infrastructure investment – benefit of 1PT vs. 900TF computer etc. Capturing the data over long time periods

slide-38
SLIDE 38

Summary

For effective evaluation of impact need to use a range of approaches to be able to capture the benefit of investment in e infrastructures For both quantitative and qualitative information need to start capturing this from the start of projects and programmes – trying to find information retrospectively is hard and extremely time consuming Need to include both quantitative and qualitative metrics to be able to demonstrate the whole range of impacts Need to collect this information often beyond the life of project so an on going activity For e-infrastructure can exploit e-tools to capture data if plan for this in advance.

slide-39
SLIDE 39

And finally

The more you know the user base of the infrastructure the easier identifying evidence of impact be it academic, social or economic will be. Reports and Information on EPSRC available at http://www.epsrc.ac.uk and examples of case studies at http://www.impactworld.org.uk

slide-40
SLIDE 40

Thank you for your attention