Grid Projects @ Belfast e-Science Centre Ron Perrott Queens - PowerPoint PPT Presentation

Grid Projects @ Belfast e-Science Centre Ron Perrott Queen’s University, Belfast {r.perrott@qub.ac.uk} France-UK N+N November 2003 1

Edinburgh Glasgow DL Newcastle Belfast Manchester Cambridge Oxford RL Hinxton Cardiff London Soton France-UK N+N November 2003 2

Belfast e-Science Centre Projects GridCast Television/Radio broadcasting – Business change, resilience, reliability, cost, customisation, interoperability RiskGrid Financial Services – Business change, performance, cost, resilience, reliability, interoperability Geddm High-performance data mining – Performance, cost, business change, resilience, interoperability GeneGrid Bioinformatics – Performance, cost, business change, interoperability GridMil Military infrastructures – Resilience, reliability, performance, interoperability, agility, cost France-UK N+N November 2003 3

The GridCast Project Grid based Broadcast Infrastructures France-UK N+N November 2003 4

The Grid Scenario: The BBC Nations BBC NI, BBC Scotland and BBC Wales • BBC Nations provide customised services in each nation • Television programmes are distributed to BBC Nations from BBC Network (London) using dedicated leased ATM circuits . France-UK N+N November 2003 5

Grid Infrastructure • Technical – High-bandwidth network connections inter- connect broadcast locations. – Network bandwidth means geography is less of an issue. • Organisational – Less centralised France-UK N+N November 2003 6

Overview • To develop a baseline media grid to support a broadcaster – Manage distributed collections of stored media – Prototype security and access mechanisms – Integrate processing and technical resources – Integrate with media standards and hardware • To analyse Quality of Service issues – Analyse remote content distribution infrastructures – Analyse remote service provision – To analyse reactivity, reliability and resilience issues in a grid-based broadcast infrastructure France-UK N+N November 2003 7

Characteristics • Stored media files are Gbytes and increasing – 1 hour ~ 200 Gbytes; distributes 1 petabyte /year • Management and distribution is significant technically • Metadata – location, timings, artists, storage formats etc. is an integral part of broadcast structure • Content is a valuable commodity – access, modification, copying must be controlled • High levels of quality required France-UK N+N November 2003 8

High level view of the Infrastructure Live Content Live Content Live Content Transmitter Transmitter Broadcast Broadcast Broadcast Controller Controller Cable, Satellite, internet Cable, Satellite, internet Controller Output Output Output BBC Scotland Glasgow BBC Scotland Glasgow BBC Scotland Glasgow BBC BBC BBC Scotland Scotland Scotland Schedule Schedule Schedule Controller Controller Network Network Schedule Schedule BBC NI Belfast BBC NI Belfast BBC NI BBC NI Schedule Schedule Controller Controller BBC BBC BBC Wales Wales Live Content Live Content Live Content Wales Schedule Schedule Schedule Controller Controller Controller Broadcast Broadcast Broadcast Output Output Output BBC Network London BBC Network London BBC Wales Cardiff BBC Wales Cardiff BBC Wales Cardiff France-UK N+N November 2003 9

Broadcasting Grid Services Each Broadcast site is defined by its collection of available services BBC Northern •Control services Ireland BBC Scotland •Content services Local Content Controller Live Output High Bandwidth IP Network Live Output BBC Wales Network Content Controller BBC Network France-UK N+N November 2003 10

A Virtualised Infrastructure BBC Scotland Sound Improvement Image Rendering Cluster BBC NI Video Editing Suite High Bandwidth IP Network Subtitling Engine BBC Network BBC Wales France-UK N+N November 2003 11

Scenario • A Network Schedule is defined – This schedule is the framework for Nation schedules • Network Schedules are distributed to BBC Nations – Usually via email • BBC Nations formulate their schedule • A Schedule is Broadcast – By programming local network and content control automation France-UK N+N November 2003 12

Model of Broadcast • Automatic distribution of broadcast schedules – Management of schedule archives – Automatic notification • Content is copied from archives to local content storage – Content distribution defined by schedule France-UK N+N November 2003 13

Broadcast grid issues • Business change – A revised organisational model. Services and resources Each broadcast location gains control….no network schedule . – • Resilience – Resource sharing and no single programme repository – A BBC Nation can be anywhere! • Reliability Use resources available in other BBC sites or from 3 rd party suppliers – • Cost – Better use of resources and less need for backup resources – Less dependence on particular vendors or suppliers • Customisation – Schedule, local resources, local capabilities • Interoperability – Business model facilitates sharing with other broadcasters France-UK N+N November 2003 14

RiskGrid Grid Financial Services France-UK N+N November 2003 15

Background • Financial sector largely cyclical • Risk assessment calculations Investment Bank (US) Investment Bank (EU) Investment Bank (ASIA) … Traders…. … Traders…. … Traders…. Corporate Intranet France-UK N+N November 2003 16

Backgroun d • Depends heavily on calculations for competitive advantage – Compute intensive • Large amount of financial derivatives calculations – Data intensive • Data-access, bottleneck. 2Gb transactions/day on NYSE • Improve performance – Increase accuracy of potential risk in trade – Direct impact on margins – 1% improvement - $$$ France-UK N+N November 2003 17

Architecture Web/Application Mobile/GPRS Custom Presentation OGSA Adapter RiskGrid Middleware Bus Publish Bind Publish Bind Publish Bind OGSA Adapter OGSA Adapter OGSA Adapter Database Domain 1 Domain 2 Utility Computing Historical FTSE market data Portfolio databases France-UK N+N November 2003 18

Issues • Business Change – Attempt to give real time risk assessment • Performance – Harnessing resources to suit the problem • Cost – Use utility and/or unused in-house resources • Resilience – Not restricted to locally available resources • Interoperability – Provide gateways to other services or service provides France-UK N+N November 2003 19

GEDDM G rid E nabled D istributed D ata M ining and Conversion of Unstructured data France-UK N+N November 2003 20

Background • Fuzzy parallelised data-matching and transformation engine • Forensic accounting, banking, anti- terrorist, crime • Clusters: PC, Linux, supercomputers • Large volumes data France-UK N+N November 2003 21

GEDDM: Business Driver • Data sources – numerous structures, formats, locations administrative domains… • Client – US County Court: insider trading litigation case • 45Tb • Email, pdf, weblogs, RDBMS, Word, files … • How to process this data to achieve ? – Meaningful outcomes quickly – Handling multiple formats with common semantic model France-UK N+N November 2003 22

Issues • Performance – Use utility computing to improve performance • Cost – Reduce internal need for high performance computing – Reduce the need to provide on-site services • Business change – Provide a secure online automated service to companies • Resilience – Reduce reliance on internal computing resources • Interoperability – Provide mining engine as a service to other services France-UK N+N November 2003 23

GeneGrid A Virtual Bioinformatics Laboratory France-UK N+N November 2003 24

GeneGrid – Fusion Antibodies Ltd : – Amtec Medical Limited: • US – NIH, Washington • Business drivers: – Develop specialist tissue specific datasets – 3 sites, little collaboration – No dedicated HPC, low bandwidth – Economic advantage (peak demand/supply min) – MicroArray, Seq, Large volumes data… France-UK N+N November 2003 25

GeneGrid • Solution – Grid Service based architecture – Protect confidentiality – Security Model – Genome database integration • Diagnosis – Screening protocols aid customised drug targeting – Gene expression profile • Dataset Mining – NI stable gene pool & complete patient records – Correlation against various target populations France-UK N+N November 2003 26

GeneGrid External Vendor OGSA Gateway OGSA Gateway OGSA Gateway OGSA Gateway HPC A OGSA Middleware Bus (JMS) OGSA Gateway OGSA Gateway OGSA Gateway OGSA Gateway Sequence Data Biological Sequence Data HPC B Databases (Internal) (External) France-UK N+N November 2003 27