Grid Projects @ Belfast e-Science Centre Ron Perrott Queens - - PowerPoint PPT Presentation

▶
grid projects belfast e science centre
SMART_READER_LITE
LIVE PREVIEW

Grid Projects @ Belfast e-Science Centre Ron Perrott Queens - - PowerPoint PPT Presentation

Oct 01, 2022 314 likes •593 views

Grid Projects @ Belfast e-Science Centre Ron Perrott Queens University, Belfast {r.perrott@qub.ac.uk} France-UK N+N November 2003 1 Edinburgh Glasgow DL Newcastle Belfast Manchester Cambridge Oxford RL Hinxton Cardiff London

slide-1
SLIDE 1

France-UK N+N November 2003 1

Grid Projects @ Belfast e-Science Centre

Ron Perrott

Queen’s University, Belfast {r.perrott@qub.ac.uk}

slide-2
SLIDE 2

France-UK N+N November 2003 2

Cambridge Newcastle Edinburgh Oxford Glasgow Manchester Cardiff Soton London Belfast DL RL Hinxton

slide-3
SLIDE 3

France-UK N+N November 2003 3

Belfast e-Science Centre Projects GridCast

Television/Radio broadcasting

– Business change, resilience, reliability, cost, customisation, interoperability

RiskGrid

Financial Services

– Business change, performance, cost, resilience, reliability, interoperability

Geddm

High-performance data mining

– Performance, cost, business change, resilience, interoperability

GeneGrid

Bioinformatics

– Performance, cost, business change, interoperability

GridMil

Military infrastructures

– Resilience, reliability, performance, interoperability, agility, cost

slide-4
SLIDE 4

France-UK N+N November 2003 4

The GridCast Project

Grid based Broadcast Infrastructures

slide-5
SLIDE 5

France-UK N+N November 2003 5

The Grid Scenario: The BBC Nations BBC NI, BBC Scotland and BBC Wales

  • BBC Nations provide

customised services in each nation

  • Television

programmes are distributed to BBC Nations from BBC Network (London) using dedicated leased ATM circuits.

slide-6
SLIDE 6

France-UK N+N November 2003 6

Grid Infrastructure

  • Technical

– High-bandwidth network connections inter- connect broadcast locations. – Network bandwidth means geography is less of an issue.

  • Organisational

– Less centralised

slide-7
SLIDE 7

France-UK N+N November 2003 7

Overview

  • To develop a baseline media grid to

support a broadcaster

– Manage distributed collections of stored media – Prototype security and access mechanisms – Integrate processing and technical resources – Integrate with media standards and hardware

  • To analyse Quality of Service issues

– Analyse remote content distribution infrastructures – Analyse remote service provision – To analyse reactivity, reliability and resilience issues in a grid-based broadcast infrastructure

slide-8
SLIDE 8

France-UK N+N November 2003 8

Characteristics

  • Stored media files are Gbytes and increasing

– 1 hour ~ 200 Gbytes; distributes 1 petabyte /year

  • Management and distribution is significant

technically

  • Metadata – location, timings, artists, storage

formats etc. is an integral part of broadcast structure

  • Content is a valuable commodity – access,

modification, copying must be controlled

  • High levels of quality required
slide-9
SLIDE 9

France-UK N+N November 2003 9

High level view of the Infrastructure

Network Schedule

BBC Network London Controller

BBC NI Belfast

BBC NI Schedule

Controller Transmitter Cable, Satellite, internet BBC Scotland Glasgow Broadcast Output Controller Live Content

BBC Scotland Schedule

BBC Wales Cardiff Controller Live Content Broadcast Output

BBC Wales Schedule Network Schedule

BBC Network London Controller

BBC NI Belfast

BBC NI Schedule

Controller Transmitter Cable, Satellite, internet BBC Scotland Glasgow Broadcast Output Controller Live Content

BBC Scotland Schedule

BBC Wales Cardiff Controller Live Content Broadcast Output

BBC Wales Schedule

BBC Scotland Glasgow Broadcast Output Controller Live Content

BBC Scotland Schedule

BBC Wales Cardiff Controller Live Content Broadcast Output

BBC Wales Schedule

slide-10
SLIDE 10

France-UK N+N November 2003 10

Broadcasting Grid Services

Each Broadcast site is defined by its collection of available services

  • Control services
  • Content services

High Bandwidth IP Network

Local Content Controller

Live Output BBC Northern Ireland

BBC Scotland BBC Wales Controller

Live Output

BBC Network

Network Content

slide-11
SLIDE 11

France-UK N+N November 2003 11

A Virtualised Infrastructure

BBC NI BBC Scotland BBC Wales BBC Network

High Bandwidth IP Network

Image Rendering Cluster Video Editing Suite Subtitling Engine Sound Improvement

slide-12
SLIDE 12

France-UK N+N November 2003 12

Scenario

  • A Network Schedule is defined

– This schedule is the framework for Nation schedules

  • Network Schedules are distributed to BBC

Nations

– Usually via email

  • BBC Nations formulate their schedule
  • A Schedule is Broadcast

– By programming local network and content control automation

slide-13
SLIDE 13

France-UK N+N November 2003 13

Model of Broadcast

  • Automatic distribution of broadcast

schedules

– Management of schedule archives – Automatic notification

  • Content is copied from archives to local

content storage

– Content distribution defined by schedule

slide-14
SLIDE 14

France-UK N+N November 2003 14

Broadcast grid issues

  • Business change

– A revised organisational model. Services and resources – Each broadcast location gains control….no network schedule.

  • Resilience

– Resource sharing and no single programme repository – A BBC Nation can be anywhere!

  • Reliability

– Use resources available in other BBC sites or from 3rd party suppliers

  • Cost

– Better use of resources and less need for backup resources – Less dependence on particular vendors or suppliers

  • Customisation

– Schedule, local resources, local capabilities

  • Interoperability

– Business model facilitates sharing with other broadcasters

slide-15
SLIDE 15

France-UK N+N November 2003 15

RiskGrid

Grid Financial Services

slide-16
SLIDE 16

France-UK N+N November 2003 16

Background

  • Financial sector largely cyclical
  • Risk assessment calculations

Corporate Intranet Investment Bank (US)

…Traders….

Investment Bank (EU)

…Traders….

Investment Bank (ASIA)

…Traders….

slide-17
SLIDE 17

France-UK N+N November 2003 17

Background

  • Depends heavily on calculations for competitive

advantage

– Compute intensive

  • Large amount of financial derivatives calculations

– Data intensive

  • Data-access, bottleneck. 2Gb transactions/day on NYSE
  • Improve performance

– Increase accuracy of potential risk in trade – Direct impact on margins – 1% improvement - $$$

slide-18
SLIDE 18

France-UK N+N November 2003 18

Architecture

Web/Application Mobile/GPRS Custom RiskGrid Middleware Bus

Database OGSA Adapter

Historical FTSE market data Portfolio databases

Domain 1 OGSA Adapter Domain 2 OGSA Adapter OGSA Adapter Presentation

Publish Bind Publish Bind Publish Bind Utility Computing

slide-19
SLIDE 19

France-UK N+N November 2003 19

Issues

  • Business Change

– Attempt to give real time risk assessment

  • Performance

– Harnessing resources to suit the problem

  • Cost

– Use utility and/or unused in-house resources

  • Resilience

– Not restricted to locally available resources

  • Interoperability

– Provide gateways to other services or service provides

slide-20
SLIDE 20

France-UK N+N November 2003 20

GEDDM

Grid Enabled Distributed Data Mining and Conversion of Unstructured data

slide-21
SLIDE 21

France-UK N+N November 2003 21

Background

  • Fuzzy parallelised data-matching and

transformation engine

  • Forensic accounting, banking, anti-

terrorist, crime

  • Clusters: PC, Linux, supercomputers
  • Large volumes data
slide-22
SLIDE 22

France-UK N+N November 2003 22

GEDDM: Business Driver

  • Data sources

– numerous structures, formats, locations administrative domains…

  • Client

– US County Court: insider trading litigation case

  • 45Tb
  • Email, pdf, weblogs, RDBMS, Word, files …
  • How to process this data to achieve ?

– Meaningful outcomes quickly – Handling multiple formats with common semantic model

slide-23
SLIDE 23

France-UK N+N November 2003 23

Issues

  • Performance

– Use utility computing to improve performance

  • Cost

– Reduce internal need for high performance computing – Reduce the need to provide on-site services

  • Business change

– Provide a secure online automated service to companies

  • Resilience

– Reduce reliance on internal computing resources

  • Interoperability

– Provide mining engine as a service to other services

slide-24
SLIDE 24

France-UK N+N November 2003 24

GeneGrid

A Virtual Bioinformatics Laboratory

slide-25
SLIDE 25

France-UK N+N November 2003 25

GeneGrid

– Fusion Antibodies Ltd : – Amtec Medical Limited:

  • US – NIH, Washington
  • Business drivers:

– Develop specialist tissue specific datasets – 3 sites, little collaboration – No dedicated HPC, low bandwidth – Economic advantage (peak demand/supply min) – MicroArray, Seq, Large volumes data…

slide-26
SLIDE 26

France-UK N+N November 2003 26

GeneGrid

  • Solution

– Grid Service based architecture – Protect confidentiality – Security Model – Genome database integration

  • Diagnosis

– Screening protocols aid customised drug targeting – Gene expression profile

  • Dataset Mining

– NI stable gene pool & complete patient records – Correlation against various target populations

slide-27
SLIDE 27

France-UK N+N November 2003 27

GeneGrid

OGSA Middleware Bus (JMS)

OGSA Gateway OGSA Gateway OGSA Gateway

External Vendor

OGSA Gateway Sequence Data

(Internal) (External)

OGSA Gateway Biological Databases OGSA Gateway Sequence Data OGSA Gateway HPC A OGSA Gateway HPC B