[PPT] - Cees de Laat GLIF.is founding member 100000 flops/byte The PowerPoint Presentation

SLIDE 1

Cees de Laat

GLIF.is founding member
OFC 2009 – Grid vs Cloud Computing and Why This Should Concern the Optical Networking Community

Challenges in Enabling Grid Computing

ver Optical Networks

SLIDE 2

100000 flops/byte

SLIDE 3

Telescopes Input nodes Correlator nodes Output node

.....

To equal the hardware correlator we need: 16 streams of 1Gbps 16 * 1Gbps of data 2 Tflops CPU power

2 TFlop / 16 Gbps =

1000 flops/byte THIS IS A DATA FLOW PROBLEM !!!

SCARIe: a research project to create a Software Correlator for e-VLBI. VLBI Correlation: signal processing technique to get high precision image from spatially distributed radio-telescope.

The SCARIe project

SLIDE 4

The “Dead Cat” demo SC2004 & iGrid2005

SC2004,

Pittsburgh,

Nov. 6 to 12, 2004

iGrid2005, San Diego,

sept. 2005

Produced by: Michael Scarpa Robert Belleman Peter Sloot Many thanks to: AMC SARA GigaPort UvA/AIR Silicon Graphics, Inc. Zoölogisch Museum

SLIDE 5

US and International OptIPortal Sites

NCMIR

SIO UIC USGS EDC TAMU UCI SARA KISTI NCSA & TRECC CALIT2

AIST

RINCON & Nortel

SLIDE 6

IJKDIJK

SLIDE 7

Sensor grid: instrument the dikes

30000 sensors (microphones) to cover all Dutch dikes

First controlled breach occurred on sept 27th ‘08:

SLIDE 8

e-Food

e-BioScience
e-Biobanking
e-COAST
e-Ecology
e-Data-

intensive sciences

.....
Virtual Laboratory
generic e-Science services
High Performance & Distributed Computing
Web & Grid services

SLIDE 9

GLIF 2008

Visualization courtesy of Bob Patterson, NCSA Data collection by Maxine Brown.

SLIDE 10

SLIDE 11

Management Visualisation Mining Web2.0 Media Backup Security

NetherLight

Meta CineGrid Medical TV Gaming Conference RemoteControl Clouds Distributed Simulations Predictions EventProcessing StreamProcessing Workflow DataExploration

SLIDE 12

Who (just to name a few) working Interfaces - similar solutions addressing the previous challenges

CANARIE - UCLP
ESnet = OSCARS
G-LAMBDA - GNS-WSI
HPDM - VLAN based lightpaths
NORTEL - DRAC
UvA - Token based service, NDL, etc
Phosphorous - G2MPLS, UCLP2, NRPS - > NSP
GN2 - JRA3 - AutoBahn - IDM
I2 - DRAGON/ HOPI - DCN
FermiLab - Lambda Station
DoE - LambdaStation, TeraPaths

Slide: Gigi Karmous Edwards

SLIDE 13

Harmony architecture (I)

SLIDE 14

SLIDE 15

15

Grid-aware GMPLS (G2MPLS) for Grid Network Services Two models for the layering between Grid and Network resources

G2MPLS overlay model

different scope with respect to the IETF GMPLS Overlay & Peer

G2MPLS integrated model

SLIDE 16

My view

GLIF

GN2/3 Federica Phosphorus Onelab Planetlab FIRE etc.

OSI 7 6 5 4 3 2 1

needs

repeatable experiments

needs QoS &

lightpaths

needs capacity

and capability

needs

infrastructure descriptions

SLIDE 17

TeraThinking

What constitutes a Tb/s network?
CALIT2 has 8000 Gigabit drops ?->? Terabit Lan?
look at 80 core Intel processor

– cut it in two, left and right communicate 8 TB/s

think back to teraflop computing!

– MPI turns a room full of pc’s in a teraflop machine

massive parallel channels in hosts, NIC’s
TeraApps programming model supported by

– TFlops

>

MPI / Globus – TBytes

>

OGSA/DAIS – TPixels

>

SAGE – TSensors

>

LOFAR, LHC, LOOKING, CineGrid, ... – Tbit/s

>

?

ref Larry Smarr & CdL

SLIDE 18

The Problem

A G F E D C B H

I want HC and AB Success depends on the order Wouldn’t it be nice if I could request [HC, AB, ...]

?

SLIDE 19

Network Description Language

Object Subject Predicate

From semantic Web / Resource Description Framework.
The RDF uses XML as an interchange syntax.
Data is described by triplets:

Location Device Interface Link

name description locatedAt hasInterface connectedTo capacity encodingType encodingLabel

Object Subject

Subject

Object Subject Object Subject Object Subject

SLIDE 20

Network Description Language

Choice of RDF instead of XML syntax Grounded modeling based on G0805 description:

Article: F. Dijkstra, B. Andree, K. Koymans, J. van der Ham, P. Grosso,

C. de Laat, "A Multi-Layer Network

Model Based on ITU-T G.805"

SLIDE 21

NDL + PROLOG

Reason about

graphs

Find sub-graphs

that comply with rules Research Questions:

order of requests
complex requests
Usable leftovers

SLIDE 22

139.63 .145.0 139.63 .145.1 139.63 .145.15 139.63 .145.16 139.63 .145.17 139.63 .145 .18 139.63 .145.2 139.63 .145.3 139.63 .145.31 139.63 .145.32 139.63 .145.33 139.63 .145.34 139.63 .145.38 139.63 .145.40 139.63 .145.41 139.63 .145.42 139.63 .145.43 139.63 .145.44 139.63 .145 .45 139.63 .145.46 139.63 .145.49 139.63 .145.50 139.63 .145.51 139.63 .145 .52 139.63 .145.63 139.63 .145 .64 139.63 .145.65 139.63 .145.66 139.63 .145.68 139.63 .145.69 139.63 .145.70 139.63 .145 .71 139.63 .145.72 139.63 .145.73 139.63 .145.74 139.63 .145.75 139.63 .145.79 139.63 .145 .81 139.63 .145.82 139.63 .145 .83 139.63 .145.84 139.63 .145.85 139.63 .145 .86 139.63 .145.87 139.63 .145.88 139.63 .145.94 192.168 .0.1 192.168 .0.2 192.168 .0.3 192.168 .0.4 192.168 .0.5 192.168 .0.6 192.168 .1.1 192.168 .1.2 192.168 .1.3 192.168 .1.4 192.168 .2.1 192.168 .2.2 192.168 .2.3 192.168 .2.4

Visualisation Initialization and BFS discovery of NEs

95.9 94.5 95.8 96. 95.9 99.9 99. 100. 100. 94.5 99.9 99.9 100. 95.8 99. 96. 99.9 100. 100. 99.8 100. 100. 98.9 99.8 100. 100. 100. 100. 100. 98.9 101. 100. 101. 100. 100. 100. 139.63.145.94 192.168.0.1 192.168.0.2 192.168.0.3 192.168.0.4 192.168.0.5 192.168.0.6 192.168.1.1 192.168.1.2 192.168.1.3 192.168.1.4 192.168.2.1 192.168.2.2 192.168.2.3 192.168.2.4

Network flows using real-time bandwidth measurements

nodePath = ConvertIndicesToNodes[ ShortestPath[ g, Node2Index[nids,"192.168.3.4"], Node2Index[nids,"139.63.77.49"]], nids]; Print["Path: ", nodePath]; If[NetworkTokenTransaction[nodePath, "green"]==True, Print["Committed"], Print["Transaction failed"]]; Path: {192.168.3.4,192.168.3.1,139.63.77.30,139.63.77.49} Committed

Transaction on shortest path with tokens

Topology matters can be dealt with algorithmically Results can be persisted using a transaction service built in UPVN

Needs["WebServices`"] <<DiscreteMath`Combinatorica` <<DiscreteMath`GraphPlot` InitNetworkTopologyService["edge.ict.tno.nl"] Available methods: {DiscoverNetworkElements,GetLinkBandwidth,GetAllIpLinks,Remote, NetworkTokenTransaction} Global`upvnverbose = True; AbsoluteTiming[nes = BFSDiscover["139.63.145.94"];][[1]] AbsoluteTiming[result = BFSDiscoverLinks["139.63.145.94", nes];][[1]] Getting neigbours of: 139.63.145.94 Internal links: {192.168.0.1, 139.63.145.94} (...) Getting neigbours of:192.168.2.3 Internal links: {192.168.2.3}

Mathematica enables advanced graph queries, visualizations and real- time network manipulations on UPVNs

ref: Robert J. Meijer, Rudolf J. Strijkers, Leon Gommans, Cees de Laat, User Programmable Virtualiized Networks, accepted for publication to the IEEE e-Science 2006 conference Amsterdam.

SLIDE 23

Interactive programmable networks

SLIDE 24

Multi Layer Service Architecture

Network

layers

Application layers

Use Interface Control Interface (protocols API’s)

Network Service Application

5b of 6

SLIDE 25

n.a.v. interview met Kees Neggers (SURFnet) & Cees de Laat (UvA)

BSIK projects

GigaPort &

VL-e / e-

Science cookreport.com

SLIDE 26

Questions ?

Thanks: Paola Grosso & Jeroen vd Ham & Freek Dijkstra & team for several of the slides.

A Declarative Approach to Multi-Layer Path Finding Based on Semantic Network Descriptions.

http://delaat.net:/~delaat/papers/declarative_path_finding.pdf