400G Demonstrator for ISC 13 Post ISC phase 2013 Wolfgang Wnsch, - - PowerPoint PPT Presentation

▶
400g demonstrator for isc 13
SMART_READER_LITE
LIVE PREVIEW

400G Demonstrator for ISC 13 Post ISC phase 2013 Wolfgang Wnsch, - - PowerPoint PPT Presentation

Jan 01, 2023 416 likes •980 views

400G Demonstrator for ISC 13 Post ISC phase 2013 Wolfgang Wnsch, Technische Universitt Dresden Eduard Beier, T-Systems International 1 Agenda Partner Purpose Project Structure Topology just click! Turbine Development

slide-1
SLIDE 1

1

400G Demonstrator for ISC ‘13

Post ISC phase 2013 Wolfgang Wünsch, Technische Universität Dresden Eduard Beier, T-Systems International

slide-2
SLIDE 2

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13

Agenda

  • Partner
  • Purpose
  • Project Structure
  • Topology
  • Turbine Development
  • Climate Computing
  • Service Recipient Relations
  • Data Path
  • Throughput Targets
  • The Big Picture
  • Project Lifetime
  • Timeline
  • DATE
  • Test items

just click! (hype perlinked)

slide-3
SLIDE 3

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 3

Partner

Back to Agenda

slide-4
SLIDE 4

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 4

Purpose The purpose of the project is: to demonstrate, that bandwidth beyond 100GBit/s is feasible and useful

Back to Agenda

slide-5
SLIDE 5

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 5

Project Structure

Back to Agenda Project Boar ard

Prof.

  • f. Dr. A. Bod
  • de / Prof.
  • f. Dr

Dr .W. Nagel

  • Dr. A. Kluge
  • F. Schneider

Prof.

  • f. Dr. W. Ge

Gentzsc sch

  • R. Wieneke
  • M. Zappolino
  • Dr. A. Ge

Geiger

  • M. Roose

sen

  • M. Fuchs
  • A. Clauberg
  • T. Wese

selowsk ski Jan Heichler

Projec ect Man anag agem emen ent

E.Beier

  • W. Wünsc

ünsch n.n.

WP1

Perfo forma mancetT tTests sts Andy Georgi System Performance Metering

WP2

Parallel Filesyste tems ms Klaus Gottsc

  • ttscha

halk Filesystem Optim izing

WP3

Server & Sto torage Beier/Wünsc sch Server & Storage Project Management

WP4

Transp nsport Mask skos / Ma Mayer Planning / / Engineering WDM Project Management

WP5

Layer 2/3 Dani niel Nowara Router Project Management

WP7

Applicati tions ns Ferdinand Jami mitz tzky Applications Project Management

WP6

SDN & NFV Ralf f Braun SDN & NFV & Security

WP8

Public Relati tions ns Udo Schä häfe fer Project Marketing

slide-6
SLIDE 6

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 6

WP1: Performance Tests

  • Performance of subsystems (e.g. storage) and total performance

measurements

  • Feedback for subsystem optimization
  • Conformance to measurement standards
  • Input for publications
  • WP lead:

ad: Andy Georgi Back to Projec ect Structure

slide-7
SLIDE 7

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 7

WP2: Parallel File System

  • Planning, roll out, optimization and operation of the Parallel File system in

coordination with other WPs and partners

  • Configure and parameterize the Parallel File System (e.g. TCP buffers)
  • Coordinate the communication between Clusters, File System and Network (IP

Concept)

  • Input for publications
  • WP lead

ad: : Klau aus Gottschal alk Back to Projec ect Structure

slide-8
SLIDE 8

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 8

WP3: Server & Storage & IB & etc

  • Planning, roll out, optimization and operation of the server, storage and

Infiniband infrastructure in coordination with other WPs and partners

  • Input for publications
  • WP lead

ad : Projekt Man anag agement Back to Projec ect Structure

slide-9
SLIDE 9

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 9

WP4: Transport

  • Planning, roll out, optimization and operation of the fiber and WDM

infrastructure in coordination with other WPs and partners

  • Input for publications
  • WP lead:

ad: Stefan an Mas askos (Planning) / Heinz May ayer (Te Technology) Back to Projec ect Structure

slide-10
SLIDE 10

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 10

WP5: Layer 2/3

  • Planning, roll out, optimization and operation of the Router infrastructure in

coordination with other WPs and partners

  • Input for publication
  • WP

WP-Leiter: : Dan aniel Nowar ara Back to Projec ect Structure

slide-11
SLIDE 11

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 11

WP6: SDN & NFV & Security

  • attend SDN and NFV approaches
  • set up a security concept in coordination with the partners
  • Implement that concept
  • input for publications
  • WP lead

ad Ral alf Brau aun (T-Labs abs) Back to Projec ect Structure

slide-12
SLIDE 12

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 12

WP7: Applications

  • Coordination of the application teams
  • Input for publications
  • WP lead:

ad: Ferdinan and Jam amitzky Back to Projec ect Structure

slide-13
SLIDE 13

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 13

WP8: Public Relations

  • Coordinate partners and activities towards optimum project marketing
  • Coordinate press release activities
  • Produce and attend project PR material (flyer, articles, etc)
  • Coordinate ISC booth activities (flyer, logo, sessions, poster, give aways,

etc)

  • Coordinate the ISC application demonstration (incl. Internet access)
  • WP lead:

ad: Udo Schäf äfer Back to Projec ect Structure

slide-14
SLIDE 14

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 14

400G Demonstrator

Topology

10GbE for Demonstrator Computing Center

Euro Industriepark München

Back to Agenda

slide-15
SLIDE 15

Turbine Development

Back to Agenda

  • Cooperation with DLR
  • Workflow Demonstration
  • Preprocessing
  • Solver 1
  • Solver 2
  • Postprocessing
  • Turbine model calculation with n Eigenmodes and m Phase Angels
slide-16
SLIDE 16

Detai ails:

 Data volume: ~ 1TB  Overall Workflow:  Multitude of independent simulation runs (HTC).  Simulations running on HPC resources at different sites.  Every simulation produces input data for subsequent simulations.  Subsequent simulations again run at different sites.

Thus to avoid knock-on delays in workflow execution data instantly should be available at different sites! GPFS:

 Adopted feature: Active File Management (AFM) and Stretched Cluster  Cross site data replication allows running simulations without prior copying  Implicit data consistent backup via AFM data replication

Turbine Development: Benefits of GPFS Usage on 400G

Back to Tu Turbine Developm pment

slide-17
SLIDE 17
  • Turbine Development: Benefits of GPFS Usage on 400G
  • No. cores

tim e

a = 6

n * m = 28 ≥ a * b = 30 Δt Solver 1 Solver 2

b = 5

240m in

Back to Tu Turbine Developm pment

slide-18
SLIDE 18

400G: : Ban andw dwidt dth req equirem emen ents for di differ eren ent job di distribu bution set etups ps

 Extreme/HTC setup with a = n * m = 300, b = 1:

  • Assuming jobs all writing within 15min to disk an avg. file size of 150MB (i.e.

write peak):

 Required bandwidth:

400GBit/s

 Required machine size: > 19200 Cores (when single jobs run on 64 cores)  „Gentle“ setup with a = 50, b = 6:

  • Assuming jobs having an avg. runtime of 240min continously writing 150MB of

data to disk to represent runtime differences over larger values b:

 Required bandwidth:

4GBit/s

 Required machine size: > 3200 Cores (when single jobs run on 64 cores)

Turbine Development: Benefits of GPFS Usage on 400G

Back to Tu Turbine Developm pment

slide-19
SLIDE 19

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 19

Chemnitz

Turbine Development Setup

  • 1. Prepr

processing @ DSI

  • 2. Flow Model

Calculat ation (Solver 1)

  • 3. Flow Model

Calculat ation (Solver 2)

  • 4. Postpr

proces essing @ DSI Back to Tu Turbine Developm pment

slide-20
SLIDE 20

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 20

Chemnitz

Turbine Development & GPFS

Parallel Distr tribute uted File Syste tem GPFS

Back to Tu Turbine Developm pment

slide-21
SLIDE 21

 Order

er 30 differ eren ent mode dels are used ed worldw dwide de

 Exper

erimen ents with thes ese mode dels produ duce1 e10s

  • f
  • f PBytes

es

 100s of

  • f Pby

bytes es of

  • f dat

ata nee eed to to be be compar pared ed betwee een multipl ple e sites es worldw dwide de

 Movem

emen ent of

  • f dat

ata should be be within months *

Transfer Rate Time to transport 1 PB of Data 10 Mbps ~ 27 years 1 Gbps ~ 97 days 100 Gbps ~ 23 hours

* Otherwise the questions will be forgotten ;-)

Statistics taken from: „BER Network Requirements Workshop”, LBNL report LBNL-4089E 2010, P 33. Recommended Reading

Climate Computing Extremely High Bandw dwidth dth Requ quirements ‘Very Big Data‘

Back to Agenda

slide-22
SLIDE 22

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 22

Climate Computing Application Setup

Folde der 1 CMIP Folde der 2 CMIP Federat ation Preal allocat ation Model po post-pr processing an and anal alysis Visual alisiat ation @ ISC ’13 Leipz pzig Folder 3 CMIP Back to Climat ate Compu puting

slide-23
SLIDE 23

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 23

CCA & GPFS & iRODS

GPFS and/or

  • r

Global Namespace iRODS

Back to Climat ate Compu puting

slide-24
SLIDE 24

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 24

Service Recipient Relations

Distributed ed Folders Distributed ed Folders Service Rec Recipien ent Feder eration, Prea eallocation Resea earch Clien ent PREP EP & POST on Cloud Res esources es Calculation Climate e Computing TRA RACE on HPC Resources es TRACE E on HPC Resources es Turbine e Dev evel elopmen ent Client ev evaluating results, e. e.g. TEC ECPLOT 5T 5T 5T 5T 5T 5T 5T 5T 5T 5T 5T 5T Distributed ed Folders 5T 5T

Back to Agenda

slide-25
SLIDE 25

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 25

400GBit/s Data Path

Router / Switch 400G G WDM Super er Channel el (4x1 x100Gb GbE) E) Link Serve

ve r

Storage GPFS Files esystem em

Back to Agenda

Router Router

12x Server

12x40 GbE

12x Server

12x40 GbE

400 GBit/s /s

36x700G Flash 36x700G Flash ∼7000 cores

IB FDR Network

12xIB FDR

TUD Cluster

∼2000 cores

IB FDR10 Network

12xIB IB FDR10

10

slide-26
SLIDE 26

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 26

3x700G Flash Server

Router 6 6 GByte/s /s 5 5 GByte/s /s

∼7000 cores

IB FDR Network

7 7 GByte/s /s

Per Server

36x700G Flash 12x Server

Router

∼7000 cores

IB FDR Network

50

GByte/s /s

60

GByte/s /s

72 72

GByte/s /s

84 84

GByte/s /s

Throughput Targets

Back to Agenda

slide-27
SLIDE 27

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 27

The Big Picture

Back to Agenda

Firew ewall / En Encryption Router er / Switch 400G G WDM Super er Channel el (4x1 x100Gb GbE) E) Link Server er Storage Parallel el File e System

17x Server Firewall ∼200 cores

1x10 GbE

36x700G Flash 12x Server

Router

17x Server

17x10 GbE Router 2x100 GbE*

DATE Cluster 1 DATE Cluster 2

12x40 GbE 17x10 GbE

17x2T Disk 17x2T Disk

Router 1x10 GbE

400 GBit/s /s

36x700G Flash ∼2000 cores 12x Server

12x40 GbE

∼7000 cores

TUD Cluster RZG Cluster

IB FDR10 Network

12xIB FDR10

10

IB FDR Network

12xIB FDR

SGI Cluster

1x10 GbE

Firew ewall *maybe nor available e during ISC

slide-28
SLIDE 28

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 28

Connection RZG G Router – RZG-WDM and Connection TUD UD Router– TUD UD WDM Type 2 x LC (100GBaseLR4) length TUD: 10m; RZG: 10m Volume 4 each (8 total)

7750 7750 SR12E 7750 7750 SR12

1830 PSS

4x100 GbE

Router er / Switch 400G G WDM Super er Channel el Link WDM Ter erminal Amplifier er

1830 PSS

4x100 GbE

coher eren ent Super er Channel el (2 x 16QAM@ 50 GH GHzGr Grid / 2 x 200 GB GBit/s)

OLA+ DGE OLA OLA OLA OLA OLA OLA OLA

70km 70km 70km 70km 70km 70km 70km 70km 70km

400 G WDM Super Channel

Back k to Big Picture

slide-29
SLIDE 29

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 29

Connection TU TUD Server – TUD UD IB-Switch and Connection RZG G Server – RZG G IB Switch Type MPO (Infiniband FDR (56GBit/s)) length TUD: 10m, RZG: 10m Volume 12 each side (24 total)

Infiniband Connections

Mellan anox Connec ect-IB Volume:1 e:12on eac ach side de (24total al)

Back k to Big Picture

Mellan anox active Cabl able (incl. . QSFP) Volume: : 12 on eac ach side (24 total al)

slide-30
SLIDE 30

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 30

Connection TU TUD Server – TUD UD Router and Connection RZG G Server – RZG G Router Type MPO (40GBaseSR4) length TUD: 10m; RZG: 10m Volume 24x10m

Mellan anox Connec ect X3

Volume: 12 on eac ach side (24 total)

Mellan anox active e Cabl able e (incl. . QSFP) Volume: e: 12 on eac ach side de (24 total al)

40GbE Connections

Alcat atel-Lucent 3-po port 40GbE IMM

Volume: 4 @ RZG G

Alcat atel-Lucent 6-po port 40GbE IMM (no picture) Volume:2@T

@TUD UD Back k to Big Picture

slide-31
SLIDE 31

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 31

IBM iDat ataP aPlex dx360 M4 (Volume: 12

12@RZ @RZG)

Bull NovaS aScal ale R460 F3 (Volume: 12@T

@TUD UD)

Server

Back k to Big Picture

slide-32
SLIDE 32

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 32

Connection Router – WDM Type LC singelmode (100GBaseLR10) length 10m Volume 2 Back k to Big Picture

Alcat atel-Lucent 2-port 100GbE IMM

Volume me: 3 @ TUD UD

Alcat atel-Lucent 1-po port 100GbE IMM (no picture) Volume

me:4@R 4@RZG

slide-33
SLIDE 33

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 33

Connection TUD UD Router – TUD UD 10GbE Cluster Type LC duplex multimode (10GBaseSR) length ? Volume 17 Back k to Big Picture

slide-34
SLIDE 34

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 34

Alcat atel el Lucen ent 7750 SR1 R12 @RZG Alcat atel el Lucen ent 7750 SR1 R12 E @TU TUD

Back k to Big Picture

slide-35
SLIDE 35

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 35

Alcat atel Lucent 1830 PSS 32

Back k to Big Picture

slide-36
SLIDE 36

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 36

Back k to Big Picture

EMC2 XtremSF 700GB SLC

Volume me: 36 PCIe cards on each side (72 total)

slide-37
SLIDE 37

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13

closing WS WS

21.6.-?

37

Project Lifetime

Demonstrat ator Setup Getting through Te Test Item em List TI TIL

IS ISC

28.1- 15.6.

Kickoff WS WS

Back to Agenda

slide-38
SLIDE 38

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 Press Release

Timeline Rev. C

Setup tup Server, Stor

  • rage, Infi

nfini niband nd Setu tup GPFS

  • 7. 6.

final Perfo foma mance Tests sts

  • 12. 6.
  • 16. 6.

RfS WDM & IP Performance Tests done RfS GPFS

Setu tup WDM, M, Route ter DATE Applicati tions ns fina nal confi nfig

RfS Tunnel, Server RfS Applications

Back to Agenda

  • 29. 5.

no Applicati tion Tests ts possi sible GPFS opti timi mizati tion

CW 19 CW 20 CW 21 CW22 CW23 CW 24

We are here

  • 10. 6.
  • 20. 5.
  • 14. 6.

Stop DATE

slide-39
SLIDE 39

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 39

Demonstrator Application Test Environment DATE Object ctive ve

  • 3 weeks between RfS IP and the begin of the ISC is by far not enough time

to get two high sophisticated applications running @400G

  • Therefore the support of the applications as early as possible is an

integrated part of the project; the application teams get access as soon possible on new building blocks of the ‘big picture’

  • 1. April - 17. May

Back to Agenda

slide-40
SLIDE 40

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 40

DATE Phase 1

Router er / Switch 400G G WDM Super er Channel el (4x1 x100Gb GbE) E) Link

17x Server

Serve

ve r

Storage Firew ewall / En Encryption GPFS Files es System em

7750 7750 SR12

17x Server

17x10 GbE 7750 7750 SR12 2x100 GbE

DATE Cluster 1 DATE Cluster 2

17x10 GbE

17x2T Disk 17x2T Disk

  • 1. April - 29. April

Back to Ti Timeline

slide-41
SLIDE 41

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 41

DATE Phase 2

Router er / Switch 400G G WDM Super er Channel el (4x1 x100Gb GbE) E) Link Serve

ve r

Storage Firew ewall / En Encryption GPFS Files es System em

17x Server 3xGPFS Server

7750 7750 SR12

17x Server

17x10 GbE 7750 7750 SR12 2x100 GbE

DATE Cluster 1 DATE Cluster 2

17x10 GbE

17x2T Disk 17x2T Disk 2xGPFS Server 2xGPFS Server

2x40 GbE

  • 29. April - 6. May

2x40 GbE

Back to Ti Timeline

4x700G Flash 4x700G Flash

slide-42
SLIDE 42

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 42

DATE Phase 3

Router er / Switch 400G G WDM Super er Channel el (4x1 x100Gb GbE) E) Link Serve

ve r

Storage Firew ewall / En Encryption GPFS Files es System em

17x Server 3xGPFS Server

7750 7750 SR12

17x Server

17x10 GbE 7750 7750 SR12 2x100 GbE

DATE Cluster 1 DATE Cluster 2

17x10 GbE

17x2T Disk 17x2T Disk ∼7000 cores

TUD Cluster

IB FDR Network

2xGPFS Server 2xGPFS Server

2x40 GbE 23xIB FDR

  • 7. May - 16. May

2x40 GbE

Back to Ti Timeline

4x700G Flash 4x700G Flash

slide-43
SLIDE 43

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 43

Test Items Object ctive ve

  • The Demonstrator is a great opportunity for getting valuable findings in

networking , HPC, virtualization and other fields

  • the project schedule just allows some synthetic performance tests and the

implementation of the applications

  • Therefore the partners have agreed to set up a test item list where each

partner is invited to contribute proposals Back to Projec ect Lifet etime 21.6.-?

slide-44
SLIDE 44

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 44

Test Item List NFV (TSI)

Back to Agenda

Lo Loadb dbalance cer / Bundl dling / Perfo formance ce / CoS / FCAPS PS (T-Labs bs)

21.6.-?

GPFS Network (TSI-SfR fR) ) SDN (TUD) D) RDMA over Ethernet

slide-45
SLIDE 45

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 45

Network Functions Virtualisation (NFV)

Back to Te Test Item em List

slide-46
SLIDE 46

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 46

Example NFV Use Case

ESX Cluster

Firewall Firewall

Customer B

VLAN 2 VLAN 1

Each customer configures his own FW entity

Firewall

VLAN 1

Customer A

VLAN 1 VLAN 2 VLAN 2 VLAN 2 VLAN 1 VLAN 2

Internet

Router er/Switch Firew ewall VM VM Standby Fi Firewall VM VM

Back to Te Test Item em List

slide-47
SLIDE 47

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 47

NFV & ESX Test Setup

ESX Cluster

Client

40 40 GbE

Router/Switch Firewall VM VM Standard HW (2xE xE5-2670 + + 128G G RA RAM)

40 40 GbE 40 40 GbE 40 40 GbE 40 40 GbE

Client nt

40 40 GbE

Back to Te Test Item em List

slide-48
SLIDE 48

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 48

Demonstrator NFV Setup

Router er / Switch Link Serve

ve r

Storage Firew ewall / En Encryption / Compres ession GPFS Files es System em

17x Server Firewall ∼200 cores

1x10 GbE

3xGPFS Server

7750 7750 SR12

17x Server

17x10 GbE 7750 7750 SR12E 2x100 GbE

DATE Cluster 1 DATE Cluster 2

17x10 GbE

17x2T Disk 17x2T Disk SGI Clus uster

7750 7750 SR12 1x10 GbE

∼2000 cores 3xGPFS Server ∼7000 cores

TUD Cluster RZG Cluster

IB FDR10 Network

3xIB FDR10

10

IB FDR Network

3xIB FDR 1x100 GbE

3xFEC

3x40 GbE 3x40 GbE

3xFEC

3x40 GbE 3x40 GbE

9x700G Flash 9x700G Flash

1x10 GbE

Back to Te Test Item em List

slide-49
SLIDE 49

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 49

Scheduled for: 21. 1.6.-5.7.? Objectives

  • Is 5GByte/s Firewall / Encryption / Compression with standard hardware (2xE5-2670 &

PCIe3.0) feasible?

  • Even hypervisored?
  • What is the impact for the application?

NFV Objectives & Comments

Comm mments

  • In ‘real life’ network functions like encryption, firewalling and compression can become very

important , even in HPC environments

  • There is no need for 400GBit/s; 100GBit/s is equivalent

Back to Te Test Item em List

slide-50
SLIDE 50

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 50

Loadbalancer / Bundling / Performance / CoS / FCAPS Setup

Back to Te Test Item em List

slide-51
SLIDE 51

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 51

GPFS Network

17x Server Firewall ∼200 cores

1x10 GbE

3xGPFS Server

7750 7750 SR12

17x Server

17x10 GbE 7750 7750 SR12E 2x100 GbE

DATE Cluster 1 DATE Cluster 2

3x40 GbE 17x10 GbE

17x2T Disk 17x2T Disk

7750 7750 SR12 1x10 GbE

∼2000 cores 3xGPFS Server

3x40 GbE

∼7000 cores

TUD Cluster RZG Cluster

IB FDR10 Network

3xIB FDR10

10

IB FDR Network

3xIB FDR 1x100 GbE

3xGPFS Server

3x40 GbE

Router er / Switch Link Serve

ve r

Storage Firew ewall / En Encryption / Compres ession GPFS Files es System em

12x700G Flash 12x700G Flash 12x700G Flash SGI Clus uster

1x10 GbE

Back to Te Test Item em List

slide-52
SLIDE 52

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 52

Scheduled for: 21. 1.6.-5.7.? Objectives

  • Wie verhält sich GPFS,wenn die Daten nicht zwischen 2, sondern 3 Sites (Option 4 Sites, Hub

be TUD) gespiegelt werden? (Server Überlast?)

  • Was passiert, wenn z.B eine Site mit geringerer Bandbreite angebunden ist? “Kommt das

GPFS noch nach”?

Test Item GPFS Network

Comm mments

  • There is no need for 400GBit/s; 100GBit/s is equivalent

Back to Te Test Item em List

slide-53
SLIDE 53

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 53

Test Item SDN

Comm mments * Aufsetzen einer virtuellen SDN Umgebung zwischen ZIH & RZG mittels vSwitch * Wünschenswert wäre die Einbindung von aktiven Netzwerkelementen mit OpenFlow Unterstützung (bspw. auch Barracudas SDN Gateway) * Gegenüberstellung verschiedener verfügbarer OpenFlow Controller (Beacon, Floodlight, FlowER, OpenDaylight, ...) * Zeitraum: 1 Monat, kann aber parallel zu anderen Untersuchungen laufen Da vermutlich nicht besonders viel Zeit zur Verfügung stehen wird, glaube ich das dies bereits sehr ambitioniert ist. Ich muss dann schauen wieviele Controller aufgesetzt und getestet werden können. Aber ich würde die Umgebung so aufbauen das ich auch nach dem 400G Showcase noch damit arbeiten kann.

Back to Te Test Item em List

slide-54
SLIDE 54

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 54

Test Item RDMA over Ethernet

Comm mments

  • Analyse vorhandener Protokolle für RDMA über Ethernet
  • Vergleich mit RDMA über InfiniBand
  • Performanceanalyse und -optimierung innerhalb einer 40GbE Testumgebung
  • Gegenüberstellung der Ergebnisse mit FDR InfiniBand

Back to Te Test Item em List

slide-55
SLIDE 55

–public – E. Beier/ W. Wünsch 400G Demonstrator für ISC’13 55

Th Than ank You!