Experiences in deploying the high-end Experiences in deploying the - - PowerPoint PPT Presentation

▶
experiences in deploying the high end experiences in
SMART_READER_LITE
LIVE PREVIEW

Experiences in deploying the high-end Experiences in deploying the - - PowerPoint PPT Presentation

Aug 16, 2023 162 likes •429 views

Experiences in deploying the high-end Experiences in deploying the high-end visualization application in the visualization application in the transatlantic GLIF environment transatlantic GLIF environment Marek Baewicz marqs@man.poznan.pl

slide-1
SLIDE 1

Experiences in deploying the high-end Experiences in deploying the high-end visualization application in the visualization application in the transatlantic GLIF environment transatlantic GLIF environment

Marek Błażewicz

marqs@man.poznan.pl

slide-2
SLIDE 2

Outline

  • 1. PSNC – projects, infrastructures, services
  • 2. PSNC – global initiatives
  • 3. SC2010 demonstrations
  • 4. Application in GLIF environment
  • 5. Future plans
  • 6. Summary
slide-3
SLIDE 3

Outline

  • 1. PSNC – projects, infrastructures, services
  • 2. PSNC – global initiatives
  • 3. SC2010 demonstrations
  • 4. Application in GLIF environment
  • 5. Future plans
  • 6. Summary
slide-4
SLIDE 4

PIONIER – National Research and Education Network

POLAND

  • Area 312sq km
  • Population 38M
  • Main academic centers 21
  • State University 165+
  • Students 2M+
  • R&D institutions and

Universities connected within PIONIER network 700+

  • Lambdas 5300km+
slide-5
SLIDE 5

PSNC – PIONIER administrator

POZNAŃ

  • Poznań the know-how city
  • PSNC is a government research lab established in 1993 in Poznań
  • PSNC consists of 4 departments:

Networking, Networking Services, HPC and Applications

slide-6
SLIDE 6

PSNC – Applications department

  • Application Department has an access to a heterogeneous high

performance computing and storage resources with a total performance around 300 TFlops

  • Use commonly available devices, to boost up the calculations

performance

  • Hardware: FPGA devices, NVIDIA graphic cards (Big GPU cluster – 334

M2050 GPUs, but also GeForce GTX260/280/480), IBM Cell and more.

  • Application Department team helps end users - scientists - to take

advantage of new powerful computing architectures, including support for benchmarks, parallelization of legacy code, integration, etc.

  • We offer a transitional access to powerful SMP and GPU based clusters

located in leading HPC Centers in Europe and the United States

  • NDA for R&D signed with leading hardware

and software vendors, including Microsoft, IBM, Intel, etc.

slide-7
SLIDE 7

Outline

  • 1. PSNC – projects, infrastructures, services
  • 2. PSNC – global initiatives
  • 3. SC2010 demonstrations
  • 4. Application in GLIF environment
  • 5. Future plans
  • 6. Summary
slide-8
SLIDE 8

PSNC – Global initiatives

  • Taking part in automation of multi-domain lightpath provisioning system;

demonstrations of FENIUS auto provisioning tool using PerfSONAR PingER: – 10th Annual Global LambdaGrid Workshop in Geneva, October 2010 – SC'10 event in New Orleans, November 2010

slide-9
SLIDE 9

PSNC – Global initiatives

  • Designing the NSI standard (Network Service Interface) within the OGF

(Open Grid Forum).

  • Taking part in implementation of the new NSI standard within the AutoBAHN.
  • Demonstrations of NSI Connection Service (NSI CS) provisioning tools:

– GLIF meeting in Rio de Janeiro, September 2011; – Future Internet Week in Poznań, October 2011 – SC in Seattle, November 2011;

The used software packages and developing organiations:

OpenNSA – NORDUnet, Copenhagen, DK G-LAMBDA-A – AIST, Tsukuba, JP AutoBAHN – GEANT Project, Poznan, PL DRAC – SURFnet, Amsterdam, NL G-LAMBDA-K – KDDI Labs, Fujimino, JP DynamicKL – KISTI, Daejeon, KR OSCARS – Esnet, Berkeley, US

slide-10
SLIDE 10

Outline

  • 1. PSNC – projects, infrastructures, services
  • 2. PSNC – global initiatives
  • 3. SC2010 demonstrations
  • 4. Application in GLIF environment
  • 5. Future plans
  • 6. Summary
slide-11
SLIDE 11

SC2010 demonstration

Assumptions

  • SuperComputing conference at New Orleans;
  • HPC hardware at PSNC (8500 km.);
  • 100Mb/s of dedicated circuit between these institutions;

Goal

  • Perform HPC simulation at PSNC SuperComputers;
  • Visualize it;
  • Display and control the simulation using thin client on SC'10;

thin client (HP touchsmart; Windows Vista);

  • First experiment where both example data and compute

demanding applications used innovative bandwidth on- demand (BoD);

slide-12
SLIDE 12

SC2010 demonstration

Problem: finding isosurface of a 3D scalar field; Solution: high performance, multi-GPU computations (marching tetrahedrons) and visualization;

slide-13
SLIDE 13

SC2010 demonstration

Outcome: Vitrall - a distributed web based visualization system. Results: 5-10 Fps, 1-4s.

slide-14
SLIDE 14

Outline

  • 1. PSNC – projects, infrastructures, services
  • 2. PSNC – global initiatives
  • 3. SC2010 demonstrations
  • 4. Application in GLIF environment
  • 5. Future plans
  • 6. Summary
slide-15
SLIDE 15

Application in GLIF environment

Assumptions

  • Partnership between Center for Computation and

Technology, LSU (CCT) – PSNC

  • 1Gb/s of dedicated circuit between these institutions;
  • High performance computations & real-time transatlantic

visualizations; Goals

  • Maximizing the performance;
  • Minimizing the reaction time;
  • Different network protocols;
slide-16
SLIDE 16

Application in GLIF environment

Connection tests

  • RTT ~ 167ms
  • Bandwidth Delay Product (BDP) = 20.875 MB
  • TCP throughput depending on TCP buffer size (iperf)
  • UDP throughput 960Mb/s (iperf)

[ ]

seconds RTT bits Byte

  • nd

bits bandwidth BDP ∗       ∗       = 8 1 sec

Max buffer size for TCP send/receive [Mbytes] Measured maximal throughput [Mbps] 21 722 22 756 24 824 26 860 29 907 32 953

slide-17
SLIDE 17

Application in GLIF environment

Initial preparations

  • Increasing send&receive TCP buffers on both sides (PSNC, CCT)

– Possible larger bandwidth; – Possible larger latency if the bandwidth is not properly

  • ccupied (Windows thin client: low performance, limited

configurability);

  • Fall back from using Jpegs to raw images

– no compression/decompression overhead; – larger bandwidth required to sustain the number of frames per second;

  • Implementing the UDP raw transfer (packets are slices of the

image, equal or less than MTU – 1500 bytes) – lowest reaction time; – possible packets loss;

slide-18
SLIDE 18

Application in GLIF environment

Results

  • About 30-35 frames rendered and transferred (1280x800);
  • Application response:

– UDP <0.5s.; – TCP “jpeged” frames ~ 1s.; – TCP raw frames ~ 1.24s.;

  • The application look and feel: real-time, almost as rendered

locally

slide-19
SLIDE 19

Application in GLIF environment

Drawbacks of UDP

  • No information about actually available transfer – packet loss;
  • The need of implementing bandwidth control on the application

level, or...

  • … qdisc
slide-20
SLIDE 20

Application in GLIF environment

Drawbacks TCP

  • Problems with Windows + Linux cooperation; it is not

deterministic to obtain the full bandwidth;

  • Windows – problems with decompressing images in real-time:

buffer size / jpeg size / FPS = latency;

  • Higher latency;
  • Traffic errors impact the performance;
slide-21
SLIDE 21

Outline

  • 1. PSNC – projects, infrastructures, services
  • 2. PSNC – global initiatives
  • 3. SC2010 demonstrations
  • 4. Application in GLIF environment
  • 5. Future plans
  • 6. Summary
slide-22
SLIDE 22

Future plans

  • 10GE connection between CCT and PSNC;
  • More advanced visualization/cooperation (16K);
  • HD visualizations in both directions at the same time;
  • Displaying large number of simulations simultaneously;
  • Interactions on both sides;
  • Teleconference at the same time;
slide-23
SLIDE 23

Future plans

slide-24
SLIDE 24

Outline

  • 1. PSNC – projects, infrastructures, services
  • 2. PSNC – global initiatives
  • 3. SC2010 demonstrations
  • 4. Application in GLIF environment
  • 5. Future plans
  • 6. Summary
slide-25
SLIDE 25

Summary

  • A remote rendering application has been tested in GLIF

environment and large distances;

  • Different communication patterns and protocols have been tested;
  • Real-time?
  • Automated provisioning system would save-up some time

required to establish the circuit (manual vs. automatic – 4K);

slide-26
SLIDE 26

Thank you! Thank you!