Consorzio COMETA FESR Visualization Element: towards the - - PowerPoint PPT Presentation

consorzio cometa
SMART_READER_LITE
LIVE PREVIEW

Consorzio COMETA FESR Visualization Element: towards the - - PowerPoint PPT Presentation

Consorzio COMETA FESR Visualization Element: towards the definition of a new Grid service Giuseppe ANDRONICO (1) , Roberto BARBERA (1)(2) , Andrea FORNAIA (1) , Marcello IACONO MANNO (3) and


slide-1
SLIDE 1

www.consorzio-cometa.it

FESR

Consorzio COMETA

Visualization Element: towards the definition of a new Grid service

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan

Giuseppe ¡ANDRONICO ¡(1), ¡Roberto ¡BARBERA(1)(2), ¡Andrea ¡FORNAIA ¡(1), ¡ Marcello ¡IACONO ¡MANNO ¡(3) ¡and ¡Giuseppe ¡LA ¡ROCCA(1) ¡ ¡

¡

(1) ¡INFN ¡– ¡Na*onal ¡Ins*tute ¡of ¡Nuclear ¡Physics, ¡Division ¡of ¡Catania, ¡Italy ¡ ¡ (2) ¡Department ¡of ¡Physics ¡and ¡Astronomy ¡of ¡the ¡University ¡of ¡Catania, ¡Italy ¡ ¡ (3) ¡Consorzio ¡COMETA, ¡Catania, ¡Italy ¡

¡

slide-2
SLIDE 2

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 2

Outline Outline

  • The Consorzio COMETA and the Sicilian Grid

infrastructure;

  • Introduction to the Visualization Element service

– Motivations; – Design and Implementation; – Architecture; – The Stereoscopy; – Examples;

§ Blender; § VISIVO;

– Future plans.

  • Summary and Conclusions;
  • References.
slide-3
SLIDE 3

3

The Italian e-Infrastructure

(interoperable with common communication protocols)

The Network

FESR

The Grid

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan

slide-4
SLIDE 4

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 4

The COMETA Consortium

(http://www.consorzio-cometa.it)

slide-5
SLIDE 5

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 5

Objectives of an e-Infrastructure in Sicily

  • Create a Virtual Laboratory in Sicily, both for

scientific and industrial applications, built on top

  • f a grid infrastructure
  • Connect the Sicilian e-Infrastructure to those

already existing in Italy, in Europe, and in the rest of the world improving the scientific collaboration and increasing the “competitiveness” of e-Science and e-Industry “made in Sicily”

  • Disseminate the “grid paradigm” through the
  • rganization of dedicated events and training

courses

  • Trigger/foster the creation of spin-offs in the ICT

area in order to reduce the “brain drain” of brilliant young people to other parts of Italy and beyond

slide-6
SLIDE 6

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 6

The COMETA Infrastructure

Catania

~15 ¡M ¡€ ¡in ¡3 ¡years ¡ >350 ¡people ¡involved! ¡ ~2500 ¡CPUs ¡ ~250 ¡TB ¡

slide-7
SLIDE 7

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 7

Computing and storage resources available

  • 1. ¡~2000 ¡cores ¡AMD ¡Opteron ¡2218 ¡
  • 2. ¡2 ¡GB ¡of ¡RAM ¡per ¡core ¡ ¡
  • 3. ¡LSF ¡as ¡LRMS ¡everywhere ¡
  • 4. ¡Infiniband-­‑4X ¡at ¡all ¡sites ¡
  • 1. ¡200+ ¡TB ¡of ¡raw ¡disk ¡storage ¡ ¡
  • 2. Distributed/parallel ¡GPFS ¡filesystem ¡
slide-8
SLIDE 8

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 8

Commercial software available

(http://www.pi2s2.it/tecn/index.php?option=com_content&task=view&id=27&Itemid=58)

slide-9
SLIDE 9

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 9

Motivations

  • Several Grid applications (e.g.: FLUENT) running in the

COMETA Infrastructure produce media content (e.g.: images) that can be collected and encoded as a unique video to be visualized also when the job that produces the images is still running.

– This feature maybe helpful for both demonstration purposes or to control job evolution.

  • Visualiza*on ¡Element is a post-processing tool allowing to

show, starting from a list of images sequence, the video

  • utput produced by a producer application.

Fig.1 ¡-­‑ ¡Some ¡frames ¡produced ¡by ¡FLUENT ¡and ¡streamed ¡by ¡the ¡VisualElement ¡service. ¡ ¡

64 cores for 14 CPU days Environment protection

slide-10
SLIDE 10

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 10

Deploying the VisualElement on Grid

  • VisualElement uses mencoder, VLC ¡VideoLAN ¡Client, ¡Xva and

FFMPEG software packages to encode the media content and produce a video that will be send in streaming.

Top BDII

  • Published a new software tag,

VO-cometa-VLC-0.9.9a, for the service.

  • The needed packages for Visualization Element can be

downloaded from repos.

  • A JDL file is created on-the-fly using a Grid portal to

call the service with the corresponding inputs.

  • A first prototype of the Visualization Element on-line

streaming tool has been deployed for the COMETA Grid infrastructure on board of the GENIUS Portal powered by EnginFrame from NICE srl

slide-11
SLIDE 11

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 11

Integration of VisualElement in COMETA

  • Two different implementations of the service are

available as shown in fig. 1 and fig. 2

  • Fig. ¡1 ¡– ¡Genera*on ¡of ¡the ¡video ¡streaming ¡ ¡with ¡a ¡single ¡job

¡

  • Fig. ¡2 ¡– ¡Genera*on ¡of ¡the ¡video ¡output ¡using ¡a ¡DAG

¡

Based ¡on ¡LCG ¡Grid ¡CE ¡ Based ¡on ¡gLite ¡3.2 ¡CREAM ¡CE ¡

slide-12
SLIDE 12

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 12 12

Commercial & industrial HPC applications

  • n Grid: CFD simulations with Fluent

64 ¡cores ¡for ¡14 ¡CPU ¡days ¡ Environment ¡protec*on ¡ (“Le ¡Marmore” ¡falls) ¡ 48 ¡cores ¡for ¡7 ¡CPU ¡days ¡ Engine ¡design ¡

slide-13
SLIDE 13

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 13

What’s next ? The Stereoscopy

  • It is the ability of creating, or enhancing, the illusion of

depth in a flat image:

  • by presenting two offset images, respectively for left and right eye,

combined together in a single one, using some optical technique;

  • This is exactly what our brain does with the “images” taken from

the eyes.

  • To implement the Stereoscopy, we can use three

different methods: – Using two separated sources:

– the user wears eyeglasses to combine the images taken from them;

– Using only one source:

– the user wears eyeglasses to split the image taken from it;

– Using a self-stereoscopic source:

– that splits the images directing the light into the viewer's eye, with no need of eyeglasses.

slide-14
SLIDE 14

Stereoscopic streams in practice

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 14

  • We had to produce for each eye, two separated

streams of images

  • observing the simulation from two different point of view;
  • Then, we have to combine them in a single stream
  • using appropriated video drivers, in according to the chosen

stereoscopic technique; e.g., the anaglyph one.

slide-15
SLIDE 15

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 15

First use case

  • The service has been tested for the first time encoding a

BLENDER application.

– The

The application The reconstructed video will be send in streaming via UDP , generates for each frame 2 different images; (default port 1234) to the client’s IP using VLC.

slide-16
SLIDE 16

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 16

Second use case

  • VIsIVO Server for COMETA Grid can create a 3D

view from data tables;

  • It runs on every Linux based system:

– The main characteristic is that VisIVO Server gives support for huge data; – No limits (in principle) are fixed to visualize your data. Some limits exist up to now, but the next version will solve these problems; – The FITS and HDF5 data format will be included among supported formats; – Output data from Gadget code will be soon supported; – The project is completed since October 2008.

slide-17
SLIDE 17

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 17

Any questions, comments or remarks are very welcome. Please contact: giuseppe. andronico@ct.infn.it

slide-18
SLIDE 18

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 18

  • G. Andronico, R. Barbera, M. Iacono Manno and G. La Rocca.

VisualGrid: On-Line Video Streaming for Application Control and Demonstration. Proceedings of the Final Workshop of Grid Projects “PON Ricerca 2000-2006, Avviso 1575”, 2009

References & Links