Consorzio COMETA FESR Visualization Element: towards the - - PowerPoint PPT Presentation

www.consorzio-cometa.it

FESR

Consorzio COMETA

Visualization Element: towards the definition of a new Grid service

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan

Giuseppe ¡ANDRONICO ¡(1), ¡Roberto ¡BARBERA(1)(2), ¡Andrea ¡FORNAIA ¡(1), ¡ Marcello ¡IACONO ¡MANNO ¡(3) ¡and ¡Giuseppe ¡LA ¡ROCCA(1) ¡ ¡

¡

(1) ¡INFN ¡– ¡Na*onal ¡Ins*tute ¡of ¡Nuclear ¡Physics, ¡Division ¡of ¡Catania, ¡Italy ¡ ¡ (2) ¡Department ¡of ¡Physics ¡and ¡Astronomy ¡of ¡the ¡University ¡of ¡Catania, ¡Italy ¡ ¡ (3) ¡Consorzio ¡COMETA, ¡Catania, ¡Italy ¡

¡

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 2

Outline Outline

• The Consorzio COMETA and the Sicilian Grid

infrastructure;

• Introduction to the Visualization Element service

– Motivations; – Design and Implementation; – Architecture; – The Stereoscopy; – Examples;

§ Blender; § VISIVO;

– Future plans.

• Summary and Conclusions;
• References.

3

The Italian e-Infrastructure

(interoperable with common communication protocols)

The Network

FESR

The Grid

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 4

The COMETA Consortium

(http://www.consorzio-cometa.it)

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 5

Objectives of an e-Infrastructure in Sicily

• Create a Virtual Laboratory in Sicily, both for

scientific and industrial applications, built on top

• f a grid infrastructure
• Connect the Sicilian e-Infrastructure to those

already existing in Italy, in Europe, and in the rest of the world improving the scientific collaboration and increasing the “competitiveness” of e-Science and e-Industry “made in Sicily”

• Disseminate the “grid paradigm” through the
• rganization of dedicated events and training

courses

• Trigger/foster the creation of spin-offs in the ICT

area in order to reduce the “brain drain” of brilliant young people to other parts of Italy and beyond

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 6

The COMETA Infrastructure

Catania

~15 ¡M ¡€ ¡in ¡3 ¡years ¡ >350 ¡people ¡involved! ¡ ~2500 ¡CPUs ¡ ~250 ¡TB ¡

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 7

Computing and storage resources available

• 1. ¡~2000 ¡cores ¡AMD ¡Opteron ¡2218 ¡
• 2. ¡2 ¡GB ¡of ¡RAM ¡per ¡core ¡ ¡
• 3. ¡LSF ¡as ¡LRMS ¡everywhere ¡
• 4. ¡Infiniband-­‑4X ¡at ¡all ¡sites ¡
• 1. ¡200+ ¡TB ¡of ¡raw ¡disk ¡storage ¡ ¡
• 2. Distributed/parallel ¡GPFS ¡filesystem ¡

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 8

Commercial software available

(http://www.pi2s2.it/tecn/index.php?option=com_content&task=view&id=27&Itemid=58)

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 9

Motivations

• Several Grid applications (e.g.: FLUENT) running in the

COMETA Infrastructure produce media content (e.g.: images) that can be collected and encoded as a unique video to be visualized also when the job that produces the images is still running.

– This feature maybe helpful for both demonstration purposes or to control job evolution.

• Visualiza*on ¡Element is a post-processing tool allowing to

show, starting from a list of images sequence, the video

• utput produced by a producer application.

Fig.1 ¡-­‑ ¡Some ¡frames ¡produced ¡by ¡FLUENT ¡and ¡streamed ¡by ¡the ¡VisualElement ¡service. ¡ ¡

64 cores for 14 CPU days Environment protection

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 10

Deploying the VisualElement on Grid

• VisualElement uses mencoder, VLC ¡VideoLAN ¡Client, ¡Xva and

FFMPEG software packages to encode the media content and produce a video that will be send in streaming.

Top BDII

• Published a new software tag,

VO-cometa-VLC-0.9.9a, for the service.

• The needed packages for Visualization Element can be

downloaded from repos.

• A JDL file is created on-the-fly using a Grid portal to

call the service with the corresponding inputs.

• A first prototype of the Visualization Element on-line

streaming tool has been deployed for the COMETA Grid infrastructure on board of the GENIUS Portal powered by EnginFrame from NICE srl

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 11

Integration of VisualElement in COMETA

• Two different implementations of the service are

available as shown in fig. 1 and fig. 2

• Fig. ¡1 ¡– ¡Genera*on ¡of ¡the ¡video ¡streaming ¡ ¡with ¡a ¡single ¡job

¡

• Fig. ¡2 ¡– ¡Genera*on ¡of ¡the ¡video ¡output ¡using ¡a ¡DAG

¡

Based ¡on ¡LCG ¡Grid ¡CE ¡ Based ¡on ¡gLite ¡3.2 ¡CREAM ¡CE ¡

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 12 12

Commercial & industrial HPC applications

• n Grid: CFD simulations with Fluent

64 ¡cores ¡for ¡14 ¡CPU ¡days ¡ Environment ¡protec*on ¡ (“Le ¡Marmore” ¡falls) ¡ 48 ¡cores ¡for ¡7 ¡CPU ¡days ¡ Engine ¡design ¡

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 13

What’s next ? The Stereoscopy

• It is the ability of creating, or enhancing, the illusion of

depth in a flat image:

• by presenting two offset images, respectively for left and right eye,

combined together in a single one, using some optical technique;

• This is exactly what our brain does with the “images” taken from

the eyes.

• To implement the Stereoscopy, we can use three

different methods: – Using two separated sources:

– the user wears eyeglasses to combine the images taken from them;

– Using only one source:

– the user wears eyeglasses to split the image taken from it;

– Using a self-stereoscopic source:

– that splits the images directing the light into the viewer's eye, with no need of eyeglasses.

Stereoscopic streams in practice

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 14

• We had to produce for each eye, two separated

streams of images

• observing the simulation from two different point of view;
• Then, we have to combine them in a single stream
• using appropriated video drivers, in according to the chosen

stereoscopic technique; e.g., the anaglyph one.

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 15

First use case

• The service has been tested for the first time encoding a

BLENDER application.

– The

• –

The application The reconstructed video will be send in streaming via UDP , generates for each frame 2 different images; (default port 1234) to the client’s IP using VLC.

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 16

Second use case

• VIsIVO Server for COMETA Grid can create a 3D

view from data tables;

• It runs on every Linux based system:

– The main characteristic is that VisIVO Server gives support for huge data; – No limits (in principle) are fixed to visualize your data. Some limits exist up to now, but the next version will solve these problems; – The FITS and HDF5 data format will be included among supported formats; – Output data from Gadget code will be soon supported; – The project is completed since October 2008.

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 17

Any questions, comments or remarks are very welcome. Please contact: giuseppe. andronico@ct.infn.it

ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 18

• G. Andronico, R. Barbera, M. Iacono Manno and G. La Rocca.

VisualGrid: On-Line Video Streaming for Application Control and Demonstration. Proceedings of the Final Workshop of Grid Projects “PON Ricerca 2000-2006, Avviso 1575”, 2009

References & Links