An open source framework for graphical related tasks Clment Forest - - PowerPoint PPT Presentation

An open source framework for graphical related tasks

Clément Forest Digital-Trainers www.digital-trainers.com

VgSDK in short

External dependencies

Supported hardware

Base principles

The user describes the

• bjects in a scene graph

The engine manages and optimises all the low level GPU related tasks

Base principles

• Architecture similar to aspect-based engine architecture in

'GPU Pro 4'.

The world is described as an

• riented direct acyclic graph

(DAG) The engine exposes a state and update it during the graph traversal

Root

Setup

Camera

Drawstyle

Scene

Object 1 Material Geometry Object 2

Material

Geometry

Specialised components (aspects) use that state to do their respective tasks (OpenGL rendering, scene manipulation, bounding boxes computation, ...)

Abstraction of the 3D API

• OpenGL is one aspect among others (vgeGL)
• vgeDirectX, vgeOpenGLES, vgeMantle could be implemented in

the same way.

• It is not necessary to implement the totality of the nodes in a given

aspect.

Modularity

• Most of the external dependencies are wrapped and
• ptional.

core

vgBase, vgd, vgDebug, vgm

engines

vge, vgBullet, vgeGL

ui

vgUI, vgGTK, vgQt, vgSDL, vgWx

io

vgio, vgPhysfs, vgOpenAssetImport, vgTrian

algorithms

vgAlg

images

vgCairo, vgFFmpeg vgITK

Modularity

2003 2008 2010 2004 2014

Functionalities

Usually provided by a node in the scene graph.

• Displacement
• Offscreen rendering

Functionalities

• Tesselation
• Adaptative and uniform Phong subdivision
• Stereoscopy (prototype)

Functionalities

• Image processing (basic filters)
• Arithmetics, resize, blur, edge detection, ...
• Complex filters
• antialiasing FXAA, depth of field, noise, ...

Functionalities

• Fluid simulation

Shaders

• Can be defined in a specialised node
• vertex, geometry, tesselation control, tessellation evaluation,

fragment shaders.

• Rapid prototyping,
• But hard to maintain (new shader for every combination of light

and of materials, ...).

Shp< Program > program = Program::create( "myShader" ); program->setVertex( "....." ); program->setFragment( "....." );

Shaders

• Can be automatically generated by the engine
• The shaders are only created on demand (lazy evaluation)
• The combinatorial explosion is manage by the engine
• It is easy to add new functionalities
• A cache system limits unnecessary recompilations

Example of a custom shader

Binding Python

• The nodes and the graphical canvas are exposed in

Python via Swig

• Swig can also be used to enable other programming

languages (C#, java, Lua, javaScript)

• Eg: loading of a scene and modification of the scene graph using

a Python file.

I/O

• 3D data: dae, Blender, 3DSMax or Wavefront files
• Images: png, openexr, jpeg, psd, targa, tiff.
• Image and sound extraction from videos with FFmpeg.
• Loading and display of polices using cairo.
• Data encryption (blowfish).

Examples of uses (Ircad)

Negatoscope 2D and 3D Patient reconstruction

Examples of uses (Ircad)

Ultrasound simulation Surgical planning

Viewer Qt

• Scene graph editor
• Inspection and modification of the automatics shaders.
• Image and video capture
• Camera information capture

Viewer Sofa

• Simulation and rendering in two different threads
• Synchronisation of the visual scene on the simulation
• Visual mappings done in the rendering thread

Laparoscopic simulator

Contact

• Clement.Forest@digital-trainers.com
• https://code.google.com/p/vgsdk/
• Liste de diffusion
• Source, Binairies, SDK, dependancies, ....

Scene description

• Data are stored in nodes
• Nodes are organised

hierarchically in a scene graph (DAG)

• Scene nodes are attached

to each other and follow each others movements

Node types

• Groups to construct a scene

graph (Switch, Separator,

GeoMorph ...)

• Shapes to define 3d objects

(VertexShape, Box, Quad, Grid, ...)

• Attributes to set materials,

lights, post-processing, rigid transformations and so on

• Kits to build higher level nodes

by hiding a sub-scene graph

(Dragger, ...)

Creating Nodes

• Creating node is done using static factories

vgd::Shp< vgd::node::Material> material = vgd::node::Material::create("redMaterial");

• vgd::Shp<> is an alias of boost::shared_ptr<>

Scene graph

• Creating a group node

vgd::Shp< Group > group = Group::create( "rootOfScene" );

• Adding a child

group->addChild( material )

Or group->insertChild( material )

• Access to the first child

vgd::Shp< Material > m = group->getChild<Material>(0);

Canvas

• Search node

result = findFirstByType<VertexShape>(); result = findFirstByRegex("tumor.*");

• Canvas refresh

refresh( REFRESH_IF_NEEDED, ASYNCHRONOUS ); refresh( REFRESH_FORCED, SYNCHRONOUS );

• Ray casting

hitNode = castRay(xMousePosition, yMousePosition); HotNode = castRay(raySource, rayDirection, outputHitTriangle)

Lights

• Create a directional light

light1 = DirectionalLight::create("light1");

• Switch on/off the light

light1->setOn( false );

• Set the direction of the light

light1->setDirection( vgm::Vec3f(0,0, -1) );

Lights (continuation)

• Create a second light

light2 = SpotLight::create("light2", 1/*light index*/);

• Change properties of the second light

light2->setOn(true); light2->setPosition( vgm::Vec3f(0,0, 5) ); light2->setDirection( vgm::Vec3f(0,0, -1) ) light2->setCastShadow( true );

Fields

• Nodes store their data in fields.
• Fields are all dynamic

node->addField( newField );

• Introspection
• Retrieve all field names node->getFieldNames()
• Access to a field node->getField( FieldName );

Fields (continuation)

• Access control
• Read only access to a field

EditorRO<FieldType > editorRO = getFieldRO(name)

• Read write access to a field

EditorRW<FieldType > editorRW = getFieldRW(name)

• Design pattern observer applied to field access.

Loading and modifying meshes

• Loading a mesh

vgOpenAssetImport::Loader loader; Shp< VertexShape > shape = loader.load( filename );

• Accessing its positions and primitives

EditorRW< TMultiField< Vec3f > > editor_pos = shape->getVertexRW(); EditorRW< TMultiField< unsigned long > > editor_ind = shape->getVertexIndexRW(); EditorRW< TMultiField< Primitive > > editor_pri = shape->getPrimitiveRW();

Adaptive Phong Tessellation

// Enables phong tessellation tessProp->setTessellation( TessellationProperties::PHONG ); // Specifies the minimum (1) and maximum (32) tessellation level used by the tessellation control shaders to tessellate the incoming primitive. tessProp->setRange( vgm::Vec2f(1.f, 32.f) ); // When tessellation method is set to PHONG and TessellationLevel.method==PIXELS_PER_EDGE, then this field specifies the desired number of pixels per edge tessProp->setPixelsPerEdge( 15.f ); // Chooses adaptive tessellation using the number of pixels per edge desired as criterion tessLevel->setMethod( TessellationLevel::PIXELS_PER_EDGE );