[PPT] - Topics Scenarios and Motivation Image Formats Processing Pipelines PowerPoint Presentation

SLIDE 1

Web3D 2013 Tutorial:

Medical and Volume Rendering with X3D

Nicholas F. Polys, Ph.D. Andy Wood Virginia Tech Peter Leskovsky, John Congote, Esther Novo, Luis Kabongo, Ph.D. VicomTech

SLIDE 2

Topics

Scenarios and Motivation
Image Formats
Processing Pipelines
Presenting and Publishing

(X3D, X3DOM)

SLIDE 3

Volume Rendering more than medical imaging

Geology
Meteorology
FlowVisualization
Microscopy
Paleontology
Non-invasive sensing:

– Transportation security – Manufacturing QA

….

SLIDE 4

Web3D Medical Working Group

Reproducible rendering for stakeholders

throughout the healthcare enterprise

An n-D Presentation must include:

– Structured and interactive virtual environment display (2D & 3D objects and time series) – Platform-independent, royalty-free technology to enable vendor innovation – Can be rendered with or without stereoscopy – Openly-published

SLIDE 5

X3D Volume Rendering

Necessary and Sufficient node set for

industry’s Greatest Common Denominator:

– Volume Component: render styles – X3D version 3.3

Two independent implementations:

– www.h3d.org – www.instantreality.org

SLIDE 6

Volume Processing and Presentation Tools

Data

– Sample xxxxx.dcm – X3D Content Examples http://www.web3d.org/x3d/content/examples/Basic/VolumeRendering/index.html – Volvis.org – http://www.osirix-viewer.com/datasets/

Warning: some are compressed w/ jpg2000 !
Tools

– ImageJ : http://rsbweb.nih.gov/ij/

Plugins: DICOM reader, DICOM header inspector

– Seg3D.org – Slicer.org; ITK-Snap – X3D-Edit 3.3

Viewers

– H3D.org – InstantReality.org – MedX3DOM

SLIDE 7

X3D Volume Rendering

Composable Render Styles covering the state of the art

– Formalizes parameters and transfer functions for 3D rendering & blending

BoundaryEnhancementVolumeStyle
CartoonVolumeStyle
ComposedVolumeStyle
EdgeEnhancementVolumeStyle
OpacityMapVolumeStyle
ProjectionVolumeStyle
ShadedVolumeStyle
SilhouetteEnhancementVolumeStyle
ToneMappedVolumeStyle

– Greatest Common Denominator

Assign different RenderStyles to different segments, blend two volumes

– BlendedVolumeStyle – SegmentedVolumeData – IsoSurfaceVolumeData

Clipping Planes are already specified in X3D 3.2 Rendering Component

Silhouette Opacity Map Cartoon

SLIDE 8

Medical Interchange Profile

X3D 3.3
Minimal X3D node set (footprint) to meet DICOM requirements:

– Core – Time – Networking – Grouping – Rendering – Shape – Geometry3D – Geometry2D – Text – Lighting – Texturing – Interpolation – Navigation – Environmental effects – Event utilities – Texturing3D – Volume rendering Includes polygon, line and point rendering; metadata on any node

SLIDE 9

ISO Process

[ X3D 3.3 ] ISO/IEC DIS 1 19775-1:2008

–Clause 33 : Texturing3D Component –Clause 41 : Volume Rendering Component –Annex L: Medical Interchange Profile

Final Draft International Standard (FDIS)
Change document and unified spec drafts

available to Web3D & DICOM members (* now published)

SLIDE 10

Online Videos & Examples

Web Video summary:

– Extensible 3D (X3D) Volume Rendering

https://snoid.sv.vt.edu/medical/X3DVolumes/videos/Volume

Vis-X3D-collected.mp4 (65 mb)

X3D Examples

– http://www.web3d.org/x3d/content/examples/Basic/ VolumeRendering/index.html

For other other Videos, Images and Scenes using

the VolumeData and VolumeRenderStyles of X3D 3.3 Clause 41, please visit:

– https://snoid.sv.vt.edu/medical/X3DVolumes/

SLIDE 11

Volume Presentation

Many techniques:

Volume rendering

– 3DSplatting, ray tracing, pixelshaders – Established CPU and GPU algorithms

Surfaces – actual meshes
Segments – identifying voxels as groups
ISOSurfaces – rendered at a threshold

I: Scenarios and Motivation

SLIDE 12

X3D Presentation Demo

Nicholas Polys, Andy Wood Virginia Tech

Screenshots

SLIDE 13

Example Volume Rendering Styles

(Head MRI, XML encoding) <Transform DEF='backdrop' > <VolumeData dimensions='.75 1 1' >

SLIDE 14

Example Volume Rendering Styles

(Head MRI, optic segment) <ISOSurfaceVolumeData surfaceValues='.15' dimensions='.75 1 1' > <ImageTexture3D containerField="voxels" url='"./Segments/masked-optic.nrrd"'/> <CartoonVolumeStyle /> </ISOSurfaceVolumeData> (Head MRI, cerebrum segment)

I: Scenarios and Motivation

SLIDE 15

Image Formats

2D image slices with order and spacing define

the voxels

Typical image stacks as:

– Tiff – Png – dcm

Sizing, smoothing, resampling can happen in

2.5D space (image stack)

SLIDE 16

Image J

Many useful (and programmable) functions to

deal with image stacks:

Invert, Blur, resample, resize reduce bit depth
ImageJ : http://rsbweb.nih.gov/ij/
Lots of Plugins: eg DICOM reader, DICOM header

inspector

III: Processing Pipelines

SLIDE 17

DICOM.dcm

Medical.nema.org
Ongoing work Item for n-Dimensional

Presentation State

Working Group 11 (Display)
Extensive use cases documented
MPR and CPR parameters of members

surveyed

Supplement draft in preparation

SLIDE 18

NRRD

Nearly Raw Raster Data
http://teem.sourceforge.net/nrrd/
Nrrd and the teem library facilitates the pre-

processing tasks (cropping, quantizing, resampling), as well as the basic analysis tasks which help to understand the structure of a dataset (histograms, maximum intensity projections, connected components)

II: Image Formats

SLIDE 19

Seg3D

Segment3D – a Gui tool With multiple methods for segmentation :

Neighborhood Growth
Thresholding

Export .nrrd

all-in-one
separate

files Seg3d.org

III: Processing Pipelines

SLIDE 20

Slicer

Slicer.org

SLIDE 21

ITKSnap

http://www.itksnap.org

SLIDE 22

X3D Edit 3.3

X3D-Edit is an open-source Extensible 3D

(X3D) Graphics authoring tool for simple error- free editing, authoring and validation of X3D scenes.

Lots of translation and QA features
Stand-alone or Netbeans plug-in
https://savage.nps.edu/X3D-Edit/

SLIDE 23

Volume acquisition DICOM data Surfacing Viewer workstation Compose WCS Model (Scene) for Rendering; assign appearances, views, etc. Segmentation nDPS

Processing Scheme for DICOM Use Cases

X3D

Segmentation: Supp 111 Mesh: Supp 132 Other 3D data

(e.g. CAD, X3D)

SLIDE 24

Volume Processing Pipelines

Source Data X3D

<ImageTexture3D> -voxels

<ImageTexture3D> -voxels <SegmentedVolumeData> <ImageTexture3D> -voxels <ImageTexture3D>

segment identifiers

Image Processor (ImageJ) Volume Data Processing (TEEM) Segmentation

Seg3D
Slicer
ITKSnap

DICOM Plugin Image Stack DICOM Stack Raw Volume NRRD Volume

Segment IDs Individual Segments

SLIDE 25

Volume Image Data

Source Data X3D

<ImageTexture3D> -voxels

<ImageTexture3D> -voxels <SegmentedVolumeData> <ImageTexture3D> -voxels <ImageTexture3D>

segment identifiers

Image Processor (ImageJ) Volume Data Processing (TEEM) Segmentation

Seg3D
Slicer
ITKSnap

DICOM Plugin Image Stack DICOM Stack Raw Volume NRRD Volume

Segment IDs Individual Segments

SLIDE 26

Source Data

Stacks of 2D images with order and spacing data

generated by imaging machinery

Some typical file formats for image stacks:

– Tiff – Png – Dcm

Image processing techniques can accomplish

much of the work for rendering in 2.5D

– Smoothing, (simple) segmentation, resampling, contrast, etc.

SLIDE 27

Working with Images

Resolution – number of pixels in x and y dimensions
Channels – color images are 3 or 4 channel (RGB or RGBA), typical

volume stack is single channel (greyscale)

Bit Depth – number of bits to encode value of a pixel, per channel
Volume data becomes large quickly:

– 512 512x512 8-bit greyscale images: 134 million voxels and 128MB (raw) – 1024^3 volume, 16-bit RGB: 1.07 billion voxels and 48GB (raw)

ImageJ

– Image processor designed to work with stacks – Many useful (and scriptable) functions: invert, blur, resample, resize, mask, etc. – Website: http://rsbweb.nih.gov/ij/

SLIDE 28

Volume Data

Source Data X3D

<ImageTexture3D> -voxels

<ImageTexture3D> -voxels <SegmentedVolumeData> <ImageTexture3D> -voxels <ImageTexture3D>

segment identifiers

Image Processor (ImageJ) Volume Data Processing (TEEM) Segmentation

Seg3D
Slicer
ITKSnap

DICOM Plugin Image Stack DICOM Stack Raw Volume NRRD Volume

Segment IDs Individual Segments

SLIDE 29

Volume Data Formats

Raw voxel data (image stack)

– Requires external metadata

DICOM (.dcm)

– Digital Imaging and Communication in Medicine: http://medical.nema.org – [MORE POINTS HERE]

NRRD (.nrrd)

– Nearly Raw Raster Data: http://teem.sourceforge.net/nrrd – Wrapper for raw or compressed image stacks with plain text metadata – TEEM library for pre-processing and analysis

SLIDE 30

Segmentation

Source Data X3D

<ImageTexture3D> -voxels

<ImageTexture3D> -voxels <SegmentedVolumeData> <ImageTexture3D> -voxels <ImageTexture3D>

segment identifiers

Image Processor (ImageJ) Volume Data Processing (TEEM) Segmentation

Seg3D
Slicer
ITKSnap

DICOM Plugin Image Stack DICOM Stack Raw Volume NRRD Volume

Segment IDs Individual Segments

SLIDE 31

Segmentation

Transfer functions and rendering techniques

are powerful, but not always sufficient

Segmentation - mark features of interest for

special rendering treatment

Some techniques:

– Hand segmentation (painting) – Threshold (used in many approaches) – Volume growing – Connected components

SLIDE 32

Segmentation Tools

Seg3D

– Large selection of segmentation algorithms and tools – Layered segment masks with multi-layer operations and export flexibility – http://www.sci.utah.edu/cibc-software/seg3d.html

ITKSnap

– Active contour segmentation (volume growing) and manual tools – http://www.itksnap.org/pmwiki/pmwiki.php

3D Slicer

– Volume manipulation and segmentation, with a focus on registration (multiple volumes) and rendering – http://slicer.org/

SLIDE 33

Volume Rendering With X3D

Source Data X3D

<ImageTexture3D> -voxels

<ImageTexture3D> -voxels <SegmentedVolumeData> <ImageTexture3D> -voxels <ImageTexture3D>

segment identifiers

Image Processor (ImageJ) Volume Data Processing (TEEM) Segmentation

Seg3D
Slicer
ITKSnap

DICOM Plugin Image Stack DICOM Stack Raw Volume NRRD Volume

Segment IDs Individual Segments

SLIDE 34

Volume Rendering With X3D

Volume rendering component added in X3D 3.3

– [ISO document info here]

Examples found at:

http://www.web3d.org/x3d/content/examples/B asic/VolumeRendering/

Browsers:

– H3D (full support): http://www.h3dapi.org/ – InstantReality (partial support): http://instantreality.org/

[not sure about officially supported file types, per

browser]

SLIDE 35

Rendering a Volume: VolumeData Node

Most basic volume rendering node in X3D
Contains the voxels to be rendered and render

styles to do so

SLIDE 36

Rendering a Volume: IsoSurfaceVolumeData

Similar to the basic VolumeData node, but

renders a surface across voxel gradients

SLIDE 37

Rendering a Volume: SegmentedVolumeData

Allows different render styles for segments of

interest

More on this later!

SLIDE 38

Applying Distinctive Rendering Techniques

VolumeStyle nodes allow different rendering

techniques/filters

– Of the form *VolumeStyle – Example: <ToneMapVolumeStyle />

Child to a volume data node

– Exception: composing and blending

Default style – OpacityMapVolumeStyle with a

linear, alpha-only transfer function

– Interpolates voxel opacity linearly by intensity

SLIDE 39

Opacity Map VolumeStyle

The default style, the basis for all enhancement styles
Has only one field, “transferFunction”

– Two or three dimensional texture – One to four components

Default With Transfer Function

SLIDE 40

Projection VolumeStyle

A raycasting technique
Depending on the value of the “type” field, returns either the MAX,

MIN, or AVERAGE of the voxel values along the ray

If “intensityThreshold” is specified, returns the first local min/max

above/below the threshol

Max Projection Threshold Max Default Style

SLIDE 41

Per-Voxel VolumeStyles

View-Dependent

– Opacity Map (default) – Enhancement Styles

Boundary, Edge, Silhouette

– Cartoon

Lighting-Dependent

– Tone Map – Shaded

Opacity Map Edge Enhanced Cartoon Shaded Tone Map

SLIDE 42

Boundary Enhancement Style

Modifies voxels based on how quickly their

surface normals are changing:

Default Boundary Enhanced

SLIDE 43

Edge Enhancement

Voxels are colored based on how close to perpendicular

their normal are to the view, outside of a threshold.

Useful for surface features, not internal features.

Default Edge Enhanced

SLIDE 44

Silhouette Enhancement

Modifies the color and opacity of voxels based on their normal values
Unlike edge enhancement, it can be used to reveal internal features

Default Silhouette Enhanced

SLIDE 45

Cartoon VolumeStyle

Renders voxels based on the normal value as one of a specified

number of color steps between an orthogonal (plane surface) color and parallel color:

2 color steps 4 color steps 8 color steps

SLIDE 46

Shaded VolumeStyle

Voxel appearance is controlled by a material node, similar to normal

geometry (relative to light source)

Can be computationally expensive

Default Shaded

SLIDE 47

Tone Mapped VolumeStyle

Uses the Gooch shading model to color voxels based on

their orientations relative to a light source, between a warm (facing light) and cool (facing away) color

Default Tone Mapped

SLIDE 48

Combining Styles: ComposedVolumeStyle

Apply multiple rendering styles to a volume
Styles are applied strictly in the order they

appear

All per-voxel style nodes are composable

– ProjectionVolumeStyle is not composable

SLIDE 49

Combining Styles: ComposedVolumeStyle

+

Style1 (Edge Enhance) Style2 (Silhouette) Composed Styles

SLIDE 50

Multipart Volumes: SegmentedVolumeData

Requires two textures: voxels (as normal) and

segment identifiers for each voxel

Specify multiple render styles, in

identifier/segment order (default is opacity map)

– If more segments than styles, last style applies to remaining segments

Individual segments may be turned on or off

using “segmentEnabled” field

– Styles still required to maintain indexing

SLIDE 51

Multipart Volumes: SegmentedVolumeData

Voxels Segments Styles Default Segmented Volume

SLIDE 52

Multipart Volumes: Multiple Volumes

Use different VolumeData nodes to represent

each segment

More computationally expensive than

SegmentedVolumeData, but more flexible

– Can manipulate individual segments – More fine-grained rendering control

Requires masked volumes (more pre-

processing)

SLIDE 53

Multipart Volumes: Multiple Volumes

Segment1 Segment2 Style 1 Style 2 <VolumeData dimensions='.75 1 1' > <ImageTexture3D containerField="voxels" url='"./Segments/masked-halfhead.nrrd"'/> <OpacityMapVolumeStyle /> </VolumeData> <VolumeData dimensions='.75 1 1' > <ImageTexture3D containerField="voxels" url='"./Segments/masked-cerebrum.nrrd"'/> <ShadedVolumeStyle lighting="TRUE" shadows="TRUE" > <Material diffuseColor='0 .5 1' specularColor='1 1 1' ambientIntensity='0.8' shininess='0.08' /> </ShadedVolumeStyle> </VolumeData>

1 2 3 4 5

Composited Volume

SLIDE 54

Multipart Volumes: Blended Render Style

Blends multiple volumes/segments in one

VolumeData node

BlendedVolumeStyle node contains its own

voxels and render style nodes

– Contained voxels rendered according to styles, then blended with parent – Weight of blending is configurable

BlendedVolumeStyle is composable and

nestable!

SLIDE 55

Multipart Volumes: BlendedVolumeStyle

+

Volume1 Volume2 Style2 Default Style1 Volume1 (Default) Volume2 (Tone Map) Blended Volume <VolumeData dimensions='512 512 452' containerField='children'> <BlendedVolumeStyle weightConstant1='0.51' enabled='true' weightConstant2='0.5' weightFunction1='CONSTANT' weightFunction2='CONSTANT' containerField='renderStyle'> <ToneMappedVolumeStyle enabled='true' coolColor='0 0 1 0' warmColor='1 1 0 0' containerField='renderStyle'/> <ImageTexture3D containerField='voxels' repeatS='false' repeatT='false' repeatR='false' url=' "internals.nrrd" '/> </BlendedVolumeStyle> <ImageTexture3D containerField='voxels' repeatS='false' repeatT='false' repeatR='false' url=' "body.nrrd" '/> </VolumeData>

SLIDE 56

Multichannel Microscopy

Using Channels as segments, each with its own style

https://www.cellimagelibrary.org/
Online Survey:

– Which of the renderings is most helpful to understand the…?

SLIDE 57

Preliminary Results

Proximity

Connectivity Containment

N= 50

SLIDE 58

Presenting & Publishing

X3D WebGL based Volume Rendering

John Congote, Luis Kabongo, Esther Novo, Peter Leskovsky (pleskovsky@vicomtech.org)

SLIDE 59

Welcome to San Sebastian

SLIDE 60

WHERE WE CAME FROM

SLIDE 61

Volume Visualization (back in 2011)

Pure WebGL implementation

<HTML><HEAD><META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> <TITLE>Real-Time Interactive Visualization of Volumetric Data with WebGL</TITLE> <SCRIPT type="text/javascript" src="sylvester.js"></SCRIPT> <SCRIPT type="text/javascript" src="glUtils.js"></SCRIPT> <SCRIPT type="text/javascript" src="glAux.js"></SCRIPT> <SCRIPT type="text/javascript" src="volumerc.js"></SCRIPT> </HEAD><BODY onload="volumerc_main('./raycast-color.frag','./aorta- low.jpg','./tf.png');" bgcolor="white"> <CENTER> <H1>Real-Time Interactive Visualization of Volumetric Data with WebGL</H1> <canvas id="canvas_win" width="512" height="512" style="border:1px solid black"></canvas> </br> <button type="button" onclick="volumerc_main('./raycast- bw.frag','./aorta-low.jpg','./tf.png');">Aorta LowRes B/W</button> <button type="button" onclick="volumerc_main('./raycast- color.frag','./aorta-low.jpg','./tf.png');">Aorta LowRes Color</button> <button type="button" onclick="volumerc_main('./raycast- bw.frag','./aorta-high.jpg','./tf.png');">Aorta HighRes B/W</button> <button type="button" onclick="volumerc_main('./raycast- color.frag','./aorta-high.jpg','./tf.png');">Aorta HighRes Color</button> <P>Rotation with the mouse, and Alt+Mouse to Zoom</P> </BR> …

SLIDE 62

WebGL Solution

Achievements:

– volume rendering of 3D texture adapted into a 2D atlas – Use of color/opacity transfer function – Existing VR shaders adapted & simplified for performance and compatibility issues

SLIDE 63

From WebGL to …

Drawbacks:

– Hard to maintain as the code was specific to each site- page-solution-dataset … – Existing shaders were not necessarily and easily portable – Hard to integrate in applications, limited interaction, …

We needed something to:

– Facilitate loading volume information – Integrate existing basic interaction solutions – Let us focus on rendering styles instead of rendering tricks – Use a standardised structure to allow portability

SLIDE 64

MedX3DOM (2012)

Implementation to support advanced medical

visualization on the Web without plugins

MEDX3D standard implemented into the X3DOM

framework

– MEDX3D: extension of the X3D ISO standard to support advanced medical visualization functionality – X3DOM: framework for integrating and manipulating X3D scenes as HTML5/DOM elements

SLIDE 65

Methodology for MedX3DOM

Node generation for two components

Texturing3D
X3DTexture3DNode
ComposedTexture3D
ImageTexture3D
PixelTexture3D
TextureCoordinate3D
TextureCoordinate4D
TextureTransformMatrix3D
TextureTransform3D
ImageTextureAtlas
VolumeRendering
X3DComposableVolumeRenderStyleNode
X3DVolumeDataNode
X3DVolumeRenderStyleNode
BlendedVolumeStyle
BoundaryEnhancementVolumeStyle
CartoonVolumeStyle
ComposedVolumeStyle
EdgeEnhancementVolumeStyle
IsoSurfaceVolumeData
OpacityMapVolumeStyle
MPRVolumeStyle
ProjectionVolumeStyle
SegmentedVolumeData
ShadedVolumeStyle
SilhouetteEnhancementVolumeStyle
ToneMappedVolumeStyle
VolumeData

Implemented/In progress Defined Not implemented Abstract

SLIDE 66

Atlas Image Type

Atlas

– Composed image – Parameters

Number of slices
Number of rows
Number of columns

Ex: Brain Atlas Image The texture atlas is the mosaic of all the slices of a volume in one image, the order of the images is given by the rows and columns.

SLIDE 67

New Node ImageTextureAtlas

ImageTextureAtlas node defines a texture map by specifying a single image file that contains slides of complete 3D data. Parameters indicate the information necessary for mapping voxels to the geometry.

ImageTextureAtlas: ImageTexture {

SFInt32 [in] numberOfSlices 0 SFInt32 [in] slicesOverX 0 SFInt32 [in] slicesOverY 0

}

SLIDE 68

Modified OpacityMapVolumeStyle

OpacityFactor: Global contrast value of the volume data LightFactor: Global radiance of light value of the volume data

OpacityMapVolumeStyle : X3DComposableVolumeRenderStyleNode {

SFBool [in,out] enabled TRUE SFNode [in,out] metadata NULL [X3DMetadataObject] SFNode [in,out] transferFunction NULL [X3DTexture2DNode,X3DTexture3DNode] SFFloat [in,out] opacityFactor 0.01 [0,1] SFFloat [in,out] lightFactor 0.3 [0,1]

}

SLIDE 69

X3DOM (Simple Example)

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1- strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="X-UA-Compatible" content="chrome=1" /> <meta http-equiv="Content-Type" content="text/html;charset=utf-8" /> <title>Hello World</title> <link rel="stylesheet" type="text/css" href="x3dom.css" /> <script type="text/javascript" src="x3dom.js"></script> </head> <body> <h1>XHTML Hello World</h1> <p>With X3D-namespace and case senstive element/node names. Works with self-closing tags (e.g. Viewpoint and Material)</p> <X3D xmlns="http://www.web3d.org/specifications/x3d-namespace" showStat="false" showLog="false" x="0px" y="0px" width="400px" height="400px“ altImg="helloX3D-alt.png"> <Scene> <Viewpoint position='0 0 10' /> <Shape> <Appearance> <Material diffuseColor='0.603 0.894 0.909' /> </Appearance> <Box DEF='box'/> </Shape> </Scene> </X3D> </body> </html>

SLIDE 70

X3DOM (Simple Example)

SLIDE 71

MEDX3D

<?xml version="1.0" encoding="utf-8"?> <X3D version="3.2" profile="MedicalInterchange"> <Scene> <Group> <NavigationInfo type='"EXAMINE" "WALK"'/> <Viewpoint description="first" position="0 0 0.6"/> <Background skyColor="1 1 1"/> <Transform scale=".001 .001 .001">  <VolumeData DEF="volume” rayStep="0.01“ useSlicing="false“ dimensions="250 250 156"> <Image3DTexture containerField="voxels“ url='"data/S76280/IM00010“> </VolumeData> </Transform> </Group> </Scene> </X3D>

SLIDE 72

MedX3DOM

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN“ "http://www.w3.org/TR/xhtml1/DTD/xhtml1- strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="X-UA-Compatible" content="chrome=1" /> <meta http-equiv="Content-Type" content="text/html;charset=utf-8" /> <title>Hello World</title> <link rel="stylesheet" type="text/css" href="x3dom.css" /> <script type="text/javascript" src="x3dom.js"></script> </head> <body> <X3D xmlns='http://www.web3d.org/specifications/x3d-namespace' showStat='true' showLog='true' width='500px' height='500px'> <Scene> <Background skyColor='0.0 0.3 0.65'/> <Viewpoint description='Default' zNear='0.0001' zFar='100'/> <Transform> <VolumeData id='volume' dimensions='4.0 4.0 4.0'> <ImageTextureAtlas containerField='voxels' url='media/volume/aorta4096.png' numberOfSlices='96' slicesOverX='10' slicesOverY='10'/> <OpacityMapVolumeStyle lightFactor='0.01'> <ImageTexture containerField='transferFunction' url='transfer.png'/> </OpacityMapVolumeStyle> </VolumeData> </Transform> </Scene> </X3D> </body> </html>

SLIDE 73

Volume Visualization 2012

SLIDE 74

Aorta Example

<X3D xmlns='http://www.web3d.org/specifications/x3d-namespace’ showStat='true' showLog='true' width='500px' height='500px'> <Scene> <Background skyColor='0.0 0.0 0.0'/> <Viewpoint description='Default' zNear='0.0001' zFar='100'/> <Transform> <VolumeData id='volume' dimensions='4.0 4.0 4.0'> <ImageTextureAtlas containerField='voxels' url='media/volume/aorta4096.png' numberOfSlices='96' slicesOverX='10' slicesOverY='10'/> <OpacityMapVolumeStyle lightFactor='0.02'

pacityFactor='0.4'>

http://x3dom.org/x3dom/test/functional/volrenOpacityTestTF_aorta.xhtml

SLIDE 75

Skull Example

SLIDE 76

Brain Example

SLIDE 77

Aorta Example

SLIDE 78

MPR Style (2012)

<X3D xmlns='http://www.web3d.org/specifications/x3d- namespace’ showStat='true' showLog='true' width='500px' height='500px'> <Scene> <Background skyColor='0.0 0.0 0.0'/> <Viewpoint description='Default' zNear='0.0001' zFar='100’ position="0.85713 1.49578 9.85028"

rientation="-0.85521 0.50269 0.12614 0.17466" />

http://x3dom.org/x3dom/test/functional/volrenMPR_aorta.xhtml

SLIDE 79

WORK IN PROGRESS

SLIDE 80

How to get an Atlas? …

Do it by hand with an image editor

– Time consuming

Use an image viewer that can create a

thumbnail atlas for you

– A bit tricky to use, little flexibility

Write your own C++/… application

– A bit long, requires effort and little glory

Use a python script like the one we made for

you

SLIDE 81

NEW Python Script for Atlas

Get it here:

http://www.volumerc.org/documentation.html

Requires Python, PIL
Use it as is:

– Parses all slices in a folder and merges then into a PNG atlas at different resolutions (full, 81922, 40962, 20482, 10242) – Automatic data rescale – DICOM compatibility, uses Window/Level values (requires pydicom and numpy)

Personalize it for your own data

format:

– Implement loadMyData function

Not optimised!! (memory)

Usage: command <InputFolder> <OutputFilename>

<InputFolder> must contain only one set of

PNG files to be processed

<OutputFilename> must contain the path and

base name of the desired output, extension will be added automatically

SLIDE 82

How to extend it …

Follow X3DOM extension guide:

– http://x3dom.org/docs/dev/components.html

Example:

– Implement the iso surface volume raycasting

Process:

– Check the supporting nodes within X3D specification

Found! -> ISOSurfaceVolumeData

– Take the X3DOM source code from github – Extend functionality

SLIDE 83

VolumeRendering.js BEFORE

/* ### ISOSurfaceVolumeData ### */ x3dom.registerNodeType( "ISOSurfaceVolumeData", "VolumeRendering", defineClass(x3dom.nodeTypes.X3DVolumeDataNode, function (ctx) { x3dom.nodeTypes.ISOSurfaceVolumeData.superClass.call(this, ctx); this.addField_MFNode('renderStyle', x3dom.nodeTypes.X3DVolumeRenderStyleNode); this.addField_SFNode('gradients', x3dom.nodeTypes.X3DTexture3DNode); this.addField_MFFloat(ctx, 'surfaceValues', []); this.addField_SFFloat(ctx, 'contourStepSize', 0); this.addField_SFFloat(ctx, 'surfaceTolerance', 0); }, { nodeChanged: function() {}, fieldChanged: function(fieldName) {} } ) );

SLIDE 84

VolumeRendering.js AFTER

/* ### ISOSurfaceVolumeData ### */ x3dom.registerNodeType( "ISOSurfaceVolumeData", "VolumeRendering", defineClass(x3dom.nodeTypes.X3DComposableVolumeRenderStyleNode, function (ctx) { //Some initialization code…

}, {

nodeChanged: function() {}, fieldChanged: function(fieldName) {}, uniforms: function() { //Define the uniforms for the shaders

},

textures: function() { //Prepare the textures

},

vertexShaderText : function() { //Generate vertex shader

},

fragmentShaderText : function (numberOfSlices, slicesOverX, slicesOverY) { //Generate fragment shader

} } ) );

SLIDE 85

Usage

SLIDE 86

IN PROGRESS Result

ISO=20% ISO=40% ISO=60% ISO=80% ISO=80% Lighting OFF

SLIDE 87

FUTURE

SLIDE 88

DICOM, NRRD, RAW Format??

Javascript is very slow to process binary data
The loaders of this kind of volume data are

based in C and the conversion or reimplementation is difficult

Other types of volume formats exists, like

MHD, NetCDF, …., imposible to implement all.

MEDX3DOM is for the web, using web formats

SLIDE 89

MEDICAL IMAGING

DICOM format is ´´the´´ standard for medical

imaging, but internally is a container of several kind of data, sound, 2D images, 3D images, metadata,etc.

WADO: It is a webservice which provides

medical images through the web stored in a PACS server.

MEDX3DOM: support WADO (Web Access to

DICOM Objects)

SLIDE 90

MEDX3DOM Future …

Next improvements

– Integration Combination with mesh models – Implementation of different styles (MIP, X- Ray, Composed …) – Lighting (Phong, Global illumination …) – Animation (4D timesteps, video, flow animation …) – Data transfer optimization (streaming, compression,…) … collaborative visualization

SLIDE 91

Volume Rendering beyond medical imaging

Geovisualization
Volume Attention

Maps

Meteorology
FlowVisualization
Confocal microscopy
….

SLIDE 92

Keep updated

Links:

– http://demos.vicomtech.org – http://www.volumerc.org – http://www.web3d.org/x3d/specifications/x3d – http://www.x3dom.org

SLIDE 93

Thank you!

Acknowledgements:

– Web3D Consortium (X3D / MEDX3D) – Fraunhofer IGD (X3DOM) – John Edgar CONGOTE CALLE (main architect and developer of this initiative) ... now Developer Relations Engineer @Unity3D