Volume Analysis Using Multimodal Surface Similarity Multimodal - - PowerPoint PPT Presentation

▶

Jul 17, 2023 458 likes •762 views

Volume Analysis Using Multimodal Surface Similarity Multimodal Surface Similarity Martin Haidacher Stefan Bruckner and Martin Haidacher, Stefan Bruckner, and M. Eduard Grller Institute of Computer Graphics and Algorithms Vienna University

SLIDE 1

Volume Analysis Using Multimodal Surface Similarity Multimodal Surface Similarity

Martin Haidacher Stefan Bruckner and Martin Haidacher, Stefan Bruckner, and

M. Eduard Gröller

Institute of Computer Graphics and Algorithms Vienna University of Technology

SLIDE 2

Motivation Multimodal data:

Same object different acquisition techniques Same object, different acquisition techniques One modality evens out drawback of the other

Martin Haidacher 1

SLIDE 3

Motivation Multimodal data:

Same object different acquisition techniques Same object, different acquisition techniques One modality evens out drawback of the other

Martin Haidacher 2

SLIDE 4

Motivation Multimodal visualization:

Side by side view Side-by-side view

Difficult for comparison of both modalities

Volumetric fusion

Differences and/or similarities between Differences and/or similarities between modalities vanish through fusion

Using similarity information to analyze and Using similarity information to analyze and visualize multimodal data

Similarity of isosurfaces for combinations of isovalues

Martin Haidacher 3

SLIDE 5

Multimodal Similarity Map (MSM)

Martin Haidacher 4

SLIDE 6

Multimodal Similarity Map (MSM)

Martin Haidacher 5

SLIDE 7

MSM Calculation

Martin Haidacher 6

SLIDE 8

MSM Calculation

Martin Haidacher 7

SLIDE 9

MSM Calculation

Martin Haidacher 8

SLIDE 10

MSM Calculation

Martin Haidacher 9

SLIDE 11

MSM Calculation

Martin Haidacher 10

SLIDE 12

MSM Calculation

Martin Haidacher 11

SLIDE 13

MSM Example

Martin Haidacher 12

SLIDE 14

Using Multimodal Similarity Maps Applications for multimodal similarity map:

Similarity based exploration Similarity-based exploration

Multimodal similarity map as guidance map

Maximum similarity isosurfaces

Comparison of isosurfaces from two Comparison of isosurfaces from two modalities

Similarity based classification Similarity-based classification

Directly classify multimodal data based on the l i d l i il i multimodal similarity map

Martin Haidacher 13

SLIDE 15

Similarity-Based Exploration The multimodal similarity map can be used to detect important structures detect important structures

E.g. regions of high similarity

Guidance map for the classification CT MRI

Martin Haidacher 14

CT MRI

SLIDE 16

Similarity-Based Exploration

Martin Haidacher 15

SLIDE 17

Similarity-Based Exploration

Martin Haidacher 16

SLIDE 18

Similarity-Based Exploration Similarity-Based Weighting

Use similarity value to manipulate opacity Use similarity value to manipulate opacity

Martin Haidacher 17

SLIDE 19

Similarity-Based Exploration Similarity-Based Weighting

Use similarity value to manipulate opacity Use similarity value to manipulate opacity

Martin Haidacher 18

SLIDE 20

Maximum Similarity Isosurfaces Using multimodal similarity map to find most similar isosurface similar isosurface

One isovalue for one modality is given Lookup in the MSM provides isovalue for most similar isosurface in modality 2 y

Useful for finding differences in both modalities modalities

E.g. artifacts

Martin Haidacher 19

SLIDE 21

Maximum Similarity Isosurfaces

Martin Haidacher 20

SLIDE 22

Similarity-Based Classification Classify multimodal data directly in the multimodal similarity map multimodal similarity map Individual transfer functions are not necessary User defines set of control points Combination of isovalues is classified with Combination of isovalues is classified with control point which is most similar

Metric is based on similarity values

Martin Haidacher 21

SLIDE 23

Similarity-Based Classification hi hi

..

l k

Martin Haidacher 22

k hi

.

SLIDE 24

Similarity-Based Classification

Martin Haidacher 23

SLIDE 25

Similarity-Based Classification Generate clusters based on user-specified control points c control points ci Calculate the cluster centroids hi and use these points to finally generate the clusters The original control point ci is the centroid of The original control point ci is the centroid of this cluster

M i t iti i t ti More intuitive user interaction

Martin Haidacher 24

SLIDE 26

Similarity-Based Classification

Martin Haidacher 25

SLIDE 27

Similarity-Based Classification

Martin Haidacher 26

SLIDE 28

Conclusion Multimodal similarity map can be used to analyze multimodal data analyze multimodal data

Detect similarities/differences in two modalities

A sub-sampled version of the volumes can be used for calculation used for calculation

Reduce calculation time to seconds

MSM can either be used as guidance map in an existing framework or to classify multimodal g y data directly

Martin Haidacher 27

SLIDE 29

Volume Analysis Using Multimodal Surface Similarity Multimodal Surface Similarity

Martin Haidacher Stefan Bruckner and Martin Haidacher, Stefan Bruckner, and

Institute of Computer Graphics and Algorithms Vienna University of Technology

Motivation Multimodal data:

Same object different acquisition techniques Same object, different acquisition techniques One modality evens out drawback of the other

Motivation Multimodal data:

Same object different acquisition techniques Same object, different acquisition techniques One modality evens out drawback of the other

Motivation Multimodal visualization:

Side by side view Side-by-side view

Difficult for comparison of both modalities

Volumetric fusion

Differences and/or similarities between Differences and/or similarities between modalities vanish through fusion

Using similarity information to analyze and Using similarity information to analyze and visualize multimodal data

Similarity of isosurfaces for combinations of isovalues

Multimodal Similarity Map (MSM)

Multimodal Similarity Map (MSM)

MSM Calculation

MSM Calculation

MSM Calculation

MSM Calculation

MSM Calculation

MSM Calculation

MSM Example

Using Multimodal Similarity Maps Applications for multimodal similarity map:

Similarity based exploration Similarity-based exploration

Multimodal similarity map as guidance map

Maximum similarity isosurfaces

Comparison of isosurfaces from two Comparison of isosurfaces from two modalities

Similarity based classification Similarity-based classification

Directly classify multimodal data based on the l i d l i il i multimodal similarity map

Similarity-Based Exploration The multimodal similarity map can be used to detect important structures detect important structures

E.g. regions of high similarity

Guidance map for the classification CT MRI

CT MRI

Similarity-Based Exploration

Similarity-Based Exploration

Similarity-Based Exploration Similarity-Based Weighting

Use similarity value to manipulate opacity Use similarity value to manipulate opacity

Similarity-Based Exploration Similarity-Based Weighting

Use similarity value to manipulate opacity Use similarity value to manipulate opacity

Maximum Similarity Isosurfaces Using multimodal similarity map to find most similar isosurface similar isosurface

One isovalue for one modality is given Lookup in the MSM provides isovalue for most similar isosurface in modality 2 y

Useful for finding differences in both modalities modalities

E.g. artifacts

Maximum Similarity Isosurfaces

Metric is based on similarity values

Similarity-Based Classification hi hi

..

l k

k hi

.

Similarity-Based Classification

M i t iti i t ti More intuitive user interaction

Similarity-Based Classification

Similarity-Based Classification

Conclusion Multimodal similarity map can be used to analyze multimodal data analyze multimodal data

Detect similarities/differences in two modalities

A sub-sampled version of the volumes can be used for calculation used for calculation

Reduce calculation time to seconds

MSM can either be used as guidance map in an existing framework or to classify multimodal g y data directly

Thank you! Thank you!

Questions?