- An overview - Marcel Lthi Graphics and Vision Research Group - - PowerPoint PPT Presentation

β–Ά
an overview
SMART_READER_LITE
LIVE PREVIEW

- An overview - Marcel Lthi Graphics and Vision Research Group - - PowerPoint PPT Presentation

Apr 30, 2023 180 likes β€’454 views

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL Probabilistic Morphable Models - An overview - Marcel Lthi Graphics and Vision Research Group Department of Mathematics and Computer Science University of Basel >

slide-1
SLIDE 1

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Probabilistic Morphable Models

  • An overview -

Marcel LΓΌthi

Graphics and Vision Research Group Department of Mathematics and Computer Science University of Basel

slide-2
SLIDE 2

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Outline

Analysis by synthesis Medical image analysis verse computer vision The space of images

Parame ters πœ„

Comparison Update πœ„ Synthesis πœ’(πœ„)

slide-3
SLIDE 3

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Summer school Online Course

Probabilistic Morphable Models

Shape Modelling Model fitting

Scalismo

slide-4
SLIDE 4

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Conceptual Basis: Analysis by synthesis

Parameters πœ„

Comparison Update πœ„ Synthesis πœ’(πœ„)

  • If we are able to synthesize an image, we can explain it.
  • We can explain unseen parts and reason about them
slide-5
SLIDE 5

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Synthesizing images

Parameters πœ„

Comparison Update πœ„ Synthesis πœ’(πœ„)

Computer graphics

slide-6
SLIDE 6

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Synthesizing images

Parameters πœ„

Comparison Update πœ„ Synthesis πœ’(πœ„)

  • Warping atlas images
  • Simulating DRRs
  • Virtual CTs
slide-7
SLIDE 7

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Mathematical Framework: Bayesian inference

  • Principled way of dealing with uncertainty.

Parameters πœ„

Comparison: π‘ž Image πœ„) Update using π‘ž(πœ„|Image) Synthesis πœ’(πœ„)

Prior π‘ž(πœ„)

slide-8
SLIDE 8

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Algorithmic implementation: MCMC

Parameters πœ„

Comparison: π‘ž Image πœ„) Sample from π‘ž(πœ„|Image) Synthesis πœ’(πœ„)

Prior π‘ž(πœ„)

slide-9
SLIDE 9

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Course programme

Mor

  • rning

Aft fternoon

Tuesday

  • Introduction
  • The course at a glance
  • Basics of Co

Computer Graphics

  • Basic tasks in Scalismo-faces
  • Ba

Bayesian mod

  • del

elli ling

  • Welcome reception

Wednesday

  • Probabilistic mod
  • del

l fit fitti ting usin ing g MCM CMC

  • Exercises: MCMC Fitting
  • Introduction to course project

Thursday

  • Face im

image analy lysis

  • Course project

Friday

  • Connections to medical image

analysis

  • Advanced topics in Gaussian

processes

  • Course project

Saturday

  • Project presentation
  • Social event
slide-10
SLIDE 10

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Pattern Theory

Computational anatomy Research at Gravis

The course in context

11 This course / PMM Text Music Natural language Medical Images Fotos Speech Ulf Grenander

slide-11
SLIDE 11

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Pattern theory vs PMM

  • Pattern theory is about developing a theory for understanding real-

world signals

  • Probabilistic Morphable Models are about usin

ing theoretical well founded concepts to analyse images.

  • GPs for modelling
  • MCMC for model fitting
  • Working software

12

slide-12
SLIDE 12

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Images: Medical Image Analysis vs Computer Vision

Source: OneYoungWorld.com

slide-13
SLIDE 13

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Images in medical image analysis

Goal: Measure and visualize the unseen

  • Acquired with specific purpose
  • Controlled measurement
  • Done by experts
  • Calibrated, specialized devices

18

Source: www.siemens.com

slide-14
SLIDE 14

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Images in medical image analysis

19

  • Images live in a coordinate system (units: mm)

(0,0,0)

slide-15
SLIDE 15

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Images in medical image analysis

20 (0,0,0) (100,720, 800) 300 𝑛𝑛 280 𝑛𝑛

slide-16
SLIDE 16

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Images in medical image analysis

Values measure properties of the patient’s tissue

  • Usually scalar-valued
  • Often calibrated
  • CT Example:
  • 1000 HU -> Air

3000 HU -> cortical bone

21

slide-17
SLIDE 17

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Images in computer vision

Goal: Capture what we see in a realistic way

  • Perspective projection from 3D object

to 2D image

  • Many parts are occluded

24

slide-18
SLIDE 18

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Images in computer vision

  • Can be done by anybody
  • Acquisition device usually unknown
  • Uncontrolled background, lighting, …
  • No clear scale
  • What is the camera distance?
  • No natural coordinate system
  • Unit usually pixel

25

Source: twitter.com

slide-19
SLIDE 19

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Images in computer vision

  • Pixels represent RGB values
  • Values are measurement of light
  • Reproduce what the human eye would

see

  • Exact RGB value depends strongly on

lighting conditions

  • Shadows
  • Ambient vs diffuse light

26

slide-20
SLIDE 20

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Medical l im image

  • Controlled measurement
  • Values have (often) clear

interpretation

  • Explicit setup to visualize unseen
  • Coordinate system with clear scale

Computer vis vision

  • Uncontrolled snapshot
  • Values are mixture of different

(unknown factors)

  • Many occlusion due to perspective
  • Scale unknown

Images: Medical Image analysis vs Computer Vision

Many complications of computer vision arise in different form also in a medical setting.

slide-21
SLIDE 21

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

The space of images

28

slide-22
SLIDE 22

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

The space of images

29

slide-23
SLIDE 23

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

The space of images

  • 10 x 10 image
  • 2 colours

How long would you need to watch TV (24 fps) until you have seen all such images?

  • Number of images: 2100 β‰ˆ 1.2e30
  • Watching 100 years continuously

24 x 60 x 60 x 24 x 365 x 100 β‰ˆ7.5e10

30

Source: bbc.co.uk

slide-24
SLIDE 24

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

The space of images

  • Most images are uninteresting
  • Only very few of all possible

images are of interest to us

Think of interesting images like this Not this ℝ100

slide-25
SLIDE 25

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Structure in images

  • Images have a rich structure
  • Images are depictions of ob
  • bjects in the

world

  • Structure is due to objects, laws and

processes

32

Our mission:

  • Model this structure
  • Needs only few parameters
  • Explain image by finding appropriate

parameters that reflect objects / laws / processes

slide-26
SLIDE 26

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 | BASEL

Next stop: Computer graphics

Image Parameters πœ„

Comparison: π‘ž Image πœ„) Sample from π‘ž(πœ„|Image) Synthesis πœ’(πœ„)

  • Computer graphics