Simultaneous Localization and Mapping for Minimally Invasive Surgery - - PowerPoint PPT Presentation

▶
simultaneous localization and mapping for minimally
SMART_READER_LITE
LIVE PREVIEW

Simultaneous Localization and Mapping for Minimally Invasive Surgery - - PowerPoint PPT Presentation

Jun 09, 2023 340 likes •754 views

Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery Simultaneous Localization and Mapping for Minimally Invasive Surgery Introduction University of Bristol using particle filters to track football players from a single static

slide-1
SLIDE 1

Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery

Simultaneous Localization and Mapping for Minimally Invasive Surgery

slide-2
SLIDE 2

Introduction

  • University of Bristol – using particle filters to track football players

from a single static camera.

  • Imperial College - Simultanous Localization and Mapping (SLAM)

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

with by Guang-Zhong Yang and Andrew Davison.

  • Siemens Corporate Research – Valve modelling
slide-3
SLIDE 3

Current Situation

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-4
SLIDE 4

Why is Camera Motion Important

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-5
SLIDE 5

Estimating Camera Motion & Building a 3D Map

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-6
SLIDE 6

Problem: Noisy Sensor Measurements

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

1 mm = Many Pixels

slide-7
SLIDE 7

Noise in the Camera Image

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-8
SLIDE 8

Noisy System

How to we get accurate models and estimates from noisy measurements?

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

Structure From Motion V Simultaneous Localization And Mapping (SLAM)

slide-9
SLIDE 9

Structure From Motion and Simultaneous Localization And Mapping

Structure From Motion

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-10
SLIDE 10

Structure From Motion and Simultaneous Localization And Mapping

SLAM – Filtering Approach

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-11
SLIDE 11

Simultaneous Localization And Mapping Extended Kalman Filter Unscented Kalman Filter Particle Filter Information Filter FastSLAM RatSLAM Augmented Reality Robotic Navigation Wearable Computers Autonomous Vehicles/ Helicopters / Planes / Cars / boats / submarines

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

RatSLAM Submapping Hybrid Systems Dense Sparse Large Scale / boats / submarines Compute Games Games Controllers Smart Phones Space Exploration Surgery

slide-12
SLIDE 12

EKF SLAM [1] - State Extended Kalman Filter SLAM – Davison et al

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

[1] MonoSLAM: Real-Time Single Camera SLAM Andrew Davison, Ian Reid, Nicholas Molton and Oliver Strasse. IEEE PAMI June 2007 (vol. 29 no. 6) pp. 1052-1067

slide-13
SLIDE 13

EKF SLAM – Uncertainty

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-14
SLIDE 14

EKF SLAM - Prediction

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-15
SLIDE 15

EKF SLAM - Prediction

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-16
SLIDE 16

EKF SLAM - Measurement

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-17
SLIDE 17

EKF SLAM – Active Search

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-18
SLIDE 18

EKF SLAM - Update

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-19
SLIDE 19

EKF SLAM - Update

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-20
SLIDE 20

EKF SLAM Adding new features

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-21
SLIDE 21

EKF SLAM Adding new features

  • 1. Feature/Map initialisation
  • 2. Predict motion
  • 3. Feature matching
  • 4. Update Pose and map

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

[1]

slide-22
SLIDE 22

SLAM for MIS

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery. Peter Mountney, Danail Stoyanov, Andrew Davison, Guang-Zhong Yang. MICCAI (1) 2006: pp. 347-354

slide-23
SLIDE 23

SLAM for MIS - Validation

CT

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

A Stereoscopic Fibroscope for Camera Motion and 3D Depth Recovery During Minimally Invasive Surgery. David Noonan, Peter Mountney, Daniel Elson, Ara Darzi and Guang-Zhong Yang. In proc ICRA 2009 , pp. 4463-4468

SLAM

slide-24
SLIDE 24

Dynamic View Expansion

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

Dynamic View Expansion for Minimally Invasive Surgery using Simultaneous Localization And Mapping. Peter Mountney and Guang- Zhong Yang. In Proc EMBC 2009: pp. 1184-1187

slide-25
SLIDE 25

Dense Stereo

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

Real-time Stereo Reconstruction in Robotic Assisted Minimally Invasive Surgery. Danail Stoyanov, Marco Visentini Scarzanella, Philip Pratt and Guang-Zhong Yang. In Proc MICCAI 2010

slide-26
SLIDE 26

SLAM Dense Mapping

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

Dense Surface Reconstruction for Enhanced Navigation in Minimally Invasive Surgery. Johannes Totz, Peter Mountney, Danail Stoyanov, and Guang-Zhong Yang. Accepted to MICCAI 2011§ § § §z

slide-27
SLIDE 27

Optical Biopsy Mapping

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

Mauna Kea - www.maunakeatech.com/

slide-28
SLIDE 28

Optical Biopsy Mapping

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

Optical Biopsy Mapping for Minimally Invasive Cancer Screening. Peter Mountney, Stamatia Giannarou, Daniel Elson and Guang- Zhong Yang. In proc MICCAI(1), 2009, pp. 483-490

slide-29
SLIDE 29

Optical Biopsy Mapping

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-30
SLIDE 30

Motion Compensated SLAM in Dynamic Environments

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-31
SLIDE 31

Motion Compensated SLAM in Dynamic Environments

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

Motion Compensated SLAM for Image Guided Surgery. Peter Mountney and Guang-Zhong Yang. In Proc MICCAI 2010

slide-32
SLIDE 32

Motion Compensated SLAM in Dynamic Environments

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-33
SLIDE 33

Motion Compensated SLAM in Dynamic Environments

2 4 6 8 x 10

  • 3

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

20 40 60 80 100 120 140 160 180 200

  • 10
  • 8
  • 6
  • 4
  • 2

PCA

slide-34
SLIDE 34

Motion Compensated SLAM in Dynamic Environments

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-35
SLIDE 35

Motion Compensated SLAM in Dynamic Environments

Respiration Model EKF state

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

EKF Prediction Model EKF Measurement Model

slide-36
SLIDE 36

Motion Compensated SLAM in Dynamic Environments

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-37
SLIDE 37

Motion Compensated SLAM in Dynamic Environments

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-38
SLIDE 38

Motion Compensated SLAM in Dynamic Environments

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

slide-39
SLIDE 39

Future Work in this Area PTAM DTAM Non rigid structure from motion Incorporation of sensors

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada

Incorporation of sensors Incorporating pre operative models Parallel Approaches Biomechanical Models

slide-40
SLIDE 40

Questions Thank you for listening

Peter Mountney – SLAM for MIS Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

22ndSeptember ,Toronto, Canada