Augmented Reality for Port Placement and Navigation in Robotically - PowerPoint PPT Presentation

Augmented Reality for Port Placement and Navigation in Robotically Assisted Minimally Invasive Cardiovascular Surgery Marco Feuerstein Fakultät für Informatik, Technische Universität München Klinik für Herz- und Gefäßchirurgie, Deutsches Herzzentrum München feuerste@cs.tum.edu June 26, 2004

Acknowledgements • Co-Authors: – Jörg Traub – Martin Bauer – Eva Schirmbeck – Hesam Najafi – Robert Bauernschmitt – Gudrun Klinker • A.R.T. GmbH – Marc Schneberger • SFB 453 Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 2/12

Problem Statement • Surgeon on a master console controls a slave teleoperator • Where to place the ports for the teleoperator arms? • Where are the teleoperator arms inside the patient’s body? Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 3/12

Workflow Perception Port Planning Validation/Simulation Patient Registration Augmented Reality Analysis Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 4/12

Perception • Acquisition of cross-sectional image slices (CT) • Image segmentation • Reconstruction of a polygonal triangle mesh, which is used to perform the planning Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 5/12

Planning the Teleoperator Arms • Definition of a region of interest by landmarks • Placing of landmarks for the ports of the endoscope and instrument arms • Visualization of corresponding teleoperator arms in a collision-free way • Possibility to interactively adjust all landmarks Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 6/12

Validation of Planned Data • Simulation of the view of the endoscope arm’s camera by a virtual endoscope view • Collision detection between teleoperator arms and critical organs or bones Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 7/12

Port Placement Movie Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 8/12

Patient Registration • Estimation of a rigid transformation from the virtual model to the real patient • Point based registration to match fiducials • Subsampling to find the correspondences Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 9/12

Augmented Reality • Intra-operative port placement by overlaying virtual planned teleoperator arms • Endosopic view additionally showing the real teleoperator arm position Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 10/12

“Global Positioning System” Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 11/12

Thank you for your attention! Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 12/12