A Deep Learning Approach for Motion Forecasting Using 4D OCT Data - - PowerPoint PPT Presentation

Institute of Medical Technology and Intelligent Systems

A Deep Learning Approach for Motion Forecasting Using 4D OCT Data

Marcel Bengs, Nils Gessert, Alexander Schlaefer

Hamburg University of Technology, Institute of Medical Technology and Intelligent Systems

July 2020 Slide 2

Motivation: Motion Forecasting

Radiotherapy Intraoperative Imaging Motion Motion ROI

July 2020 Slide 3

Deep Learning and Spatio-temporal OCT

t OCT volumes over time template 4D deep learning We propose a deep learning approach for motion estimation and forecasting using sequences of OCT volumes deep learning (CNN) motion estimation motion estimation motion forecasting

1

1 Gessert, Nils, et al. Two-path 3D CNNs for calibration of system parameters for OCT-based motion compensation. In: Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and

• Modeling. International Society for Optics and Photonics, 2019. S. 1095108.

Optical Coherence Tomography (OCT) lag between the adjustment and the motion estimation moving state

July 2020 Slide 4

Deep Learning Methods and Data Set

1 Gessert, Nils, et al. Two-path 3D CNNs for calibration of system parameters for OCT-based motion compensation. In: Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and

• Modeling. International Society for Optics and Photonics, 2019. S. 1095108.

Data Set

• Swept-source OCT device (OMES, OptoRes)
• 40 different ROIs of a chicken breast sample
• 100 different trajectories each

1

July 2020 Slide 5

Results and Discussion

Using a stream of volumes improves estimation performance and allows for forecasting

2-Path-CNN3D (2-P-3D) n-Path-CNN3D (n-P-3D) CNN4D (CNN4D) n-Path-CNN4D (n-P-4D) GRU-CNN3D (GRUC3D)

Motion Estimation tn Motion Forecasting tn+1 Motion Forecasting tn+2