Simultaneous segmentation and correspondence improvement using - - PowerPoint PPT Presentation

Simultaneous segmentation and correspondence improvement using statistical modes

Ayu yushi i Si Sinhaa, Austin Reitera, Simon Leonarda, Masaru Ishiib, Gregory D. Hagera, Russell H. Taylora

• aDept. of Computer Science, the Johns Hopkins University
• bDept. of Otolaryngology-Head and Neck Surgery, Johns Hopkins Medical Institutions

1 14th February, 2017 Orlando, Florida

Functional Endoscopic Sinus Surgery

• What is it?
• Minimally invasive procedure
• Chronic sinusitis, nasal polyps, etc.
• 600,000 procedures in the US per year[1]
• 5-7% result in minor complications[2]
• ~1% result in major complications[2]

Image from Toffelcenter.com

WHY?

2

[1] Bhattacharyya, N.: Ambulatory sinus and nasal surgery in the United States: Demographics and perioperative

• utcomes. The Laryngoscope. 120, 635-638 (2010)

[2] Dalziel, Kim; Stein, Ken; Round, Ali; Garside, Ruth; Royle, P.: Endoscopic sinus surgery for the excision of nasal polyps: A systematic review of safety and effectiveness. American Journal of Rhinology. 20(5), 506-519 (2006)

Frontal sinus Ethmoid sinuses Maxillary sinuses

Sinuses & Nasal Airway: Complex structures with thin boundaries

Fovea eth thmoidalis is: separates the ethmoid cells from the anterior cranial fossa Thickness: ~

~ 0.5

0.5 mm[3] Boundary between the sinuses and the orbit Thickness: ~

~ 0.91

0.91 mm[4]

3

[3] Kainz, J. and Stammberger, H., “The roof of the anterior ethmoid: A place of least resistance in the skull base,” American Journal of Rhinology 3(4), 191-199 (1989). [4] Tao, H., Ma, Z., Dai, P., and Jiang, L., “Computer-aided three-dimensional reconstruction and measurement

• f the optic canal and intracanalicular structures,” The Laryngoscope 109(9), 1499-1502 (1999).

Enhanced Endoscopic Navigation

4

Pre-op CT Intra-op video Labeled Template

De Deformable Reg egis istration Str tructure fr from mot

• tion

Reg egis istration

(ICP[13]/IMLP[14]/IMLOP[15]/ V-IMLOP[16]/etc.)

[13] P.J. Besl, H.D. McKay, A method for registration of 3-D shapes, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992 [14] Billings SD, Boctor EM, Taylor RH. Iterative Most-Likely Point Registration (IMLP): A Robust Algorithm for Computing Optimal Shape Alignment. PLOS ONE 10(3): e0117688, 2015 [15] Seth D. Billings, RH Taylor. Iterative Most Likely Oriented Point Registration. MICCAI, Boston, Proceedings, Part I. Vol. 8673: pp. 178-185, 2014 [16] Seth D. Billings, A Sinha, A. Reiter, S Leonard, M Ishii, GD Hager, RH Taylor. Anatomically constrained Video- CT registration via the V-IMLOP algorithm. MICCAI, Athens, Proceedings, Part III. Vol. 9902: pp. 133-141, 2016

Segmentation & Statistics

5

Statistics

Se Set t of

• f CTs

Seg egmen entation Statis istic ics

Our paper: Better segmentation & statistics

6

Be Before Aft fter Be Before Aft fter Be Before Aft fter

Mes esh Qualit ality

Statistical Shape Model (SSM)[5]

𝑊

1

𝑊

2

𝑊

3

𝑊

𝑜𝑡

𝑊 = 1 𝑜𝑡

𝑗=1 𝑜𝑡

𝑊

𝑗

Σ = 1 𝑜𝑡

𝑗=1 𝑜𝑡

𝑊

𝑗 −

𝑊 𝑈(𝑊

𝑗 −

𝑊) Σ = [𝑛1 ⋯ 𝑛𝑜𝑡] 𝜇1 ⋱ 𝜇𝑜𝑡 𝑛1 ⋯ 𝑛𝑜𝑡 𝑈

7

[5] Cootes, T., Taylor, C., Cooper, D., and Graham, J., "Active shape models-their training and application," Computer Vision and Image Understanding 61(1), 38-59 (1995).

Correspondence Improvement[8]

8

Project shape onto the modes

𝑐𝑗 = 𝑛𝑗

𝑈(𝑊 𝑗 −

𝑊)

Compute estimate shape

𝑊 = 𝑊 +

𝑗=1 𝑜𝑡

𝑐𝑗𝑛𝑗

Move vertices of original shape along the surface toward the corresponding vertex on estimated shape[8]

[8] Seshamani, S., Chintalapani, G., and Taylor, R., "Iterative refinement of point correspondences for 3D statistical shape models," in Medical Image Computing and Computer-Assisted Intervention, 417-425 (2011).

Assumption

• High accuracy segmentations
• Segmentation improvement
• E.g.: Using gradient vector flow (GVF) snakes[6][7]
• Use gradient in corresponding CT image
• Move mesh vertices toward structure boundaries
• Correspondences between shapes
• Lost during segmentation improvement

[6] Xu, C. and Prince, J. L., "Gradient vector flow: A new external force for snakes," in Computer Vision and Pattern Recognition, IEEE Computer Society Conference on, 66-71 (1997). [7] Xu, C. and Prince, J., “Snakes, shapes, and gradient vector ow," Image Processing, IEEE Transactions on, 7, 359- 369 (Mar 1998).

9

Simultaneous segmentation and correspondence improvement

10

Constrained segmentation improvement

• Using GVF
• Move vertices toward large gradients in

image to obtain new surface, 𝜚

• Estimate 𝜚 using pre-existing SSM
• Slide vertices on 𝜚 along the surface toward

corresponding vertices on estimated shape

11

Simultaneous segmentation and correspondence improvement

12

𝑄 = 5 𝑅 = 3 5 iterations

Results

From 52 publicly available CTs[9][10][11][12]

13

[9] Bosch, W. R., Straube, W. L., Matthews, J. W., and Purdy, J. A., “Data from head-neck-cetuximab. The cancer imaging archive.," (2015). [10] Beichel, R. R., Ulrich, E. J., Bauer, C., Wahle, A., Brown, B., Chang, T., Plichta, K. A., Smith, B. J., Sunderland, J. J., Braun, T., Fedorov, A., Clunie, D., Onken, M., Riesmeier, J., Pieper, S., Kikinis, R., Graham, M. M., Casavant, T. L., Sonka, M., and Buatti, J. M., “Data from qin-headneck. The cancer imaging archive.," (2015). [11] Fedorov, A., Clunie, D., Ulrich, E., Bauer, C., Wahle, A., Brown, B., Onken, M., Riesmeier, J., Pieper, S., Kikinis, R., Buatti, J., and Beichel, R. R., “DICOM for quantitative imaging biomarker development: a standards based approach to sharing clinical data and structured pet/ct analysis results in head and neck cancer research," PeerJ 4(e2057) (2016). [12] Clark, K., Vendt, B., Smith, K., Freymann, J., Kirby, J., Koppel, P., Moore, S., Phillips, S., Matt, D., Pringle, M., Tarbox, L., and Prior, F., \The cancer imaging archive (tcia): Maintaining and operating a public information repository," Journal of Digital Imaging 26(6), 1045{1057 (2013).

Results: Segmentation

14

Red contour: Segmentation via label transfer using deformable registration Blue contour: Hand-labeled gold standard Green contour: Improved segmentation using our method

Results: Segmentation

Mea ean Err Error ± St

• Std. De
• Dev. (m

(mm) Max Err Error (m (mm) Deformable registration 0.3327 ± 0.3147 2.338 GVF 0.1135 ± 0.1316 1.1548 GVF + SSM (our method) 0.09 0.0985 ± 0.12 0.128 1.03 1.0364

15

Truth De Deformable Reg egistr tratio ion GVF VF GVF VF+SSM (ou (our meth thod) Segmentation errors compared against hand-segmented gold standard computed using the Hausdorff distance metric.

0.2 0.4 0.6 0.8 1

Results: Correspondence

16

Results: Mesh Quality

17

Segmentation improved using GVF Segmentation improved using our method

Triangle Quality

Conclusion

• Our method improves segmentation while maintaining

correspondences

• Demonstrate improved segmentation and correspondence

18

Our shape model contains more accurate information Our shape model is able to estimate a new shape accurately

Thank you!

Facult lty

Russ Taylor Greg Hager Masaru Ishii Austin Reiter Simon Leonard

Framework

Rob Grupp cisst Developers

19

Fundin ing

NIH R01R01-EB015530: Enhanced Navigation for Endoscopic Sinus Surgery through Video Analysis (PI: Hager) Johns Hopkins University internal funds

References

[1] Bhattacharyya, N.: Ambulatory sinus and nasal surgery in the United States: Demographics and perioperative outcomes. The Laryngoscope. 120, 635-638 (2010) [2] Dalziel, Kim; Stein, Ken; Round, Ali; Garside, Ruth; Royle, P.: Endoscopic sinus surgery for the excision of nasal polyps: A systematic review of safety and effectiveness. American Journal of Rhinology. 20(5), 506- 519 (2006) [3] Kainz, J. and Stammberger, H., “The roof of the anterior ethmoid: A place of least resistance in the skull base,” American Journal of Rhinology 3(4), 191-199 (1989). [4] Tao, H., Ma, Z., Dai, P., and Jiang, L., “Computer-aided three-dimensional reconstruction and measurement

• f the optic canal and intracanalicular structures,” The Laryngoscope 109(9), 1499-1502 (1999).

[5] Cootes, T., Taylor, C., Cooper, D., and Graham, J., "Active shape models-their training and application," Computer Vision and Image Understanding 61(1), 38-59 (1995). [6] Xu, C. and Prince, J. L., "Gradient vector flow: A new external force for snakes," in Computer Vision and Pattern Recognition, IEEE Computer Society Conference on, 66-71 (1997). [7] Xu, C. and Prince, J., “Snakes, shapes, and gradient vector ow," Image Processing, IEEE Transactions on, 7, 359-369 (Mar 1998). [8] Seshamani, S., Chintalapani, G., and Taylor, R., "Iterative refinement of point correspondences for 3D statistical shape models," in Medical Image Computing and Computer-Assisted Intervention, 417-425 (2011). [9] Bosch, W. R., Straube, W. L., Matthews, J. W., and Purdy, J. A., “Data from head-neck-cetuximab. The cancer imaging archive.," (2015). [10] Beichel, R. R., Ulrich, E. J., Bauer, C., Wahle, A., Brown, B., Chang, T., Plichta, K. A., Smith, B. J., Sunderland, J. J., Braun, T., Fedorov, A., Clunie, D., Onken, M., Riesmeier, J., Pieper, S., Kikinis, R., Graham, M. M., Casavant, T. L., Sonka, M., and Buatti, J. M., “Data from qin-headneck. The cancer imaging archive.," (2015). [11] Fedorov, A., Clunie, D., Ulrich, E., Bauer, C., Wahle, A., Brown, B., Onken, M., Riesmeier, J., Pieper, S., Kikinis, R., Buatti, J., and Beichel, R. R., “DICOM for quantitative imaging biomarker development: a standards based approach to sharing clinical data and structured pet/ct analysis results in head and neck cancer research," PeerJ 4(e2057) (2016). [12] Clark, K., Vendt, B., Smith, K., Freymann, J., Kirby, J., Koppel, P., Moore, S., Phillips, S., Matt, D., Pringle, M., Tarbox, L., and Prior, F., \The cancer imaging archive (tcia): Maintaining and operating a public information repository," Journal of Digital Imaging 26(6), 1045{1057 (2013). [13] P.J. Besl, H.D. McKay, A method for registration of 3-D shapes, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992 [14] Billings SD, Boctor EM, Taylor RH. Iterative Most-Likely Point Registration (IMLP): A Robust Algorithm for Computing Optimal Shape Alignment. PLOS ONE 10(3): e0117688, 2015 [15] Seth D. Billings, RH Taylor. Iterative Most Likely Oriented Point Registration. MICCAI, Boston, Proceedings, Part I. Vol. 8673: pp. 178-185, 2014 [16] Seth D. Billings, A Sinha, A. Reiter, S Leonard, M Ishii, GD Hager, RH Taylor. Anatomically constrained Video-CT registration via the V-IMLOP algorithm. MICCAI, Athens, Proceedings, Part III. Vol. 9902: pp. 133- 141, 2016

20

21

Questions?

Code will be available on github soon!