High quality ultrasonic multi-line transmission through deep - - PowerPoint PPT Presentation

High quality ultrasonic multi-line transmission through deep learning

Sanketh Vedula Technion, Israel

Machine Learning for Medical Image Reconstruction Workshop

Ortal Senouf Technion Grigoriy Zurakhov Technion Alex Bronstein Technion Michael Zibulevsky Technion Oleg Michailovich U of Waterloo Diana Gaitini Technion Dan Adam Technion

Joint work with:

This research was partially supported by ERC StG RAPID

Ultrasound imaging

• Non-invasive, cheap, no ionising radiation, and recently,

also portable.

• But, all the above advantages come at the expense of

image quality.

• “Point-of-care ultrasound (POCUS): the visual stethoscope
• f the 21st century” [Gillman et al., 2012]

[Gillman et al., 2012], Portable bedside ultrasound: the visual stethescope of the 21st century, J. Trauma. Resusc. Emerg. Med., 2012 Image credits: GE VScan, Butterfly I/Q

Ultrasound acquisition pipeline

Full/reduced transmissions Receive (Rx) Time-delay & phase rotation Reconstruction/ artifact correction

Apodization/ element summation

Post-processing Envelope detection

Ultrasound image

Ultrasound acquisition pipeline

Full/reduced transmissions Receive (Rx) Time-delay & phase rotation Reconstruction/ artifact correction

Apodization/ element summation

Post-processing Envelope detection

Ultrasound image Our goal: Design end-to-end fast learning-based algorithms to improve the quality of point-of-care ultrasound imaging (POCUS).

Ultrasound acquisition pipeline

Full/reduced transmissions Receive (Rx) Time-delay & phase rotation Reconstruction/ artifact correction

Apodization/ element summation

Post-processing Envelope detection

Ultrasound image Our goal: Design end-to-end fast learning-based algorithms to improve the quality of point-of-care ultrasound imaging (POCUS).

[Vedula et al., 2017]

[Vedula et al., 2017] Towards CT-quality ultrasound imaging with deep learning, arXiv: 1710.06304

Ultrasound acquisition pipeline

Full/reduced transmissions Receive (Rx) Time-delay & phase rotation Reconstruction/ artifact correction

Apodization/ element summation

Post-processing Envelope detection

Ultrasound image Our goal: Design end-to-end fast learning-based algorithms to improve the quality of point-of-care ultrasound imaging (POCUS).

[Vedula et al., 2017]

[Vedula et al., 2017] Towards CT-quality ultrasound imaging with deep learning, arXiv: 1710.06304 [Senouf et al., 2018] High frame-rate cardiac ultrasound imaging using deep learning, MICCAI 2018.

This work & [Senouf et al., 2018]

Motivation

High frame-rate is a crucial consideration while performing

• Echocardiography: for functional analysis of heart
• 3D-sonography: to scan large volumes

Existing methods for increasing frame-rate provide low- quality images that suffer from poor resolution and contrast

Multi-line transmission

[Mallart et al., 1992] Improved imaging rate through simultaneous transmission of several ultrasound beams

MLT artifacts

Single-line transmission Multi-line transmission linear phased array

• f transducers

Loss of contrast due to cross-talk between the lines transmitted simultaneously

Traditional methods

• Constant and adaptive apodization (Tukey, = 0.5) [Tong et al., 2013]
• Filtered delay-multiply-and-sum (FDMAS) [Matrone et al., 2017]

Limitations:

• Apodization affects resolution
• FDMAS exhibits poor contrast-to-noise ratio

α

[Tong et al., 2013] Multi-transmit beam forming for fast cardiac imaging-a simulation study, IEEE T-UFFC, 2013. [Matrone et al., 2017] High frame-rate, high resolution ultrasound imaging with multi-line transmission and filtered-delay multiply and sum beamforming, IEEE TMI, 2017.

Proposed CNN-based pipeline

Conv 3x3 BatchNorm Stride 1 ReLU Conv 3x3 BatchNorm Stride 1 ReLU Conv 3x3 BatchNorm Stride 1 ReLU Strided- Conv 3x3 BatchNorm Stride 1 ReLU Strided- Conv 3x3 BatchNorm Stride 1 ReLU Strided- Conv 3x3 BatchNorm Stride 1 ReLU

M/2 x N/2 x 256 M x N x 64 M/4 x N/4 x1024 M/4 x N/4 x 2048 M/2 x N/2 x 512 M x N x 128

Conv 3x3 BatchNorm Stride 1 ReLU Strided- Conv 3x3 BatchNorm Stride 1 ReLU

M/2^b x N/2^b x 64*2^b*2^b M/2^b x N/2^b x 64*2^b*2^b

b

Inputs: time-delayed MLT element-wise I/Q data

Apodization Layer:

Summing over elements Outputs: Corresponding SLT I/Q images

b

= Concatenate = Average pooling = Skip connections = Number of bifurcations

M x N x 64

Dataset

• In-vivo data was collected from quasi-static organs (e.g. bladder,

kidney) from 6 volunteers.

• MLT data (with beam-separation angle of at least 15 degrees) can

be approximated through summation of the corresponding sequentially transmitted lines from SLT [Prieur et al., 2013].

• In total, 750 frames were used for training.

[Prieur et al., 2013] Multi-Line Transmission in Medical Imaging Using the Second-Harmonic Signal, IEEE UFFC, 2013

Results (phantom)

Results (phantom)

Results (in-vivo)

The cine-loop of a scan taken the abdominal region

Results (in-vivo)

The cine-loop of a scan taken the abdominal region

Results (in-vivo)

Takeaway (this paper)

• We proposed a CNN-based approach for MLT artifact

correction.

• First use of deep learning for MLT artifact correction.
• The proposed approach reconstructs the SLT data well

and generalises across new patients, anatomies and to the phantom data.

Multi-line acquisition

We show that similar approach can be used to improve the quality of another high-frame rate acquisition mode called Multi-line acquisition (MLA)

• and
• (MLT)
• 20

• 80
• 60
• 40
• 20

• 80
• 60
• 40
• 20

• 80
• 60
• 40
• 20

• /

• ()
• ()
• 80
• 60
• 40
• 20

• 80
• 60
• 40
• 20

• 80
• 60
• 40
• 20

• 80
• 60
• 40
• 20

• /-0.04

• /

• Come visit our poster (M-15) at the main conference!

Conclusion

• One major disadvantage of current POCUS scanners:

they support only hardware-based classical beamforming.

• GPU-enabled ultrasound, that allows the fusion of deep

learning techniques with ultrasound imaging, seems promising!

Marcin Lewandowski, Ultrasound medical imaging in the GPU era, NVIDIA GTC 2018

Thanks!

Questions?