SLIDE 1
Differentiable Mutual Information and Matrix Exponential for Multi-Resolution Image Registration
Abhishek Nan, Matthew Tennant, Uriel Rubin, Nilanjan Ray
Medical Imaging with Deep Learning, 2020
https://github.com/abnan/DRMIME
Differentiable Mutual Information and Matrix Exponential for - - PowerPoint PPT Presentation
Differentiable Mutual Information and Matrix Exponential for - - PowerPoint PPT Presentation
Differentiable Mutual Information and Matrix Exponential for Multi-Resolution Image Registration Abhishek Nan, Matthew Tennant, Uriel Rubin, Nilanjan Ray Medical Imaging with Deep Learning, 2020 https://github.com/abnan/DRMIME Registration
SLIDE 2
SLIDE 3
With MSE
Source: https://anhir.grand-challenge.org/
SLIDE 4
Problems?
- Multi-modal images
- MSE won’t work
Source: https://www.mathworks.com/discovery/image-registration.html
SLIDE 5
Solution?
- Mutual Information
SLIDE 6
Issues?
- Mutual Information for images is computed using joint histograms.
- Histograms are not differentiable.
- No gradient descent?
SLIDE 7
Differentiable mutual information
- The function T is realized by a neural
network with parameter θ.
- V(θ) is differentiable and can be used as a
- bjective function in place of MI.
SLIDE 8
MINE for images
SLIDE 9
Currently
SLIDE 10
Matrix exponential
- Matrix exponential of a square matrix A is given by the following:
- Geometric transformation matrices can be obtained by exponential of a linear combination of basis
matrices.
SLIDE 11
Matrix Exponential (Examples)
SLIDE 12
Why matrix exponentials?
SLIDE 13
So far...
SLIDE 14
More problems?
- Medical/Microscopy images often are extremely high resolution. So gradient descent can be
extremely slow.
- Optimization for neural networks is non-convex.
SLIDE 15
Solution?
- Gaussian Pyramids
Source: https://en.wikipedia.org/wiki/Pyramid_(image_processing)
SLIDE 16
SLIDE 17
- Can we do better?
○ What if we did simultaneous optimization for all levels?
- Each level of optimization is for different images. So MI between them changes as well. Solution?
○ A single MINE can be trained for all of these!
- How?
○ Mini-batches can be constructed by sampling from all levels
SLIDE 18
SLIDE 19
What about the loss?
- Since we are doing simultaneous optimization, with modern deep learning frameworks, it’s very
easy to combine the loss from each level and perform joint optimization.
- For eg, for just 1 level:
○ Loss = MI(F, Gv(M))
- For 4 levels:
○ Loss = (¼) * [MI(F1, Gv(M1)) + MI(F2, Gv(M2)) + MI(F3, Gv(M3)) + MI(F4, Gv(M4))]
SLIDE 20
SLIDE 21
Evaluation
- Public datasets
- Available ground truth
SLIDE 22
Results
SLIDE 23
Source: https://projects.ics.forth.gr/cvrl/fire/
SLIDE 24