SLIDE 1
Microscopic Advances with Large-Scale Learning: Stochastic Optimization for Cryo-EM
Ali Punjani, Marcus Brubaker University of Toronto Department of Computer Science
Microscopic Advances with Large-Scale Learning: Stochastic - - PowerPoint PPT Presentation
Microscopic Advances with Large-Scale Learning: Stochastic - - PowerPoint PPT Presentation
Microscopic Advances with Large-Scale Learning: Stochastic Optimization for Cryo-EM Ali Punjani, Marcus Brubaker University of Toronto Department of Computer Science Structure Determination } Macromolecules } Protein structure determines
SLIDE 2
SLIDE 3
Electron Cryo-Microscopy (Cryo-EM)
} No crystals needed, large molecules and complexes
Low dose electron beam Particles in unknown 3D pose Ice Transfer Function Corrupted Noisy Integral Projections Film/CCD
Computational Task: Recover 3D Electron Density
SLIDE 4
Cryo-EM Image Formation
} Challenges for reconstruction:
} Destructive CTF } Low SNR } Unknown pose
Low dose electron beam Particles in unknown 3D pose Ice Transfer Function Corrupted Noisy Integral Projections Film/CCD Corruption by CTF
=
2D Particle Images
SLIDE 5
Cryo-EM Image Formation
K
p(I|θ, R, t, V) = N(I|StCθPRV, σ2I)
I θ Rt V
SLIDE 6
Cryo-EM Image Formation
K
p(I|θ, R, t, V) = N(I|StCθPRV, σ2I)
I θ Rt V
Linear Voxels Integral Projection
SLIDE 7
Cryo-EM Image Formation
K
p(I|θ, R, t, V) = N(I|StCθPRV, σ2I)
I θ Rt V
p(˜ I|θ, R, t, ˜ V) = N(˜ I|˜ St ˜ Cθ ˜ PR ˜ V, σ2I)
In Fourier Domain: Diagonal Linear Voxels Fourier Coefficients Integral Projection Slicing
SLIDE 8
Marginalization for Latent Variables
K
I θ Rt V
p(˜ I|θ, ˜ V) = Z
R2
Z
SO(3)
p(˜ I|θ, R, t, ˜ V)p(R)p(t)dRdt
SLIDE 9
Marginalization for Latent Variables
K
I θ Rt V
p(˜ I|θ, ˜ V) = Z
R2
Z
SO(3)
p(˜ I|θ, R, t, ˜ V)p(R)p(t)dRdt
} Numerical Quadrature
≈
M
X
j=1
wjp(˜ I|θ, Rj, tj, ˜ V)
SLIDE 10
Maximum-a-Posteriori Estimation
K
I θ Rt V
p(V|D) ∝ p(V)
K
Y
i=1
p(˜ Ii|θi, ˜ V)
SLIDE 11
Optimization Problem
K
I θ Rt V
p(V|D) ∝ p(V)
K
Y
i=1
p(˜ Ii|θi, ˜ V) arg min
V − K
X
i=1
⇣ log p(˜ I|θ, ˜ V) + K−1 log p(V) ⌘
SLIDE 12
Stochastic Optimization for Cryo-EM
arg min
V − K
X
i=1
⇣ log p(˜ I|θ, ˜ V) + K−1 log p(V) ⌘
} Expensive to compute objective with large K } Stochastic Optimization:
} Approximate objective with subset of images } Update based on approximate gradient
} Various Algorithms (vary by update rule) } Advantages: speed, random initialization
SLIDE 13
Experiments: Datasets
} Real Dataset:
} 46K Images of ATP Synthase from Thermus Thermophilius } Low SNR and known CTF parameters
SLIDE 14
Experiments: Datasets
} Synthetic Dataset:
} 50,000 Projections of known artificial density } Low SNR and realistic CTF parameters
SLIDE 15
Experiments: Seven Methods
} Vanilla Stochastic Gradient Descent (SGD) } Momentum Methods:
} Classical Momentum } Nesterov’s Accelerated Gradient
} Adaptive Methods:
} AdaGrad } TONGA
} Quasi-Second Order Methods:
} Online L-BFGS } Hessian Free
SLIDE 16
Experiments: Results
} Identical random initialization in all
experiments
SLIDE 17
Experiments: Results
} Simplest Method
SLIDE 18
Experiments: Results
} Momentum Method
SLIDE 19
Experiments: Results
} Adaptive Step-size
SLIDE 20
Experiments: Results
} Quasi-second order
SLIDE 21
Experiments: Results
} Qualitatively Similar } Reasonable in one pass through data
SLIDE 22
Experiments: Results
SLIDE 23
Experiments: Results
SLIDE 24
Experiments: Comparison
Projection Matching RELION (E-M) Proposed Approach
3 Hours – 1 Epochs
SLIDE 25
Experiments: Comparison
Projection Matching
24 Hours – 5 Epochs
RELION (E-M)
24 Hours – 5 Epochs
Proposed Approach
3 Hours – 1 Epochs
SLIDE 26
Experiments: Comparison
Projection Matching
24 Hours – 5 Epochs
RELION (E-M)
24 Hours – 5 Epochs
Proposed Approach
3 Hours – 1 Epochs
SLIDE 27
Experiments: Comparison
} Random Initialization is difficult for other
methods
Projection Matching
24 Hours – 5 Epochs
RELION (E-M)
24 Hours – 5 Epochs
Proposed Approach
3 Hours – 1 Epochs
SLIDE 28
Conclusions
} Introduced Cryo-EM Structure Determination } Stochastic Optimization solution } Simple methods are best } State of the art speed and robustness
SLIDE 29
Recent Progress
} Higher resolution reconstructions } Importance Sampling: 100,000x speedup
SLIDE 30
Recent Progress
} Higher resolution reconstructions } Importance Sampling: 100,000x speedup } Forward:
} Heterogeneous mixtures of particles } Better priors } Video exposure