SLIDE 1
High resolution holographic image synthesis for future display eyeglasses
Praneeth Chakravarthula
UNC Chapel Hill
Future of Personal Computing
2
High resolution holographic image synthesis for future display - - PowerPoint PPT Presentation
High resolution holographic image synthesis for future display - - PowerPoint PPT Presentation
High resolution holographic image synthesis for future display eyeglasses Praneeth Chakravarthula UNC Chapel Hill Future of Personal Computing 2 Eyeglasses-Style Near-Eye Display Optics Wide field of view High resolution Accommodation
SLIDE 2
SLIDE 3
3
Wide field of view High resolution Accommodation support
Eyeglasses-Style Near-Eye Display Optics
SLIDE 4
Eyeglasses-Style Near-Eye Display Optics
4
Wide field of view Moderate resolution Accommodation support
Holography is the only demonstrated technology for getting everything
Maimone et al. 2017
SLIDE 5
Basics of Digital Holography
5
SLIDE 6
Principle of Holography
6
SLIDE 7
Principle of Holography
7
SLIDE 8
Principle of Holography
8
Step 1: Recording
SLIDE 9
Principle of Holography
9
Step 1: Recording
SLIDE 10
Principle of Holography
10
Step 1: Recording Step 2: Playback
SLIDE 11
Principle of Holography
11
Step 1: Recording
SLIDE 12
Principle of Holography
12
Step 2: Playback Step 1: Recording
SLIDE 13
13
Incident light modulated by Hologram H
Holographic Image Formation
SLIDE 14
14
Propagates to result in the final field z and final image |z|2
Holographic Image Formation
SLIDE 15
Phase Hologram Reconstructed Image Reference
Heuristic Hologram Phase Retrieval
Reconstruction
SLIDE 16
Double Phase Encoding Hologram
Phase Hologram Reconstructed Image Reference Reconstruction
SLIDE 17
Wirtinger Holography
Phase Hologram Reconstructed Image Reference Reconstruction
SLIDE 18
Wirtinger Holography
18
SLIDE 19
19
Differentiable forward model Compute complex Wirtinger gradients Optimize for phase holograms
Wirtinger Holography Overview
SLIDE 20
20
Differentiable forward model
Step 1: For a penalty function f, compute the error between the holographic reconstruction and the target image
Compute complex Wirtinger gradients Optimize for phase holograms
SLIDE 21
21
Differentiable forward model Compute complex Wirtinger gradients Optimize for phase holograms
Step 1: For a penalty function f, compute the error between the holographic reconstruction and he target image
SLIDE 22
22
Differentiable forward model
Step 1: For a penalty function f, compute the error between the holographic reconstruction and the target image
Compute complex Wirtinger gradients Optimize for phase holograms
SLIDE 23
23
Differentiable forward model
Step 1: For a penalty function f, compute the error between the holographic reconstruction and the target image
Compute complex Wirtinger gradients Optimize for phase holograms
SLIDE 24
24
Differentiable forward model
Step 1: For a penalty function f, compute the error between the holographic reconstruction and the target image
Compute complex Wirtinger gradients Optimize for phase holograms
SLIDE 25
25
Differentiable forward model
Step 1: For a penalty function f, compute the error between the holographic reconstruction and the target image
Compute complex Wirtinger gradients Optimize for phase holograms
SLIDE 26
26
Differentiable forward model Compute complex Wirtinger gradients Optimize for phase holograms
Step 2: Construct the cost function to minimize the error with optional regularizer to obtain the
- ptimal phase
SLIDE 27
27
Differentiable forward model Compute complex Wirtinger gradients Optimize for phase holograms
Step 2: Construct the cost function to minimize the error with optional regularizer to obtain the
- ptimal phase
SLIDE 28
28
Differentiable forward model Compute complex Wirtinger gradients Optimize for phase holograms
Step 2: Construct the cost function to minimize the error with optional regularizer to obtain the
- ptimal phase
SLIDE 29
29
Differentiable forward model Compute complex Wirtinger gradients Optimize for phase holograms
Step 2: Construct the cost function to minimize the error with optional regularizer to obtain the
- ptimal phase
SLIDE 30
30
Differentiable forward model Compute complex Wirtinger gradients Optimize for phase holograms
Step 2: Construct the cost function to minimize the error with optional regularizer to obtain the
- ptimal phase
We can use standard optimizers if there is a gradient
SLIDE 31
31
Differentiable forward model Compute complex Wirtinger gradients Optimize for phase holograms
Holomorphic function: Complex function that is complex differentiable Derivative of holomorphic real-valued function is always ZERO Derivatives of any order are NOT DEFINED for our objective function
Real-valued function Complex-valued argument
SLIDE 32
Optimize for phase holograms Differentiable forward model Compute complex Wirtinger gradients
32
Two important properties of gradient: 1) Direction of maximal rate of change 2) Is zero at stationary points Step 3: Define approximate gradient and compute Wirtinger derivatives for each propagation model REFER TO MY SIGGRAPH Asia 2019 PAPER AND SUPPLEMENT
SLIDE 33
33
Differentiable forward model Compute complex Wirtinger gradients Optimize for phase holograms
Standard off-the-shelf optimization methods 1) Quasi-Newton 2) Stochastic gradient descent Step 4: Optimize for holograms using off-the-shelf methods
SLIDE 34
Simulation Results
34
SLIDE 35
Reference Reconstruction
SLIDE 36
Reference Reconstruction
SLIDE 37
37
Simulation Results
Target Modified GS
(Peng et al. 2017)
Double phase
(Maimone et al. 2017)
Our Method
SLIDE 38
38
Simulation Results
Target Modified GS
(Peng et al. 2017)
Double phase
(Maimone et al. 2017)
Our Method
SLIDE 39
SLIDE 40
Prototype Hardware Results
SLIDE 41
41
Prototype Hardware Results
Target Modified GS
(Peng et al. 2017)
Double phase
(Maimone et al. 2017)
Our Method
SLIDE 42
42
Target Modified GS
(Peng et al. 2017)
Double phase
(Maimone et al. 2017)
Our Method
Prototype Hardware Results
SLIDE 43
43
Target Modified GS
(Peng et al. 2017)
Double phase
(Maimone et al. 2017)
Our Method
Prototype Hardware Results
SLIDE 44
Results
Reference Experiment
SLIDE 45
Reference Experiment
SLIDE 46
46
Perceptually Optimized Holography
Learned Perceptual Image Patch Similarity (LPIPS)
( Zhang et al. 2018 )
Optimize for deep learning based perceptual losses
SLIDE 47
47
Perceptually Optimized Holography
Target LPIPS optimized MS-SSIM optimized L2 optimized
SLIDE 48
48
Cascaded Superresolution Holography
SLIDE 49
49
Target Modified GS
(Peng et al. 2017)
Double phase
(Maimone et al. 2017)
Our Method
Prototype Hardware Results
SLIDE 50
Real World Deviations
SLIDE 51
51
Ideal Wave Propagation
SLIDE 52
52
Real World Wave Propagation
SLIDE 53
53
Compensating real world deviations via Hardware-in-the-loop phase retrieval
Target
(Chakravarthula et al. 2019)
Wirtinger Holography Double Phase Encoding
(Maimone et al. 2017)
Our Method
Upcoming at SIGGRAPH Asia 2020
SLIDE 54
54
Target
(Chakravarthula et al. 2019)
Wirtinger Holography Double Phase Encoding
(Maimone et al. 2017)
Our Method
Compensating real world deviations via Hardware-in-the-loop phase retrieval Upcoming at SIGGRAPH Asia 2020
SLIDE 55
Differentiable forward model Compute complex Wirtinger gradients Optimize for phase holograms
Wirtinger Holography
www.cs.unc.edu/~cpk Praneeth Chakravarthula