High resolution holographic image synthesis for future display - PowerPoint PPT Presentation

High resolution holographic image synthesis for future display eyeglasses Praneeth Chakravarthula UNC Chapel Hill

Future of Personal Computing 2

Eyeglasses-Style Near-Eye Display Optics Wide field of view High resolution Accommodation support 3

Eyeglasses-Style Near-Eye Display Optics Wide field of view Moderate resolution Accommodation support Holography is the only demonstrated technology for getting everything Maimone et al. 2017 4

Basics of Digital Holography 5

Principle of Holography 6

Principle of Holography 7

Principle of Holography Step 1: Recording 8

Principle of Holography Step 1: Recording 9

Principle of Holography Step 1: Recording Step 2: Playback 10

Principle of Holography Step 1: Recording 11

Principle of Holography Step 1: Recording Step 2: Playback 12

Holographic Image Formation Incident light modulated by Hologram H 13

Holographic Image Formation Propagates to result in the final field z and final image |z| 2 14

Heuristic Hologram Phase Retrieval Phase Hologram Reconstructed Image Reconstruction Reference

Double Phase Encoding Hologram Phase Hologram Reconstructed Image Reconstruction Reference

Wirtinger Holography Phase Hologram Reconstructed Image Reconstruction Reference

Wirtinger Holography 18

Wirtinger Holography Overview Differentiable Compute complex Optimize for forward model Wirtinger gradients phase holograms 19

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Step 1: For a penalty function f , compute the error between the holographic reconstruction and the target image 20

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Step 1: For a penalty function f , compute the error between the holographic reconstruction and he target image 21

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Step 1: For a penalty function f , compute the error between the holographic reconstruction and the target image 22

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Step 1: For a penalty function f , compute the error between the holographic reconstruction and the target image 23

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Step 1: For a penalty function f , compute the error between the holographic reconstruction and the target image 24

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Step 1: For a penalty function f , compute the error between the holographic reconstruction and the target image 25

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Step 2: Construct the cost function to minimize the error with optional regularizer to obtain the optimal phase 26

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Step 2: Construct the cost function to minimize the error with optional regularizer to obtain the optimal phase 27

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Step 2: Construct the cost function to minimize the error with optional regularizer to obtain the optimal phase 28

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Step 2: Construct the cost function to minimize the error with optional regularizer to obtain the optimal phase 29

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients We can use standard optimizers if there is a gradient Step 2: Construct the cost function to minimize the error with optional regularizer to obtain the optimal phase 30

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Holomorphic function: Complex function that is complex differentiable Derivative of holomorphic real-valued function is always ZERO Real-valued function Complex-valued argument Derivatives of any order are NOT DEFINED for our objective function 31

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Two important properties of gradient: 1) Direction of maximal rate of change 2) Is zero at stationary points REFER TO MY SIGGRAPH Asia 2019 PAPER AND SUPPLEMENT Step 3: Define approximate gradient and compute Wirtinger derivatives for each propagation model 32

Compute complex Differentiable Optimize for Wirtinger forward model phase holograms gradients Standard off-the-shelf optimization methods 1) Quasi-Newton 2) Stochastic gradient descent Step 4: Optimize for holograms using off-the-shelf methods 33

Simulation Results 34

Reference Reconstruction

Reference Reconstruction

Simulation Results Modified GS Double phase Target Our Method (Peng et al. 2017) (Maimone et al. 2017) 37

Simulation Results Modified GS Double phase Target Our Method (Peng et al. 2017) (Maimone et al. 2017) 38

Prototype Hardware Results

Prototype Hardware Results Modified GS Double phase Target Our Method (Peng et al. 2017) (Maimone et al. 2017) 41

Prototype Hardware Results Modified GS Double phase Target Our Method (Peng et al. 2017) (Maimone et al. 2017) 42

Prototype Hardware Results Modified GS Double phase Target Our Method (Peng et al. 2017) (Maimone et al. 2017) 43

Reference Results Experiment

Reference Experiment

Perceptually Optimized Holography Optimize for deep learning based perceptual losses Learned Perceptual Image Patch Similarity (LPIPS) ( Zhang et al. 2018 ) 46

Perceptually Optimized Holography Target L2 optimized MS-SSIM optimized LPIPS optimized 47

Cascaded Superresolution Holography 48

Prototype Hardware Results Modified GS Double phase Target Our Method (Peng et al. 2017) (Maimone et al. 2017) 49

Real World Deviations

Ideal Wave Propagation 51

Real World Wave Propagation 52

Compensating real world deviations via Hardware-in-the-loop phase retrieval Double Phase Encoding Wirtinger Holography Target Our Method (Chakravarthula et al. 2019) (Maimone et al. 2017) Upcoming at SIGGRAPH Asia 2020 53

Compensating real world deviations via Hardware-in-the-loop phase retrieval Double Phase Encoding Wirtinger Holography Target Our Method (Chakravarthula et al. 2019) (Maimone et al. 2017) Upcoming at SIGGRAPH Asia 2020 54

Praneeth Chakravarthula Differentiable Compute complex Optimize for forward model Wirtinger gradients phase holograms www.cs.unc.edu/~cpk Wirtinger Holography