richardson lucy deblurring for moving light field cameras
play

Richardson-Lucy Deblurring for Moving Light Field Cameras Donald - PowerPoint PPT Presentation

Jan 12, 2024 •129 likes •597 views

Richardson-Lucy Deblurring for Moving Light Field Cameras Donald Dansereau 1 , Anders Eriksson 2 and Jrgen Leitner 2,3 1 Stanford University, 2 Queensland University of Technology, 3 ARC Centre of Excellence for Robotic Vision CVPR:LF4CV

  1. Richardson-Lucy Deblurring for Moving Light Field Cameras Donald Dansereau 1 , Anders Eriksson 2 and Jürgen Leitner 2,3 1 Stanford University, 2 Queensland University of Technology, 3 ARC Centre of Excellence for Robotic Vision CVPR:LF4CV Workshop 2017 July 26

  2. 3D Motion Complicates Vision Scene-dependent nonuniform apparent motion 2 ComputationalImaging.org

  3. 3D is Easier in 4D We have 6-DOF virtual camera control Video Stabilization [video] http://pages.cs.wisc.edu/~lizhang/projects/lfstable/ [Smith2009] 3 ComputationalImaging.org

  4. 3D is Easier in 4D We can fix the camera’s position Closed-Form Change Detection Per-pix ixel l stil still- l-ca camera meth thods ● Change detection ● Tracking/segmentation ● Velocity & temporal filtering [dansereau2016] http://dgd.vision/Projects/LFChangeDet/ 4 ComputationalImaging.org

  5. 3D is Easier in 4D We can fix the camera’s position Closed-Form Change Detection Per-pix ixel l stil still- l-ca camera meth thods ● Change detection ● Tracking/segmentation ● Velocity & temporal filtering [dansereau2016] http://dgd.vision/Projects/LFChangeDet/ 5 ComputationalImaging.org

  6. 3D is Easier in 4D We can fix the camera’s position Closed-Form Change Detection Per-pix ixel l stil still- l-ca camera meth thods ● Change detection ● Tracking/segmentation ● Velocity & temporal filtering [dansereau2016] http://dgd.vision/Projects/LFChangeDet/ 6 ComputationalImaging.org

  7. 3D is Easier in 4D Lukas-Kanade optical flow generalizes to 6-DOF Linearize Apparent Motion Closed-form 6-DOF Odometry L x L y T r u e P l e n o p t i c P o i n t w i s e F e a t u r e 2 ) m ( − 2 z L y L x 1 0 1 4 1 2 5 1 0 0 8 − 5 6 4 − 1 0 y ( m ) x ( m ) 2 − 1 5 L z L z [Neumann2002, Dansereau2011, Dong2013] 7 ComputationalImaging.org

  8. 3D is Easier in 4D Lukas-Kanade optical flow generalizes to 6-DOF Linearize Apparent Motion Closed-form 6-DOF Odometry L x L y T r u e P l e n o p t i c P o i n t w i s e F e a t u r e 2 ) m ( − 2 z L y L x 1 0 1 4 1 2 5 1 0 0 8 − 5 6 4 − 1 0 y ( m ) x ( m ) 2 − 1 5 L z L z [Neumann2002, Dansereau2011, Dong2013] 8 ComputationalImaging.org

  9. Blur in 3D Scenes Convolution models blurring in 2D… Can we replace convolution with LF rendering in 3D scenes? 9 ComputationalImaging.org

  10. Related Work “Light Field Blind Motion Deblurring” [Srinivasan 2017] ● 3-DOF ● Insights on blur manifestation in LF ● Blind ● Modern optimization (ADAM) LF-RL ● Requires extension to be blind ● 6-DOF ● Proof of convergence to ML estimate (see paper) ● New LF equiparallax regularizer 10 ComputationalImaging.org

  11. Related Work “Richardson-Lucy Deblurring for Scenes under a Projective Motion Path” [Tai et al. 2011] 11 ComputationalImaging.org

  12. Richardson-Lucy Deblurring IN 12 ComputationalImaging.org

  13. Light Field Richardson-Lucy IN FWD BLUR REV BLUR Still LF 13 ComputationalImaging.org

  14. Light Field Richardson-Lucy IN FWD BLUR REV BLUR Still LF Simulated Motion 14 ComputationalImaging.org

  15. Light Field Richardson-Lucy IN FWD BLUR REV BLUR Still LF Simulated Motion Simulated Blur 15 ComputationalImaging.org

  16. Regularization Anisotropic total variation Favour textural edges [Goldluecke & Wanner 2013, Heber2013] Equiparallax Favour equal slopes in s,u and t,v 16 ComputationalImaging.org

  17. Rendered Results: Rot about y Ori riginal Ori riginal Ori riginal 17 ComputationalImaging.org

  18. Rendered Results: Rot about y Blurre Ori riginal red Ori Bl Blurre riginal red Ori Bl Blurre riginal red 18 ComputationalImaging.org

  19. Rendered Results: Rot about y Deblurre Blurre Ori riginal red rred Deblurr Ori Bl Blurre riginal red rred Deblurr Blurre Bl Ori riginal red rred 19 ComputationalImaging.org

  20. Rendered Results: Rot about z rotz Bl Blurre rred Bl Blurr rred Bl Blurr rred 20 ComputationalImaging.org

  21. Rendered Results: Rot about z rotz Bl Blurre Deblurre rred red Blurr Deblurre Bl rred rred Deblurre Bl Blurr rred rred 21 ComputationalImaging.org

  22. Rendered Results: Trans along x Bl Blurre rred Bl Blurr rred Bl Blurr rred 22 ComputationalImaging.org

  23. Rendered Results: Trans along x Bl Blurre Deblurre rred red Blurr Deblurre Bl rred rred Deblurre Blurr Bl rred rred 23 ComputationalImaging.org

  24. Rendered Results: Rot about z rotz Bl Blurre rred Bl Blurr rred Bl Blurr rred 24 ComputationalImaging.org

  25. Rendered Results: Rot about z rotz Bl Blurre Deblurre rred red Blurr Deblurre Bl rred rred Deblurre Bl Blurr rred rred 25 ComputationalImaging.org

  26. Rendered Results: Trans along z Bl Blurre rred Blurr Bl rred Bl Blurr rred 26 ComputationalImaging.org

  27. Rendered Results: Trans along z Deblurre Bl Blurre rred red Bl Blurr Deblurre rred rred Bl Blurr Deblurre rred rred 27 ComputationalImaging.org

  28. Rendered Results: Trans along z Deblurre Blurre Bl rred red Without Regulariza Bl Blurr Deblurre rred rred zation Without Regulariza Deblurre Blurr Bl rred rred zation 28 ComputationalImaging.org

  29. Results: Captured Quantitative Evaluation Repeatable camera motion Isolated dimensions Known magnitudes George 29 ComputationalImaging.org

  30. Validating Calibration & Rendering Still Camera calibration, rectification Metric blur simulation Metric robot motion Camera-to-robot calibration 30 ComputationalImaging.org

  31. Validating Calibration & Rendering Measured Still Blur Camera calibration, rectification Metric blur simulation Metric robot motion Camera-to-robot calibration 31 ComputationalImaging.org

  32. Validating Calibration & Rendering Simulated Blur Measured Still Blur Camera calibration, rectification Metric blur simulation Metric robot motion Camera-to-robot calibration 32 ComputationalImaging.org

  33. Validating Calibration & Rendering Simulated Blur Measured Still Still Blur Camera calibration, rectification Metric blur simulation Metric robot motion Camera-to-robot calibration 33 ComputationalImaging.org

  34. Results: Captured No increase in noise Large increase in sharpness Regularization is helping 34 ComputationalImaging.org

  35. Results: Captured Measured Blur 35 ComputationalImaging.org

  36. Results: Captured Measured Blur Deblurred 36 ComputationalImaging.org

  37. Results: Captured 37 ComputationalImaging.org

  38. Results: Captured 38 ComputationalImaging.org

  39. Results: Captured 39 ComputationalImaging.org

  40. Results: Captured 40 ComputationalImaging.org

  41. Results: Captured 41 ComputationalImaging.org

  42. Results: Captured 42 ComputationalImaging.org

  43. Summary & Future Work Generaliz lized co convolu lutio tional l blu lur usin sing LF Renderin ing Applied to RL deblurring 3D scenes, 6-DOF camera motion Proof of convergence to ML estimate Equiparallax regularization Next: Equiparallax regularization: applications Beyond 6-DOF, defocus Blind deblurring 43 ComputationalImaging.org

  44. Acknowledgments QUT HPC Group George 44 ComputationalImaging.org

  45. Light Field T oolbox for MATLAB Load Gantry and Lytro imagery Calibrate and rectify Lytro imagery Linear depth, volume fjlters Denoising: low-light, fog, dust, murky water Occluder removal: rain, snow, silty water LF Synth: Bare-Bones Rendering 45 ComputationalImaging.org

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

1 Prior Work: Depth from Defocus Prior Work: Modifying Cameras DSLR with a focusing mechanism

1 Prior Work: Depth from Defocus Prior Work: Modifying Cameras DSLR with a focusing mechanism

Light fields Light Fields: From Shape Recovery to Sparse Reconstruction Ravi Ramamoorthi University of California, San Diego Refocusing Viewpoint Change L4CV Keynote, Jul 26, 2017 Consumer light field cameras Outline Motivation Light

113 views • 10 slides
Image Deblurring Seungyong Lee POSTECH 1 Contents Fast Motion Deblurring (Siggraph Asia

Image Deblurring Seungyong Lee POSTECH 1 Contents Fast Motion Deblurring (Siggraph Asia

Image Deblurring Seungyong Lee POSTECH 1 Contents Fast Motion Deblurring (Siggraph Asia 2009) Non-uniform Motion Deblurring for Camera Shakes using Image Registration (Siggraph 2011 Talks) Text Deblurring (an ongoing project) 2

1.35k views • 87 slides
Light Field Depth Estimation Challenge LF4CV@CVPR2017 Katrin Honauer, Ole Johannsen, Bastian

Light Field Depth Estimation Challenge LF4CV@CVPR2017 Katrin Honauer, Ole Johannsen, Bastian

Light Field Depth Estimation Challenge LF4CV@CVPR2017 Katrin Honauer, Ole Johannsen, Bastian Goldluecke July 26, 2017 4D Light Fields 2D grid of 2D views all cameras are parallel and identical (except shift: negative disparity) [Figure:

479 views • 22 slides
Field of a moving particle in a dielectric The E- and B-field associated to a uniformly

Field of a moving particle in a dielectric The E- and B-field associated to a uniformly

Field of a moving particle in a dielectric The E- and B-field associated to a uniformly moving particle in vacuum are given by and where Previously we considered the case b <<1 P. Piot, PHYS 571 Fall 2007 Field

504 views • 14 slides
Spinning Parallelogram Operator (SPO) for Light Field Depth Estimation 4D Light Field Benchmark

Spinning Parallelogram Operator (SPO) for Light Field Depth Estimation 4D Light Field Benchmark

Spinning Parallelogram Operator (SPO) for Light Field Depth Estimation 4D Light Field Benchmark Challenge at 2 nd Workshop on LF4CV, CVPR 2017 Shuo Zhang 1 , Hao Sheng 1 , Chao Li 1 , Jun Zhang 2 , Zhang Xiong 1 1 Beihang University 2

419 views • 12 slides
High-Fidelity Light Field VR Playback Using NVIDIA GPUs Tim Milliron, Vice President of

High-Fidelity Light Field VR Playback Using NVIDIA GPUs Tim Milliron, Vice President of

High-Fidelity Light Field VR Playback Using NVIDIA GPUs Tim Milliron, Vice President of Engineering Nikhil Karnad, Architect for Image-Based Rendering 1 An Introduction to Light Field 2 THE LIGHT FIELD HOW IT WORKS KEY BENEFITS Capture

449 views • 34 slides
Cameras, Light and Shading CS 543 / ECE 549 Saurabh Gupta Spring 2020, UIUC

Cameras, Light and Shading CS 543 / ECE 549 Saurabh Gupta Spring 2020, UIUC

Cameras, Light and Shading CS 543 / ECE 549 Saurabh Gupta Spring 2020, UIUC http://saurabhg.web.illinois.edu/teaching/ece549/sp2020/ Many slides adapted from S. Seitz, L. Lazebnik, D. Hoiem, D. Forsyth Recap P Y Z f O y &' ( ,

765 views • 48 slides
What is Light ? Discussion Questions: 1) What is light? 2) How fast does light travel? 3) What

What is Light ? Discussion Questions: 1) What is light? 2) How fast does light travel? 3) What

What is Light ? Discussion Questions: 1) What is light? 2) How fast does light travel? 3) What is visible light? 4) What is our planets main source of light? electric field travels in a vertical plane What is Light? direction of travel

523 views • 4 slides
Ford Site Design Standards April 23, 2019 City of Saint Paul Mike Richardson Lucy Thompson

Ford Site Design Standards April 23, 2019 City of Saint Paul Mike Richardson Lucy Thompson

Ford Site Design Standards April 23, 2019 City of Saint Paul Mike Richardson Lucy Thompson Project Manager Principal City Planner Planning & Economic Development Menaka Mohan Ford Site Planner LHB, Inc. Michael Lamb Lydia Major

372 views • 19 slides
Cameras EECS 442 Prof. David Fouhey Winter 2019, University of Michigan

Cameras EECS 442 Prof. David Fouhey Winter 2019, University of Michigan

Cameras EECS 442 Prof. David Fouhey Winter 2019, University of Michigan http://web.eecs.umich.edu/~fouhey/teaching/EECS442_W19/ Next Few Classes Tuesday: Cameras (Projective Geometry) Thursday: Cameras (Light, Lenses) Next

1.41k views • 95 slides
Intelligent Transportation Systems James Marty Farris P.E. ODOT ITS Field Equipment

Intelligent Transportation Systems James Marty Farris P.E. ODOT ITS Field Equipment

Intelligent Transportation Systems James Marty Farris P.E. ODOT ITS Field Equipment Operations Center Websites Cities Field Equipment DMS Cameras RWIS Snowplow Cameras Portable DMS Dynamic Message Signs

856 views • 23 slides
Insights from PhenoCam g Andrew D. Richardson Harvard University I thank my PhenoCam

Insights from PhenoCam g Andrew D. Richardson Harvard University I thank my PhenoCam

Using digital cameras to monitoring vegetation phenology: Insights from PhenoCam g Andrew D. Richardson Harvard University I thank my PhenoCam collaborators for their contributions to this work. I gratefully acknowledge funding support from

344 views • 22 slides
Gesture Detection A camera is used to detect gestures made in the cameras field of view using

Gesture Detection A camera is used to detect gestures made in the cameras field of view using

Gesture Detection A camera is used to detect gestures made in the cameras field of view using infrared LEDs. Detected gestures will be used to control a music player. Andrew Zhong Patrick Kuczera Shahzeb Asif 1 Motivation/Goals The

230 views • 10 slides
belly buttons, absent cameras, Linda C.H. LAI moving house, The City University of Hong Kong

belly buttons, absent cameras, Linda C.H. LAI moving house, The City University of Hong Kong

Documenting Sentiments in Hong Kong 1997 Video Diaries: belly buttons, absent cameras, Linda C.H. LAI moving house, The City University of Hong Kong November 3, 2012 fooling around 1997 A relatively recent conclusion of HK's British

443 views • 32 slides
Origin of Magnetic field A current (or moving charge) experience a magnetic force when it is in a

Origin of Magnetic field A current (or moving charge) experience a magnetic force when it is in a

Origin of Magnetic field A current (or moving charge) experience a magnetic force when it is in a magnetic field. The magnetic field is the result of another current (or moving charge). If electric field describes the interaction between two

344 views • 8 slides
Body Worn Cameras and Transparency Increase in use of cameras Expectation of

Body Worn Cameras and Transparency Increase in use of cameras Expectation of

10/10/2016 Body Cameras: The Intersection of Public Records and Law Enforcement Frayda Bluestein, Jeff Welty, Tom Carruthers Body Worn Cameras and Transparency Increase in use of cameras Expectation of accountability and transparency

348 views • 9 slides
Making VR Inclusive for People with Low Vision Meredith Ringel Morris Microsoft Research

Making VR Inclusive for People with Low Vision Meredith Ringel Morris Microsoft Research

Making VR Inclusive for People with Low Vision Meredith Ringel Morris Microsoft Research http://aka.ms/seeingvr http://aka.ms/vraccess Low Vision VR apps under simulated low vision augmented by SeeingVR: (A) Waltz of the Wizard [Aldin Dynamics,

172 views • 13 slides
CASE NUMBER ONE DIAGNOSIS? 8 year boy referred from school- headaches, reading

CASE NUMBER ONE DIAGNOSIS? 8 year boy referred from school- headaches, reading

12/4/2015 CONVERGENCE DIFFICIENCIES Children vs. Adults Insufficiency vs. Paralysis CASE NUMBER ONE DIAGNOSIS? 8 year boy referred from school- headaches, reading CONVERGENCE INSUFFICIENCY difficulties and blurred vision MY

345 views • 7 slides
Welcome to Sept 12 th , 2020 House Keeping Rules Keep your microphone on mute 4 Rounds of

Welcome to Sept 12 th , 2020 House Keeping Rules Keep your microphone on mute 4 Rounds of

Welcome to Sept 12 th , 2020 House Keeping Rules Keep your microphone on mute 4 Rounds of 10 questions each Honor system marking if you cheat, youre only cheating yourself Dont type the answers out in the chat Dry Note

696 views • 32 slides
Module Sustainable Menu Project ReKuK Vocational Training for Chefs and Executive Chefs

Module Sustainable Menu Project ReKuK Vocational Training for Chefs and Executive Chefs

Module Sustainable Menu Project ReKuK Vocational Training for Chefs and Executive Chefs 2016-1-AT01-KA202-016677 Project ReKuK 2016-1-AT01-KA202-016677 Vocational Training for Chefs and Executive Chefs ModuleSustainable Menu Disclaimer: The

1.2k views • 73 slides
CMSC 20370/30370 Winter 2020 Understanding human abilities for inclusive technology Case Study:

CMSC 20370/30370 Winter 2020 Understanding human abilities for inclusive technology Case Study:

CMSC 20370/30370 Winter 2020 Understanding human abilities for inclusive technology Case Study: Accessibility Feb 5, 2020 Quiz Time (5-7 minutes). Quiz on SeeingVR Principles of Good Design Administrivia Midterm on Friday Feedback

645 views • 62 slides
Disease Monitoring James W. Stark, MD, FAAN TISCH MS RESEARCH CENTER OF NEW YORK 22 ND ANNUAL

Disease Monitoring James W. Stark, MD, FAAN TISCH MS RESEARCH CENTER OF NEW YORK 22 ND ANNUAL

Disease Monitoring James W. Stark, MD, FAAN TISCH MS RESEARCH CENTER OF NEW YORK 22 ND ANNUAL PATIENT SYMPOSIUM Timeline of MS Disease Modifying Therapies 1980s 1990s 2000s 2010s Future Gilenya Ibudilast Aubagio Novantrone

410 views • 10 slides
Lecture 4: Image pyramids PS1 due at midnight PS2 out, due next Tues. No Thursday

Lecture 4: Image pyramids PS1 due at midnight PS2 out, due next Tues. No Thursday

Lecture 4: Image pyramids PS1 due at midnight PS2 out, due next Tues. No Thursday office hours this week If you're on the waitlist, submit your PS1 via email to the course staff. Today Image pyramids Texture We want

801 views • 62 slides
A Robust Recursive Partitioning Algorithm for Mining Multiple Populations Jose Alvir 1 Javier

A Robust Recursive Partitioning Algorithm for Mining Multiple Populations Jose Alvir 1 Javier

A Robust Recursive Partitioning Algorithm for Mining Multiple Populations Jose Alvir 1 Javier Cabrera 2 Frank Caridi 1 Ha Nguyen 1 Pfizer Inc 1 & Rutgers University 2 Rutgers Biostatistics Day, 4/25/2008 The Challenge of Personalized

641 views • 25 slides

More recommend