of Light Perception in Virtual Reality ISMAR 2020 Laura R. Luidolt - PowerPoint PPT Presentation

Gaze-Dependent Simulation of Light Perception in Virtual Reality ISMAR 2020 Laura R. Luidolt 1 Michael Wimmer 1 Katharina Krösl 2 1 2 1 TU Wien , 2 VRVis Forschungs-GmbH Research Division of Computer Graphics Institute of Visual Computing & Human-Centered Technology TU Wien, Austria

Introduction brightness range < tone mapping Motivation ▪ Overview ▫ Methodology ▫ Evaluation ▫ Conclusion ▫ 2 L. R. Luidolt

Introduction Motivation ▪ Overview ▫ Methodology ▫ Evaluation ▫ Conclusion ▫ 3 L. R. Luidolt

Introduction ▪ → Perceptual algorithms necessary! ▪ Medically based ▪ Account for viewing direction, pupil size Motivation ▪ Overview ▫ Methodology ▫ Evaluation ▫ Conclusion ▫ 4 L. R. Luidolt

Contribution ▪ Post-processing workflow ▪ Accurate simulation of light perception in VR/AR ▪ Medically-based, perceptual effects ▪ In real-time VR/AR ▪ Following optometrist advice ▪ Eye tracking for measuring light incidence Motivation ▪ ▪ Pilot user study, comparison of Overview ▫ ▪ Real-world low-light situation Methodology ▫ Evaluation ▫ ▪ And VR simulation Conclusion ▫ 5 L. R. Luidolt

Temporal Eye Perceptual Visual Acuity Scotopic Adaptation Glare Reduction Color Vision Visual adjustment to Colorful patterns Blurred details in low Color shift towards bright and dark when viewing bright light scenes blue in low light light sources scenes Adaptation of rods Scattering of light in Rods not present in Rods more sensitive and cones over time the eye fovea (point of to longer wavelength Motivation ▫ sharpest vision) light than cones Overview ▪ Methodology ▫ Evaluation ▫ Conclusion ▫ Based on Krawczyk et al., 2005 6 L. R. Luidolt and Ritschel et al., 2009

Temporal Eye Perceptual Visual Acuity Scotopic Adaptation Glare Reduction Color Vision Visual adjustment to Colorful patterns Blurred details in low Color shift towards bright and dark when viewing bright light scenes blue in low light light sources scenes Adaptation of rods Scattering of light in Rods not present in Rods more sensitive and cones over time the eye fovea (point of to longer wavelength Motivation ▫ sharpest vision) light than cones Overview ▪ Methodology ▫ Evaluation ▫ Conclusion ▫ L. R. Luidolt0 0 2 1,.4 - Based on Krawczyk et al., 2005 8 0.02 0.15 2.2 2.9 and Ritschel et al., 2009

Temporal Eye Perceptual Visual Acuity Scotopic Adaptation Glare Reduction Color Vision Visual adjustment to Colorful patterns Blurred details in low Color shift towards bright and dark when viewing bright light scenes blue in low light light sources scenes Adaptation of rods Scattering of light in Rods not present in Rods more sensitive and cones over time the eye fovea (point of to longer wavelength Motivation ▫ sharpest vision) light than cones Overview ▪ Methodology ▫ Evaluation ▫ Conclusion ▫ Based on Krawczyk et al., 2005 10 L. R. Luidolt and Ritschel et al., 2009

Temporal Eye Perceptual Visual Acuity Scotopic Adaptation Glare Reduction Color Vision Visual adjustment to Colorful patterns Blurred details in low Color shift towards bright and dark when viewing bright light scenes blue in low light light sources scenes Adaptation of rods Scattering of light in Rods not present in Rods more sensitive and cones over time the eye fovea (point of to longer wavelength Motivation ▫ sharpest vision) light than cones Overview ▪ Methodology ▫ Evaluation ▫ Conclusion ▫ Based on Krawczyk et al., 2005 12 L. R. Luidolt and Ritschel et al., 2009

Temporal Eye Perceptual Visual Acuity Scotopic Adaptation Glare Reduction Color Vision Visual adjustment to Colorful patterns Blurred details in low Color shift towards bright and dark when viewing bright light scenes blue in low light light sources scenes Adaptation of rods Scattering of light in Rods not present in Rods more sensitive and cones over time the eye fovea (point of to longer wavelength Motivation ▫ sharpest vision) light than cones Overview ▪ Methodology ▫ Evaluation ▫ Conclusion ▫ Based on Krawczyk et al., 2005 14 L. R. Luidolt and Ritschel et al., 2009

Temporal Eye Adaptation ▪ 𝑀 𝑗 = 𝑀 𝑗−1 + (𝑍 − 𝑀 𝑗−1 ) ∙ ft 𝜐(𝑍) 1 − 𝑓 − ൗ ▪ Target luminance Y Motivation ▫ ▪ Temporally filtered Overview ▫ luminance L i of frame i Methodology ▼ Adaptation ▪ ▪ Photoreceptor adaptation Glare ▫ VA reduction ▫ times τ Color shift ▫ Evaluation ▫ Conclusion ▫ 15 L. R. Luidolt

Perceptual Glare Motivation ▫ Overview ▫ Methodology ▼ Adaptation ▫ Glare ▪ VA reduction ▫ Color shift ▫ Evaluation ▫ Conclusion ▫ Image adapted from 16 L. R. Luidolt commons.wikimedia.org/wiki/File:Eyesection.svg

Perceptual Glare Motivation ▫ Overview ▫ Methodology ▼ Adaptation ▫ Glare ▪ VA reduction ▫ Color shift ▫ Evaluation ▫ Conclusion ▫ Image adapted from 17 L. R. Luidolt commons.wikimedia.org/wiki/File:Eyesection.svg

Perceptual Glare Motivation ▫ Overview ▫ Methodology ▼ Adaptation ▫ Glare ▪ VA reduction ▫ 2 1 1 Color shift ▫ 𝑂 ∙ ℱ 𝑄(𝑦, 𝑧) ∙ 𝑓i 𝜌 𝜇𝑒 𝑦 2 +𝑧 2 𝑁 𝑦, 𝑧 = 𝜇𝑒 2 Evaluation ▫ After Ritschel et al., 2009 Conclusion ▫ Image adapted from 18 L. R. Luidolt commons.wikimedia.org/wiki/File:Eyesection.svg

Perceptual Glare Monochromatic PSF Diffraction on the retina of a Motivation ▫ single wavelength light source Overview ▫ Methodology ▼ Spectral PSF Adaptation ▫ Glare ▪ Combination of multiple VA reduction ▫ Color shift ▫ wavelengths to simulate spectral Evaluation ▫ light Conclusion ▫ 19 L. R. Luidolt

Perceptual Glare Monochromatic PSF Diffraction on the retina of a Motivation ▫ single wavelength light source Overview ▫ Methodology ▼ Spectral PSF Adaptation ▫ Glare ▪ Combination of multiple VA reduction ▫ Color shift ▫ wavelengths to simulate spectral Evaluation ▫ light Conclusion ▫ 20 L. R. Luidolt

Perceptual Glare · (1 - ) + · Motivation ▫ Overview ▫ = Methodology ▼ Adaptation ▫ Glare ▪ VA reduction ▫ Color shift ▫ Evaluation ▫ Conclusion ▫ 21 L. R. Luidolt

Perceptual Glare Motivation ▫ Overview ▫ Methodology ▼ Adaptation ▫ Glare ▪ VA reduction ▫ Color shift ▫ Evaluation ▫ Conclusion ▫ 22 L. R. Luidolt

Visual Acuity Reduction ▪ 𝜏 𝑀 = max(1 − 𝑀, 0) ▪ Gaussian variance 𝜏 Motivation ▫ Overview ▫ ▪ Pixel’s lightness L Methodology ▼ Adaptation ▫ Glare ▫ VA reduction ▪ Color shift ▫ Evaluation ▫ Conclusion ▫ 23 L. R. Luidolt

Scotopic Color Vision Motivation ▫ Overview ▫ Methodology ▼ Adaptation ▫ Glare ▫ VA reduction ▫ Color shift ▪ Off On Evaluation ▫ Conclusion ▫ 24 L. R. Luidolt

Evaluation Motivation ▫ Qualitative user study with 5 Overview ▫ participants Methodology ▫ Evaluation ▪ Conclusion ▫ 25 L. R. Luidolt

Conclusion Motivation ▫ ▪ temporal eye adaptation Real-time VR/AR post-processing workflow Overview ▫ ▪ perceptual glare Using eye tracking ▪ visual acuity reduction Methodology ▫ Based on medical research ▪ scotopic color vision Evaluation ▫ Pilot user study Conclusion ▪ Related article: “ CatARact : Simulating Cataracts in Augmented Reality”, 26 L. R. Luidolt Krösl et al., 2020

Thank you for your attention! Gaze-Dependent Simulation of Light Perception in Virtual Reality Laura R. Luidolt, luidolt@cg.tuwien.ac.at Michael Wimmer, wimmer@cg.tuwien.ac.at Katharina Krösl, kroesl@vrvis.at