Modern Display Technology - Rendering Challenges - Philipp - - PowerPoint PPT Presentation

▶
modern display technology
SMART_READER_LITE
LIVE PREVIEW

Modern Display Technology - Rendering Challenges - Philipp - - PowerPoint PPT Presentation

Mar 24, 2023 307 likes •838 views

Modern Display Technology - Rendering Challenges - Philipp Slusallek Karol Myszkowski Gurprit Singh Realistic Image Synthesis SS18 Modern Display Technologies Karol Myszkowski Outline Binocular 3D displays Color Anaglyph

slide-1
SLIDE 1

Realistic Image Synthesis SS18 – Modern Display Technologies

Modern Display Technology

  • Rendering Challenges -

Philipp Slusallek Karol Myszkowski Gurprit Singh

Karol Myszkowski

slide-2
SLIDE 2

Realistic Image Synthesis SS18 – Modern Display Technologies

Outline

  • Binocular 3D displays

– Color Anaglyph – Polarization – Active Shutter Glasses – Head-Mounted Displays

  • Autostereoscopic (Glass-free 3D) Displays

– Parallax Barriers – Integral Imaging – Multi-layer displays

  • Light field displays
  • Multi-projector displays
  • HDR displays
slide-3
SLIDE 3

Realistic Image Synthesis SS18 – Modern Display Technologies

Binocular Stereovision

slide-4
SLIDE 4

Realistic Image Synthesis SS18 – Modern Display Technologies

Binocular 3D Displays

  • Capable of providing sense of 3D by simulating

binocular disparity

– Color Anaglyphs – Polarization – Shutter Glasses – Head-Mounted Displays

  • They mostly do not provide accommodation depth cue
slide-5
SLIDE 5

Realistic Image Synthesis SS18 – Modern Display Technologies

Color Anaglyphs

  • Left and right images are filtered using different colors (usually

complementary):

– Red – Green, Red – Cyan, Green – Magenta – Amber – Blue (ColorCode 3D, patented [Sorensen et al. 2004])

  • Limited color perception (since each eye sees only a subset of whole

colorspace)

Images adapted from http://axon.physik.uni-bremen.de/research/stereo/color_anaglyph/

slide-6
SLIDE 6

Realistic Image Synthesis SS18 – Modern Display Technologies

Polarization

  • Usually a wire grid filter converts the unpolarized light beam to a

polarized one

Projector Polarizing Filter Screen (preserving polarization) Glasses with polarizing filters

Images adapted from https://cpinettes.u-cergy.fr/S6-Electromag_files/fig1.pdf

Unpolarized light source Wire grid filter

slide-7
SLIDE 7

Realistic Image Synthesis SS18 – Modern Display Technologies

Shutter Glasses

  • Exploits the “memory effect” of the Human Visual System [Coltheart

1980]

  • Glasses have shutters which operate in synchronization with the display

system

  • Left and right eye images are shown in alternation
  • Color neutral; however, temporal resolution is reduced

IR receiver for synchronization

Images adapted from https://en.wikipedia.org/wiki/Active_shutter_3D_system

slide-8
SLIDE 8

Realistic Image Synthesis SS18 – Modern Display Technologies

Head-Mounted Displays

  • Separate displays for the left and right eye
  • May provide current orientation of the head (and update the

stimuli accordingly to provide a VR)

Images adapted from http://www.oculus.com

slide-9
SLIDE 9

Realistic Image Synthesis SS18 – Modern Display Technologies

Reflections and Refractions in S3D

slide-10
SLIDE 10

Realistic Image Synthesis SS18 – Modern Display Technologies Rivalry

Reflections and Refractions in S3D

slide-11
SLIDE 11

Realistic Image Synthesis SS18 – Modern Display Technologies glossy surface light source eyes virtual image

  • n-surface

highlights

matte look

see: G. Wendt et al., 2008 Highlight disparity contributes to the authenticity and strength of perceived glossiness glossy surface light source eyes

Reflections and Refractions in S3D

slide-12
SLIDE 12

Realistic Image Synthesis SS18 – Modern Display Technologies

Correct highlights

[K. Templin et al., ACM SIGGRAPH 2012]

Highlights in Stereo 3D: Microdisparity

Possible binocular rivalry

Even more annoying for HDR displays Moving head does not help

Highlight microdisparity solution improves viewing comfort while maintaining glossy look

On-surface highlight Our solution

slide-13
SLIDE 13

Realistic Image Synthesis SS18 – Modern Display Technologies Dąbała et al. Manipulating refractive and reflective binocular disparity, Eurographics 2014, Strasbourg / France Physical Ours

Refractions in Stereo 3D

slide-14
SLIDE 14

Realistic Image Synthesis SS18 – Modern Display Technologies

… …

1st diffuse Reflection Refraction

Reflections and Refractions in S3D

slide-15
SLIDE 15

Realistic Image Synthesis SS18 – Modern Display Technologies

𝑔 𝑒, 𝑠, 𝑥 = 𝛽𝑒𝑔

𝑒 𝑒, 𝑥 + 𝛽𝑏𝑔 𝑏 𝑒, 𝑥 + 𝛽𝑞𝑔 𝑞 𝑒, 𝑥 + 𝛽𝑠𝑔 𝑠 𝑠, 𝑥

Data term Absolute disparity Relative disp. Rivalry

Reflections and Refractions in S3D

slide-16
SLIDE 16

Realistic Image Synthesis SS18 – Modern Display Technologies

Optimizing Eye Vergence – Film Cuts

slide-17
SLIDE 17

Realistic Image Synthesis SS18 – Modern Display Technologies

Cut in a Regular Film

Shot 1 Shot 2 Cut

Source: Big Buck Bunny CC-BY Blender Foundation, Janus B. Kristensen

slide-18
SLIDE 18

Realistic Image Synthesis SS18 – Modern Display Technologies

Saccades

Left eye Right eye 2D Display

slide-19
SLIDE 19

Realistic Image Synthesis SS18 – Modern Display Technologies

Optimizing Eye Vergence – Film Cuts

slide-20
SLIDE 20

Realistic Image Synthesis SS18 – Modern Display Technologies

Cut in a Stereoscopic 3D Film

Shot 1 Shot 2 Cut

Source: Big Buck Bunny CC-BY Blender Foundation, Janus B. Kristensen

Left eye Right eye

slide-21
SLIDE 21

Realistic Image Synthesis SS18 – Modern Display Technologies

Vergence

Left eye Right eye 3D Display

slide-22
SLIDE 22

Realistic Image Synthesis SS18 – Modern Display Technologies

3D cuts are challenging

  • Unexpected change
  • Different view-point/scene
  • Missing depth cues
  • Retinal blur
  • Head motion parallax

Vergence vs. Film Editing

1930 1970 2010 25 s 10 s 4 s 2 s 1 s

Cutting et al. 2011. Quicker, faster, darker: Changes in Hollywood film over 75 years

Average shot length

Rocky IV

slide-23
SLIDE 23

Realistic Image Synthesis SS18 – Modern Display Technologies

Autostereoscopic Displays

  • Stereo displays which are viewable without special glasses or

head-wear equipment

  • Simulate an approximate lightfield with a finite number of views

– Parallax Barriers – Integral Imaging – Multi-layer Displays

Image adapted from Geng, Jason. "Three-dimensional display technologies." Advances in optics and photonics 5.4 (2013): 456-535.

slide-24
SLIDE 24

Realistic Image Synthesis SS18 – Modern Display Technologies

Parallax Barriers

  • Occlusion-based working principle and key features

[Ives 1903]:

Reduced resolution and brightness

There is an “optimal” distance for observation

If this aperture is too small, diffraction effects are introduced. This is a problem for high- resolution displays.

slide-25
SLIDE 25

Realistic Image Synthesis SS18 – Modern Display Technologies

Parallax Barriers

Video adapted from: http://www.youtube.com/watch?v=sxF9PGRiabw “Glasses-Free 3D Gaming for $5 (Parallax Barrier)”

slide-26
SLIDE 26

Realistic Image Synthesis SS18 – Modern Display Technologies

Parallax Barriers

Video adapted from: http://www.youtube.com/watch?v=sxF9PGRiabw “Glasses-Free 3D Gaming for $5 (Parallax Barrier)”

slide-27
SLIDE 27

Realistic Image Synthesis SS18 – Modern Display Technologies

Parallax Barriers

  • It is possible to switch between 2D and 3D modes
  • Parallax barrier of Nintendo 3DS turning on/off under microscope:

Video adapted from: https://www.youtube.com/watch?v=D-LzRT7Bvc0

slide-28
SLIDE 28

Realistic Image Synthesis SS18 – Modern Display Technologies

Integral Imaging

  • Refraction-based working principle [Lippmann 1908]:

Images adapted from http://www.3d-forums.com/threads/autostereoscopic-displays.1/

It is possible to reproduce parallax, perspective shift and accommodation depth cues. Reduction in resolution and brightness is still a problem.

There is an “optimal” distance for viewing

slide-29
SLIDE 29

Realistic Image Synthesis SS18 – Modern Display Technologies

Integral Imaging

3D Scene

Array of lenses (multiple cameras each with a single lens [Wilburn 2005] or a single camera with multiple lenses in front of the sensor [Ng 2005])

Elemental Images

Images adapted from Martınez-Corral, Manuel, et al. "3D integral imaging monitors with fully programmable display parameters."

slide-30
SLIDE 30

Realistic Image Synthesis SS18 – Modern Display Technologies

Integral Imaging

Images adapted from Martınez-Corral, Manuel, et al. "3D integral imaging monitors with fully programmable display parameters."

Integral Image as seen by the observer

slide-31
SLIDE 31

Realistic Image Synthesis SS18 – Modern Display Technologies

  • Smooth transitions

Multi-view Autostereoscopic Display

Multi-view autostereoscopic display

View 1 View 2 View 3 View 4 „Antialiasing for automultiscopic 3D displays” [Zwicker et al. 2006]

slide-32
SLIDE 32

Realistic Image Synthesis SS18 – Modern Display Technologies

  • Smooth transitions
  • Blur increases with depth

Multi-view autostereoscopic display

View 1 View 2 View 3 View 4

Weaker depth percept

„Antialiasing for automultiscopic 3D displays” [Zwicker et al. 2006]

Multi-view Autostereoscopic Display

slide-33
SLIDE 33

Realistic Image Synthesis SS18 – Modern Display Technologies

Input disparity Output disparity Input disparity Output disparity

  • Big disparity range
  • Large part of the scene out of focus
  • Everything stays in focus
  • Disparity range reduced

Multi-view Autostereoscopic Display

slide-34
SLIDE 34

Realistic Image Synthesis SS18 – Modern Display Technologies

Multi-layer Displays

  • Improved resolution over parallax barriers and lenslet arrays
  • Provides a solution to accommodation-vergence conflict

Images adapted from Wetzstein, Gordon, et al. "Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays." ACM Transactions on Graphics (ToG). Vol. 30. No. 4. ACM, 2011.

slide-35
SLIDE 35

Realistic Image Synthesis SS18 – Modern Display Technologies

Tensor Displays

  • Lightfield emitted by a multi-layer display is represented by a tensor where rays

span a 2D plane in 3D tensor space

  • Target lightfield is decomposed into Rank-1 tensors using Nonnegative Tensor

Factorization

  • Rank-1 tensors are shown in quick succession with a high refresh rate, which are

perceptually averaged over time by the Human Visual System

Video adapted from Wetzstein, Gordon, et al. "Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting." (2012).

slide-36
SLIDE 36

Realistic Image Synthesis SS18 – Modern Display Technologies

Accomodation-Vergence Conflict

Visuals adapted from Akeley, Kurt, et al. "A stereo display prototype with multiple focal distances." ACM transactions on graphics (TOG). Vol. 23. No. 3. ACM,

  • 2004. and Narain, Rahul, et al. "Optimal presentation of imagery with focus cues on multi-plane displays." ACM Transactions on Graphics (TOG) 34.4 (2015): 59.
slide-37
SLIDE 37

Realistic Image Synthesis SS18 – Modern Display Technologies

Multi-focal Plane Displays

  • A display prototype with multiple focal distances using

beam-splitters

Images adapted from Akeley, Kurt, et al. "A stereo display prototype with multiple focal distances." ACM transactions on graphics (TOG).

  • Vol. 23. No. 3. ACM, 2004.
slide-38
SLIDE 38

Realistic Image Synthesis SS18 – Modern Display Technologies

Multi-focal Plane Displays

Prototype introduced by Love et al [2009]

Images adapted from Narain, Rahul, et al. "Optimal presentation of imagery with focus cues on multi-plane displays." ACM Transactions

  • n Graphics (TOG) 34.4 (2015): 59.

Narain et al. [2015] optimize the focus cues for improved realism. Halo artifacts

slide-39
SLIDE 39

Realistic Image Synthesis SS18 – Modern Display Technologies

Deformable Beamsplitter

See-through Dynamic focal depth: objects at any depth Wide field of view Optics are simple

Membrane AR – Dunn et al.

slide-40
SLIDE 40

Realistic Image Synthesis SS18 – Modern Display Technologies

Membrane AR – Dunn et al.

Deformable Beamsplitter

slide-41
SLIDE 41

Realistic Image Synthesis SS18 – Modern Display Technologies

Membrane AR – Dunn et al.

slide-42
SLIDE 42

Realistic Image Synthesis SS18 – Modern Display Technologies

Accommodation Response

  • Step change of fixated object depth

– Smooth and steady accommodation increase

  • up to 1 second to achieve the full accommodation state
  • ~300 ms latency

Bharadwaj and Schor, Vision Research 2004

slide-43
SLIDE 43

Realistic Image Synthesis SS18 – Modern Display Technologies

Lightfield Displays

4 5

slide-44
SLIDE 44

Realistic Image Synthesis SS18 – Modern Display Technologies

Displays Comparison

4 6

2D Display Stereoscopic Display Autostereoscopic Display Automultiscopic Display Light field Display Pictorial Cues Disparity Motion Parallax Accommodation Head-mounted Display Glasses-free

slide-45
SLIDE 45

Realistic Image Synthesis SS18 – Modern Display Technologies

Multi-projector Displays

  • Mainly used to provide a wide panoramic display
  • Edge blending, color/contrast/brightness matching between
  • verlapping regions is an issue (the transition must be seamless)
  • The display surface may be curved

Images adapted from http://www.matrox.com

slide-46
SLIDE 46

Realistic Image Synthesis SS18 – Modern Display Technologies

Multi-projector Displays

Video adapted from https://www.youtube.com/watch?v=dOY2lREuwjU

slide-47
SLIDE 47

Realistic Image Synthesis SS18 – Modern Display Technologies

HDR Displays

  • Instead of using a single constant backlight source, an

array of LEDs is used

  • The LEDs may be individually adjusted for different

brightness levels

Images adapted from http://www.bit-tech.net/hardware/2005/10/04/brightside_hdr_edr/6

slide-48
SLIDE 48

Realistic Image Synthesis SS18 – Modern Display Technologies

HDR Displays

  • Comparison of LDR display (left) with Brightside HDR

display (right)

Images adapted from http://www.bit-tech.net/hardware/2005/10/04/brightside_hdr_edr/8

slide-49
SLIDE 49

Realistic Image Synthesis SS18 – Modern Display Technologies

HDR Displays

  • Comparison of LDR display (left) with Brightside HDR

display (right)

Images adapted from http://www.bit-tech.net/hardware/2005/10/04/brightside_hdr_edr/8

slide-50
SLIDE 50

Realistic Image Synthesis SS18 – Modern Display Technologies

HDR Displays

  • Comparison of LDR display (left) with Brightside HDR

display (right)

Images adapted from http://www.bit-tech.net/hardware/2005/10/04/brightside_hdr_edr/8

slide-51
SLIDE 51

Realistic Image Synthesis SS18 – Modern Display Technologies

References

  • Geng, Jason. "Three-dimensional display technologies." Advances in optics and photonics 5.4

(2013): 456-535.

  • Ives, Frederic E. "Parallax stereogram and process of making same." U.S. Patent No. 725,567. 14 Apr.

1903.

  • Lippmann, Gabriel. "Epreuves reversibles donnant la sensation du relief." J. Phys. Theor. Appl. 7.1

(1908): 821-825.

  • Wilburn, Bennett, et al. "High performance imaging using large camera arrays." ACM Transactions on

Graphics (TOG). Vol. 24. No. 3. ACM, 2005.

  • Ng, Ren, et al. "Light field photography with a hand-held plenoptic camera." Computer Science

Technical Report CSTR 2.11 (2005): 1-11.

  • Coltheart, M. "The persistences of vision." Philosophical Transactions of the Royal Society of

London B: Biological Sciences 290.1038 (1980): 57-69.

  • Akeley, Kurt, et al. "A stereo display prototype with multiple focal distances." ACM transactions on

graphics (TOG). Vol. 23. No. 3. ACM, 2004.

  • Sorensen, Svend Erik Borre, Per Skafte Hansen, and Nils Lykke Sorensen. "Method for recording and

viewing stereoscopic images in color using multichrome filters." U.S. Patent No. 6,687,003. 3 Feb. 2004.

  • Love, Gordon D., et al. "High-speed switchable lens enables the development of a volumetric

stereoscopic display." Optics express 17.18 (2009): 15716-15725.

  • Wetzstein, Gordon, et al. "Layered 3D: tomographic image synthesis for attenuation-based light field

and high dynamic range displays." ACM Transactions on Graphics (ToG). Vol. 30. No. 4. ACM, 2011.

  • Wetzstein, Gordon, et al. "Tensor displays: compressive light field synthesis using multilayer

displays with directional backlighting." (2012).

  • Narain, Rahul, et al. "Optimal presentation of imagery with focus cues on multi-plane displays." ACM

Transactions on Graphics (TOG) 34.4 (2015): 59.

slide-52
SLIDE 52

Realistic Image Synthesis SS18 – Modern Display Technologies

Acknowledgements

  • I would like to thank Okan Tursun for help in preparing

those slides.

Karol Myszkowski