See in 3D: how it works stereoscopic rendering Vincent Nozick - IGM - - PowerPoint PPT Presentation

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

stereoscopic rendering

Vincent Nozick - IGM / A3SI

Vincent Nozick - IGM / A3SI stereoscopic rendering 1 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

See in 3D: how it works

Vincent Nozick - IGM / A3SI stereoscopic rendering 2 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

See in 3D

• we have 2 eyes
• we get two images that are similar but not identical
• triangulation

Vincent Nozick - IGM / A3SI stereoscopic rendering 3 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Monocular clues

vanishing points, perspective

Vincent Nozick - IGM / A3SI stereoscopic rendering 4 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Monocular clues

Lighting and shadow

Vincent Nozick - IGM / A3SI stereoscopic rendering 5 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Monocular clues

Occlusions (including participating medium)

Vincent Nozick - IGM / A3SI stereoscopic rendering 6 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Monocular clues

Occlusions

Vincent Nozick - IGM / A3SI stereoscopic rendering 7 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Monocular clues

Eye accomodation

Vincent Nozick - IGM / A3SI stereoscopic rendering 8 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Monocular clues

Focus

Vincent Nozick - IGM / A3SI stereoscopic rendering 9 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Monocular clues

Motion parallax

Vincent Nozick - IGM / A3SI stereoscopic rendering 10 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Monocular clues

Experience and knowledge

Vincent Nozick - IGM / A3SI stereoscopic rendering 11 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Monocular clues

• perspective (vanishing points)
• lighting and shadow
• occlusions (participating medium ...)
• eye focus
• motion parallax
• experience

Vincent Nozick - IGM / A3SI stereoscopic rendering 12 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic vision

To see in 3D, we need:

• correct eyes
• visual fields to cover each other
• images fusion

Vincent Nozick - IGM / A3SI stereoscopic rendering 13 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic vision

identical images : no 3D

Vincent Nozick - IGM / A3SI stereoscopic rendering 14 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic vision

small shift

Vincent Nozick - IGM / A3SI stereoscopic rendering 15 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic vision

superposition without fusion

Vincent Nozick - IGM / A3SI stereoscopic rendering 16 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic vision

superposition then suppression

Vincent Nozick - IGM / A3SI stereoscopic rendering 17 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Restitution devices

purpose: to provide to each eye the corresponding image.

Vincent Nozick - IGM / A3SI stereoscopic rendering 18 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Cross vision

Vincent Nozick - IGM / A3SI stereoscopic rendering 19 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Parallel vision

Vincent Nozick - IGM / A3SI stereoscopic rendering 20 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Anaglyph

Vincent Nozick - IGM / A3SI stereoscopic rendering 21 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Inficolor

TriOviz

Vincent Nozick - IGM / A3SI stereoscopic rendering 22 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Chromadepth

Vincent Nozick - IGM / A3SI stereoscopic rendering 23 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Pulfrich

Vincent Nozick - IGM / A3SI stereoscopic rendering 24 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Single Image Random Dot Stereogram

Vincent Nozick - IGM / A3SI stereoscopic rendering 25 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

SIRDS

Vincent Nozick - IGM / A3SI stereoscopic rendering 26 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Visiocask

Vincent Nozick - IGM / A3SI stereoscopic rendering 27 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Polarized glasses

Also exists with circular polarisation

Vincent Nozick - IGM / A3SI stereoscopic rendering 28 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Obturators

synchronized with a high frequency display

Vincent Nozick - IGM / A3SI stereoscopic rendering 29 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

HMD

Head Mounted Display

Vincent Nozick - IGM / A3SI stereoscopic rendering 30 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic displays

Vincent Nozick - IGM / A3SI stereoscopic rendering 31 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic displays

Vincent Nozick - IGM / A3SI stereoscopic rendering 32 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic displays

Vincent Nozick - IGM / A3SI stereoscopic rendering 33 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Ghost

Failure in the 2 images separation:

• crosstalk: screen latency (obturators, ...)
• bad image filtering (anaglyphs, polarization, ..)

Vincent Nozick - IGM / A3SI stereoscopic rendering 34 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Holograms

Vincent Nozick - IGM / A3SI stereoscopic rendering 35 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Others

Vincent Nozick - IGM / A3SI stereoscopic rendering 36 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Others

Vincent Nozick - IGM / A3SI stereoscopic rendering 37 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Others

Vincent Nozick - IGM / A3SI stereoscopic rendering 38 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Parallax

distance in pixel between 2 representations of the same object. ⇒ horizontal and vertical components.

Vincent Nozick - IGM / A3SI stereoscopic rendering 39 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Horizontal positive parallax

Vincent Nozick - IGM / A3SI stereoscopic rendering 40 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Horizontal negative parallax

Vincent Nozick - IGM / A3SI stereoscopic rendering 41 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Horizontal zero parallax

Vincent Nozick - IGM / A3SI stereoscopic rendering 42 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Parallax angle

Vincent Nozick - IGM / A3SI stereoscopic rendering 43 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Vertical parallax

Optical axis: should be parallel?

Vincent Nozick - IGM / A3SI stereoscopic rendering 44 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Vertical parallax

Vincent Nozick - IGM / A3SI stereoscopic rendering 45 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Vertical parallax

To avoid!

Vincent Nozick - IGM / A3SI stereoscopic rendering 46 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic images and computer graphics

Problem: how to generate a pair of stereoscopic images?

Vincent Nozick - IGM / A3SI stereoscopic rendering 47 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Asymetric frustum

Vincent Nozick - IGM / A3SI stereoscopic rendering 48 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Rendering

Vincent Nozick - IGM / A3SI stereoscopic rendering 49 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Analogy

screen ↔ virtual screen distance between eyes ↔ distance between cameras distance between the

• bserver and the screen

↔ distance between the cameras and the virtual screen

Vincent Nozick - IGM / A3SI stereoscopic rendering 50 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Hardware: Nvidia

• Primitive duplication by the drivers
• Programming rules
• Kit all in one

Vincent Nozick - IGM / A3SI stereoscopic rendering 51 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

GPU: multi-view rendering in pass

• Primitive duplication in the geometry shader.
• Available for more than two views (with few additional cost).

Vincent Nozick - IGM / A3SI stereoscopic rendering 52 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic images with real cameras

Vincent Nozick - IGM / A3SI stereoscopic rendering 53 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic images with real cameras

Vincent Nozick - IGM / A3SI stereoscopic rendering 54 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic images with a single camera

Vincent Nozick - IGM / A3SI stereoscopic rendering 55 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Hardware: Depth + texture

+ = ⇒ kinect 2D+depth

Vincent Nozick - IGM / A3SI stereoscopic rendering 56 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic images with real cameras

Vincent Nozick - IGM / A3SI stereoscopic rendering 57 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic images with real cameras

Vincent Nozick - IGM / A3SI stereoscopic rendering 58 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic images with real cameras

Special care about:

• the zoom → lookup table
• the focus
• the colorimetric calibration / correction
• the camera aligment
• the camera synchronisation

Vincent Nozick - IGM / A3SI stereoscopic rendering 59 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Pseudoscopic motion

Vincent Nozick - IGM / A3SI stereoscopic rendering 60 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Pseudoscopic motion

Vincent Nozick - IGM / A3SI stereoscopic rendering 61 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Pseudoscopic motion

Vincent Nozick - IGM / A3SI stereoscopic rendering 62 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Pseudoscopic motion

Vincent Nozick - IGM / A3SI stereoscopic rendering 63 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Orthostereoscopic rendering

Render the scene in the correct proportions. roundness factor = 1 (ratio radius/depth percieved of a 3D sphere)

Vincent Nozick - IGM / A3SI stereoscopic rendering 64 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Rendering

Vincent Nozick - IGM / A3SI stereoscopic rendering 65 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic images

A stereoscopic image pair is generated for a specific rendering device.

Vincent Nozick - IGM / A3SI stereoscopic rendering 66 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Giant

Hyperstereo (miniaturization)

Vincent Nozick - IGM / A3SI stereoscopic rendering 67 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Dwarf

Hypostereo (gigantism)

Vincent Nozick - IGM / A3SI stereoscopic rendering 68 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Depth

Humain: over km, we do not percieve depth.

Vincent Nozick - IGM / A3SI stereoscopic rendering 69 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Blur

Depth of field blur on the background makes the images easier to fusion.

Vincent Nozick - IGM / A3SI stereoscopic rendering 70 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

3D effects

It is recommended to render the main objects with a zero-parallax.

Vincent Nozick - IGM / A3SI stereoscopic rendering 71 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Frame effect

The screen frame (borders) is source of confusion: avoid negative parallax on the screen frame.

Vincent Nozick - IGM / A3SI stereoscopic rendering 72 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Disparity

Avoid big disparity discontinuities between successive pixels.

Vincent Nozick - IGM / A3SI stereoscopic rendering 73 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

From a sequence to the next

Avoid big depth differences between successive sequences.

Vincent Nozick - IGM / A3SI stereoscopic rendering 74 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Reflection

Limit the light reflection phenomenon on the scene (mirror, water, etc.)

Vincent Nozick - IGM / A3SI stereoscopic rendering 75 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Vertical parallax

To avoid!

Vincent Nozick - IGM / A3SI stereoscopic rendering 76 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Focus vs. convergence

• convergence in space (where the triangulation occurs)
• focus on the screen

⇒ headache.

Vincent Nozick - IGM / A3SI stereoscopic rendering 77 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Parallax

To make the image fusion easier: parallax limitation

Vincent Nozick - IGM / A3SI stereoscopic rendering 78 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

3D effects

It is easier to see 3D objects behind the screen.

Vincent Nozick - IGM / A3SI stereoscopic rendering 79 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

3D effects

It is difficult to see 3D objects in front of the screen. ⇒ limit spectacular effects. ⇒ object depth should not exceed the third of the disdance between the observer and the screen.

Vincent Nozick - IGM / A3SI stereoscopic rendering 80 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Binocular mask

• one of the two images is noisy.
• during the stereoscopic images fusion, the sharp image

corrects the noisy image.

Vincent Nozick - IGM / A3SI stereoscopic rendering 81 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Fusion

Some people can not make stereoscopic images fusion.

Vincent Nozick - IGM / A3SI stereoscopic rendering 82 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic images with real cameras

using calibrated cameras (projected textures)

• r point correspondences.

Vincent Nozick - IGM / A3SI stereoscopic rendering 83 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic images: image rectification

Vincent Nozick - IGM / A3SI stereoscopic rendering 84 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Stereoscopic images: image rectification

Vincent Nozick - IGM / A3SI stereoscopic rendering 85 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Colorimetric Correction

↓

Vincent Nozick - IGM / A3SI stereoscopic rendering 86 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Radial distortion

Should be corrected

Vincent Nozick - IGM / A3SI stereoscopic rendering 87 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Radial distortion

Vincent Nozick - IGM / A3SI stereoscopic rendering 88 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Lense flare

No 3D meaning → to avoid.

Vincent Nozick - IGM / A3SI stereoscopic rendering 89 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Noise

Noise: should be corrected (or added on the other views).

Vincent Nozick - IGM / A3SI stereoscopic rendering 90 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

HDR

High Dynamic Range should be the same on each image.

Vincent Nozick - IGM / A3SI stereoscopic rendering 91 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Video coding

Video compression: inter-frame and intra-frame.

Vincent Nozick - IGM / A3SI stereoscopic rendering 92 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

3D reconstruction

computes 2 images from 4 input images in real-time (gpu).

Vincent Nozick - IGM / A3SI stereoscopic rendering 93 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

3D reconstruction

Plane-sweep

Vincent Nozick - IGM / A3SI stereoscopic rendering 94 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Billboards

In 3D, we can seen it → to avoid.

Vincent Nozick - IGM / A3SI stereoscopic rendering 95 / 97 See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Graphic interface

Zero-parallax, or set up according to the scene.

Vincent Nozick - IGM / A3SI stereoscopic rendering 96 / 97

See in 3d Devices stereoscopic images Orthostereoscopy Shooting Post-production

Conclusion

Any questions?

Vincent Nozick - IGM / A3SI stereoscopic rendering 97 / 97