3D VIDEO SYSTEMS Fernando Pereira Audio and Video Communication, - PowerPoint PPT Presentation

3D VIDEO SYSTEMS Fernando Pereira Audio and Video Communication, Fernando Pereira, 2014/2015

It’s a 3D World ! Audio and Video Communication, Fernando Pereira, 2014/2015

Context and Motivation • Strong interest in 3D services • Increasing production of premium content, e.g. movies and sports • Numerous devices supporting stereoscopic displaying available to the consumer including mobile • Substantial investments being made to upgrade digital cinema theaters with 3D capabilities • Many new standards being developed, e.g. production, distribution, digital interfaces Audio and Video Communication, Fernando Pereira, 2014/2015

Stereoscopic Displays Sales Forecast Source: DisplaySearch 3D Display Technology and Market Forecast Report Audio and Video Communication, Fernando Pereira, 2014/2015

Critical Success Factors • High quality experience not burdened with high transition costs or turned off by viewing discomfort or fatigue • Usability and consumer acceptance of 3D viewing technology, e.g., glasses vs no glasses • Availability of premium 3D content in the home • Availability of an appropriate data format providing interoperability through the delivery chain and taking into consideration the constraints imposed by each delivery channel Audio and Video Communication, Fernando Pereira, 2014/2015

3D Perception Basics Audio and Video Communication, Fernando Pereira, 2014/2015

The Human Eye Rod and cone cells in the retina allow conscious light perception and vision including color differentiation and the perception of depth. The crystalline lens changes/focus for the light to strike the retina Audio and Video Communication, Fernando Pereira, 2014/2015

Human Visual System Audio and Video Communication, Fernando Pereira, 2014/2015

Stereoscopic Vision • Accommodation, a monocular cue, refers to the variation of the crystalline lens shape and thickness (and thus its focal length), to allow the eye to focus on an object as its distance varies to maintain a clear image or focus. • Vergence, a binocular cue, refers to the muscular rotation of the eye balls, which is used to converge both eyes on the same object. • Under normal conditions, changing the focus of the eyes to look at an object at a different distance will automatically cause vergence and accommodation, sometimes known as the accommodation-convergence reflex . • In reality, the viewer eyes accommodate (focus) and converge (point) to the depth of the object . Audio and Video Communication, Fernando Pereira, 2014/2015

Retinal Disparity • The retinal images are different in the left and right eyes depending on the object distance and angle. • The retinal disparity is the slight difference in the two retinal images due to the angle from which each eye views an object. • The retinal disparity is an importante depth cue. Audio and Video Communication, Fernando Pereira, 2014/2015

Positive and Negative Disparities Negative Disparity Location of Stereoscopic “3D” Display Image Positive Disparity • Negative and positive disparities are not a natural situation as normally both accommodation and vergence occur at the same depth. • Too large a disparity may cause eyestrain (especially in older viewers). Audio and Video Communication, Fernando Pereira, 2014/2015

Depth Perception Control • In the human visual system, perception of depth is greatly controlled by 2 parameters: retinal disparity (controlled by vergence) and focus (controlled by accommodation). • Accommodation cannot be controlled in a stereo display situation as humans focus the eyes on the screen surface, even if objects are positioned in 3D in front or behind the screen surface. • Current stereo display technologies only have indirect control over vergence by presenting slightly different images to the left and right eyes (disparity). Audio and Video Communication, Fernando Pereira, 2014/2015

Accomodation-Vergence Conflict • In natural viewing, the vergence stimulus and focal stimulus are always at the same distance and, therefore, are consistent with one another. • Stereo displaying create (varying) inconsistencies between vergence and focal distances because the vergence distance varies depending on the image contents while the focal distance remains constant (in the screen). • The accommodation-vergence conflicts lead to problems, notably 3D structure distortions and visual fatigue. Audio and Video Communication, Fernando Pereira, 2014/2015

Depth Perception: the Comfort Zone • Due to the accommodation-vergence conflict, there is a limited disparity range allowing proper stereo vision and depth perception. In content production, the admissible disparity range is called comfort zone . • 3D video production has to map the arbitrary depth range of the real world into this comfort zone by carefully modifying the stereo camera baseline and convergence settings. Audio and Video Communication, Fernando Pereira, 2014/2015

Camera Baseline • The camera baseline or base is the distance between the 2 stereoscopic lenses. This distance has a profound effect on stereo content. • For most images, the baseline is close to the distance between the human eyes, which is around 65 mm. • However, it is possible to use a shorter baseline for closeup photography or a longer baseline when shooting distant subjects such as the moon or mountains. This is critical to ‘put’ the real world in the comfort zone. • Especially for 3D home entertainment, newer stereoscopic displays can vary the baseline between the views to adapt to different viewing distances. Audio and Video Communication, Fernando Pereira, 2014/2015

Screen Disparity … Screen Disparity = distance between P left and P right Left P left Eye P right Right Object with Eye Positive P right Disparity P left Object with Negative Disparity Display Screen Audio and Video Communication, Fernando Pereira, 2014/2015

Disparity, Depth & Perceived Depth Perceived depth depends on ds on - be: eye distance - D: viewer distance to display d’: screen parallax - (function of disparity) b D  f b e Z ' d   ( b d ' ) Z e • Disparity depends on - f: focal length of cameras - b: distance between cameras (baseline) - Z: depth of the object Audio and Video Communication, Fernando Pereira, 2014/2015

Disparity, Depth & Perceived Depth The left and right views are shot with some disparity ! The left and right views are consumed with some depth depending on the consumption conditions ! Audio and Video Communication, Fernando Pereira, 2014/2015

Influence of Viewing Distance and Age • Perceived depth is directly proportional to viewer distance • Kid eye distance/baseline is ~84% of an adult • Therefore, for kids, closer objects look closer, far objects look farther Audio and Video Communication, Fernando Pereira, 2014/2015

Depth Cues: Monocular and Binocular • Most of the depth cues used by humans to visualize the world’s 3D structure are available in 2D projections; this is why images make sense on a TV/cinema screen. • The depth cues can be classified into oculomotor cues coming from the eye muscles, and visual cues from the scene content itself. They can also be classified into monocular and binocular cues. • Monocular cues for 3D perception include: - Occlusion - one object partially covering another - Perspective - point of view - Familiar size - we know the real-world sizes of many objects - Atmospheric haze - objects further away look more washed out - Selective focus or Accommodation of the eyeball (eyeball focus) - the eye changes optical power to maintain a clear image (focus) on an object as its distance changes. - … Audio and Video Communication, Fernando Pereira, 2014/2015

Main Monocular Depth Cues Audio and Video Communication, Fernando Pereira, 2014/2015

Main Binocular Depth Cues Some main cues are missing from 2D media: • Stereo parallax - seeing a different image with each eye, thus different aspects of the same object • Motion parallax - seeing different perspective images when we move our heads; nearby objects appear to move faster across the view • Vergence - muscular rotation of the eye balls, which is used to converge both eyes on the same object Audio and Video Communication, Fernando Pereira, 2014/2015

Range of Effectiveness of Depth Cues • Not all cues have the same importance in the visual system, and their relative importance depends on the viewing distance, among other factors • Some depth cues are independent of distance, such as occlusion or relative size, where as others are distance-dependent, such as disparity or vergence Audio and Video Communication, Fernando Pereira, 2014/2015

3D Systems Audio and Video Communication, Fernando Pereira, 2014/2015