In the name of Allah the compassionate, the merciful Digital Video - - PowerPoint PPT Presentation

In the name of Allah

the compassionate, the merciful

Digital Video Systems

• S. Kasaei
• S. Kasaei

Room: CE 307 Department of Computer Engineering Sharif University of Technology E-Mail: skasaei@sharif.edu Webpage: http://sharif.edu/~skasaei

• Lab. Website: http://ipl.ce.sharif.edu

Acknowledgment

Most of the slides used in this course have been provided by: Prof. Yao Wang (Polytechnic University, Brooklyn) based on the book: Video Processing & Communications written by: Yao Wang, Jom Ostermann, & Ya-Oin Zhang Prentice Hall, 1st edition, 2001, ISBN: 0130175471. [SUT Code: TK 5105 .2 .W36 2001]

Chapter 5

Video Modeling

Outline

Camera model Object model

Shape model Motion model

Scene model 2-D motion model

Why Modeling?

To describe various events in a video processing

system in parametric forms.

To enable estimation of these parameters.

Pinhole Camera

Perfect image if the hole is infinitely small. Pure geometric optics. No depth of field issue.

Simplified Pinhole Camera

Eye-image pyramid (frustum). Note that the distance/size of image is arbitrary.

Pinhole Camera

Y Y X Z C X X x x y x y F Z 2-D image 3-D point

The image of an object is reversed from its 3-D position. The object appears smaller when it is farther away. Focal Length Focal /Camera Center Imaging Plane

Perspective vs. Orthographic

Perspective Orthographic

Parallel Lines

Pinhole Camera Model: Perspective Projection

Z y x Z Y F y Z X F x Z Y F y Z X F x to related inversely are , , , = = ⇒ = =

All points in this ray will have the same image.

Approximate Model: Orthographic Projection

When the object is very far ( ) , Can be used as long as the depth variation within the object is small compared to the distance of the object. Z x X y Y → ∞ = =

Perspective Projection

Rigid Object Motion

z y x z y x

T T T , , : ; , , : ] [ ; ) ]( [ ' : center

• bject

the

• n wrp.

translati and Rotation T R C T C X R X θ θ θ + + − =

Flexible Object Motion

Two ways to describe it:

Decompose into multiple, but connected rigid

sub-objects.

Ex., Human body consists of many parts each

undergoes a rigid motion.

Global motion plus local motion in sub-

• bjects.

Ex., Global camera motion plus local object

motions.

Scene Models

A scene is determined by:

Illumination Objects in the scene (their shape, motion, & relative

positions)

Camera

3-D scene model 2.5-D scene model 2-D scene model

3-D Scene Model

Perspective camera

(object image depends

• n depth).

Ambient illumination. Objects at varying

depth.

Used for 3-D

motion/structure estimation.

2.5-D Scene Model

Orthographic camera

(depth has no effect on

• bject image).

Ambient illumination. Objects at varying depth

(layered objects, MPEG-4).

2-D Scene Model

Orthographic camera. Ambient illumination. Objects are flat & at the

same depth (H.261, H.263, MPEG-1, MPEG-2).

2-D Scene Model

Projection of 3-D motion Camera motion Rigid object motion

Projective mapping

Approximation of projective mapping

Affine model Bilinear model

3-D Motion -> 2-D Motion

2-D MV 3-D MV

Sample Motion Field

Motion Field Motion Vector

Occlusion Effect

Motion is undefined in occluded regions.

Typical Camera Motions

Mounted Camera

2-D Motion Corresponding to Camera Motion

Camera zoom Camera rotation around Z-axis (roll)

Linear Transformations

Affine Transformations

Preserves parallel lines.

• P. Distances

& Angles

• P. Angles

Projective Transformations

Preserves lines.

Projective Mapping

Sampled Perspective Affine Translation

Projective Mapping

Affine Rigid Projective Nonlinear

Motion Field Corresponding to Different 2-D Motion Models

Translation Bilinear Perspective Affine

Projective Mapping

Two features of projective mapping:

Chirping: increasing perceived spatial frequency for far away objects [ ]. Converging (Keystone): parallel lines converge in distance [ ].

nonparallel nonequal parallel Co Ch&Co Ch&Co Ch&Co Ch

2-D Motion Corresponding to Rigid Object Motion

General case: Prospective mapping:

F T Z F r y r x r F T Z F r y r x r F y F T Z F r y r x r F T Z F r y r x r F x T T T Z Y X r r r r r r r r r Z Y X

z y z x z y x

+ + + + + + = + + + + + + =       →            +                     =           ) ( ) ( ' ) ( ) ( ' ' ' '

9 8 7 6 5 4 9 8 7 3 2 1 Projection e Perspectiv 9 8 7 6 5 4 3 2 1

y c x c y b x b b y y c x c y a x a a x

2 1 2 1 2 1 2 1

1 ' , 1 ' : c) bY aX (Z planar is surface

• bject

When the + + + + = + + + + = + + =

8-parameter 12-parameter

Affine and Bilinear Models

Affine (6-parameter):

Good for mapping triangles to triangles. Cannot capture either chirping or converging

effect.

      + + + + =       y b x b b y a x a a y x d y x d

y x 2 1 2 1

) , ( ) , (

Affine and Bilinear Models

Bilinear (8-parameter):

Good for mapping blocks to quadrangles. Can capture converging effect of projective

mapping but not chirping effect.

      + + + + + + =       xy b y b x b b xy a y a x a a y x d y x d

y x 3 2 1 3 2 1

) , ( ) , (

Homework 3

Reading assignment:

Chapter 5: Sec. 5.1, 5.4, & 5.5

Written assignment:

• Prob. 5.2, 5.3, 5.4, 5.5, & 5.6

Correction to Problems:

5.3: Show that the projected 2-D motion of a 3-D object

undergoing rigid motion can be described by Eq.(5.5.13).

5.4: Change aX+bY+cZ=1 to Z=aX+bY+c.

Other corrections:

P.125, Fig. 5.11 caption: “diffuse”->”ambient”.

The End