In the name of Allah the compassionate, the merciful Digital Video - - PowerPoint PPT Presentation

In the name of Allah

the compassionate, the merciful

Digital Video Systems

• S. Kasaei
• S. Kasaei

Room: CE 307 Department of Computer Engineering Sharif University of Technology E-Mail: skasaei@sharif.edu Webpage: http://sharif.edu/~skasaei

• Lab. Website: http://ipl.ce.sharif.edu

Acknowledgment

Most of the slides used in this course have been provided by: Prof. Yao Wang (Polytechnic University, Brooklyn) based on the book: Video Processing & Communications written by: Yao Wang, Jom Ostermann, & Ya-Oin Zhang Prentice Hall, 1st edition, 2001, ISBN: 0130175471. [SUT Code: TK 5105 .2 .W36 2001]

Chapter 6

2-D Motion Estimation

Part I: Fundamentals & Basic Techniques

6 Kasaei

Outline

2-D motion vs. optical flow Optical flow equation & ambiguity in motion

estimation

General methodologies in motion estimation

Motion representation Motion estimation criterion Optimization methods Gradient descent methods

Pixel-based motion estimation Block-based motion estimation

EBMA algorithm

7 Kasaei

2-D Motion Estimation

Motion estimation (ME) is an important part of

many video processing tasks.

ME main applications are video compression,

sampling rate conversion, filtering, …

For computer vision, motion vectors (MV) are used to

deduce 3-D structure & motion parameters (sparse but accurate set of MVs are required).

For video coding, MVs are used to produce motion-

compensated predicted frame to reduce required bitrate for coding MVs & prediction errors (tense and accurate set of MVs are required).

8 Kasaei

2-D Motion Estimation

An ME problem is converted to an

• ptimization problem that involves key

components of:

Parameterization of motion field. Formulation of optimization criterion. Searching for optimal parameters.

Optimal Motion Parameters Optimization Criteria Motion Field Input Frames

9 Kasaei

2-D Motion vs. Optical Flow

(a) A sphere is rotating under a constant ambient illumination, but observed image does not change. (b) A point light source is rotating around a stationary sphere, causing highlight point on sphere to rotate.

2-D Motion: Projection of 3-D motion. Depends on 3-D object motion &

projection operator (physical aspects).

Optical flow: “Perceived” 2-D motion based on changes in image pattern,

also depends on illumination & object surface texture. (a) (b)

10 Kasaei

Correspondence & Optical Flow

2-D displacement & velocity fields are projections of

respective 3-D fields into image plane.

Correspondence field & optical flow field are

displacement & velocity functions “perceived” from the time-varying image intensity pattern.

Correspondence field & optical flow field are also

called “apparent 2-D displacement” field & “apparent 2-D velocity” field.

11 Kasaei

Correspondence & Optical Flow

Since we can only observe correspondence & optical

flow fields, we assume that they are the same as the 2-D motion field.

When illumination condition is unknown, the best one

can do is to estimate the optical flow.

Constant intensity assumption: The image of the same

• bject point at different time intervals have the same

luminance value.

Constant intensity assumption (CIA) Optical flow

(OF) equation.

12 Kasaei

Optical Flow Equation

• r
• r

: equation flow

• ptical

the have we two, above the Compare ) , , ( ) , , ( : expansion s Taylor' using But, ) , , ( ) , , ( : " assumption intensity constant " Under = ∂ ∂ + ∇ = ∂ ∂ + ∂ ∂ + ∂ ∂ = ∂ ∂ + ∂ ∂ + ∂ ∂ ∂ ∂ + ∂ ∂ + ∂ ∂ + = + + + = + + + t t v y v x d t d y d x d y d y d x t y x d t d y d x t y x d t d y d x

T y x t y x t y x t y x t y x

ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ v

[The velocity vector (flow vector), v, is the unknown parameter. One equation with two unknowns.] [(x,y,t) (x+dx, y+dy, t+dt)]

spatial gradient vector

13 Kasaei

Ambiguities in Motion Estimation

Optical flow equation only

constrains the flow vector in the gradient direction ( ).

The flow vector in the tangent

direction ( ) is under- determined (aperture problem).

Also, in regions with constant

brightness ( ), the flow is indeterminate Motion estimation is unreliable in regions with flat texture, but more reliable near edges.

n

v = ∇ψ

= ∂ ∂ + ∇ + = t v v v

n t t n n

ψ ψ e e v

t

v

? ? v

aperture problem

If:

gradient vector magnitude no vt!

14 Kasaei

Ambiguities in Motion Estimation

To solve the undetermined component problem

( ) of OFE, one must impose additional constraints.

The most common constraints is that the flow

vectors should vary smoothly spatially (to estimate the motion vector).

t

v

?

• k

15 Kasaei

General Considerations for ME

Two categories of approaches:

Feature-based: More often used in object tracking & 3-D

reconstruction from 2-D (least-squares fitting of features, good for global motions).

Intensity-based: Based on CIA (no simple model). More

• ften used for motion compensated prediction (required in

video coding), frame interpolation Our focus.

Three important questions:

How to represent (parameterize) the motion field? What criteria to use to estimate motion parameters? How to search for optimal motion parameters?

16 Kasaei

Motion Representation

Global: Entire motion field is represented by a few global parameters (global motion representation; camera motion). Pixel-based: One MV at each pixel, with some smoothness constraint between adjacent MVs (very time consuming). Region-based: Entire frame is divided into regions, then each region corresponding to an object (or sub-

• bject) with consistent

motion, is represented by a few parameters (requires region segmentation map, which pels have similar motions?). Block-based: Entire frame is divided into non-overlapping blocks, then motion in each block is characterized by a few parameters (good compromise between accuracy & complexity, discontinuous across blocks, no multiple

• bjects, scale, or

rotation).

17 Kasaei

Motion Representation

Other representation: mesh-based

representation.

Underlying image frame is partitioned into nonoverlapping

polygonal elements.

Mvs at the corners of polygonal elements determine the

entire motion field.

Mvs at the interior points of an element are interpolated

from the nodal MVs.

Induces a motion field that is continuous everywhere. Adaptive methods allow discontinuities when necessary

(on object boundaries).

18 Kasaei

Notations

Anchor frame: Target frame: Motion parameters: Motion vector at a pixel in the anchor frame: Motion field: Mapping function: ) (

1 x

ψ ) (

2 x

ψ ) (x d Λ ∈ x a x d ), ; ( a Λ ∈ + = x a x d x a x w ), ; ( ) ; (

reference frame in video coding current frame in video coding

19 Kasaei

Regularization Theory

• Ill-posed problems.
• Regularization methods.
• Stochastic regularization methods.
• Relaxation labeling.

Discrete relaxation labeling. Stochastic relaxation labeling.

20 Kasaei

Well-Posed Problems

• A mathematical problem is well-posed

when its solution:

1.

Exists.

2.

Is unique.

3.

Is robust to noise.

• Physical simulation problems are well-

posed, but “inverse” problems are usually ill-posed.

21 Kasaei

Regularization Methods

• Basis idea behind regularization is:

1.

to restrict the space of acceptable solutions, and

2.

by choosing the function that minimizes an appropriate functional.

• For solving an ill-posed problem, regularization

theory provides the mathematical function for choosing the norm and stabilizing functional that together characterize the global constraints for the problem.

• i.e., finding x that satisfies: ||G(x)||<C & min ||F(x)-y||.

22 Kasaei

Stochastic Regularization Methods

Instead of regularization theory, a Bayesian formulation

can be used to transform ill-posed inverse problems into the functional optimization framework.

Looking for the most likely model given a set of data.

Likelihood: evaluates how well the model describes the data

(stabilizing functional ||F(x)-y||).

A priori: evaluates the model (norm ||G(x)||).

Other modeling approaches:

Minimum description length also describe the model constraints.

Other stochastic or probabilistic optimization methods:

Simulated annealing, Genetic algorithms/ evolutionary strategy, Expectation-maximization.

23 Kasaei

Consistence Labeling

• How to infer smooth features, detect discontinuities,

and identify outliers.

• As each location only assumes one of these roles, we need a

consistence labeling framework.

• A labeling problem is characterized by:

1.

a set of objects (pixels),

2.

a set of possible labels (edges with orientations, discontinuities, gray-levels, regions, line matches) for each

• bject,

3.

a neighbor relation over objects, and

4.

a compatibility relation over labels at pairs of neighboring

• bjects.
• The goal is to assign a label to each object such that the labeling

is consistent with respect to the compatibility relation (4).

24 Kasaei

Relaxation Labeling

• A natural extension of regularization operation

to the class of problem whose solution involves symbols rather than functions.

• Structure of relaxation labeling is motivated by

two basis concerns:

1.

Decomposition of the complex computation into a network of simple “myopic” or local computations, and

2.

Requisite use of context in resolving ambiguities.

25 Kasaei

Relaxation Labeling

Three main types of relaxation labeling

methods:

Discrete relaxation labeling. Continuous relaxation labeling. Stochastic relaxation labeling.

26 Kasaei

Discrete Relaxation Labeling

Assigns labels to graph nodes. Is governed by the label discarding rule:

Discard a label at a node if there exists a neighbor

such that every label currently assigned to the neighbor is incompatible with the label.

Discard process is applied iteratively. Applied in parallel at each node, until one or more

limiting label sets are obtained.

Main issues in iterated process:

initialization, updating, and stopping condition.

27 Kasaei

Stochastic Relaxation Labeling

Labeling weights and constraints preference weights are

replaced by probability distributions.

Is based on the use of a stochastic modeling of physical

phenomenon called Markov random fields (MRF).

MRF is often combined with the Bayesian estimation

techniques known as maximum a posteriori (MAP), forming MRF-MAP.

It involves solving an energy minimization problem. Typically, one uses a global minimum seeking algorithms such

as simulated annealing, evolutionary algorithms, or expectation- maximization to minimize the often nonconvex energy functions.

28 Kasaei

Motion Estimation Criteria

To minimize the displaced frame difference (DFD): To satisfy the optical flow equation:

MSE : 2 MAD; : 1 min ) ( )) ; ( ( ) (

1 2 DFD

= = → − + = ∑

Λ ∈

P p E

x p

x a x d x a ψ ψ

( )

min ) ( ) ( ) ; ( ) ( ) (

1 2 1 OF

→ − + ∇ = ∑

Λ ∈ x p T

E x x a x d x a ψ ψ ψ

29 Kasaei

Motion Estimation Criteria

To impose additional smoothness constraint using

regularization technique (important in pixel- & block- based representation):

Lower penalty weights at object boundaries.

Bayesian (MAP) criterion: to maximize the a posteriori

probability:

max ) , (

1 2

→ = ψ ψ d D P

min ) ( ) ( ) ; ( ) ; ( ) (

DFD 2

→ + − = ∑ ∑

Λ ∈ ∈

a a a y d a x d a

x y s s DFD N s

E w E w E

x

smoothness constraint

30 Kasaei

Relation Among Different Criteria

OF criterion is good only if motion is small. OF criterion can often yield closed-form solution as the

• bjective function is quadratic in MVs.

When the motion is not small, one can iterate the

solution based on OF criterion to satisfy DFD criterion.

Bayesian criterion can be reduced to DFD criterion plus

motion smoothness constraint.

More in the textbook.

[DFD: displaced frame difference]

31 Kasaei

Optimization Methods

Exhaustive search:

Typically used for the DFD criterion with p=1 (MAD). Guarantees reaching the global optimal. Required computation may be unacceptable when number

• f parameters to search simultaneously is large!

Fast search algorithms reach sub-optimal solution in a

shorter time.

32 Kasaei

Optimization Methods

Gradient-based search:

Typically used for the DFD or OF criterion with p=2

(MSE).

The gradient can often be calculated analytically. When used with the OF criterion, closed-form solution may be

• btained.

Reaches the local optimal point closest to the initial

solution.

Multi-resolution search:

Searches from coarse-to-fine resolution. Is faster than exhaustive search. Avoids being trapped into a local minimum.

33 Kasaei

Gradient Descent Method

Iteratively updates the current estimate in the

direction opposite to the gradient direction.

Not a good initial. A good initial. Appropriate stepsize. Too big stepsize.

34 Kasaei

Gradient Descent Method

The solution depends on the initial condition.

Reaches the local minimum closest to the initial condition.

You can start with several different initial solutions.

Choice of stepsize:

Fixed stepsize: Stepsize must be small to avoid

• scillation (requires many iterations).

Steepest gradient descent: A 1st order gradient decent

method that uses a variable stepsize (adjusts the stepsize optimally).

Converges in few iterations, but with more computations.

35 Kasaei

Newton’s Method

Newton’s method uses the first- & second-order

derivatives:

Hessian matrix

36 Kasaei

Newton’s Method

Converges faster than the 1st order method (i.e., requires

fewer number of iterations to reach convergence).

Requires more calculations in each iteration. More prone to noise (gradient calculation is subject to noise

more with 2nd order than with 1st order).

Uses a constant stepsize (a) smaller that 1.

May not converge, if a >=1.

Should choose the stepsize appropriate to reach a good

compromise between guaranteeing convergence & convergence rate.

37 Kasaei

Newton-Raphson Method

Newton-Ralphson method:

Approximates 2nd order gradient by a product of 1st order

gradients.

Applicable when the objective function is a sum of squared

errors.

Only needs to calculate 1st order gradients, yet converges at

a rate similar to Newton’s method.

38 Kasaei

Newton-Raphson Method

If:

1st order gradients

39 Kasaei

Pixel-Based Motion Estimation

Horn-Schunck method:

OF + smoothness criterion.

Multipoint neighborhood method:

Assumes that every pixel in a small block surrounding a

pixel has the same MV.

Pel-recurrsive method:

MV for a current pel is updated from those of its previous

pels, so that the MV does not need to be coded.

Developed for early generation of video coders.

40 Kasaei

Multipoint Neighborhood Method

Estimates the MV at each pixel independently, by

minimizing the DFD error over a neighborhood surrounding this pixel.

Every pixel in the neighborhood is assumed to

have the same MV.

Minimizing (cost) function:

min ) ( ) ( ) ( ) (

) ( 2 1 2 n DFD

→ − + = ∑

∈

n

B n

w E

x x

x d x x d ψ ψ

41 Kasaei

Multipoint Neighborhood Method

Optimization method:

Exhaustive search (feasible as one only needs to search

• ne MV at a time).

Needs to select the appropriate search range & the search step-

size.

Gradient-based method.

42 Kasaei

Example: Gradient Descent Method

[ ]

[ ]

) ( ) ( : method Raphson

• Newton

) ( : descent gradient

• rder

First ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( min ) ( ) ( ) ( ) (

) ( n 1 ) ( n ) ( n ) 1 ( n ) ( n ) ( n ) 1 ( n 2 2 ) ( 2 2 2 2 2 ) ( 2 n 2 n 2 ) ( n n ) ( 2 1 2 n DFD l l l l l l l T B n T B B n B n

n n n n n n n n

w e w w E e w E w E d g d H d d d g d d x x x x d x x x x x d d H x d x x d d g x d x x d

d x x x d x d x x x d x x x x x − + + + ∈ + + ∈ + ∈ ∈

− = − =       ∂ ∂ ∂ ∂ ≈ ∂ ∂ + +       ∂ ∂ ∂ ∂ = ∂ ∂ = ∂ ∂ + = ∂ ∂ = → − + =

∑ ∑ ∑ ∑

α α ψ ψ ψ ψ ψ ψ ψ ψ

43 Kasaei

Simplification using OF Criterion

( ) ( )

( )

( ) ( )

        ∇ −         ∇ ∇ = = ∇ − + ∇ = ∂ ∂ → − + ∇ =

∑ ∑ ∑ ∑

∈ − ∈ ∈ ∈ ) ( 1 2 1 1 ) ( 1 1

• pt

n, 1 ) ( 1 2 1 n ) ( 2 1 2 1 n OF

) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( min ) ( ) ( ) ( ) ( ) (

n n n n

B B T B n T B n T

w w w E w E

x x x x x x x x

x x x x x x x d x x x d x x d x x d x x d ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ

This solution is good only if the actual MV is small. When this is not the case, one should iterate the above solution, with the following update: iteration at that found MV the denote ) ( ) (

) 1 ( ) 1 ( ) ( n ) 1 ( n ) ( n 2 ) 1 ( 2 + + + +

∆ ∆ + = + =

l n l n l l l l

where d d d x x ψ ψ

44 Kasaei

Block-Based Motion Estimation (A Brief Overview)

Assumes that all pixels in a block undergo a coherent

motion & searches for the motion parameters for each block independently.

Block matching algorithm (BMA): assumes a

translational motion, 1 MV per block (2 parameters):

Exhaustive BMA (EBMA). Fast algorithms.

Deformable block matching algorithm (DBMA): allows

more complex motion (affine, bilinear); to be discussed later.

45 Kasaei

Block-Based Motion Estimation (A Brief Overview)

46 Kasaei

Block Matching Algorithm

Overview:

Assumes that all pixels in a block undergo a translation,

denoted by a single MV.

Estimate the MV for each block independently, by

minimizing the DFD error over this block.

Results in non-smooth MVs, but better handles the object

boundaries, new appearing objects, & occlusion problem.

Minimizing function:

min ) ( ) ( ) (

1 2 m DFD

→ − + = ∑

∈

m

B p m

E

x

x d x d ψ ψ

47 Kasaei

Block Matching Algorithm

Optimization method:

Exhaustive search (feasible as one only needs to search one

MV at a time), using MAD criterion (p=1).

Fast search algorithms. Integer- vs. fractional-pel accuracy search.

48 Kasaei

Exhaustive Block Matching Algorithm (EBMA)

49 Kasaei

Complexity of Integer-Pel EBMA

Assumption:

Image size: MxM. Block size: NxN. Search range: (-R, R) in each dimension. Search stepsize: 1 pixel (assuming integer MV).

Operation counts (1 operation=1 “-”, 1 “+”, 1 “*”):

Each candidate position: N^2. Each block going through all candidates: (2R+1)^2 N^2. Entire frame: (M/N)^2 (2R+1)^2 N^2=M^2 (2R+1)^2.

Independent of block size!

50 Kasaei

Complexity of Integer-Pel EBMA

Example: M=512, N=16, R=16, 30 fps.

Total operation count = 2.85x10^8/frame

=8.55x10^9/second.

Regular structure suitable for VLSI implementation. Challenging for software-only implementation.

51 Kasaei

Sample Matlab Script for Integer-Pel EBMA

%f1: anchor frame; f2: target frame, fp: predicted image; %mvx,mvy: store the MV image %widthxheight: image size; N: block size, R: search range for i=1:N:height-N, for j=1:N:width-N %for every block in the anchor frame MAD_min=256*N*N;mvx=0;mvy=0; for k=-R:1:R, for l=-R:1:R %for every search candidate MAD=sum(sum(abs(f1(i:i+N-1,j:j+N-1)-f2(i+k:i+k+N-1,j+l:j+l+N-1)))); % calculate MAD for this candidate if MAD<MAX_min MAD_min=MAD,dy=k,dx=l; end; end;end; fp(i:i+N-1,j:j+N-1)= f2(i+dy:i+dy+N-1,j+dx:j+dx+N-1); %put the best matching block in the predicted image iblk=(floor)(i-1)/N+1; jblk=(floor)(j-1)/N+1; %block index mvx(iblk,jblk)=dx; mvy(iblk,jblk)=dy; %record the estimated MV end;end;

Note: A real working program needs to check whether a pixel in the candidate matching block falls

• utside the image boundary and such pixel should not count in MAD. This program is meant to

illustrate the main operations involved. Not the actual working Matlab script.

52 Kasaei

Fractional Accuracy EBMA

Real MV may not always be multiples of pixels. To

allow sub-pixel MV, the search stepsize must be less than 1 pixel.

Half-pel EBMA: stepsize=1/2 pixel in both dimensions. Difficulty:

Target frame only has integer pels.

Solution:

Interpolate the target frame by a factor of two before searching. Bilinear interpolation is typically used.

53 Kasaei

Fractional Accuracy EBMA

Complexity:

4-times of integer-pel, plus additional operations for

interpolation.

Fast algorithms:

Searches in integer precisions first, then refines in a small

search region in half-pel accuracy.

54 Kasaei

Half-Pel Accuracy EBMA

55 Kasaei

Bilinear Interpolation

(x+1, y) (x, y) (x+1, y+1) (x, y+1) (2x, 2y) (2x+1, 2y) (2x, 2y+1) (2x+1, 2y+1)

O[2x, 2y]=I[x, y] O[2x+1, 2y]=(I[x, y]+I[x+1, y])/2 O[2x, 2y+1]=(I[x, y]+I[x, y+1])/2 O[2x+1, 2y+1]=(I[x, y]+I[x+1, y]+I[x, y+1]+I[x+1, y+1])/4

56 Kasaei

Predicted Anchor Frame (29.86 dB) Anchor Frame Target Frame Motion Field Example: Half-pel EBMA.

57 Kasaei

Pros & Cons with EBMA

Blocking artifacts (discontinuity across block

boundary) in the predicted image:

Because the block-wise translation model is not accurate. Fix: Deformable BMA (next lecture).

Motion field somewhat chaotic:

Because MVs are estimated independently from block to

block.

Fix 1: Mesh-based motion estimation (next lecture). Fix 2: Imposing smoothness constraint explicitly.

58 Kasaei

Pros & Cons with EBMA

Wrong MV in flat regions:

Because motion is indeterminate when spatial gradient is

near zero.

Nonetheless, widely used for motion compensated

prediction in video coding.

Because of its simplicity & optimality in minimizing

prediction error.

59 Kasaei

Fast Algorithms for BMA

Key idea to reduce the computation in EBMA:

Reduce the number of search candidates:

Only search for those that are likely to produce small errors. Predict possible remaining candidates, based on previous search

results.

Simplify the error measure (DFD) to reduce the

computation involved for each candidate.

Classical fast algorithms:

Three-step. 2-D log. Conjugate direction.

60 Kasaei

Fast Algorithms for BMA

Many new fast algorithms have been developed since

then.

Some suitable for software implementation, others for

VLSI implementation (memory access, etc).

61 Kasaei

2-D Log Search

Best matching MVs in steps 1-5 are: (0,2), (0,4), (2,4), (2,6), & (2,6).

final match

• Each step tests 5 diamond search

points.

• Initial stepsize is half of the search

range.

• Search stepsize reduces if the best

matching point is:

• the center point, or
• on the border of the max

search range.

• Final step is reached when:
• stepsize is reduced to 1 pel, &
• 9 search points are examined

at this last step.

• No. of steps cannot be determined.

62 Kasaei

Three-Step Search Algorithm

Best matching MVs in steps 1–3 are: (3,3), (3,5), & (2,6).

final match decreasing steps

• Each step tests 8 search points.
• At first, it tests 9 search points.
• Initial stepsize is half of the search

range.

• Search stepsize reduces by half

after each step.

• Final step is reached when:
• stepsize is reduced to 1 pel, &
• 8 search points are examined

at this last step.

• No. of steps is (8L+1).
• Proper for VLSI implementation.

63 Kasaei

Three-Step Search Algorithm

final match decreasing steps

64 Kasaei

VcDemo Example

VcDemo: Image & Video Compression Learning Tool Developed at Delft University of Technology: http://www-ict.its.tudelft.nl/~inald/vcdemo/ Use the ME tool to show the motion estimation results with different parameter choices.

65 Kasaei

Summary

Optical flow equation:

Derived from constant intensity & small motion

assumptions.

Ambiguity in motion estimation.

How to represent motion:

Pixel-based, block-based, region-based, mesh-based,

global, etc.

Estimation criterion:

DFD (constant intensity). OF (constant intensity+small motion). Bayesian (MAP, DFD+motion smoothness).

66 Kasaei

Summary

Search method:

Exhaustive search, gradient-descent, multi-resolution (next

lecture).

Basic motion estimation techniques:

Pixel-based:

Most accurate representation, but also most costly to estimate.

Block-based:

EBMA, integer-pel vs. half-pel accuracy, fast algorithms Good trade-off between accuracy & speed. EBMA (and its fast but suboptimal variant) is widely used in video

coding for motion-compensated temporal prediction.

67 Kasaei

Homework 4

Reading assignment:

Chap 6: Sec. 6.1-6.4 (Sec. 6.4.5,6.4.6 not required), & Apx.

A & B.

Written assignment:

• Prob. 6.4, 6.5, 6.6

68 Kasaei

Homework 4

Computer assignment:

• Prob. 6.12, 6.13

Optional: Prob.6.14 Note: you can download sample video frames from the

course webpage. When applying your motion estimation algorithm, you should choose two frames that have sufficient motion in between so that it is easy to observe effect of motion estimation inaccuracy. If necessary, choose two frames that are several frames apart. For example, foreman: frame 100 & frame 103.

The End