Geometric Transformations, RANSAC and Morphing
CS448V — Computational Video Manipulation April 2019
Geometric Transformations, RANSAC and Morphing CS448V Computational - - PowerPoint PPT Presentation
Geometric Transformations, RANSAC and Morphing CS448V Computational Video Manipulation April 2019 In previous lecture In previous lecture Feature detection In previous lecture Feature detection [0.1, 0.6, ...] Feature description
CS448V — Computational Video Manipulation April 2019
Feature detection
Feature detection Feature description
[0.1, 0.6, ...] [0.0, 0.01, ...] [0.54, -0.3, ...]
Feature detection Feature description Feature matching
[0.1, 0.6, ...] [0.0, 0.01, ...] [0.54, -0.3, ...]
Feature detection Feature description Feature matching
[0.1, 0.6, ...] [0.0, 0.01, ...] [0.54, -0.3, ...]
ratio distance = SSD(f1, f2) SSD(f1, f′
2)
f1 f2 f2'
http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html
Relation between photos
Image from OpenCV documentation
Relation between photos
Image from OpenCV documentation
Warping
Relation between photos
Image from OpenCV documentation
(last lecture)
Relation between photos
Image from OpenCV documentation
(last lecture)
Relation between photos
Image from OpenCV documentation
(last lecture)
Relation between photos
Image from OpenCV documentation
RANSAC
(last lecture)
Relation between photos
Image from OpenCV documentation
homography RANSAC
RANdom SAmple Consensus
RANdom SAmple Consensus
RANdom SAmple Consensus
cs131
RANdom SAmple Consensus
RANdom SAmple Consensus
RANdom SAmple Consensus
RANdom SAmple Consensus
RANdom SAmple Consensus
RANdom SAmple Consensus
RANdom SAmple Consensus
RANdom SAmple Consensus
RANdom SAmple Consensus
RANdom SAmple Consensus
RANdom SAmple Consensus
RANdom SAmple Consensus
2 (tx, ty)
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
tx = ? ty = ?
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
tx = ? ty = ?
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
RANdom SAmple Consensus
2 (tx, ty) 1 pair
Use best result
RANdom SAmple Consensus
How many iterations?
RANdom SAmple Consensus
How many iterations?
p probability that a given data point is valid (an inlier)
RANdom SAmple Consensus
How many iterations?
p probability that a given data point is valid (an inlier) n amount of data points that define a transformation
RANdom SAmple Consensus
How many iterations?
p probability that a given data point is valid (an inlier) n amount of data points that define a transformation M number of iterations
RANdom SAmple Consensus
How many iterations?
p probability that a given data point is valid (an inlier) n amount of data points that define a transformation M number of iterations S probability of success after M iterations
RANdom SAmple Consensus
How many iterations?
Probability of a successful iteration (all points are inliers)
p probability that a given data point is valid (an inlier) n amount of data points that define a transformation M number of iterations S probability of success after M iterations
RANdom SAmple Consensus
How many iterations?
Probability of a successful iteration (all points are inliers) pn
p probability that a given data point is valid (an inlier) n amount of data points that define a transformation M number of iterations S probability of success after M iterations
RANdom SAmple Consensus
How many iterations?
Probability of a successful iteration (all points are inliers) pn Probability of a failed iteration
p probability that a given data point is valid (an inlier) n amount of data points that define a transformation M number of iterations S probability of success after M iterations
RANdom SAmple Consensus
How many iterations?
Probability of a successful iteration (all points are inliers) pn Probability of a failed iteration 1 − pn
p probability that a given data point is valid (an inlier) n amount of data points that define a transformation M number of iterations S probability of success after M iterations
RANdom SAmple Consensus
How many iterations?
Probability of a successful iteration (all points are inliers) pn Probability of a failed iteration 1 − pn Probability of all M iterations to fail
p probability that a given data point is valid (an inlier) n amount of data points that define a transformation M number of iterations S probability of success after M iterations
RANdom SAmple Consensus
How many iterations?
Probability of a successful iteration (all points are inliers) pn Probability of a failed iteration 1 − pn Probability of all M iterations to fail (1 − pn)M
p probability that a given data point is valid (an inlier) n amount of data points that define a transformation M number of iterations S probability of success after M iterations
RANdom SAmple Consensus
How many iterations?
Probability of a successful iteration (all points are inliers) pn Probability of a failed iteration 1 − pn Probability of all M iterations to fail (1 − pn)M Probability of success after M iterations
p probability that a given data point is valid (an inlier) n amount of data points that define a transformation M number of iterations S probability of success after M iterations
RANdom SAmple Consensus
How many iterations?
Probability of a successful iteration (all points are inliers) pn Probability of a failed iteration 1 − pn Probability of all M iterations to fail (1 − pn)M Probability of success after M iterations S = 1 − (1 − pn)M
p probability that a given data point is valid (an inlier) n amount of data points that define a transformation M number of iterations S probability of success after M iterations
RANdom SAmple Consensus
How many iterations?
Probability of a successful iteration (all points are inliers) pn Probability of a failed iteration 1 − pn Probability of all M iterations to fail (1 − pn)M Probability of success after M iterations S = 1 − (1 − pn)M
p probability that a given data point is valid (an inlier) n amount of data points that define a transformation M number of iterations S probability of success after M iterations
M = log(1 − S) log(1 − pn)
10% inliers (p = 0.1)
Number of iterations M 22.5 45 67.5 90 Probability of success S 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.99 0.999 0.9999
n = 1
M = log(1 − S) log(1 − pn)
10% inliers (p = 0.1)
Number of iterations M 2500 5000 7500 10000 Probability of success S 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.99 0.999 0.9999
n = 1 n = 3
M = log(1 − S) log(1 − pn)
10% inliers (p = 0.1)
Number of iterations M 250000 500000 750000 1000000 Probability of success S 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.99 0.999 0.9999
n = 1 n = 3 n = 5
M = log(1 − S) log(1 − pn)
10% inliers (p = 0.1)
Number of iterations M 1 1000 1000000 Probability of success S 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.99 0.999 0.9999
n = 1 n = 3 n = 5
M = log(1 − S) log(1 − pn)
10% inliers (p = 0.1)
Number of iterations M 1 1000 1000000 Probability of success S 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.99 0.999 0.9999
n = 1 n = 3 n = 5
M = log(1 − S) log(1 − pn)
44 4603 460,514
Cartesian coordinates
Cartesian coordinates Point at infinity
Cartesian coordinates Point at infinity Omitted
Translation as matrix multiplication
1 0 tx 1 ty 1
Translation
cos θ −sin θ tx sin θ cos θ ty 1
Translation + rotation = Rigid, Euclidean
Translation + rotation + uniform scale = Similarity
s cos θ −s sin θ tx s sin θ s cos θ ty 1
Translation + rotation + scale + shear = Affine
a b tx c d ty 1
Projective, Perspective, Homography
a b tx c d ty e f 1
Transformation Preserves Degrees of freedom Translation Rigid Similarity Affine Projective Transformation Preserves Degrees of freedom
Projective Straight lines Transformation Preserves Degrees of freedom Translation Rigid Similarity Affine Projective Transformation Preserves Degrees of freedom
Projective Straight lines Affine Parallelism Transformation Preserves Degrees of freedom Translation Rigid Similarity Affine Projective Transformation Preserves Degrees of freedom
Projective Straight lines Affine Parallelism Similarity Angles Transformation Preserves Degrees of freedom Translation Rigid Similarity Affine Projective Transformation Preserves Degrees of freedom
Projective Straight lines Affine Parallelism Similarity Angles Rigid Length Transformation Preserves Degrees of freedom Translation Rigid Similarity Affine Projective Transformation Preserves Degrees of freedom
Projective Straight lines Affine Parallelism Similarity Angles Rigid Length Translation Orientation Transformation Preserves Degrees of freedom Translation Rigid Similarity Affine Projective Transformation Preserves Degrees of freedom
Projective Straight lines Affine Parallelism Similarity Angles Rigid Length Translation Orientation Transformation Preserves Degrees of freedom Translation Rigid Similarity Affine Projective Transformation Preserves Degrees of freedom Translation 2
Projective Straight lines Affine Parallelism Similarity Angles Rigid Length Translation Orientation Transformation Preserves Degrees of freedom Translation Rigid Similarity Affine Projective Transformation Preserves Degrees of freedom Translation 2 Rigid 3
Projective Straight lines Affine Parallelism Similarity Angles Rigid Length Translation Orientation Transformation Preserves Degrees of freedom Translation Rigid Similarity Affine Projective Transformation Preserves Degrees of freedom Translation 2 Rigid 3 Similarity 4
Projective Straight lines Affine Parallelism Similarity Angles Rigid Length Translation Orientation Transformation Preserves Degrees of freedom Translation Rigid Similarity Affine Projective Transformation Preserves Degrees of freedom Translation 2 Rigid 3 Similarity 4 Affine 6
Projective Straight lines Affine Parallelism Similarity Angles Rigid Length Translation Orientation Transformation Preserves Degrees of freedom Translation Rigid Similarity Affine Projective Transformation Preserves Degrees of freedom Translation 2 Rigid 3 Similarity 4 Affine 6 Projective 8
connect between 3D scenes viewed by a rotating camera
[Szeliski & Shum ’97]
moving camera, plane connect between planes seen by different cameras
Image from OpenCV documentation
connect between 3D scenes viewed by a rotating camera
[Szeliski & Shum ’97]
moving camera, plane connect between planes seen by different cameras
Image from OpenCV documentation
connect between 3D scenes viewed by a rotating camera
[Szeliski & Shum ’97]
We’ll revisit these in a later class (structure from motion, scene building)
Polynomial
[ ̂ x ̂ y] = [ a0 a1 a2 a3 a4 a5 b0 b1 b2 b3 b4 b5] 1 x y xy x2 y2
Quadratic Or higher orders
Radial Polynomial
[ ̂ x ̂ y] = [ a0 a1 a2 a3 a4 a5 b0 b1 b2 b3 b4 b5] 1 x y xy x2 y2
Quadratic Or higher orders
Radial Polynomial
[ ̂ x ̂ y] = [ a0 a1 a2 a3 a4 a5 b0 b1 b2 b3 b4 b5] 1 x y xy x2 y2
Quadratic Or higher orders
Deformation fields
Destination Image Source Image
Q Q′ X X′ P P′ u u v v
weight = ( line_lenp a + pixel_dist )
b
Forward sampling Reverse sampling
“Where should this pixel go?” “Where does this pixel come from?”
Forward sampling Reverse sampling
“Where should this pixel go?” “Where does this pixel come from?”
Advantage of reverse sampling?
What about in-between landmarks?
Sparse vector field
Dense vector field Sparse vector field
scipy.interpolate.griddata
scipy.interpolate.griddata
barycentric coordinates
p1 p2 p3 p
p = λ1p1 + λ2p2 + λ3p3
λ1 + λ2 + λ3 = 1 λi ≥ 0
scipy.interpolate.griddata
interpolate vectors barycentric coordinates
p1 p2 p3 p
p = λ1p1 + λ2p2 + λ3p3
λ1 + λ2 + λ3 = 1 λi ≥ 0
scipy.interpolate.griddata
interpolate vectors barycentric coordinates
p1 p2 p3 p
p = λ1p1 + λ2p2 + λ3p3
λ1 + λ2 + λ3 = 1 λi ≥ 0
scipy.interpolate.griddata
interpolate vectors barycentric coordinates
p1 p2 p3 p
p = λ1p1 + λ2p2 + λ3p3
λ1 + λ2 + λ3 = 1 λi ≥ 0
triangulate & interpolate
Dense vector field Sparse vector field
Dense vector field Sparse vector field Can be linear, cubic, …