Poisson Image Editing and opaque source image regions into a - - PDF document

SLIDE 1

1

Poisson Image Editing

Patrick Perez, Michel Gangnet and Andrew Blake Microsoft Research UK

Goals

Seamlessly importing (cloning) transparent

and opaque source image regions into a destination image.

Seamless modification of appearance of the

image within a selected region.

Input Images

source image target image

Simple Cloning Result

Poisson Seamless Cloning Result

Some More Results

source images target image

SLIDE 2

2 Some More Results

source images simple cloning

Some More Results

source images Poisson seamless cloning

Interpolation

• S: a closed subset of R2
• Ω: a closed subset of S, with boundary ∂Ω
• f* : known scalar function over S\Ω
• f: unknown scalar function over Ω

Membrane Interpolant

Solve the following minimization problem: subject to Dirichlet boundary conditions:

2

min∫∫

Ω

∇f

f Ω ∂ Ω ∂ = *

f f

Euler-Lagrange Equation

A fundamental equation of calculus of variations,

which states that if J is defined by an integral of the form:

then J has a stationary value if the following

differential equation is satisfied:

dx f f x F J

x)

, , (

∫

=

= ∂ ∂ − ∂ ∂

x

f F dx d f F

Euler-Lagrange (continued)

In our case: The equation is:

• and

So we get:

= ∂ ∂ − ∂ ∂ − ∂ ∂

y x

f F dy d f F dx d f F

( )

2 2 2 y x

f f f F + = ∇ =

= ∂ ∂ f F

2 2

2 2 x f f dx d f F dx d

x x

∂ ∂ = = ∂ ∂

2 2 2 2

= Δ = ∂ ∂ + ∂ ∂ f y f x f

SLIDE 3

3

Smooth Image Completion

Euler-Lagrange:

Ω ∂ Ω ∂ Ω

= ∇

∫∫

* 2

. .

min arg

f f t s f

f

Ω ∂ Ω ∂ =

Ω = Δ

*

. . f f t s

• ver

f

The minimum is achieved when:

Discrete Approximation

2 2 2 2

y f x f f ∂ ∂ + ∂ ∂ = Δ

y x y x

f f x f

, , 1 −

≅ ∂ ∂

+

1, , 1, , 1 , , 1 1, 1, , 1 , 1 ,

( , ) 2 2 4

x y x y x y x y x y x y x y x y x y x y x y

f x y f f f f f f f f f f f

+ − + − + − + −

Δ ≅ − + + − + = = + + + + − =

Ω ∂ Ω ∂ =

Ω = Δ

*

. . f f t s

• ver

f

y x y x y x

f f f x f

, 1 , , 1 2 2

2

− +

+ − ≅ ∂ ∂

Solving

Each fx,y is an unknown variable xi, total of

N variables (covering the unknown pixels)

Reduces to the sparse algebraic system: N x N = b1 b2 b3 1 1 -4 1 1 1 1 -4 1 1 1 1 -4 1 1 Known values of f() contribute to the left side xi-w+xi-1-4xi+xi+1=-f(x,y+1)

fx,y-1+fx-1,y-4fx,y+fx+1,y+fx,y+1=0 or xi-w+xi-1-4xi+xi+1+xi+w=0

x1 x2 … xN

Wow result

input result ground truth

Guided Interpolation

Laplace equation yields a smooth

interpolant.

Idea: use a guiding vector field v.

Poisson Cloning: “Guiding” the completion

We can guide the completion to fill the hole

using gradients from another source image

Reverse: Seek a function f whose gradients

are closest to the gradients of the source image

SLIDE 4

4 Guided Interpolant

Solve the following minimization problem: subject to Dirichlet boundary conditions: 2

min

f

f G

Ω

∇ −

∫∫

Ω ∂ Ω ∂ = *

f f

Poisson Equation

This time the Euler-Lagrange equation

reduces to the Poisson equation:

written more concisely as: 2 2 2 2 y x

G G f f x y x y ∂ ∂ ∂ ∂ + = + ∂ ∂ ∂ ∂

div f G Δ =

Poisson Cloning

Ω ∂ Ω ∂ Ω

= − ∇

∫∫

* 2

. .

min arg

f f t s G f

f

Ω ∂ Ω ∂ =

Ω = Δ

*

. . f f t s

• ver

G div f

) 1 , ( ) , ( ) , 1 ( ) , ( − − + − − ≅ ∂ + ∂ = y x G y x G y x G y x G dy G dx G G div

y y x x

G source image = ∇

(backward difference)

Denote

Poisson Cloning (Solving)

Each fx,y is a variable xi as before, solving

fx,y-1+fx-1,y-4fx,y+fx+1,y+fx,y+1=divG(x,y) Or xi-w+xi-1-4xi+xi+1+xi+w=divG(x,y)

As before it reduces to a sparse algebraic system

Seamless Cloning

Choose the gradient field of the source

image as the guiding vector field G.

Concealment

SLIDE 5

5 Insertion Feature Exchange Texture Swapping Mixing Gradients

Sometimes it is desirable to define the

guiding field as a mixture of source and target gradient fields.

Option A: linear combination of the gradient

fields

Option B: strongest gradient wins:

⎩ ⎨ ⎧ ∇ ∇ > ∇ ∇ =

• therwise

) ( ) ( ) ( if ) ( ) (

* *

x x x x x v g g f f

SLIDE 6

6 More Mixtures More Mixtures Selection Editing

Texture flattening: use target gradient field

• nly where edges are detected:

) ( ) ( ) (

* x

x x v f M ∇ =

Local Illumination Changes Local Color Changes Seamless Tiling