Convolution Pyramids Zeev Farbman , Raanan Fattal and Dani Lischinski - - PowerPoint PPT Presentation

▶
convolution pyramids
SMART_READER_LITE
LIVE PREVIEW

Convolution Pyramids Zeev Farbman , Raanan Fattal and Dani Lischinski - - PowerPoint PPT Presentation

Jan 13, 2024 350 likes •1.05k views

Convolution Pyramids Zeev Farbman , Raanan Fattal and Dani Lischinski Motivation SIGGRAPH Asia Conference (2011) Convolution Pyramids Application 1 - Gaussian Kernels presented by: Julian Steil Application 2 - Boundary Interpolation

slide-1
SLIDE 1

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Convolution Pyramids

Zeev Farbman, Raanan Fattal and Dani Lischinski

SIGGRAPH Asia Conference (2011) presented by:

Julian Steil

supervisor:

  • Prof. Dr. Joachim Weickert
  • Fig. 1.1: Gradient integration example
  • Fig. 1.2: Reconstruction result of Fig. 1.1

Seminar - Milestones and Advances in Image Analysis

  • Prof. Dr. Joachim Weickert, Oliver Demetz

Mathematical Image Analysis Group Saarland University

13th of November, 2012

slide-2
SLIDE 2

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Overview

  • 1. Motivation
  • 2. Convolution Pyramids
  • 3. Application 1 - Gaussian Kernels
  • 4. Application 2 - Boundary Interpolation
  • 5. Application 3 - Gradient Integration
  • 6. Summary
slide-3
SLIDE 3

Motivation

Convolution Gaussian Pyramid Gaussian Pyramid - Example From Gaussian to Laplacian Pyramid

Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Overview

  • 1. Motivation

Convolution Gaussian Pyramid Gaussian Pyramid - Example From Gaussian to Laplacian Pyramid

  • 2. Convolution Pyramids
  • 3. Application 1 - Gaussian Kernels
  • 4. Application 2 - Boundary Interpolation
  • 5. Application 3 - Gradient Integration
  • 6. Summary

2 / 22

slide-4
SLIDE 4

Motivation

Convolution Gaussian Pyramid Gaussian Pyramid - Example From Gaussian to Laplacian Pyramid

Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Motivation

Convolution

Two-Dimensional Convolution:

  • discrete convolution of two images

g = (gi,j)i,j∈Z and w = (wi,j)i,j∈Z : (g ∗ w)i,j :=

  • k∈Z
  • ℓ∈Z

gi−k,j−ℓwk,ℓ (1)

  • components of convolution kernel w can be regarded as mirrored

weights for averaging the components of g

  • the larger the kernel size the larger the runtime
  • ordinary convolution implementation needs O(n2)

3 / 22

slide-5
SLIDE 5

Motivation

Convolution Gaussian Pyramid Gaussian Pyramid - Example From Gaussian to Laplacian Pyramid

Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Motivation

Gaussian Pyramid

  • sequence of images g0, g1, ..., gn
  • computed by a filtering procedure equivalent to convolution with a

local, symmetric weighting function = ⇒ e.g. a Gaussian kernel Procedure:

  • image initialised by array g0 which contains C columns and R rows
  • each pixel represents the light intensity I between 0 and 255

= ⇒ g0 is the zero level of Gaussian Pyramid

  • each pixel value in level i is computed as a weighting average of

level i − 1 pixel values

  • Fig. 2: One-dimensional graphic representation of the Gaussian pyramid

4 / 22

slide-6
SLIDE 6

Motivation

Convolution Gaussian Pyramid Gaussian Pyramid - Example From Gaussian to Laplacian Pyramid

Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Motivation

Gaussian Pyramid - Example

  • Fig. 3: First six levels of the Gaussian pyramid for the “Lena” image. The original image, level 0, measures 257x257 pixels =

⇒ level 5 measures just 9x9 pixels

Remark: density of pixels is reduced by half in one dimension and by fourth in two dimensions from level to level

5 / 22

slide-7
SLIDE 7

Motivation

Convolution Gaussian Pyramid Gaussian Pyramid - Example From Gaussian to Laplacian Pyramid

Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Motivation

From Gaussian to Laplacian Pyramid

  • Fig. 4: First four levels of the Gaussian and Laplacian pyramid of Fig.3.
  • each level of Laplacian pyramid is the difference between the

corresponding and the next higher level of the Gaussian pyramid

  • full expansion is used in Fig. 4 to help visualise the contents the

pyramid images

6 / 22

slide-8
SLIDE 8

Motivation Convolution Pyramids

Approach Forward and Backward Transform Flow Chart and Pseudocode Optimisation

Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Overview

  • 1. Motivation
  • 2. Convolution Pyramids

Approach Forward and Backward Transform Flow Chart and Pseudocode Optimisation

  • 3. Application 1 - Gaussian Kernels
  • 4. Application 2 - Boundary Interpolation
  • 5. Application 3 - Gradient Integration
  • 6. Summary

6 / 22

slide-9
SLIDE 9

Motivation Convolution Pyramids

Approach Forward and Backward Transform Flow Chart and Pseudocode Optimisation

Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Convolution Pyramids

Approach

Task:

  • approximate effect of convolution with large kernels

= ⇒ higher spectral accuracy + translation-invariant operation

  • Is it also possible in O(n)?

Idea:

  • use of repeated convolution with small kernels on multiple scales
  • disadvantage: not translation-invariant due to subsampling
  • peration to reach O(n) performance

Method:

  • pyramids rely on a spectral “divide-and-conquer” strategy
  • no subsampling of the decomposed signal increases the

translation-invariance

  • use finite impulse response filters to achieve some spacial

localisation and runtime O(n)

7 / 22

slide-10
SLIDE 10

Motivation Convolution Pyramids

Approach Forward and Backward Transform Flow Chart and Pseudocode Optimisation

Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Convolution Pyramids

Approach

Task:

  • approximate effect of convolution with large kernels

= ⇒ higher spectral accuracy + translation-invariant operation

  • Is it also possible in O(n)?

Idea:

  • use of repeated convolution with small kernels on multiple scales
  • disadvantage: not translation-invariant due to subsampling
  • peration to reach O(n) performance

Method:

  • pyramids rely on a spectral “divide-and-conquer” strategy
  • no subsampling of the decomposed signal increases the

translation-invariance

  • use finite impulse response filters to achieve some spacial

localisation and runtime O(n)

7 / 22

slide-11
SLIDE 11

Motivation Convolution Pyramids

Approach Forward and Backward Transform Flow Chart and Pseudocode Optimisation

Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Convolution Pyramids

Approach

Task:

  • approximate effect of convolution with large kernels

= ⇒ higher spectral accuracy + translation-invariant operation

  • Is it also possible in O(n)?

Idea:

  • use of repeated convolution with small kernels on multiple scales
  • disadvantage: not translation-invariant due to subsampling
  • peration to reach O(n) performance

Method:

  • pyramids rely on a spectral “divide-and-conquer” strategy
  • no subsampling of the decomposed signal increases the

translation-invariance

  • use finite impulse response filters to achieve some spacial

localisation and runtime O(n)

7 / 22

slide-12
SLIDE 12

Motivation Convolution Pyramids

Approach Forward and Backward Transform Flow Chart and Pseudocode Optimisation

Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Convolution Pyramids

Forward and Backward Transform

Forward Transform - Analysis Step:

  • convolve a signal with a first filter h1
  • subsample the result by a factor of two
  • process is repeated on the subsampled data
  • an unfiltered and unsampled copy of the signal is kept at each level

al

0 = al

(2) al+1 = ↓ (h1 ∗ al) (3) Backward Transform - Synthesis Step:

  • upsample by inserting a zero between every two samples
  • convolve the result with a second filter h2
  • combine upsampled signal with the signal stored at each level after

convolving with a third filter g ˆ al = h2 ∗ (↑ ˆ al+1) + g ∗ al (4)

8 / 22

slide-13
SLIDE 13

Motivation Convolution Pyramids

Approach Forward and Backward Transform Flow Chart and Pseudocode Optimisation

Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Convolution Pyramids

Forward and Backward Transform

Forward Transform - Analysis Step:

  • convolve a signal with a first filter h1
  • subsample the result by a factor of two
  • process is repeated on the subsampled data
  • an unfiltered and unsampled copy of the signal is kept at each level

al

0 = al

(2) al+1 = ↓ (h1 ∗ al) (3) Backward Transform - Synthesis Step:

  • upsample by inserting a zero between every two samples
  • convolve the result with a second filter h2
  • combine upsampled signal with the signal stored at each level after

convolving with a third filter g ˆ al = h2 ∗ (↑ ˆ al+1) + g ∗ al (4)

8 / 22

slide-14
SLIDE 14

Motivation Convolution Pyramids

Approach Forward and Backward Transform Flow Chart and Pseudocode Optimisation

Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Convolution Pyramids

Flow Chart and Pseudocode

  • Fig. 5: Flow Chart to visualise pyramid structure, source taken from [1]

Algorithm 1 Multiscale Transform

1: Determine the number of levels L 2: {Forward transform (analysis)} 3: a0 = a 4: for each level l = 0...L − 1 do 5:

al

0 = al

6:

al+1 = ↓ (h1 ∗ al)

7: end for 8: {Backward transform (synthesis)} 9:

ˆ aL = g ∗ aL

10: for each level l = L − 1...0 do 11:

ˆ al = h2 ∗ (↑ ˆ al+1) + g ∗ al

12: end for

9 / 22

slide-15
SLIDE 15

Motivation Convolution Pyramids

Approach Forward and Backward Transform Flow Chart and Pseudocode Optimisation

Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Convolution Pyramids

Optimisation

Kernel Determination:

  • target kernel f is given
  • seek a set of kernels F = {h1, h2, g} that minimise

arg min

F

  • ˆ

a0

F

  • result of

multiscale transform

− f

  • target

kernel

∗ a

  • input

signal

  • (5)
  • kernels in F should be small and separable
  • use larger and/or non-separable filters increase accuracy

= ⇒ specific choice depends on application requirements

  • remarkable results using separable kernels in F for non-separable

target filters f

  • target filters f with rotational and mirroring symmetries enforce

symmetry on h1, h2, g

10 / 22

slide-16
SLIDE 16

Motivation Convolution Pyramids Application 1 - Gaussian Kernels

Gaussian Kernel Convolution Example - Gaussian Filter Example - Scattered Data Interpolation

Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Overview

  • 1. Motivation
  • 2. Convolution Pyramids
  • 3. Application 1 - Gaussian Kernels

Gaussian Kernel Convolution Example - Gaussian Filter Example - Scattered Data Interpolation

  • 4. Application 2 - Boundary Interpolation
  • 5. Application 3 - Gradient Integration
  • 6. Summary

10 / 22

slide-17
SLIDE 17

Motivation Convolution Pyramids Application 1 - Gaussian Kernels

Gaussian Kernel Convolution Example - Gaussian Filter Example - Scattered Data Interpolation

Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Application 1 - Gaussian Kernels

Gaussian Kernel Convolution

Task:

  • approximate Gaussian kernels e

x2 2σ2 at the original fine grid in O(n)

  • no truncated filter support

Determination of F = {h1, h2, g}: arg min

F

  • ˆ

a0

F

  • result of

multiscale transform

− f

  • target

Gaussian kernel

∗ a

  • image

to convolve

  • (5)

Problem:

  • Gaussians are rather efficient low-pass filters
  • pyramid contains high-frequent components coming from finer

levels introduced by convolution with g

11 / 22

slide-18
SLIDE 18

Motivation Convolution Pyramids Application 1 - Gaussian Kernels

Gaussian Kernel Convolution Example - Gaussian Filter Example - Scattered Data Interpolation

Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Application 1 - Gaussian Kernels

Gaussian Kernel Convolution

Task:

  • approximate Gaussian kernels e

x2 2σ2 at the original fine grid in O(n)

  • no truncated filter support

Determination of F = {h1, h2, g}: arg min

F

  • ˆ

a0

F

  • result of

multiscale transform

− f

  • target

Gaussian kernel

∗ a

  • image

to convolve

  • (5)

Problem:

  • Gaussians are rather efficient low-pass filters
  • pyramid contains high-frequent components coming from finer

levels introduced by convolution with g

11 / 22

slide-19
SLIDE 19

Motivation Convolution Pyramids Application 1 - Gaussian Kernels

Gaussian Kernel Convolution Example - Gaussian Filter Example - Scattered Data Interpolation

Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Application 1 - Gaussian Kernels

Gaussian Kernel Convolution

Task:

  • approximate Gaussian kernels e

x2 2σ2 at the original fine grid in O(n)

  • no truncated filter support

Determination of F = {h1, h2, g}: arg min

F

  • ˆ

a0

F

  • result of

multiscale transform

− f

  • target

Gaussian kernel

∗ a

  • image

to convolve

  • (5)

Problem:

  • Gaussians are rather efficient low-pass filters
  • pyramid contains high-frequent components coming from finer

levels introduced by convolution with g

11 / 22

slide-20
SLIDE 20

Motivation Convolution Pyramids Application 1 - Gaussian Kernels

Gaussian Kernel Convolution Example - Gaussian Filter Example - Scattered Data Interpolation

Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Application 1 - Gaussian Kernels

Example - Gaussian Filter

Solution:

  • modulation of g at each level l
  • higher wl at the levels closest to the target size
  • for different σ different sets of kernels F are necessary
  • Fig. 6.1: Original image,

source: taken from [1]

  • Fig. 6.2: Exact convolution with a Gaussian filter

(σ = 4), source: taken from [1]

  • Fig. 6.3: Convolution using optimization

approach for σ = 4, source: taken from [1]

  • Fig. 7.1: Exact kernels (in red) with

approximated kernels (in blue), source: taken from [1]

  • Fig. 7.2: Exact Gaussian (red), approximation

using 5x5 kernels (blue) and 7x7 kernel (green) , source: taken from [1]

  • Fig. 7.3: Magnification of Fig. 7.2 shows better

accuracy of larger kernels, source: taken from [1]

used kernels

12 / 22

slide-21
SLIDE 21

Motivation Convolution Pyramids Application 1 - Gaussian Kernels

Gaussian Kernel Convolution Example - Gaussian Filter Example - Scattered Data Interpolation

Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Application 1 - Gaussian Kernels

Example - Scattered Data Interpolation

  • Fig. 8.4: Approximation with

wider Gaussian, source: taken from [1]

  • Fig. 8.5: Approximation with

narrower Gaussian, source: taken from [1]

  • Fig. 8.6: Exact results

corresponding to red wider Gaussian , source: taken from [1]

  • Fig. 8.7: Exact results

corresponding to red narrower Gaussian, source: taken from [1]

  • Fig. 8.1: Horizontal slice through exact

wider Gaussian (red) and approximation (blue), source: taken from [1]

  • Fig. 8.2: Horizontal slice through exact

narrower Gaussian (red) and approximation (blue), source: taken from [1]

  • Fig. 8.3: Scattered data

interpolation input , source: taken from [1]

13 / 22

slide-22
SLIDE 22

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation

How to use boundary interpolation? Example - Seamless Cloning

Application 3 - Gradient Integration Summary

Overview

  • 1. Motivation
  • 2. Convolution Pyramids
  • 3. Application 1 - Gaussian Kernels
  • 4. Application 2 - Boundary Interpolation

How to use boundary interpolation? Example - Seamless Cloning

  • 5. Application 3 - Gradient Integration
  • 6. Summary

13 / 22

slide-23
SLIDE 23

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation

How to use boundary interpolation? Example - Seamless Cloning

Application 3 - Gradient Integration Summary

Application 2 - Boundary Interpolation

How to use boundary interpolation?

Seamless Image Cloning:

  • formulation as boundary value problem
  • effectively solved by constructing a smooth membrane
  • interpolation of differences along a seam between two images

Shepard’s Method:

  • Ω is region of interest and boundary values are given by b(x)
  • smoothly interpolates boundary values to all grid points inside Ω
  • defines interpolant r at x as weighted average of boundary values:

r(x) =

  • k wk(x)b(xk)
  • k wk(x)

= ⇒ r(xi) = n

j=0 w(xi, xj)ˆ

r(xj) n

j=0 w(xi, xj)χˆ r(xj) = w ∗ ˆ

r w ∗ χˆ

r (6)

  • xk = boundary points, b(xk) = boundary values
  • weight function wk(x) is given by

wk(x) = w(xk, x) = 1 d(xk, x)3 (7)

  • strong spike at xk and decays rapidly away from it
  • computational cost O(Kn), K boundary values and n points in Ω

14 / 22

slide-24
SLIDE 24

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation

How to use boundary interpolation? Example - Seamless Cloning

Application 3 - Gradient Integration Summary

Application 2 - Boundary Interpolation

How to use boundary interpolation?

Seamless Image Cloning:

  • formulation as boundary value problem
  • effectively solved by constructing a smooth membrane
  • interpolation of differences along a seam between two images

Shepard’s Method:

  • Ω is region of interest and boundary values are given by b(x)
  • smoothly interpolates boundary values to all grid points inside Ω
  • defines interpolant r at x as weighted average of boundary values:

r(x) =

  • k wk(x)b(xk)
  • k wk(x)

= ⇒ r(xi) = n

j=0 w(xi, xj)ˆ

r(xj) n

j=0 w(xi, xj)χˆ r(xj) = w ∗ ˆ

r w ∗ χˆ

r (6)

  • xk = boundary points, b(xk) = boundary values
  • weight function wk(x) is given by

wk(x) = w(xk, x) = 1 d(xk, x)3 (7)

  • strong spike at xk and decays rapidly away from it
  • computational cost O(Kn), K boundary values and n points in Ω

14 / 22

slide-25
SLIDE 25

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation

How to use boundary interpolation? Example - Seamless Cloning

Application 3 - Gradient Integration Summary

Application 2 - Boundary Interpolation

Example - Seamless Cloning

Determination of F = {h1, h2, g}: arg min

F

  • ˆ

a0

F

  • result of

multiscale transform

− f ∗ a

exact membrane r(x)

  • (5)
  • Fig. 9.1: Source image,

source: taken from [2]

  • Fig. 9.2: Membrane mask,

source: taken from [2]

  • Fig. 9.3: Target image,

source: taken from [2]

  • Fig. 9.4: Approximated membrane

source: taken from [1]

  • Fig. 9.5: Superimposed image with a cloned

patch, source: taken from [1]

  • Fig. 9.6: Result of applying Fig. 9.4 to Fig. 9.5,

source: taken from [1]

Used Kernels

15 / 22

slide-26
SLIDE 26

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration

Kernel Detection Example - Gradient Integration How does the target filter look like? Reconstruction of Target Filter

Summary

Overview

  • 1. Motivation
  • 2. Convolution Pyramids
  • 3. Application 1 - Gaussian Kernels
  • 4. Application 2 - Boundary Interpolation
  • 5. Application 3 - Gradient Integration

Kernel Detection Example - Gradient Integration How does the target filter look like? Reconstruction of Target Filter

  • 6. Summary

15 / 22

slide-27
SLIDE 27

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration

Kernel Detection Example - Gradient Integration How does the target filter look like? Reconstruction of Target Filter

Summary

Application 3 - Gradient Integration

Kernel Detection

Determination of F = {h1, h2, g}:

  • choose a natural image I
  • a is the divergence of its gradient field:

a = div ∇I (8) I = f ∗ a (9) arg min

F

  • ˆ

a0

F

  • result of

multiscale transform

− f ∗ a

natural image I

  • (5)
  • Fig. 10.1: Natural image I,

source: taken from [1]

  • Fig. 10.2: Corresponding gradient image a of
  • Fig. 10.1, source: taken from [1]

16 / 22

slide-28
SLIDE 28

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration

Kernel Detection Example - Gradient Integration How does the target filter look like? Reconstruction of Target Filter

Summary

Application 3 - Gradient Integration

Example - Gradient Integration

  • Fig. 11.1: Gradient image of Fig 11.4,

source: taken from [1]

  • Fig. 11.2: Reconstruction of Fig. 11.1 with

F5,3 , source: taken from [1]

  • Fig. 11.3: Reconstruction of Fig. 11.1 with

F7,5, source: taken from [1]

  • Fig. 11.4: Original image (512x512),

source: taken from [1]

  • Fig. 11.5: Absolute errors of Fig. 11.2

(magnified by x50), source: taken from [1]

  • Fig. 11.6: Absolute errors of Fig. 11.3

(magnified by x50), source: taken from [1]

Used Kernels

17 / 22

slide-29
SLIDE 29

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration

Kernel Detection Example - Gradient Integration How does the target filter look like? Reconstruction of Target Filter

Summary

Application 3 - Gradient Integration

How does the target filter look like?

Task:

  • recover image u (here: u = ˆ

a0

F) by solving the Poisson equation

△u = div v (10)

  • v = gradient field

Solution:

  • Green’s functions

G(x, x′) = G(x − x′) = 1 2π

log

1 x − x′ (11) define fundamental solutions to the Poisson equation △G(x, x′) = δ(x, x′) (12)

  • δ = discrete delta function
  • (10) is defined over an infinite domain with no boundary constraints

= ⇒ Laplace operator becomes spatially invariant = ⇒ Green’s function becomes translation invariant

  • solution of (10) is given by the convolution

u = G ∗ div v (13)

18 / 22

slide-30
SLIDE 30

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration

Kernel Detection Example - Gradient Integration How does the target filter look like? Reconstruction of Target Filter

Summary

Application 3 - Gradient Integration

How does the target filter look like?

Task:

  • recover image u (here: u = ˆ

a0

F) by solving the Poisson equation

△u = div v (10)

  • v = gradient field

Solution:

  • Green’s functions

G(x, x′) = G(x − x′) = 1 2π

log

1 x − x′ (11) define fundamental solutions to the Poisson equation △G(x, x′) = δ(x, x′) (12)

  • δ = discrete delta function
  • (10) is defined over an infinite domain with no boundary constraints

= ⇒ Laplace operator becomes spatially invariant = ⇒ Green’s function becomes translation invariant

  • solution of (10) is given by the convolution

u = G ∗ div v (13)

18 / 22

slide-31
SLIDE 31

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration

Kernel Detection Example - Gradient Integration How does the target filter look like? Reconstruction of Target Filter

Summary

Application 3 - Gradient Integration

Reconstruction of Target Filter

Target Filter Determination:

  • using results of previous F = {h1, h2, g}
  • a is a centered delta function

a = div ∇I (8) I = f ∗ a (9)

  • Green’s function provides a suitable result for f
  • Fig. 12.1: Reconstruction of the Green’s function,

source: taken from [1]

  • Fig. 12.2: space invariant corresponding kernel of Fig. 12.1,

source: taken from [1]

19 / 22

slide-32
SLIDE 32

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Summary

Overview

  • 1. Motivation
  • 2. Convolution Pyramids
  • 3. Application 1 - Gaussian Kernels
  • 4. Application 2 - Boundary Interpolation
  • 5. Application 3 - Gradient Integration
  • 6. Summary

Summary

19 / 22

slide-33
SLIDE 33

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

Summary

Summary

Summary

  • approximation of large convolution filters in O(n)

= ⇒ using kernels of small support F = {h1, h2, g} + multiscale pyramid scheme

  • kernel determination by optimization:

arg min

F

  • ˆ

a0

F

  • result of

multiscale transform

− f

  • target

kernel

∗ a

  • input

signal

  • suitable for different applications like...
  • gradient integration
  • seamless cloning
  • scattered data interpolation

20 / 22

slide-34
SLIDE 34

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

References

References

[1] ZEEV FARBMAN, RAANAN FATTAL, DANI LISCHINSKI Convolution pyramids

  • Proc. 2011 SIGGRAPH Asia Conference, Article No. 175

The Hebrew University (2011) [2] COMPUTER GRAPHICS & COMPUTATIONAL PHOTOGRAPHY LAB Supplementary Materials of the paper “Convolution pyramids” The Hebrew University (2011) http://www.cs.huji.ac.il/labs/cglab/projects/convpyr/ [3] MATHEMATICAL IMAGE ANALYSIS GROUP Lecture notes of the “Image Processing and Computer Vision” lecture Saarland University. Winter term (2011) http://www.mia.uni-saarland.de/Teaching/ipcv06.shtml [4] PETER J. BURT, EDWARD H. ADELSON The Laplacian Pyramid as a Compact Image Code IEEE Transcriptions on Communications

  • Vol. COM-31, No. 4, (April 1983)

21 / 22

slide-35
SLIDE 35

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

References

Thank you for your attention!

22 / 22

slide-36
SLIDE 36

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

I

slide-37
SLIDE 37

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

II

slide-38
SLIDE 38

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

III

slide-39
SLIDE 39

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

IV

slide-40
SLIDE 40

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

V

slide-41
SLIDE 41

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

VI

slide-42
SLIDE 42

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

VII

slide-43
SLIDE 43

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

VIII

slide-44
SLIDE 44

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

IX

slide-45
SLIDE 45

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

X

slide-46
SLIDE 46

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XI

slide-47
SLIDE 47

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XII

slide-48
SLIDE 48

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XIII

slide-49
SLIDE 49

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XIV

slide-50
SLIDE 50

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XV

slide-51
SLIDE 51

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XVI

slide-52
SLIDE 52

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XVII

slide-53
SLIDE 53

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XVIII

slide-54
SLIDE 54

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XIX

slide-55
SLIDE 55

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XX

slide-56
SLIDE 56

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXI

slide-57
SLIDE 57

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXII

slide-58
SLIDE 58

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXIII

slide-59
SLIDE 59

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXIV

slide-60
SLIDE 60

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXV

slide-61
SLIDE 61

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXVI

slide-62
SLIDE 62

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXVII

slide-63
SLIDE 63

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXVIII

slide-64
SLIDE 64

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXIX

slide-65
SLIDE 65

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXX

slide-66
SLIDE 66

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXXI

slide-67
SLIDE 67

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXXII

slide-68
SLIDE 68

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXXIII

slide-69
SLIDE 69

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXXIV

slide-70
SLIDE 70

Motivation Convolution Pyramids Application 1 - Gaussian Kernels Application 2 - Boundary Interpolation Application 3 - Gradient Integration Summary

XXXV