SLIDE 1
Vectorising Bitmaps into Semi-Transparent Gradient Layers
Christian Richardt 1,2 Jorge Lopez-Moreno 1,3 Adrien Bousseau 1 Maneesh Agrawala 4 George Drettakis 1
1
1 2 3 4
2
Vectorising Bitmaps into Semi-Transparent Gradient Layers 1 - - PowerPoint PPT Presentation
Vectorising Bitmaps into Semi-Transparent Gradient Layers 1 - - PowerPoint PPT Presentation
Vectorising Bitmaps into Semi-Transparent Gradient Layers 1 Christian Richardt 1,2 2 Jorge Lopez-Moreno 1,3 Adrien Bousseau 1 3 Maneesh Agrawala 4 4 George Drettakis 1 1 photos drawings vector art 2 Vector art representations 3 Vector
SLIDE 2
SLIDE 3
Vector art representations
3
SLIDE 4
Vector art representations
3
single layer
SLIDE 5
Vector art representations
3
single layer multiple layers
SLIDE 6
Image vectorisation
[Sun+ 2007]
Gradient meshes
4
[Lecot & Lévy 2006]
Ardeco
[Orzan+ 2008]
Diffusion curves
SLIDE 7
5
Our interactive workflow
Shutterstock/George Dolgikh
SLIDE 8
5
Our interactive workflow
Shutterstock/George Dolgikh
SLIDE 9
5
Our interactive workflow
Shutterstock/George Dolgikh
SLIDE 10
6
Our interactive workflow
SLIDE 11
7
Vectorised result
SLIDE 12
8
Editing result
SLIDE 13
Similarity to matting
9
I = α · F + (1 − α) · B
+ =
compositing equation
[Porter & Duff 1984]
SLIDE 14
The matting problem
[Smith & Blinn 1996]
10
I = α · F + (1 − α) · B
we have 3 equations: one each for R, G, B
SLIDE 15
The matting problem
[Smith & Blinn 1996]
10
I = α · F + (1 − α) · B
we have 3 equations: one each for R, G, B
α F B α F
solve for # unknowns
7 4
I B I
know
underconstrained underconstrained
SLIDE 16
Solving the matting problem
[Smith & Blinn 1996]
11
I1 = α · F + (1 − α) · B1 I2 = α · F + (1 − α) · B2
we know:
I1 I2 B1 B2 α F
solve for: 6 equations, 4 unknowns
great!
B1 B2
SLIDE 17
Image decompositions
Alpha matting
[e.g. Smith+ 1996, Chuang+ 2001, Levin+ 2008]
Reflection separation
[e.g. Levin+ 2004/2007, Kim+ 2013, Li & Brown 2014]
Intrinsic images
[e.g. Bousseau+ 2009, Carroll+ 2011]
12
SLIDE 18
Decompositing
13
SLIDE 19
Decompositing
13
SLIDE 20
Decompositing
13
I1
SLIDE 21
Decompositing
13
I1 B1
SLIDE 22
Decompositing
13
B2 I2 I1 B1
SLIDE 23
Decompositing
14
SLIDE 24
Decompositing
14
SLIDE 25
Decompositing
14
SLIDE 26
Decompositing
15
SLIDE 27
Decompositing
15
SLIDE 28
Decompositing
15
SLIDE 29
Parametric gradient functions
16
f = c g = c(g(x, θ), θ)
SLIDE 30
gradial(x, θ) = kx θpk θr glinear(x, θ) = x · θv kθvk2 + θo
Parametric gradient functions
16
Radial gradient
1
Linear gradient
1
gradient function
f = c g = c(g(x, θ), θ) g(x, θ)
SLIDE 31
f = c g = c(g(x, θ), θ)
c3(β, θ) = 8 < : mix ⇣ θc1, θc2,
β θs2
⌘ β ≤ θs2 mix ⇣ θc2, θc3, β−θs2
1−θs2
⌘ β > θs2
Three-stop gradient
1 s1 c2 c3 c1 s2 s3
- Two-stop gradient
1 s1 c2 c1 s2
- Parametric gradient functions
17
c2(β, θ) = mix(θc1, θc2, β) mix(a, b, t)=(1− t) · a + t · b
c(β, θ)
colour function
SLIDE 32
Foreground estimation
18
I = α · F + (1 α) · B = F B I(x) = F(x) B(x) I(x) = f(x, θ) B(x) arg min
θ,B
X
x∈R
- I(x) f(x, θ) B(x)
2
pixel position gradient parameters selected image region R θ x
SLIDE 33
Foreground estimation
18
arg min
θ,B
X
x∈R
- I(x) f(x, θ) B(x)
2 I(x) I(b(x))
background sample pixel position gradient parameters selected image region R θ x b
SLIDE 34
Foreground estimation
18
arg min
θ
X
x∈∂R
- I(x) f(x, θ) I(b(x))
2 I(x) I(b(x)) ≈ B(x)
background sample pixel position gradient parameters selected image region region boundary ∂R R θ x b
SLIDE 35
Foreground estimation
18
arg min
θ
X
x∈∂R
- I(x) f(x, θ) I(b(x))
2
background sample pixel position gradient parameters selected image region region boundary ∂R R θ x b
SLIDE 36
Background estimation
19
Input photo (slightly blurred)
Shutterstock/Picsfive
SLIDE 37
Background estimation
19
Input photo (slightly blurred)
Shutterstock/Picsfive
SLIDE 38
Background estimation
19
Input photo (slightly blurred)
Shutterstock/Picsfive
SLIDE 39
Background estimation
20
with hard region boundary
Shutterstock/Picsfive
SLIDE 40
Background estimation
21
hard region boundary
Shutterstock/Picsfive
SLIDE 41
Background estimation
22
trimap from hard region boundary
Shutterstock/Picsfive
SLIDE 42
Background estimation
23
matted region boundary
Shutterstock/Picsfive
SLIDE 43
Background estimation
24
with matted region boundary
Shutterstock/Picsfive
SLIDE 44
Background estimation
25
plus TV-smoothed region
Shutterstock/Picsfive
SLIDE 45
Background estimation
26
plus Poisson blending
Shutterstock/Picsfive
SLIDE 46
Background estimation
26
plus Poisson blending
Shutterstock/Picsfive
SLIDE 47
Summary
- 1. joint decompositing and vectorisation of foreground:
solving the matting problem around region boundary strong prior on foreground + user input
- 2. background estimation by optimisation:
similar to inversion of compositing equation additional terms to remove residuals: TV smoothness + Poisson blending
27
SLIDE 48
Input photo
28
Shutterstock/Givaga
SLIDE 49
Vectorised result
29
SLIDE 50
Editing result
30
SLIDE 51
Input photo
31
Flickr/squinza (CC BY-SA 2.0)
SLIDE 52
Vectorised result
32
SLIDE 53
Input drawing
33
Spencer Nugent
SLIDE 54
Vectorised result
34
SLIDE 55
Editing result
35
SLIDE 56
Limitation: few iso-contours
36
input image ground truth
- ur decomposition
foreground background
SLIDE 57
Limitation: background textures
37
input photo
- ur recomposited result
SLIDE 58
Limitation: background textures
37
input photo estimated background
SLIDE 59
Future work
more complex semi-transparent vector primitives automatic segmentation and decompositing extract Vector Shade Trees [Lopez-Moreno+ 2013] from exemplar materials
38
SLIDE 60
Conclusion
key insight: complex images can often be explained by stacking simple layers first approach creating layered vector art from bitmaps:
- paque and semi-transparent gradient layers
produces a simple, editable stack of vector layers valuable for professionals and novices alike
39
We thank: Inria CRISP associate team, ANR-12-JS02-003-01 DRAO, research donation from Adobe.