Seam Carving for Content-Aware Finding Optimal Seams Image - - PowerPoint PPT Presentation

▶
seam carving for content aware
SMART_READER_LITE
LIVE PREVIEW

Seam Carving for Content-Aware Finding Optimal Seams Image - - PowerPoint PPT Presentation

Jan 21, 2023 163 likes •610 views

Seam Carving for Image Resizing Overview Definitions Seam Carving for Content-Aware Finding Optimal Seams Image Resizing Optimal Seams Order Results Video Synopsis Ricardo David Casta neda Marin Experiments References Computer

slide-1
SLIDE 1

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Carving for Content-Aware Image Resizing

Ricardo David Casta˜ neda Marin

Computer Graphics at Instituto de Matematica Pura e Aplicada VISGRAF

November 29, 2007

slide-2
SLIDE 2

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Overview

Content Aware − → Energy Preservation

Figure: Image and its Energy

slide-3
SLIDE 3

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Overview

Content Aware − → Energy Preservation

Figure: Different strategies for energy preservation

slide-4
SLIDE 4

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Definitions

◮ Image: I = I(i, j) of size n × m.

slide-5
SLIDE 5

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Definitions

◮ Image: I = I(i, j) of size n × m. ◮ Vertical Seam: {(i, S(i))}n i=1 , such that

|S(i) − S(i − 1)| ≤ 1 , ∀i > 1.

Figure: Vertical Seam

slide-6
SLIDE 6

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Definitions

◮ Image: I = I(i, j) of size n × m. ◮ Vertical Seam: {(i, S(i))}n i=1 , such that

|S(i) − S(i − 1)| ≤ 1 , ∀i > 1.

◮ Energy: e1(I) = | ∂I ∂x | + | ∂I ∂y | = E.

slide-7
SLIDE 7

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Definitions

◮ Image: I = I(i, j) of size n × m. ◮ Vertical Seam: {(i, S(i))}n i=1 , such that

|S(i) − S(i − 1)| ≤ 1 , ∀i > 1.

◮ Energy: e1(I) = | ∂I ∂x | + | ∂I ∂y | = E. ◮ Seam Cost: cost(S) = Σn i=1E(i, S(i)).

slide-8
SLIDE 8

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Definitions

◮ Image: I = I(i, j) of size n × m. ◮ Vertical Seam: {(i, S(i))}n i=1 , such that

|S(i) − S(i − 1)| ≤ 1 , ∀i > 1.

◮ Energy: e1(I) = | ∂I ∂x | + | ∂I ∂y | = E. ◮ Seam Cost: cost(S) = Σn i=1E(i, S(i)). ◮ Optimal Seam: S∗ = minS{cost(S)}.

slide-9
SLIDE 9

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

How to find Optimal Seams?

◮ M(i, j) =Minimum Cost from Row 1 to (i, j) in E.

slide-10
SLIDE 10

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

How to find Optimal Seams?

◮ M(i, j) =Minimum Cost from Row 1 to (i, j) in E.

= E(i, j) + min{M(i − 1, j − 1), M(i − 1, j), M(i − 1, j + 1)}

slide-11
SLIDE 11

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

How to find Optimal Seams?

◮ M(i, j) =Minimum Cost from Row 1 to (i, j) in E.

= E(i, j) + min{M(i − 1, j − 1), M(i − 1, j), M(i − 1, j + 1)}

◮ S∗(n) = minArgjM(n, j)

slide-12
SLIDE 12

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

How to find Optimal Seams?

◮ M(i, j) =Minimum Cost from Row 1 to (i, j) in E.

= E(i, j) + min{M(i − 1, j − 1), M(i − 1, j), M(i − 1, j + 1)}

◮ S∗(n) = minArgjM(n, j) ◮ S∗(i) = minArgS(i+1)−1≤j≤S(i+1)+1M(i, j)

slide-13
SLIDE 13

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

How to find Optimal Seams?

◮ M(i, j) =Minimum Cost from Row 1 to (i, j) in E.

= E(i, j) + min{M(i − 1, j − 1), M(i − 1, j), M(i − 1, j + 1)}

◮ S∗(n) = minArgjM(n, j) ◮ S∗(i) = minArgS(i+1)−1≤j≤S(i+1)+1M(i, j)

Figure: Finding Optimal Seams

slide-14
SLIDE 14

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

How to find Optimal Seams?

◮ M(i, j) =Minimum Cost from Row 1 to (i, j) in E.

= E(i, j) + min{M(i − 1, j − 1), M(i − 1, j), M(i − 1, j + 1)}

◮ S∗(n) = minArgjM(n, j) ◮ S∗(i) = minArgS(i+1)−1≤j≤S(i+1)+1M(i, j)

Figure: Finding Optimal Seams

slide-15
SLIDE 15

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Optimal Seams Order

◮ Transport Map: T(i, j) = Minimum cost of removing i

rows and j columns.

slide-16
SLIDE 16

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Optimal Seams Order

◮ Transport Map: T(i, j) = Minimum cost of removing i

rows and j columns.

◮ T(r, c) = min{T(r −1, c)+X(r −1, c), T(r, c −1)+Y (r, c −1)}.

Figure: Computing T

slide-17
SLIDE 17

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Optimal Seams Order

◮ Transport Map: T(i, j) = Minimum cost of removing i

rows and j columns.

◮ T(r, c) = min{T(r −1, c)+X(r −1, c), T(r, c −1)+Y (r, c −1)}. ◮ Backtracking 0 = row, 1 = column.

Figure: Backtracking

slide-18
SLIDE 18

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Removal

Figure: Seam Removal

slide-19
SLIDE 19

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Removal

Figure: Seam Removal

slide-20
SLIDE 20

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Removal

Figure: Seam Removal

slide-21
SLIDE 21

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Removal

Figure: Seam Removal

slide-22
SLIDE 22

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Removal

Figure: Seam Removal

slide-23
SLIDE 23

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Removal

Figure: Seam Removal

slide-24
SLIDE 24

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Removal

Figure: Seam Removal

slide-25
SLIDE 25

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Insertion

Figure: Seam Insertion

slide-26
SLIDE 26

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Insertion

Figure: Seam Insertion

slide-27
SLIDE 27

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Insertion

Figure: Seam Insertion

slide-28
SLIDE 28

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Insertion

Figure: Seam Insertion

slide-29
SLIDE 29

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Insertion

Figure: Seam Insertion

slide-30
SLIDE 30

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Amplification

Figure: Amplification

slide-31
SLIDE 31

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Amplification

Figure: Amplification

slide-32
SLIDE 32

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Amplification

Figure: Amplification

slide-33
SLIDE 33

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Video Synopsis

⇒Deleting redundancies in time.

Figure: Synopsis video: The baby and the child appear at different times in the original video, while the synopsis video can represent them at the same time.

slide-34
SLIDE 34

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Carving in Time

◮ Sequence of Images: f (i, j, t), with k frames of size

n × m

slide-35
SLIDE 35

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Carving in Time

◮ Sequence of Images: f (i, j, t), with k frames of size

n × m

◮ Seam Frame: {(i, j, S(i, j))}(n,m) (i,j)=(1,1).

slide-36
SLIDE 36

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Carving in Time

◮ Sequence of Images: f (i, j, t), with k frames of size

n × m

◮ Seam Frame: {(i, j, S(i, j))}(n,m) (i,j)=(1,1). ◮ Energy: E(i, j, t) = |∂f (i,j,t) ∂t

|.

slide-37
SLIDE 37

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Carving in Time

◮ Sequence of Images: f (i, j, t), with k frames of size

n × m

◮ Seam Frame: {(i, j, S(i, j))}(n,m) (i,j)=(1,1). ◮ Energy: E(i, j, t) = |∂f (i,j,t) ∂t

|.

◮ Seam Frame Cost: cost(S) = Σn i=1Σm j=1E(i, j, S(i, j)).

slide-38
SLIDE 38

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Seam Carving in Time

◮ Sequence of Images: f (i, j, t), with k frames of size

n × m

◮ Seam Frame: {(i, j, S(i, j))}(n,m) (i,j)=(1,1). ◮ Energy: E(i, j, t) = |∂f (i,j,t) ∂t

|.

◮ Seam Frame Cost: cost(S) = Σn i=1Σm j=1E(i, j, S(i, j)). ◮ Optimal Seam Frame: S∗ = minS{cost(S)}.

slide-39
SLIDE 39

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Reformulation

◮ Transform the problem into:

  • Minimize F(V )

Subject to: |AV | ≤ 1

slide-40
SLIDE 40

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Reformulation

◮ Transform the problem into:

  • Minimize F(V )

Subject to: |AV | ≤ 1

◮ S = V ∈ ℜnm. V (m(i − 1) + j) = S(i, j).

slide-41
SLIDE 41

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Reformulation

◮ Transform the problem into:

  • Minimize F(V )

Subject to: |AV | ≤ 1

◮ S = V ∈ ℜnm. V (m(i − 1) + j) = S(i, j). ◮ Constraints:

|aij, V | ≤ 1 |bij, V | ≤ 1 |cij, V | ≤ 1 |dij, V | ≤ 1

slide-42
SLIDE 42

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Cont.→

◮

|AijV | ≤ 1 , with Aij =     aij bij cij dij    

4×nm ◮

A =       A11 A12 A13 · · · Anm      

4nm×nm ◮

F(V ) = Σn

i=1Σm j=1E(i, j, Vm(i−1)+j)

slide-43
SLIDE 43

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

Simplification

Remove whole columns or rows.

Figure: Simplification: On the left we calculate M averaging the energy cube by column in all the frames, and then to right we apply seam carving.

slide-44
SLIDE 44

Seam Carving for Image Resizing Overview Definitions Finding Optimal Seams Optimal Seams Order Results Video Synopsis Experiments References

References

Shai Avidan, Ariel Shamir: Seam Carving for Content-Aware Image Resizing. Proceedings of SIGGRAPH 2007. Alex Rav-Acha, Yael Pritch, Shmuel Peleg: Making a Long Video Short: Dynamic Video Sypnosis. Proceedings of the IEEE CVPR 2006. Alex Rav-Acha, Yael Pritch, Dani Lischinski, Shmuel Peleg: Dynamosaics: Video Mosaics with Non-Chronological Time. Proceedings of the IEEE CVPR 2006.