Submodularity beyond submodular energies: Coupling edges in graph - - PowerPoint PPT Presentation

▶
submodularity beyond submodular energies coupling edges
SMART_READER_LITE
LIVE PREVIEW

Submodularity beyond submodular energies: Coupling edges in graph - - PowerPoint PPT Presentation

Dec 24, 2022 28 likes •419 views

Graph Cuts Cooperative Cuts Optimization Applications Submodularity beyond submodular energies: Coupling edges in graph cuts Stefanie Jegelka and Jeff Bilmes Max Planck Institute for Intelligent Systems T ubingen, Germany University of

slide-1
SLIDE 1

Graph Cuts Cooperative Cuts Optimization Applications

Submodularity beyond submodular energies: Coupling edges in graph cuts

Stefanie Jegelka and Jeff Bilmes

Max Planck Institute for Intelligent Systems T¨ ubingen, Germany University of Washington Seattle, USA

1 / 18

slide-2
SLIDE 2

Graph Cuts Cooperative Cuts Optimization Applications

local pairwise random fields . . .

2 / 18

slide-3
SLIDE 3

Graph Cuts Cooperative Cuts Optimization Applications 3 / 18

slide-4
SLIDE 4

Graph Cuts Cooperative Cuts Optimization Applications Random Walker Curvature reg. Graph Cut 3 / 18

slide-5
SLIDE 5

Graph Cuts Cooperative Cuts Optimization Applications Random Walker Curvature reg. Graph Cut 3 / 18

slide-6
SLIDE 6

Graph Cuts Cooperative Cuts Optimization Applications

Markov Random Fields and Energies

p(x | z) ∝ exp(−EΨ(x; z)) MAP x∗ = arg min

x EΨ(x; z)

s t

4 / 18

slide-7
SLIDE 7

Graph Cuts Cooperative Cuts Optimization Applications

Markov Random Fields and Energies

p(x | z) ∝ exp(−EΨ(x; z)) MAP x∗ = arg min

x EΨ(x; z)

E(x; z) =

  • i

Ψi(xi) +

  • (i,j)∈N

Ψij(xi, xj)

s t

4 / 18

slide-8
SLIDE 8

Graph Cuts Cooperative Cuts Optimization Applications

Markov Random Fields and Energies

p(x | z) ∝ exp(−EΨ(x; z)) MAP x∗ = arg min

x EΨ(x; z)

E(x; z) =

  • i

Ψi(xi) +

  • (i,j)∈N

Ψij(xi, xj) E(x; z) =

  • e∈Γx∩Et

we +

  • e∈Γx∩En

we

s t

4 / 18

slide-9
SLIDE 9

Graph Cuts Cooperative Cuts Optimization Applications

Markov Random Fields and Energies

p(x | z) ∝ exp(−EΨ(x; z)) MAP x∗ = arg min

x EΨ(x; z)

E(x; z) =

  • i

Ψi(xi) +

  • (i,j)∈N

Ψij(xi, xj) E(x; z) =

  • e∈Γx∩Et

we +

  • e∈Γx∩En

we

1

s t

1 1 1 1 1

4 / 18

slide-10
SLIDE 10

Graph Cuts Cooperative Cuts Optimization Applications

1

s t

1 1 1 1 1

5 / 18

slide-11
SLIDE 11

Graph Cuts Cooperative Cuts Optimization Applications

1

s t

1 1 1 1 1

5 / 18

slide-12
SLIDE 12

Graph Cuts Cooperative Cuts Optimization Applications

1

s t

1 1 1 1 1

5 / 18

slide-13
SLIDE 13

Graph Cuts Cooperative Cuts Optimization Applications

1

s t

1 1 1 1 1

Couple edges globally

5 / 18

slide-14
SLIDE 14

Graph Cuts Cooperative Cuts Optimization Applications

Richer Cuts: Cooperative Cuts

1

s t

1 1 1 1 1

E(x) =

  • e∈Γx

w(e) = w(Γx)

6 / 18

slide-15
SLIDE 15

Graph Cuts Cooperative Cuts Optimization Applications

Richer Cuts: Cooperative Cuts

1

s t

1 1 1 1 1

E(x) =

  • e∈Γx

w(e) = w(Γx)

Ef (x) = f (Γx)

submodular function

  • n edges

6 / 18

slide-16
SLIDE 16

Graph Cuts Cooperative Cuts Optimization Applications

Richer Cuts: Cooperative Cuts

1

s t

1 1 1 1 1

E(x) =

  • e∈Γx

w(e) = w(Γx)

Ef (x) = f (Γx)

submodular function

  • n edges

non-submodular & global energy

6 / 18

slide-17
SLIDE 17

Graph Cuts Cooperative Cuts Optimization Applications

Coupling via Submodularity

s t

7 / 18

slide-18
SLIDE 18

Graph Cuts Cooperative Cuts Optimization Applications

Coupling via Submodularity

A B e A B e

f (A ∪ e) − f (A) ≥ f (A ∪ B ∪ e) − f (A ∪ B)

s t Graph Cuts: LHS = RHS “it does not matter which other edges are cut”

8 / 18

slide-19
SLIDE 19

Graph Cuts Cooperative Cuts Optimization Applications

Coupling via Submodularity

A B e A B e

f (A ∪ e) − f (A) ≥ f (A ∪ B ∪ e) − f (A ∪ B)

s t Graph Cuts: LHS = RHS “it does not matter which other edges are cut”

submodularity: reward co-occurrence structure

8 / 18

slide-20
SLIDE 20

Graph Cuts Cooperative Cuts Optimization Applications

Generality

1

s t

1 1 1 1 1

labels features ... boundary

Special cases of cooperative cuts: (robust) Pn potentials (Kohli et al. ’07,’09) label costs (Delong et al. ’11) discrete versions of norm-based cuts (Sinop & Grady ’07) . . .

9 / 18

slide-21
SLIDE 21

Graph Cuts Cooperative Cuts Optimization Applications

Optimization?

10 / 18

slide-22
SLIDE 22

Graph Cuts Cooperative Cuts Optimization Applications

Optimization?

(s, t)-cut Γ ⊆ E with min cost f (Γ).

Theorem

Minimum Cooperative Cut is NP-hard.

10 / 18

slide-23
SLIDE 23

Graph Cuts Cooperative Cuts Optimization Applications

Optimization

Γ0 = ∅; repeat compute upper bound ˆ fi ≥ f based on Γi−1; until convergence ; ˆ fi(Γi−1) = f (Γi−1)

11 / 18

slide-24
SLIDE 24

Graph Cuts Cooperative Cuts Optimization Applications

Optimization

Γ0 = ∅; repeat compute upper bound ˆ fi ≥ f based on Γi−1; Γi ∈ argmin{ ˆ fi(Γ) | Γ a cut } ; // Min-cut! i = i + 1; until convergence ; ˆ fi(Γi−1) = f (Γi−1)

11 / 18

slide-25
SLIDE 25

Graph Cuts Cooperative Cuts Optimization Applications

Optimization

Γ0 = ∅; repeat compute upper bound ˆ fi ≥ f based on Γi−1; Γi ∈ argmin{ ˆ fi(Γ) | Γ a cut } ; // Min-cut! i = i + 1; until convergence ; Worst-case approximation bound: Ef (x) ≤

|Γ∗| 1+(|Γ∗|−1)ν Ef (x∗)

for ν = mine∈Γ∗ ρe(E\e)

maxe∈C∗ f (e)

11 / 18

slide-26
SLIDE 26

Graph Cuts Cooperative Cuts Optimization Applications

Image Segmentation

Random Walker Curvature reg. Graph Cut 12 / 18

slide-27
SLIDE 27

Graph Cuts Cooperative Cuts Optimization Applications

Image Segmentation

Random Walker Curvature reg. Graph Cut

prefer congruous boundaries

12 / 18

slide-28
SLIDE 28

Graph Cuts Cooperative Cuts Optimization Applications

Selective Discount for Congruous Boundaries

s t

Ew(x) =

  • e∈Γ∩Et

we +λ

  • e∈Γ∩En

we Ef (x) =

  • e∈Γ∩Et

we +λ f (Γ ∩ En)

13 / 18

slide-29
SLIDE 29

Graph Cuts Cooperative Cuts Optimization Applications

Selective Discount for Congruous Boundaries

s t

Ew(x) =

  • e∈Γ∩Et

we +λ

  • e∈Γ∩En

we Ef (x) =

  • e∈Γ∩Et

we +λ f (Γ ∩ En) discount for co-occurring similar edges no discount for dissimilar edges

13 / 18

slide-30
SLIDE 30

Graph Cuts Cooperative Cuts Optimization Applications

Structured Discounts

groups Si of edges f (Γ) =

  • i fi(Γ∩Si)

100 200 300 400 50 100 150

14 / 18

slide-31
SLIDE 31

Graph Cuts Cooperative Cuts Optimization Applications

Structured Discounts

groups Si of edges f (Γ) =

  • i fi(Γ∩Si)

100 200 300 400 50 100 150

14 / 18

slide-32
SLIDE 32

Graph Cuts Cooperative Cuts Optimization Applications

Structured Discounts

groups Si of edges f (Γ) =

  • i fi(Γ∩Si)

100 200 300 400 50 100 150

14 / 18

slide-33
SLIDE 33

Graph Cuts Cooperative Cuts Optimization Applications

Some Results: Shading

Graph Cut CoopCut

7.39% 2.23% 7.65% 3.50%

15 / 18

slide-34
SLIDE 34

Graph Cuts Cooperative Cuts Optimization Applications

Some Results: Shading

gray color high-freq Graph Cut: no discount 14.03 3.41 2.56 CoopCut (1 group): discount 11.58 2.95 1.49 CoopCut (15 groups): structured 3.63 1.69 1.27 discount

Graph Cut CoopCut 5.08% 0.64%

16 / 18

slide-35
SLIDE 35

Graph Cuts Cooperative Cuts Optimization Applications

Shrinking bias

0.5 1 1.5 2 2.5 2 4 6 8 10 12 14 λ

total error (%)

GC twig CoopC twig GC total CoopC total

0.5 1 1.5 2 2.5 0 10 20 30 40 50

twig error (%)

  • i

ψi(xi) + λ

  • e∈Γx

we

Graph Cut

17 / 18

slide-36
SLIDE 36

Graph Cuts Cooperative Cuts Optimization Applications

Shrinking bias

0.5 1 1.5 2 2.5 2 4 6 8 10 12 14 λ

total error (%)

GC twig CoopC twig GC total CoopC total

0.5 1 1.5 2 2.5 0 10 20 30 40 50

twig error (%)

  • i

ψi(xi) + λ

  • e∈Γx

we

Graph Cut

17 / 18

slide-37
SLIDE 37

Graph Cuts Cooperative Cuts Optimization Applications

Shrinking bias

0.5 1 1.5 2 2.5 2 4 6 8 10 12 14 λ

total error (%)

GC twig CoopC twig GC total CoopC total

0.5 1 1.5 2 2.5 0 10 20 30 40 50

twig error (%)

  • i

ψi(xi) + λ f (Γx)

CoopCut Graph Cut

17 / 18

slide-38
SLIDE 38

Graph Cuts Cooperative Cuts Optimization Applications

Shrinking bias

0.5 1 1.5 2 2.5 2 4 6 8 10 12 14 λ

total error (%)

GC twig CoopC twig GC total CoopC total

0.5 1 1.5 2 2.5 0 10 20 30 40 50

twig error (%)

  • i

ψi(xi) + λ f (Γx)

CoopCut Graph Cut

17 / 18

slide-39
SLIDE 39

Graph Cuts Cooperative Cuts Optimization Applications

Summary: Coupling Edges in Graph Cuts

global, non-submodular family of energies NP-hard, but . . .

graph structure indirect submodularity

→ efficient approximation algorithm applications

guide segmentations via edge coupling

18 / 18