Nisarg Shah 373F20 - Nisarg Shah 1 Recap Some more DP Traveling - - PowerPoint PPT Presentation

CSC373 Week 5: Network Flow (contd)

373F20 - Nisarg Shah 1

Nisarg Shah

Recap

373F20 - Nisarg Shah 2

• Some more DP

➢ Traveling salesman problem (TSP)

• Start of network flow

➢ Problem statement ➢ Ford-Fulkerson algorithm ➢ Running time ➢ Correctness using max-flow, min-cut

This Lecture

373F20 - Nisarg Shah 3

• Network flow in polynomial time

➢ Edmonds-Karp algorithm (shortest augmenting path)

• Applications of network flow

➢ Bipartite matching & Hall’s theorem ➢ Edge-disjoint paths & Menger’s theorem ➢ Multiple sources/sinks ➢ Circulation networks ➢ Lower bounds on flows ➢ Survey design ➢ Image segmentation

Ford-Fulkerson Recap

373F20 - Nisarg Shah 4

• Define the residual graph 𝐻𝑔 of flow 𝑔

➢ 𝐻𝑔 has the same vertices as 𝐻 ➢ For each edge e = (𝑣, 𝑤) in 𝐻, 𝐻𝑔 has at most two edges

• Forward edge 𝑓 = (𝑣, 𝑤) with capacity 𝑑 𝑓 − 𝑔 𝑓
• We can send this much additional flow on 𝑓
• Reverse edge 𝑓𝑠𝑓𝑤 = (𝑤, 𝑣) with capacity 𝑔(𝑓)
• The maximum “reverse” flow we can send is the maximum

amount by which we can reduce flow on 𝑓, which is 𝑔(𝑓)

• We only add each edge if its capacity > 0

Ford-Fulkerson Recap

373F20 - Nisarg Shah 5

• Example!

s t u v

20/20 20/30 20/20 0/10 0/10

s t u v

𝟑𝟏 𝟐𝟏 𝟑𝟏 𝟐𝟏 𝟐𝟏 𝟑𝟏

Flow 𝑔 Residual graph 𝐻𝑔

Ford-Fulkerson Recap

373F20 - Nisarg Shah 6

MaxFlow(𝐻):

// initialize: Set 𝑔 𝑓 = 0 for all 𝑓 in 𝐻 // while there is an 𝑡-𝑢 path in 𝐻𝑔: While 𝑄 = FindPath(s, t,Residual(𝐻, 𝑔))!=None: 𝑔 = Augment(𝑔, 𝑄) UpdateResidual(𝐻,𝑔) EndWhile Return 𝑔

Ford-Fulkerson Recap

373F20 - Nisarg Shah 7

• Running time:

➢ #Augmentations:

• At every step, flow and capacities remain integers
• For path 𝑄 in 𝐻𝑔, bottleneck 𝑄, 𝑔 > 0 implies bottleneck 𝑄, 𝑔 ≥ 1
• Each augmentation increases flow by at least 1
• At most 𝐷 = σ𝑓 leaving 𝑡 𝑑(𝑓) augmentations

➢ Time for an augmentation:

• 𝐻𝑔 has 𝑜 vertices and at most 2𝑛 edges
• Finding an 𝑡-𝑢 path in 𝐻𝑔 takes 𝑃(𝑛 + 𝑜) time

➢ Total time: 𝑃( 𝑛 + 𝑜 ⋅ 𝐷)

Edmonds-Karp Algorithm

373F20 - Nisarg Shah 8

• At every step, find the shortest path from 𝑡 to 𝑢 in

𝐻𝑔, and augment.

MaxFlow(𝐻): // initialize: Set 𝑔 𝑓 = 0 for all 𝑓 in 𝐻 // Find shortest 𝑡-𝑢 path in 𝐻𝑔 & augment: While 𝑄 = BFS(s, t,Residual(𝐻, 𝑔))!=None: 𝑔 = Augment(𝑔, 𝑄) UpdateResidual(𝐻,𝑔) EndWhile Return 𝑔

Minimum number of edges

Proof Overview

373F20 - Nisarg Shah 9

• Overview

➢ Lemma 1: The length of the shortest 𝑡 → 𝑢 path in 𝐻𝑔

never decreases.

• (Proof ahead)

➢ Lemma 2: After at most 𝑛 augmentations, the length of

the shortest 𝑡 → 𝑢 path in 𝐻𝑔 must strictly increase.

• (Proof ahead)

➢ Theorem: The algorithm takes 𝑃(𝑛2𝑜) time.

• Proof:
• Length of shortest 𝑡 → 𝑢 path in 𝐻𝑔 can go from 0 to 𝑜 − 1
• Using Lemma 2, there can be at most 𝑛 ⋅ 𝑜 augmentations
• Each takes 𝑃(𝑛) time using BFS. ∎

Level Graph

373F20 - Nisarg Shah 10

• Level graph 𝑀𝐻 of a directed graph 𝐻 = (𝑊, 𝐹):

➢ Level: ℓ(𝑤) = length of shortest 𝑡 → 𝑤 path ➢ Level graph 𝑀𝐻 = (𝑊, 𝐹𝑀) is a subgraph of 𝐻 where we

• nly retain edges 𝑣, 𝑤 ∈ 𝐹 where ℓ 𝑤 = ℓ 𝑣 + 1
• Intuition: Keep only the edges useful for shortest paths

Level Graph

373F20 - Nisarg Shah 11

• Level graph 𝑀𝐻 of a directed graph 𝐻 = (𝑊, 𝐹):

➢ Level: ℓ(𝑤) = length of shortest 𝑡 → 𝑤 path ➢ Level graph 𝑀𝐻 = (𝑊, 𝐹𝑀) is a subgraph of 𝐻 where we

• nly retain edges 𝑣, 𝑤 ∈ 𝐹 where ℓ 𝑤 = ℓ 𝑣 + 1
• Intuition: Keep only the edges useful for shortest paths
• Property: 𝑄 is a shortest 𝑡 → 𝑤 path in 𝐻 if and
• nly if 𝑄 is an 𝑡 → 𝑤 path in 𝑀𝐻.

Edmonds-Karp Proof

373F20 - Nisarg Shah 12

• Lemma 1:

➢ Length of the shortest 𝑡 → 𝑢 path in 𝐻𝑔 never decreases.

• Proof:

➢ Let 𝑔 and 𝑔′ be flows before and after an augmentation

step, and 𝐻𝑔 and 𝐻𝑔′ be their residual graphs.

Residual graph 𝐻𝑔 Residual graph 𝐻𝑔′

Edmonds-Karp Proof

373F20 - Nisarg Shah 13

• Lemma 1:

➢ Length of the shortest 𝑡 → 𝑢 path in 𝐻𝑔 never decreases.

• Proof:

➢ Let 𝑔 and 𝑔′ be flows before and after an augmentation

step, and 𝐻𝑔 and 𝐻𝑔′ be their residual graphs.

➢ Augmentation happens along a path in 𝑀𝐻𝑔 ➢ For each edge on the path, we either remove it, add an

• pposite direction edge, or both.

➢ Opposite direction edges can’t help reduce the length of

the shortest 𝑡 → 𝑢 path (exercise!).

➢ QED!

NOT IN SYLLABUS

Edmonds-Karp Proof

373F20 - Nisarg Shah 14

• Lemma 2:

➢ After at most 𝑛 augmentations, the length of the

shortest 𝑡 → 𝑢 path in 𝐻𝑔 must strictly increase.

• Proof:

➢ In each augmentation step, we remove at least one edge

from 𝑀𝐻𝑔

• Because we make the flow on at least one edge on the shortest

path equal to its capacity

➢ No new edges are added in 𝑀𝐻𝑔 unless the length of the

shortest 𝑡 → 𝑢 path strictly increases

➢ This cannot happen more than 𝑛 times! ∎

NOT IN SYLLABUS

Edmonds-Karp Proof Overview

373F20 - Nisarg Shah 15

• Overview

➢ Lemma 1: The length of the shortest 𝑡 → 𝑢 path in 𝐻𝑔

never decreases.

➢ Lemma 2: After at most 𝑛 augmentations, the length of

the shortest 𝑡 → 𝑢 path in 𝐻𝑔 must strictly increase.

➢ Theorem: The algorithm takes 𝑃(𝑛2𝑜) time.

Edmonds-Karp Proof Overview

373F20 - Nisarg Shah 16

• Note:

➢ Some graphs require Ω(𝑛𝑜) augmentation steps ➢ But we may be able to reduce the time to run each

augmentation step

• Two algorithms use this idea to reduce run time

➢ Dinitz’s algorithm [1970] ⇒ 𝑃(𝑛𝑜2) ➢ Sleator–Tarjan algorithm [1983] ⇒ 𝑃(𝑛 𝑜 log 𝑜)

• Using the dynamic trees data structure

373F20 - Nisarg Shah 17

Network Flow Applications

373F20 - Nisarg Shah 18

Rail network connecting Soviet Union with Eastern European countries

(Tolstoǐ 1930s)

373F20 - Nisarg Shah 19

Rail network connecting Soviet Union with Eastern European countries

(Tolstoǐ 1930s) Min-cut

Integrality Theorem

373F20 - Nisarg Shah 20

• Before we look at applications, we need the

following special property of the max-flow computed by Ford-Fulkerson and its variants

• Observation:

➢ If edge capacities are integers, then the max-flow

computed by Ford-Fulkerson and its variants are also integral (i.e. the flow on each edge is an integer).

➢ Easy to check that each augmentation step preserves

integral flow

Bipartite Matching

373F20 - Nisarg Shah 21

• Problem

➢ Given a bipartite graph 𝐻 = (𝑉 ∪ 𝑊, 𝐹), find a maximum

cardinality matching

• We do not know any efficient greedy or dynamic

programming algorithm for this problem.

• But it can be reduced to max-flow.

Bipartite Matching

373F20 - Nisarg Shah 22

• Create a directed flow graph where we…

➢ Add a source node 𝑡 and target node 𝑢 ➢ Add edges, all of capacity 1:

• 𝑡 → 𝑣 for each 𝑣 ∈ 𝑉, 𝑤 → 𝑢 for each 𝑤 ∈ 𝑊
• 𝑣 → 𝑤 for each 𝑣, 𝑤 ∈ 𝐹

𝑉 𝑊 𝑉 𝑊

Bipartite Matching

373F20 - Nisarg Shah 23

• Observation

➢ There is a 1-1 correspondence between matchings of size

𝑙 in the original graph and flows with value 𝑙 in the corresponding flow network.

• Proof: (matching ⇒ integral flow)

➢ Take a matching 𝑁 =

𝑣1, 𝑤1 , … , 𝑣𝑙, 𝑤𝑙

• f size 𝑙

➢ Construct the corresponding unique flow 𝑔

𝑁 where…

• Edges 𝑡 → 𝑣𝑗, 𝑣𝑗 → 𝑤𝑗, and 𝑤𝑗 → 𝑢 have flow 1, for all 𝑗 = 1, … , 𝑙
• The rest of the edges have flow 0

➢ This flow has value 𝑙

Bipartite Matching

373F20 - Nisarg Shah 24

• Observation

➢ There is a 1-1 correspondence between matchings of size

𝑙 in the original graph and flows with value 𝑙 in the corresponding flow network.

• Proof: (integral flow ⇒ matching)

➢ Take any flow 𝑔 with value 𝑙 ➢ The corresponding unique matching 𝑁

𝑔 = set of edges

from 𝑉 to 𝑊 with a flow of 1

• Since flow of 𝑙 comes out of 𝑡, unit flow must go to 𝑙 distinct

vertices in 𝑉

• From each such vertex in 𝑉, unit flow goes to a distinct vertex in 𝑊
• Uses integrality theorem

Bipartite Matching

373F20 - Nisarg Shah 25

• Perfect matching = flow with value 𝑜

➢ where 𝑜 = 𝑉 = 𝑊

• Recall naïve Ford-Fulkerson running time:

➢ 𝑃((𝑛 + 𝑜) ⋅ 𝐷), where 𝐷 = sum of capacities of edges

leaving 𝑡

➢ Q: What’s the runtime when used for bipartite matching?

• Some variants are faster…

➢ Dinitz’s algorithm runs in time 𝑃 𝑛 𝑜 when all edge

capacities are 1

Hall’s Marriage Theorem

373F20 - Nisarg Shah 26

• When does a bipartite graph have a perfect matching?

➢ Well, when the corresponding flow network has value 𝑜 ➢ But can we interpret this condition in terms of edges of the

• riginal bipartite graph?

➢ For 𝑇 ⊆ 𝑉, let 𝑂 𝑇 ⊆ 𝑊 be the set of all nodes in 𝑊 adjacent

to some node in 𝑇

• Observation:

➢ If 𝐻 has a perfect matching, 𝑂 𝑇

≥ |𝑇| for each 𝑇 ⊆ 𝑉

➢ Because each node in 𝑇 must be matched to a distinct node

in 𝑂(𝑇)

Hall’s Marriage Theorem

373F20 - Nisarg Shah 27

• We’ll consider a slightly different flow network,

which is still equivalent to bipartite matching

➢ All 𝑉 → 𝑊 edges now have ∞ capacity ➢ 𝑡 → 𝑉 and 𝑊 → 𝑢 edges are still unit capacity

𝑉 𝑊 𝑉 𝑊 ∞ ∞ 1 1 1 1

Hall’s Marriage Theorem

373F20 - Nisarg Shah 28

• Hall’s Theorem:

➢ 𝐻 has a perfect matching iff 𝑂 𝑇

≥ |𝑇| for each 𝑇 ⊆ 𝑊

• Proof (reverse direction, via network flow):

➢ Suppose 𝐻 doesn’t have a perfect matching ➢ Hence, max-flow = min-cut < 𝑜 ➢ Let (𝐵, 𝐶) be the min-cut

• Can’t have any 𝑉 → 𝑊 (∞ capacity edges)
• Has unit capacity edges 𝑡 → 𝑉 ∩ 𝐶 and 𝑊 ∩ 𝐵 → 𝑢

Hall’s Marriage Theorem

373F20 - Nisarg Shah 29

• Hall’s Theorem:

➢ 𝐻 has a perfect matching iff 𝑂 𝑇

≥ |𝑇| for each 𝑇 ⊆ 𝑊

• Proof (reverse direction, via network flow):

➢ 𝑑𝑏𝑞 𝐵, 𝐶 = 𝑉 ∩ 𝐶 + 𝑊 ∩ 𝐵 < 𝑜 = 𝑉 ➢ So 𝑊 ∩ 𝐵 < |𝑉 ∩ 𝐵| ➢ But 𝑂 𝑉 ∩ 𝐵 ⊆ 𝑊 ∩ 𝐵 because the cut doesn’t include any

∞ edges

➢ So 𝑂 𝑉 ∩ 𝐵

≤ 𝑊 ∩ 𝐵 < |𝑉 ∩ 𝐵|. ∎

Some Notes

373F20 - Nisarg Shah 30

• Runtime for bipartite perfect matching

➢ 1955: 𝑃(𝑛𝑜) → Ford-Fulkerson ➢ 1973: 𝑃 𝑛 𝑜 → blocking flow (Hopcroft-Karp, Karzanov) ➢ 2004: 𝑃 𝑜2.378 → fast matrix multiplication (Mucha–

Sankowsi)

➢ 2013: ෨

𝑃 𝑛

Τ 10 7 → electrical flow (Mądry)

➢ Best running time is still an open question

• Nonbipartite graphs

➢ Hall’s theorem → Tutte’s theorem ➢ 1965: 𝑃(𝑜4) → Blossom algorithm (Edmonds) ➢ 1980/1994: 𝑃 𝑛 𝑜 → Micali-Vazirani

Edge-Disjoint Paths

373F20 - Nisarg Shah 31

• Problem

➢ Given a directed graph 𝐻 = (𝑊, 𝐹), two nodes 𝑡 and 𝑢,

find the maximum number of edge-disjoint 𝑡 → 𝑢 paths

➢ Two 𝑡 → 𝑢 paths 𝑄 and 𝑄′ are edge-disjoint if they don’t

share an edge

Edge-Disjoint Paths

373F20 - Nisarg Shah 32

• Application:

➢ Communication networks

• Max-flow formulation

➢ Assign unit capacity on all edges

Edge-Disjoint Paths

373F20 - Nisarg Shah 33

• Theorem:

➢ There is 1-1 correspondence between sets of 𝑙 edge-

disjoint 𝑡 → 𝑢 paths and integral flows of value 𝑙

• Proof (paths → flow)

➢ Let 𝑄

1, … , 𝑄𝑙 be a set of 𝑙 edge-disjoint 𝑡 → 𝑢 paths

➢ Define flow 𝑔 where 𝑔 𝑓 = 1 whenever 𝑓 ∈ 𝑄𝑗 for some

𝑗, and 0 otherwise

➢ Since paths are edge-disjoint, flow conservation and

capacity constraints are satisfied

➢ Unique integral flow of value 𝑙

Edge-Disjoint Paths

373F20 - Nisarg Shah 34

• Theorem:

➢ There is 1-1 correspondence between 𝑙 edge-disjoint

𝑡 → 𝑢 paths and integral flows of value 𝑙

• Proof (flow → paths)

➢ Let 𝑔 be an integral flow of value 𝑙 ➢ 𝑙 outgoing edges from 𝑡 have unit flow ➢ Pick one such edge (𝑡, 𝑣1)

• By flow conservation, 𝑣1 must have unit outgoing flow (which we

haven’t used up yet).

• Pick such an edge and continue building a path until you hit 𝑢

➢ Repeat this for the other 𝑙 − 1 edges coming out of 𝑡

with unit flow. ∎

Edge-Disjoint Paths

373F20 - Nisarg Shah 35

• Maximum number of edge-disjoint 𝑡 → 𝑢 paths

➢ Equals max flow in this network ➢ By max-flow min-cut theorem, also equals minimum cut ➢ Exercise: minimum cut = minimum number of edges we

need to delete to disconnect 𝑡 from 𝑢

• Hint: Show each direction separately (≤ and ≥)

Edge-Disjoint Paths

373F20 - Nisarg Shah 36

• Exercise!

➢ Show that to compute the maximum number of edge-

disjoint 𝑡-𝑢 paths in an undirected graph, you can create a directed flow network by adding each undirected edge in both directions and setting all capacities to 1

• Menger’s Theorem

➢ In any directed/undirected graph, the maximum number

• f edge-disjoint (resp. vertex-disjoint) 𝑡 → 𝑢 paths equals

the minimum number of edges (resp. vertices) whose removal disconnects 𝑡 and 𝑢

Multiple Sources/Sinks

373F20 - Nisarg Shah 37

• Problem

➢ Given a directed graph 𝐻 = (𝑊, 𝐹) with edge capacities

𝑑: 𝐹 → ℕ, sources 𝑡1, … , 𝑡𝑙 and sinks 𝑢1, … , 𝑢ℓ, find the maximum total flow from sources to sinks.

Multiple Sources/Sinks

373F20 - Nisarg Shah 38

• Network flow formulation

➢ Add a new source 𝑡, edges from 𝑡 to each 𝑡𝑗 with ∞ capacity ➢ Add a new sink 𝑢, edges from each 𝑢𝑘 to 𝑢 with ∞ capacity ➢ Find max-flow from 𝑡 to 𝑢 ➢ Claim: 1 − 1 correspondence between flows in two networks

Circulation

373F20 - Nisarg Shah 39

• Input

➢ Directed graph 𝐻 = (𝑊, 𝐹) ➢ Edge capacities 𝑑 ∶ 𝐹 → ℕ ➢ Node demands 𝑒 ∶ 𝑊 → ℤ

• Output

➢ Some circulation 𝑔 ∶ 𝐹 → ℕ satisfying

• For each 𝑓 ∈ 𝐹 : 0 ≤ 𝑔 𝑓 ≤ 𝑑(𝑓)
• For each 𝑤 ∈ 𝑊 : σ𝑓 entering 𝑤 𝑔(𝑤) − σ𝑓 leaving 𝑤 𝑔 𝑤 = 𝑒(𝑤)

➢ Note that you need σ𝑤:𝑒 𝑤 >0 𝑒(𝑤) = σ𝑤:𝑒 𝑤 <0 −𝑒(𝑤) ➢ What are demands?

Circulation

373F20 - Nisarg Shah 40

• Demand at 𝑤 = amount of flow you need to take
• ut at node 𝑤

➢ 𝑒 𝑤 > 0 : You need to take some flow out at 𝑤

• So there should be 𝑒(𝑤) more incoming flow than outgoing flow
• “Demand node”

➢ 𝑒 𝑤 < 0 : You need to put some flow in at 𝑤

• So there should be 𝑒 𝑤

more outgoing flow than incoming flow

• “Supply node”

➢ 𝑒 𝑤 = 0 : Node has flow conservation

• Equal incoming and outgoing flows
• “Transshipment node”

Circulation

373F20 - Nisarg Shah 41

• Example

Circulation

373F20 - Nisarg Shah 42

• Network-flow formulation 𝐻′

➢ Add a new source 𝑡 and a new sink 𝑢 ➢ For each “supply” node 𝑤 with 𝑒 𝑤 < 0, add edge (𝑡, 𝑤)

with capacity −𝑒(𝑤)

➢ For each “demand” node 𝑤 with 𝑒 𝑤 > 0, add edge

(𝑤, 𝑢) with capacity 𝑒(𝑤)

• Claim: 𝐻 has a circulation iff 𝐻′ has max flow of

value σ𝑤:𝑒 𝑤 >0 𝑒(𝑤) = σ𝑤:𝑒 𝑤 <0 −𝑒(𝑤)

Circulation

373F20 - Nisarg Shah 43

• Example

Circulation

373F20 - Nisarg Shah 44

• Example

Circulation with Lower Bounds

373F20 - Nisarg Shah 45

• Input

➢ Directed graph 𝐻 = (𝑊, 𝐹) ➢ Edge capacities 𝑑 ∶ 𝐹 → ℕ and lower bounds ℓ ∶ 𝐹 → ℕ ➢ Node demands 𝑒 ∶ 𝑊 → ℤ

• Output

➢ Some circulation 𝑔 ∶ 𝐹 → ℕ satisfying

• For each 𝑓 ∈ 𝐹 : ℓ(𝑓) ≤ 𝑔 𝑓 ≤ 𝑑(𝑓)
• For each 𝑤 ∈ 𝑊 : σ𝑓 entering 𝑤 𝑔(𝑤) − σ𝑓 leaving 𝑤 𝑔 𝑤 = 𝑒(𝑤)

➢ Note that you still need σ𝑤:𝑒 𝑤 >0 𝑒(𝑤) = σ𝑤:𝑒 𝑤 <0 −𝑒(𝑤)

Circulation with Lower Bounds

373F20 - Nisarg Shah 46

• Transform to circulation without lower bounds

➢ Do the following operation to each edge

• Claim: Circulation in 𝐻 iff circulation in 𝐻′

➢ Proof sketch: 𝑔(𝑓) gives a valid circulation in 𝐻 iff

𝑔 𝑓 − ℓ(𝑓) gives a valid circulation in 𝐻′

Survey Design

373F20 - Nisarg Shah 47

• Problem

➢ We want to design a survey about 𝑛 products

• We have one question in mind for each product
• Need to ask product 𝑘’s question to between 𝑞𝑘 and 𝑞𝑘

′ consumers

➢ There are a total of 𝑜 consumers

• Consumer 𝑗 owns a subset of products 𝑃𝑗
• We can ask consumer 𝑗 questions about only these products
• We want to ask consumer 𝑗 between 𝑑𝑗 and 𝑑𝑗

′ questions

➢ Is there a survey meeting all these requirements?

Survey Design

373F20 - Nisarg Shah 48

• Bipartite matching is a special case

➢ 𝑑𝑗 = 𝑑𝑗

′ = 𝑞𝑘 = 𝑞𝑘 ′ = 1 for all 𝑗 and 𝑘

• Formulate as circulation with lower bounds

➢ Create a network with special nodes 𝑡 and 𝑢 ➢ Edge from 𝑡 to each consumer 𝑗 with flow ∈ [𝑑𝑗, 𝑑𝑗

′]

➢ Edge from each consumer 𝑗 to each product 𝑘 ∈ 𝑃𝑗 with

flow ∈ [0,1]

➢ Edge from each product 𝑘 to 𝑢 with flow ∈ [𝑞𝑘, 𝑞𝑘

′]

➢ Edge from 𝑢 to 𝑡 with flow in [0, ∞] ➢ All demands and supplies are 0

Survey Design

373F20 - Nisarg Shah 49

• Max-flow formulation:

➢ Feasible survey iff feasible circulation in this network

Image Segmentation

373F20 - Nisarg Shah 50

• Foreground/background segmentation

➢ Given an image, separate “foreground” from “background”

• Here’s the power of PowerPoint (or the lack thereof)

Remove background

Image Segmentation

373F20 - Nisarg Shah 51

• Foreground/background segmentation

➢ Given an image, separate “foreground” from “background”

• Here’s what remove.bg gets using AI

Remove background

Image Segmentation

373F20 - Nisarg Shah 52

• Informal problem

➢ Given an image (2D array of pixels), and likelihood

estimates of different pixels being foreground/background, label each pixel as foreground or background

➢ Want to prevent having too many

neighboring pixels where one is labeled foreground but the other is labeled background

Image Segmentation

373F20 - Nisarg Shah 53

• Input

➢ An image (2D array of pixels) ➢ 𝑏𝑗 = likelihood of pixel 𝑗 being in foreground ➢ 𝑐𝑗 = likelihood of pixel 𝑗 being in background ➢ 𝑞𝑗,𝑘 = penalty for “separating” pixels 𝑗 and 𝑘 (i.e. labeling

• ne of them as foreground and the other as background)
• Output

➢ Label each pixel as “foreground” or “background” ➢ Minimize “total penalty”

• Want it to be high if 𝑏𝑗 is high but 𝑗 is labeled background, 𝑐𝑗 is high

but 𝑗 is labeled foreground, or 𝑞𝑗,𝑘 is high but 𝑗 and 𝑘 are separated

Image Segmentation

373F20 - Nisarg Shah 54

• Recall

➢ 𝑏𝑗 = likelihood of pixels 𝑗 being in foreground ➢ 𝑐𝑗 = likelihood of pixels 𝑗 being in background ➢ 𝑞𝑗,𝑘 = penalty for separating pixels 𝑗 and 𝑘 ➢ Let 𝐹 = pairs of neighboring pixels

• Output

➢ Minimize total penalty

• 𝐵 = set of pixels labeled foreground
• 𝐶 = set of pixels labeled background
• Penalty =

෍

𝑗∈𝐵

𝑐𝑗 + ෍

𝑘∈𝐶

𝑏𝑘 + ෍

𝑗,𝑘 ∈𝐹 𝐵∩ 𝑗,𝑘 =1

𝑞𝑗,𝑘

Image Segmentation

373F20 - Nisarg Shah 55

• Formulate as a min-cut problem

➢ Want to divide the set of pixels 𝑊 into (𝐵, 𝐶) to minimize

෍

𝑗∈𝐵

𝑐𝑗 + ෍

𝑘∈𝐶

𝑏𝑘 + ෍

𝑗,𝑘 ∈𝐹 𝐵∩ 𝑗,𝑘 =1

𝑞𝑗,𝑘

➢ Nodes:

• source 𝑡, target 𝑢, and 𝑤𝑗 for each pixel 𝑗

➢ Edges:

• (𝑡, 𝑤𝑗) with capacity 𝑏𝑗 for all 𝑗
• (𝑤𝑗, 𝑢) with capacity 𝑐𝑗 for all 𝑗
• (𝑤𝑗, 𝑤𝑘) and (𝑤𝑘, 𝑤𝑗) with capacity 𝑞𝑗,𝑘 each for all neighboring (𝑗, 𝑘)

Image Segmentation

373F20 - Nisarg Shah 56

• Formulate as min-cut problem

➢ Here’s what the network looks like

Image Segmentation

373F20 - Nisarg Shah 57

➢ Consider the min-cut (𝐵, 𝐶)

𝑑𝑏𝑞 𝐵, 𝐶 = ෍

𝑗∈𝐵

𝑐𝑗 + ෍

𝑘∈𝐶

𝑏𝑘 + ෍

𝑗,𝑘 ∈𝐹 𝑗∈𝐵,𝑘∈𝐶

𝑞𝑗,𝑘

➢ Exactly what we want to minimize!

If 𝑗 and 𝑘 are labeled differently, it will add 𝑞𝑗,𝑘 exactly once

Image Segmentation

373F20 - Nisarg Shah 58

• GrabCut [Rother-Kolmogorov-Blake 2004]

Profit Maximization (Yeaa…!)

373F20 - Nisarg Shah 59

• Problem

➢ There are 𝑜 tasks ➢ Performing task 𝑗 generates a profit of 𝑞𝑗

• We allow 𝑞𝑗 < 0 (i.e. performing task 𝑗 may be costly)

➢ There is a set 𝐹 of precedence relations

• 𝑗, 𝑘 ∈ 𝐹 indicates that if we perform 𝑗, we must also perform 𝑘
• Goal

➢ Find a subset of tasks 𝑇 which, subject to the precedence

constraints, maximizes 𝑞𝑠𝑝𝑔𝑗𝑢 𝑇 = σ𝑗∈𝑇 𝑞𝑗

Profit Maximization

373F20 - Nisarg Shah 60

• We can represent the input as a graph

➢ Nodes = tasks, node weights = profits, ➢ Edges = precedence constraints ➢ Goal: find a subset of nodes 𝑇 with highest total weight

s.t. if 𝑗 ∈ 𝑇 and 𝑗, 𝑘 ∈ 𝐹, then 𝑘 ∈ 𝑇 as well

• 1

3

• 4
• 2
• 3

7 3

• 9

Profit Maximization

373F20 - Nisarg Shah 61

• Want to formulate as a min-cut

➢ Add source 𝑡 and target 𝑢 ➢ min-cut (𝐵, 𝐶) ⇒ want desired solution to be 𝑇 = 𝐵 ∖ {𝑡} ➢ Goals:

• 𝑑𝑏𝑞(𝐵, 𝐶) should nicely relate to 𝑞𝑠𝑝𝑔𝑗𝑢(𝑇)
• Precedence constraints must be respected
• “Hard” constraints are usually enforced using infinite capacity edges
• Construction:

➢ Add each 𝑗, 𝑘 ∈ 𝐹 with infinite capacity ➢ For each 𝑗:

• If 𝑞𝑗 > 0, add (𝑡, 𝑗) with capacity 𝑞𝑗
• If 𝑞𝑗 < 0, add (𝑗, 𝑢) with capacity −𝑞𝑗

Profit Maximization

373F20 - Nisarg Shah 62

• 2

3

• 3
• 1

7 3

Profit Maximization

373F20 - Nisarg Shah 63

• 2

3

• 3
• 1

7 3 s t 3 7 3 3 1 2 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞

Profit Maximization

373F20 - Nisarg Shah 64

A possible cut

QUESTION: What is the capacity of this cut?

• 2

3

• 3
• 1

7 3 s t 3 7 3 3 1 2 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞

Profit Maximization

373F20 - Nisarg Shah 65

Exercise: Show that…

• 1. A finite capacity cut exists.
• 2. If 𝑑𝑏𝑞(𝐵, 𝐶) is finite, then 𝐵\ 𝑡 is a valid solution;
• 3. Minimizing 𝑑𝑏𝑞(𝐵, 𝐶) maximizes 𝑞𝑠𝑝𝑔𝑗𝑢(𝐵\ 𝑡 )
• Show that 𝑑𝑏𝑞 𝐵, 𝐶 = constant − 𝑞𝑠𝑝𝑔𝑗𝑢 𝐵\ 𝑡

, where the constant is independent of the choice of (𝐵, 𝐶)