CS 473: Algorithms, Fall 2016
Dynamic Programming on Trees
Lecture 4
September 2, 2016
Chandra & Ruta (UIUC) CS473 1 Fall 2016 1 / 46
Dynamic Programming on Trees Lecture 4 September 2, 2016 Chandra - - PowerPoint PPT Presentation
CS 473: Algorithms, Fall 2016 Dynamic Programming on Trees Lecture 4 September 2, 2016 Chandra & Ruta (UIUC) CS473 1 Fall 2016 1 / 46 What is Dynamic Programming? Every recursion can be memoized. Automatic memoization does not help us
September 2, 2016
Chandra & Ruta (UIUC) CS473 1 Fall 2016 1 / 46
Every recursion can be memoized. Automatic memoization does not help us understand whether the resulting algorithm is efficient or not.
A recursion that when memoized leads to an efficient algorithm. Key Questions: Given a recursive algorithm, how do we analyze the complexity when it is memoized? How do we recognize whether a problem admits a dynamic programming based efficient algorithm? How do we further optimize time and space of a dynamic programming based algorithm?
Chandra & Ruta (UIUC) CS473 2 Fall 2016 2 / 46
1
Come up with a recursive algorithm to solve problem
2
Understand the structure/number of the subproblems generated by recursion
3
Memoize the recursion
set up compact notation for subproblems set up a data structure for storing subproblems
4
Iterative algorithm
Understand dependency graph on subproblems Pick an evaluation order (any topological sort of the dependency dag)
5
Analyze time and space
6
Optimize
Chandra & Ruta (UIUC) CS473 3 Fall 2016 3 / 46
Fact: Many graph optimization problems are NP-Hard Fact: The same graph optimization problems are in P on trees. Why?
Chandra & Ruta (UIUC) CS473 4 Fall 2016 4 / 46
Fact: Many graph optimization problems are NP-Hard Fact: The same graph optimization problems are in P on trees. Why? A significant reason: DP algorithm based on decomposability Powerful methodology for graph algorithms via a formal notion of decomposability called treewidth (beyond the scope of this class)
Chandra & Ruta (UIUC) CS473 4 Fall 2016 4 / 46
Given undirected graph G = (V , E) a subset of nodes S ✓ V is an independent set (also called a stable set) if for there are no edges between nodes in S. That is, if u, v 2 S then (u, v) 62 E.
A B C D E F
Some independent sets in graph above: {D}, {A, C}, {B, E, F}
Chandra & Ruta (UIUC) CS473 5 Fall 2016 5 / 46
Input Graph G = (V , E) Goal Find maximum sized independent set in G
A B C D E F
Chandra & Ruta (UIUC) CS473 6 Fall 2016 6 / 46
Input Graph G = (V , E), weights w(v) 0 for v 2 V Goal Find maximum weight independent set in G
A B C D E F
20 5 2 2 10 15
Chandra & Ruta (UIUC) CS473 7 Fall 2016 7 / 46
1
No one knows an efficient (polynomial time) algorithm for this problem
2
Problem is NP-Hard and it is believed that there is no polynomial time algorithm
Try all subsets of vertices.
Chandra & Ruta (UIUC) CS473 8 Fall 2016 8 / 46
Let V = {v1, v2, . . . , vn}. For a vertex u let N(u) be its neighbors.
Chandra & Ruta (UIUC) CS473 9 Fall 2016 9 / 46
Let V = {v1, v2, . . . , vn}. For a vertex u let N(u) be its neighbors.
v1: vertex in the graph. One of the following two cases is true Case 1 v1 is in some maximum independent set. Case 2 v1 is in no maximum independent set. We can try both cases to “reduce” the size of the problem
Chandra & Ruta (UIUC) CS473 9 Fall 2016 9 / 46
Let V = {v1, v2, . . . , vn}. For a vertex u let N(u) be its neighbors.
v1: vertex in the graph. One of the following two cases is true Case 1 v1 is in some maximum independent set. Case 2 v1 is in no maximum independent set. We can try both cases to “reduce” the size of the problem G1 = G v1 obtained by removing v1 and incident edges from G G2 = G v1 N(v1) obtained by removing N(v1) [ v1 from G MIS(G) = max{MIS(G1), MIS(G2) + w(v1)}
Chandra & Ruta (UIUC) CS473 9 Fall 2016 9 / 46
RecursiveMIS(G):
if G is empty then Output 0
a = RecursiveMIS(G v1) b = w(v1) + RecursiveMIS(G v1 N(vn)) Output max(a, b)
Chandra & Ruta (UIUC) CS473 10 Fall 2016 10 / 46
Chandra & Ruta (UIUC) CS473 11 Fall 2016 11 / 46
..for Maximum Independent Set
Running time: T(n) = T(n 1) + T ⇣ n 1 deg(v1) ⌘ + O(1 + deg(v1)) where deg(v1) is the degree of v1. T(0) = T(1) = 1 is base case. Worst case is when deg(v1) = 0 when the recurrence becomes T(n) = 2T(n 1) + O(1) Solution to this is T(n) = O(2n).
Chandra & Ruta (UIUC) CS473 12 Fall 2016 12 / 46
We can memoize the recursive algorithm. Question: Does it lead to an efficient algorithm?
Chandra & Ruta (UIUC) CS473 13 Fall 2016 13 / 46
We can memoize the recursive algorithm. Question: Does it lead to an efficient algorithm? What is number of subproblems if started on graph with n nodes? Exercise: Show that even when G is a cycle the number of subproblems is exponential in n.
Chandra & Ruta (UIUC) CS473 13 Fall 2016 13 / 46
Chandra & Ruta (UIUC) CS473 14 Fall 2016 14 / 46
Input Tree T = (V , E) and weights w(v) 0 for each v 2 V Goal Find maximum weight independent set in T
r a b c d e f g h i j 10 5 8 4 4 9 2 7 8 11 3
Maximum weight independent set in above tree: ??
Chandra & Ruta (UIUC) CS473 15 Fall 2016 15 / 46
For an arbitrary graph G:
1
Number vertices as v1, v2, . . . , vn
2
Find recursively optimum solutions without vn (recurse on G vn) and with vn (recurse on G vn N(vn) & include vn).
3
Saw that if graph G is arbitrary there was no good ordering that resulted in a small number of subproblems. What about a tree?
Chandra & Ruta (UIUC) CS473 16 Fall 2016 16 / 46
For an arbitrary graph G:
1
Number vertices as v1, v2, . . . , vn
2
Find recursively optimum solutions without vn (recurse on G vn) and with vn (recurse on G vn N(vn) & include vn).
3
Saw that if graph G is arbitrary there was no good ordering that resulted in a small number of subproblems. What about a tree? Natural candidate for vn is root r of T?
Chandra & Ruta (UIUC) CS473 16 Fall 2016 16 / 46
Natural candidate for vn is root r of T? Let O be an optimum solution to the whole problem. Case r 62 O : Then O contains an optimum solution for each subtree of T hanging at a child of r.
Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 46
Natural candidate for vn is root r of T? Let O be an optimum solution to the whole problem. Case r 62 O : Then O contains an optimum solution for each subtree of T hanging at a child of r. Case r 2 O : None of the children of r can be in O. O {r} contains an optimum solution for each subtree of T hanging at a grandchild of r.
Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 46
Natural candidate for vn is root r of T? Let O be an optimum solution to the whole problem. Case r 62 O : Then O contains an optimum solution for each subtree of T hanging at a child of r. Case r 2 O : None of the children of r can be in O. O {r} contains an optimum solution for each subtree of T hanging at a grandchild of r. Subproblems? Subtrees of T rooted at nodes in T.
Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 46
Natural candidate for vn is root r of T? Let O be an optimum solution to the whole problem. Case r 62 O : Then O contains an optimum solution for each subtree of T hanging at a child of r. Case r 2 O : None of the children of r can be in O. O {r} contains an optimum solution for each subtree of T hanging at a grandchild of r. Subproblems? Subtrees of T rooted at nodes in T. How many of them?
Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 46
Natural candidate for vn is root r of T? Let O be an optimum solution to the whole problem. Case r 62 O : Then O contains an optimum solution for each subtree of T hanging at a child of r. Case r 2 O : None of the children of r can be in O. O {r} contains an optimum solution for each subtree of T hanging at a grandchild of r. Subproblems? Subtrees of T rooted at nodes in T. How many of them? O(n)
Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 46
Chandra & Ruta (UIUC) CS473 18 Fall 2016 18 / 46
T(u): subtree of T hanging at node u OPT(u): max weighted independent set value in T(u) OPT(u) =
Chandra & Ruta (UIUC) CS473 19 Fall 2016 19 / 46
T(u): subtree of T hanging at node u OPT(u): max weighted independent set value in T(u) OPT(u) = max (P
v child of u OPT(v),
w(u) + P
v grandchild of u OPT(v)
Chandra & Ruta (UIUC) CS473 19 Fall 2016 19 / 46
1
Compute OPT(u) bottom up. To evaluate OPT(u) need to have computed values of all children and grandchildren of u
2
What is an ordering of nodes of a tree T to achieve above?
Chandra & Ruta (UIUC) CS473 20 Fall 2016 20 / 46
1
Compute OPT(u) bottom up. To evaluate OPT(u) need to have computed values of all children and grandchildren of u
2
What is an ordering of nodes of a tree T to achieve above? Post-order traversal of a tree.
Chandra & Ruta (UIUC) CS473 20 Fall 2016 20 / 46
MIS-Tree(T): Let v1, v2, . . . , vn be a post-order traversal of nodes of T
for i = 1 to n do
M[vi] = max P
vj child of vi M[vj],
w(vi) + P
vj grandchild of vi M[vj]
!
return M[vn] (* Note:
vn is the root of T *)
Chandra & Ruta (UIUC) CS473 21 Fall 2016 21 / 46
MIS-Tree(T): Let v1, v2, . . . , vn be a post-order traversal of nodes of T
for i = 1 to n do
M[vi] = max P
vj child of vi M[vj],
w(vi) + P
vj grandchild of vi M[vj]
!
return M[vn] (* Note:
vn is the root of T *)
Space:
Chandra & Ruta (UIUC) CS473 21 Fall 2016 21 / 46
MIS-Tree(T): Let v1, v2, . . . , vn be a post-order traversal of nodes of T
for i = 1 to n do
M[vi] = max P
vj child of vi M[vj],
w(vi) + P
vj grandchild of vi M[vj]
!
return M[vn] (* Note:
vn is the root of T *)
Space: O(n) to store the value at each node of T Running time:
Chandra & Ruta (UIUC) CS473 21 Fall 2016 21 / 46
MIS-Tree(T): Let v1, v2, . . . , vn be a post-order traversal of nodes of T
for i = 1 to n do
M[vi] = max P
vj child of vi M[vj],
w(vi) + P
vj grandchild of vi M[vj]
!
return M[vn] (* Note:
vn is the root of T *)
Space: O(n) to store the value at each node of T Running time:
1
Naive bound: O(n2) since each M[vi] evaluation may take O(n) time and there are n evaluations.
Chandra & Ruta (UIUC) CS473 21 Fall 2016 21 / 46
MIS-Tree(T): Let v1, v2, . . . , vn be a post-order traversal of nodes of T
for i = 1 to n do
M[vi] = max P
vj child of vi M[vj],
w(vi) + P
vj grandchild of vi M[vj]
!
return M[vn] (* Note:
vn is the root of T *)
Space: O(n) to store the value at each node of T Running time:
1
Naive bound: O(n2) since each M[vi] evaluation may take O(n) time and there are n evaluations.
2
Better bound: O(n). A value M[vj] is accessed only by its parent and grand parent.
Chandra & Ruta (UIUC) CS473 21 Fall 2016 21 / 46
Chandra & Ruta (UIUC) CS473 22 Fall 2016 22 / 46
Each node (including the root) is a separator!
Given a graph G = (V , E) a set of nodes S ⇢ V is a separator for G if G S has at least two connected components.
Chandra & Ruta (UIUC) CS473 23 Fall 2016 23 / 46
Each node (including the root) is a separator!
Given a graph G = (V , E) a set of nodes S ⇢ V is a separator for G if G S has at least two connected components.
S is a balanced separator if each connected component of G S has at most 2|V (G)|/3 nodes.
Chandra & Ruta (UIUC) CS473 23 Fall 2016 23 / 46
Each node (including the root) is a separator!
Given a graph G = (V , E) a set of nodes S ⇢ V is a separator for G if G S has at least two connected components.
S is a balanced separator if each connected component of G S has at most 2|V (G)|/3 nodes. Exercise: Prove that every tree T has a balanced separator consisting of a single node.
Chandra & Ruta (UIUC) CS473 23 Fall 2016 23 / 46
Chandra & Ruta (UIUC) CS473 24 Fall 2016 24 / 46
Given undirected graph G = (V , E) a subset of nodes S ✓ V is a dominating set if for all v 2 V , either v 2 S or a neighbor of v is in S.
A B C D E F
Some dominating sets in graph above: {A, B, C, D, E, F},
Chandra & Ruta (UIUC) CS473 25 Fall 2016 25 / 46
Input Graph G = (V , E), weights w(v) 0 for v 2 V Goal Find minimum weight dominating set in G
A B C D E F
20 5 2 2 10 15
Chandra & Ruta (UIUC) CS473 26 Fall 2016 26 / 46
Input Graph G = (V , E), weights w(v) 0 for v 2 V Goal Find minimum weight dominating set in G
A B C D E F
20 5 2 2 10 15
NP-Hard problem
Chandra & Ruta (UIUC) CS473 26 Fall 2016 26 / 46
Input Tree T = (V , E) and weights w(v) 0 for each v 2 V Goal Find minimum weight dominating set in T
r a b c d e f g h i j 10 5 8 4 4 9 2 7 8 11 3
Minimum weight dominating set in above tree: ??
Chandra & Ruta (UIUC) CS473 27 Fall 2016 27 / 46
r is root of T. Let O be an optimum solution for T. Case r 62 O : Then O must contain some child of r. Which one?
Chandra & Ruta (UIUC) CS473 28 Fall 2016 28 / 46
r is root of T. Let O be an optimum solution for T. Case r 62 O : Then O must contain some child of r. Which one? Case r 2 O : None of the children of r need to be in O because r can dominate them. However, they may have to be.
Chandra & Ruta (UIUC) CS473 28 Fall 2016 28 / 46
r is root of T. Let O be an optimum solution for T. Case r 62 O : Then O must contain some child of r. Which one? Case r 2 O : None of the children of r need to be in O because r can dominate them. However, they may have to be. In both cases it is not feasible to express |O| easily as optimum solution values of children or descendants of r. Removing r decomposes T into subtrees rooted at children of r. However, not easy to decompose problem structure recursively. Problems at children of r are dependent. Need to introduce additional variable(s).
Chandra & Ruta (UIUC) CS473 28 Fall 2016 28 / 46
Let u1, u2, . . . , uk be children of root r of T What “information” do Tu1, . . . , Tuk need to know about r’s status in an optimum solution in order to become “independent”
Chandra & Ruta (UIUC) CS473 29 Fall 2016 29 / 46
Let u1, u2, . . . , uk be children of root r of T What “information” do Tu1, . . . , Tuk need to know about r’s status in an optimum solution in order to become “independent” Whether r is included in the solution If r is not included then which of the children is going to dominate it. Equivalently, Tui needs to know whether it should cover r or some other child will.
Chandra & Ruta (UIUC) CS473 29 Fall 2016 29 / 46
u: node in tree pi: boolean variable to indicate whether parent is in solution. pi = 0 means parent is not included. pi = 1 means it is included. cp: boolean variable to indicate whether node needed to cover
means not needed. OPT(u, pi, cp): value of minimum dominating set in Tu with booleans pi and cp with meaning above. OPT(r, 0, 0): value of minimum dominating set in T OPT(u, 1, 0): value of min dominating set in Tu with parent of u included no need to cover parent.
Chandra & Ruta (UIUC) CS473 30 Fall 2016 30 / 46
Can we express OPT(u, pi, cp) recursively via children of u?
Chandra & Ruta (UIUC) CS473 31 Fall 2016 31 / 46
Can we express OPT(u, pi, cp) recursively via children of u? First consider OPT(u, 0, 0) which is the value of a minimum dominating set in Tu where we assume that u’s parent is not included and u does not need to cover its parent. Let Cu be children of u. Case u is included: Then u does not need to covered by any child and we recurse on children with u being included. OPT(u, 0, 0) = w(u) + P
v2Cu OPT(v, 1, 0)
Case u is not included: Then u needs to be covered by some child. We do a min over all children. OPT(u, 0, 0) = minv2Cu(OPT(v, 0, 1) + P
v 02Cuv OPT(v 0, 0, 0))
Since one of these cases has to be true, we take the min of the values in the above two cases to compute OPT(u, 0, 0).
Chandra & Ruta (UIUC) CS473 31 Fall 2016 31 / 46
Consider OPT(u, 1, 0) which is the value of a minimum dominating set in Tu where we assume that u’s parent is included and u does not need to cover its parent. Let Cu be children of u. Case u is included: Then u does not need to covered by any child and we recurse on children with u being included. OPT(u, 1, 0) = w(u) + P
v2Cu OPT(v, 1, 0)
Case u is not included: Since u’s parent is included u does not need to be covered by its children. Thus we have, OPT(u, 1, 0) = P
v2Cu OPT(v, 0, 0)
Since one of these cases has to be true, we take the min of the values in the above two cases to compute OPT(u, 0, 0).
Chandra & Ruta (UIUC) CS473 32 Fall 2016 32 / 46
Consider OPT(u, 1, 0) which is the value of a minimum dominating set in Tu where we assume that u’s parent is included and u does not need to cover its parent. Let Cu be children of u. Case u is included: Then u does not need to covered by any child and we recurse on children with u being included. OPT(u, 1, 0) = w(u) + P
v2Cu OPT(v, 1, 0)
Case u is not included: Since u’s parent is included u does not need to be covered by its children. Thus we have, OPT(u, 1, 0) = P
v2Cu OPT(v, 0, 0)
Since one of these cases has to be true, we take the min of the values in the above two cases to compute OPT(u, 0, 0). Caution: Not including u may appear to be always advantageous but it is not true.
Chandra & Ruta (UIUC) CS473 32 Fall 2016 32 / 46
Consider OPT(u, 0, 1) which is the value of a minimum dominating set in Tu where we assume that u’s parent is not included and u needs to cover its parent. Let Cu be children of u. Case u is included: Then u does not need to covered by any child and we recurse on children with u being included. OPT(u, 0, 1) = w(u) + P
v2Cu OPT(v, 1, 0)
Case u is not included: This does not arise because u has to cover its parent.
Chandra & Ruta (UIUC) CS473 33 Fall 2016 33 / 46
Consider OPT(u, 1, 1) which is the value of a minimum dominating set in Tu where we assume that u’s parent is included and u needs to cover its parent. This subproblem does not make sense since if u’s parent is included then u does not need to cover it. In other words it suffices to only consider the subproblems OPT(u, 0, 0), OPT(u, 1, 0), OPT(u, 0, 1).
Chandra & Ruta (UIUC) CS473 34 Fall 2016 34 / 46
Leaves are base cases. If u is a leaf. OPT(u, 0, 0) = w(u) OPT(u, 1, 0) = 0 OPT(u, 0, 1) = w(u)
Chandra & Ruta (UIUC) CS473 35 Fall 2016 35 / 46
Minimum weight dominating set value in T is OPT(r, 0, 0) To compute OPT(r, 0, 0) we need to compute recursively OPT(u, 0, 0), OPT(u, 1, 0), OPT(u, 0, 1) for all u 2 T. Thus number of subproblems is O(n). Can do this bottom up from leaves to root.
Chandra & Ruta (UIUC) CS473 36 Fall 2016 36 / 46
DominatingSet-Tree(T): Let v1, v2, . . . , vn be a post-order traversal of nodes of T Allocate array M[1..n, 0..1, 0..1] to store OPT(vi, pi, cp) values
for i = 1 to n do
Compute OPT(vi, 0, 0), OPT(vi, 1, 0) and OPT(vi, 0, 1) using values of children of vi stored in M,
Store computed values in M for use by parent of vi.
return OPT(vn, 0, 0) (* Note:
vn is the root of T *)
Exercise: Work out details and prove an O(n) time implementation.
Chandra & Ruta (UIUC) CS473 37 Fall 2016 37 / 46
To obtain recursive solution we introduced additional variables based on “information” needed to decompose Decomposition depends both on structure (trees decompose via separators) and objective function Subproblems and recursion are almost defined hand in hand
Chandra & Ruta (UIUC) CS473 38 Fall 2016 38 / 46
Chandra & Ruta (UIUC) CS473 39 Fall 2016 39 / 46
(Informal): A graph G = (V , E) is planar if it can be drawn in the plane without edges crossing. (More formal): G is planar if there is a mapping of φ : V ! R2 to distinct points in the 2-dimensional Euclidean plane and a mapping of each edge uv 2 E to a non-crossing curve ψ(uv) that connects φ(u) to φ(v) such that the curves corresponding to different edges intersect only at φ(V ). Planar graphs are very important in both theory and practice.
Chandra & Ruta (UIUC) CS473 40 Fall 2016 40 / 46
Many beautiful properties and theorems. Euler’s theorem. |E| 3|V | 6 for every planar graph. Hence there is always a node of degree at most 5. Thus planar graphs are 5-degenerate. Kuratowski’s theorem: G is planar iff it “excludes” K5 or K3,3 4-Color theorem. Every planar graph is 4-colorable. Planar Separator Theorem: Every planar graph G on n nodes has a balance separator of size O(pn). Properties are exploited to develop algorithms.
Chandra & Ruta (UIUC) CS473 41 Fall 2016 41 / 46
MIS in general graphs: best known algorithm runs in time O(1.889n). Believed that MIS in general graphs requires cn time for some fixed c > 1. However, can solve MIS in planar graphs in 2O(pn) time.
Chandra & Ruta (UIUC) CS473 42 Fall 2016 42 / 46
RecursiveDecomp(G): If |V (G)| n0 Output tree T with single root node containing V (G) Else Compute balanced separator S of G of size O( p |V (G)|) For each component Gi of G S do Ti =RecursiveDecomp(Gi) Create T with root r containing S Make each Ti a child of r Output T
Using linear-time algorithm to compute planar separator, algorithms run time is O(n).
Chandra & Ruta (UIUC) CS473 43 Fall 2016 43 / 46
Let T be the decomposition tree for planar G with n nodes T is a rooted tree where each tree-node v 2 V (T) there is a set Sv ✓ V (G) of G’s vertices associated with it. |Sv| = O(pn) for all v 2 V (T). Sv \ Su = ; for u 6= v and [v2V (T)Sv = V (G). For node v let S0
v = [v2TvSv be the nodes in sub-tree of T
rooted at v. Let Gv = G[S0
v] be the sub-graph of G induced
by S0
Depth of T is O(log n). |V (T) = O(n).
Chandra & Ruta (UIUC) CS473 44 Fall 2016 44 / 46