Dynamic Programming Has nothing to do with programming in the way - PDF document

Dynamic Programming • Has nothing to do with programming in the way we normally use that term Dynamic Programming – More like TV programming (scheduling) – Filling in a table Chapter 16 • Algorithm design technique CPTR 318 1 2 Longest Common Subsequence Longest Common Subsequence • Given two sequences • Given two sequences X = x 0 x 1 x 2 … x m-1 X = x 0 x 1 x 2 … x m-1 Y = y 0 y 1 y 2 … y n-1 Y = y 0 y 1 y 2 … y n-1 find a longest sequence common to both find a longest sequence common to both • Example: • Example: X = A B C B D A B X = A B C B D A B Y = B D C A B A Y = B D C A B A – B D A B – B D A B – B C A B – B C A B – B C B A – B C B A 3 4 Longest Common Subsequence Longest Common Subsequence • Given two sequences • Given two sequences X = x 0 x 1 x 2 … x m-1 X = x 0 x 1 x 2 … x m-1 Y = y 0 y 1 y 2 … y n-1 Y = y 0 y 1 y 2 … y n-1 find a longest sequence common to both find a longest sequence common to both • Example: • Example: X = A B C B D A B X = A B C B D A B Y = B D C A B A Y = B D C A B A – B D A B – B D A B – B C A B – B C A B – B C B A – B C B A 5 6 1

Simple Algorithm Brute Force Pseudocode • Brute force string LCS(string X[0..m-1], string Y[0..n-1]) { string result = ""; – Try all subsequences s of X to see if s is a if (|X| > 0 && |Y| > 0) { if (X[0] == Y[0]) subsequence of Y result = X[0] + LCS(X[1..m-1], Y[1, n-1]); – Select a longest such subsequence else result = maxlen(LCS(X, Y[1..n-1]), LCS(X[1..m-1], Y)); } return result; } 7 8 Brute Force Pseudocode Brute Force Pseudocode string LCS(string X[0..m-1], string Y[0..n-1]) { string LCS(string X[0..m-1], string Y[0..n-1]) { + here means + here means string result = ""; string result = ""; string string if (|X| > 0 && |Y| > 0) { if (|X| > 0 && |Y| > 0) { concatenation concatenation if (X[0] == Y[0]) if (X[0] == Y[0]) result = X[0] + LCS(X[1..m-1], Y[1, n-1]); result = X[0] + LCS(X[1..m-1], Y[1, n-1]); else else result = maxlen(LCS(X, Y[1..n-1]), LCS(X[1..m-1], Y)); result = maxlen(LCS(X, Y[1..n-1]), LCS(X[1..m-1], Y)); } } return result; return result; maxlen returns the longer of } } two strings 9 10 Brute Force Pseudocode Brute Force Pseudocode string LCS(string X[0..m-1], string Y[0..n-1]) { string LCS(string X[0..m-1], string Y[0..n-1]) { + here means string result = ""; string result = ""; string if (|X| > 0 && |Y| > 0) { if (|X| > 0 && |Y| > 0) { concatenation if (X[0] == Y[0]) if (X[0] == Y[0]) result = X[0] + LCS(X[1..m-1], Y[1, n-1]); result = X[0] + LCS(X[1..m-1], Y[1, n-1]); else else result = maxlen(LCS(X, Y[1..n-1]), LCS(X[1..m-1], Y)); result = maxlen(LCS(X, Y[1..n-1]), LCS(X[1..m-1], Y)); } } return result; maxlen returns return result; the longer of } } two strings “Absolute value” bars mean string length 11 12 2

Brute Force Complexity Dynamic Programming • Need to check each subsequence of X • Overlapping subproblems • Θ ( n ) to check a subsequence of X – The solution to a subproblem is recorded and can be used multiple times • There are Θ (2 m ) subsequences of X • Optimal substructure • Total time Θ ( n∙ 2 m ) – Optimal solutions of subproblems are used to compute the optimal solution to the overall problem 13 14 Table of Subsequence Lengths Build the DP Table • Consider length of LCS(X,Y) Table LCS_table(string X[0..m-1], string Y[0..n-1]) { Table C[0..m][0..n]; • Consider prefixes of X and Y for (int i = 0; i <= m; i++) C[i][0] = 0; • Build a table of prefix lengths for (int j = 0; j <= n; j++) C[0][j] = 0; – C[ i ][ j ] = | LCS(X[0.. i - 1], Y[0.. j - 1]) | for (int i = 0; i < m; i++) – C[ m -1][ n -1] = | LCS(X, Y) | for (int j = 0; j < n; j++) if (X[i] == Y[j]) • C[ i ][ j ] = C[ i - 1][ j - 1] + 1 , if X[ i ] = Y[ j ] C[i + 1][j + 1] = C[i][j] + 1; else or max(C[ i ][ j - 1] , C[ i -1][ j ] ), otherwise C[i + 1][j + 1] = max(C[i + 1][j], C[i][j + 1]); return C; } 15 16 Build the DP Table Build the DP Table Table LCS_table(string X[0..m-1], string Y[0..n-1]) { Table LCS_table(string X[0..m-1], string Y[0..n-1]) { Table C[0..m][0..n]; Table C[0..m][0..n]; for (int i = 0; i <= m; i++) for (int i = 0; i <= m; i++) C[i][0] = 0; C[i][0] = 0; Initialization for (int j = 0; j <= n; j++) for (int j = 0; j <= n; j++) Table construction C[0][j] = 0; C[0][j] = 0; for (int i = 0; i < m; i++) for (int i = 0; i < m; i++) for (int j = 0; j < n; j++) for (int j = 0; j < n; j++) if (X[i] == Y[j]) if (X[i] == Y[j]) C[i + 1][j + 1] = C[i][j] + 1; C[i + 1][j + 1] = C[i][j] + 1; else else C[i + 1][j + 1] = max(C[i + 1][j], C[i][j + 1]); C[i + 1][j + 1] = max(C[i + 1][j], C[i][j + 1]); return C; return C; } } 17 18 3

X Extract the Subsequence A B C B D A B string back_trace(Table C, string X, string Y, int i, int j) { 0 0 0 0 0 0 0 0 if (i < 0 || j < 0) return ""; B 0 0 1 1 1 1 1 1 else if (X[i] == Y[j]) return back_trace(C, X, Y, i - 1, j - 1) + X[i]; D 0 0 1 1 1 2 2 2 else Y if (C[i + 1][j] > C[i][j + 1]) C 0 0 1 2 2 2 2 2 return back_trace(C, X, Y, i, j - 1); else A 0 1 1 2 2 2 3 3 return back_trace(C, X, Y, i - 1, j); B 0 1 2 2 3 3 3 4 } A 0 1 2 2 3 3 4 4 19 20 X Extract the Subsequence A B C B D A B string back_trace(Table C, string X, string Y, int i, int j) { 0 0 0 0 0 0 0 0 if (i < 0 || j < 0) return ""; B 0 0 1 1 1 1 1 1 else if (X[i] == Y[j]) return back_trace(C, X, Y, i - 1, j - 1) + X[i]; D 0 0 1 1 1 2 2 2 else Y if (C[i + 1][j] > C[i][j + 1]) C 0 0 1 2 2 2 2 2 return back_trace(C, X, Y, i, j - 1); else A 0 1 1 2 2 2 3 3 return back_trace(C, X, Y, i - 1, j); } B 0 1 2 2 3 3 3 4 A 0 1 2 2 3 3 4 4 Initial call: back_trace(C, X, Y, m – 1, n – 1) 21 22 X X BCAB BCAB A B C B D A B A B C B D A B 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 B 0 0 1 1 1 1 1 1 B 0 0 1 1 1 1 1 1 D 0 0 1 1 1 2 2 2 D 0 0 1 1 1 2 2 2 Y Y C 0 0 1 2 2 2 2 2 C 0 0 1 2 2 2 2 2 A 0 1 1 2 2 2 3 3 A 0 1 1 2 2 2 3 3 B 0 1 2 2 3 3 3 4 B 0 1 2 2 3 3 3 4 A 0 1 2 2 3 3 4 4 A 0 1 2 2 3 3 4 4 23 24 4

X X A B C B D A B A B C B D A B 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 B 0 0 1 1 1 1 1 1 B 0 0 1 1 1 1 1 1 D 0 0 1 1 1 2 2 2 D 0 0 1 1 1 2 2 2 Y Y C 0 0 1 2 2 2 2 2 C 0 0 1 2 2 2 2 2 A 0 1 1 2 2 2 3 3 A 0 1 1 2 2 2 3 3 B 0 1 2 2 3 3 3 4 B 0 1 2 2 3 3 3 4 A 0 1 2 2 3 3 4 4 A 0 1 2 2 3 3 4 4 25 26 X X A B C B D A B A B C B D A B 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 B 0 0 1 1 1 1 1 1 B 0 0 1 1 1 1 1 1 D 0 0 1 1 1 2 2 2 D 0 0 1 1 1 2 2 2 Y Y C 0 0 1 2 2 2 2 2 C 0 0 1 2 2 2 2 2 A 0 1 1 2 2 2 3 3 A 0 1 1 2 2 2 3 3 B 0 1 2 2 3 3 3 4 B 0 1 2 2 3 3 3 4 A 0 1 2 2 3 3 4 4 A 0 1 2 2 3 3 4 4 27 28 X X A B C B D A B A B C B D A B 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 B 0 0 1 1 1 1 1 1 B 0 0 1 1 1 1 1 1 D 0 0 1 1 1 2 2 2 D 0 0 1 1 1 2 2 2 Y Y C 0 0 1 2 2 2 2 2 C 0 0 1 2 2 2 2 2 A 0 1 1 2 2 2 3 3 A 0 1 1 2 2 2 3 3 B 0 1 2 2 3 3 3 4 B 0 1 2 2 3 3 3 4 A 0 1 2 2 3 3 4 4 A 0 1 2 2 3 3 4 4 29 30 5

X BDAB DP Complexity A B C B D A B • Θ ( m∙n ) to fill out the table 0 0 0 0 0 0 0 0 • Θ (max( m , n )) to backtrace B 0 0 1 1 1 1 1 1 • Space: Θ ( m∙n ) D 0 0 1 1 1 2 2 2 Y C 0 0 1 2 2 2 2 2 A 0 1 1 2 2 2 3 3 B 0 1 2 2 3 3 3 4 A 0 1 2 2 3 3 4 4 31 32 6