CS 473: Algorithms Chandra Chekuri Ruta Mehta University of Illinois, Urbana-Champaign Fall 2016 Chandra & Ruta (UIUC) CS473 1 Fall 2016 1 / 57
CS 473: Algorithms, Fall 2016 Review session Lecture 99 September 30, 2016 Chandra & Ruta (UIUC) CS473 2 Fall 2016 2 / 57
What we saw so far... Fast Fourier Transform (FFT). Dynamic Programming String algorithms. Graph algorithms: shortest path, independent set, dominating set, etc. Randomozed Algorithms Quick sort, High probability analysis: Markov, Chebyshev, and Chernoff inequalities Chandra & Ruta (UIUC) CS473 3 Fall 2016 3 / 57
What we saw so far... Fast Fourier Transform (FFT). Dynamic Programming String algorithms. Graph algorithms: shortest path, independent set, dominating set, etc. Randomozed Algorithms Quick sort, High probability analysis: Markov, Chebyshev, and Chernoff inequalities Hashing, Fingerprinting Chandra & Ruta (UIUC) CS473 3 Fall 2016 3 / 57
Part I FFT Chandra & Ruta (UIUC) CS473 4 Fall 2016 4 / 57
What is Fast Fourier Transform Definition Given vector a = (a 0 , a 1 , . . . , a n − 1 ) the Discrete Fourier Transform (DFT) of a is the vector a ′ = (a ′ 0 , a ′ 1 , . . . , a ′ n − 1 ) where a ′ j = a( ω j n ) for 0 ≤ j < n . a ′ is a sample representation of polynomial with coefficient reprentation a at n ’th roots of unity. We have shown that a ′ can be computed from a in O(n log n) time. This divide and conquer algorithm is called the Fast Fourier Transform (FFT). Chandra & Ruta (UIUC) CS473 5 Fall 2016 5 / 57
Why FFT? Convolution and Polynomial Multiplication Convolution Convolution of vectors a = (a 0 , a 1 , . . . a n − 1 ) and b = (b 0 , b 1 , . . . b n − 1 ) is a vector c = (c 0 , c 1 , . . . , c 2n − 2 ) , where � c k = a i · b j i , j: i+j=k Chandra & Ruta (UIUC) CS473 6 Fall 2016 6 / 57
Why FFT? Convolution and Polynomial Multiplication Convolution Convolution of vectors a = (a 0 , a 1 , . . . a n − 1 ) and b = (b 0 , b 1 , . . . b n − 1 ) is a vector c = (c 0 , c 1 , . . . , c 2n − 2 ) , where � c k = a i · b j i , j: i+j=k Polynomial Multiplication If vectors a and b are coefficients of two n − 1 degree polynomials, i=0 b i x i then c is (abusing notation) a(x) = � n − 1 b(x) = � n − 1 i=0 a i x i , the coefficient vector of the product polynomial a(x) ∗ b(x) . Chandra & Ruta (UIUC) CS473 6 Fall 2016 6 / 57
Why FFT? Convolution and Polynomial Multiplication Convolution Given vectors a = (a 0 , a 1 , . . . a n − 1 ) and b = (b 0 , b 1 , . . . b n − 1 ) find its convolution vector c = (c 0 , c 1 , . . . , c 2n − 2 ) . Compute values of P a and P b at the 2n th roots of unity, to get 1 their sample representation a ′ and b ′ . Compute sample representation c ′ = � a ′ 0 b ′ 0 , . . . , a ′ 2n − 2 b ′ 2n − 2 of 2 product c = a · b Compute c from c ′ using inverse Fourier transform. 3 Step 1 takes O(n log n) using two FFTs Step 2 takes O(n) time Step 3 takes O(n log n) using one FFT Chandra & Ruta (UIUC) CS473 7 Fall 2016 7 / 57
Problem Suppose we are given a bit string B[1 .. n] . A triple of distinct indices 1 ≤ i < j < k ≤ n is called a well-spaced triple in B if B[i] = B[j] = B[k] = 1 and k − j = j − i . (a) Describe a brute-force algorithm to determine whether B has a well-spaced triple in O(n 2 ) time. Chandra & Ruta (UIUC) CS473 8 Fall 2016 8 / 57
Application of FFT Suppose we are given a bit string B[1 .. n] . A triple of distinct indices 1 ≤ i < j < k ≤ n is called a well-spaced triple in B if B[i] = B[j] = B[k] = 1 and k − j = j − i . (b) Describe an algorithm to determine whether B has a well-spaced triple in O(n log n) time. Chandra & Ruta (UIUC) CS473 9 Fall 2016 9 / 57
Application of FFT Suppose we are given a bit string B[1 .. n] . A triple of distinct indices 1 ≤ i < j < k ≤ n is called a well-spaced triple in B if B[i] = B[j] = B[k] = 1 and k − j = j − i . (c) Describe an algorithm to determine the number of well-spaced triples in B in O(n log n) time. Chandra & Ruta (UIUC) CS473 10 Fall 2016 10 / 57
Part II Dynamic Programming Chandra & Ruta (UIUC) CS473 11 Fall 2016 11 / 57
Recursion Reduction: Reduce one problem to another Recursion A special case of reduction reduce problem to a smaller instance of itself 1 self-reduction 2 Problem instance of size n is reduced to one or more instances 1 of size n − 1 or less. For termination, problem instances of small size are solved by 2 some other method as base cases . Chandra & Ruta (UIUC) CS473 12 Fall 2016 12 / 57
What is Dynamic Programming? Every recursion can be memoized. Automatic memoization does not help us understand whether the resulting algorithm is efficient or not. Chandra & Ruta (UIUC) CS473 13 Fall 2016 13 / 57
What is Dynamic Programming? Every recursion can be memoized. Automatic memoization does not help us understand whether the resulting algorithm is efficient or not. Dynamic Programming: A recursion that when memoized leads to an efficient algorithm. Chandra & Ruta (UIUC) CS473 13 Fall 2016 13 / 57
Edit Distance Definition Edit distance between two words X and Y is the number of letter insertions, letter deletions and letter substitutions required to obtain Y from X . Example The edit distance between FOOD and MONEY is at most 4 : FOOD → MOOD → MONOD → MONED → MONEY Chandra & Ruta (UIUC) CS473 14 Fall 2016 14 / 57
Edit Distance: Alternate View Alignment Place words one on top of the other, with gaps in the first word indicating insertions, and gaps in the second word indicating deletions. F O O D M O N E Y Chandra & Ruta (UIUC) CS473 15 Fall 2016 15 / 57
Edit Distance: Alternate View Alignment Place words one on top of the other, with gaps in the first word indicating insertions, and gaps in the second word indicating deletions. F O O D M O N E Y Formally, an alignment is a set M of pairs (i , j) such that each index appears at most once, and there is no “crossing”: i < i ′ and i is matched to j implies i ′ is matched to j ′ > j . In the above example, this is M = { (1 , 1) , (2 , 2) , (3 , 3) , (4 , 5) } . Chandra & Ruta (UIUC) CS473 15 Fall 2016 15 / 57
Edit Distance: Alternate View Alignment Place words one on top of the other, with gaps in the first word indicating insertions, and gaps in the second word indicating deletions. F O O D M O N E Y Formally, an alignment is a set M of pairs (i , j) such that each index appears at most once, and there is no “crossing”: i < i ′ and i is matched to j implies i ′ is matched to j ′ > j . In the above example, this is M = { (1 , 1) , (2 , 2) , (3 , 3) , (4 , 5) } . Cost of an alignment is the number of mismatched columns plus number of unmatched indices in both strings. Chandra & Ruta (UIUC) CS473 15 Fall 2016 15 / 57
Edit Distance Problem Problem Given two words, find the edit distance between them, i.e., an alignment of smallest cost. Chandra & Ruta (UIUC) CS473 16 Fall 2016 16 / 57
Edit Distance Basic observation Let X = α x and Y = β y α, β : strings. x and y single characters. Possible alignments between X and Y α x α x α x or or β y β y β y Observation Prefixes must have optimal alignment! Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 57
Edit Distance Basic observation Let X = α x and Y = β y α, β : strings. x and y single characters. Possible alignments between X and Y α x α x α x or or β y β y β y Observation Prefixes must have optimal alignment! EDIST( α, β ) + [x � = y] EDIST(X , Y) = min 1 + EDIST( α, Y) 1 + EDIST(X , β ) Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 57
Recursive Algorithm Assume X is stored in array A[1 .. m] and Y is stored in B[1 .. n] EDIST(A[1 .. i] , B[1 .. j]) If ( i = 0 ) return j If ( j = 0 ) return i m 1 = 1 + EDIST(A[1 .. (i − 1)] , B[1 .. j]) m 2 = 1 + EDIST(A[1 .. i] , B[1 .. (j − 1)])) If (A[m] = B[n]) then m 3 = EDIST(A[1 .. (i − 1)] , B[1 .. (j − 1)]) Else m 3 = 1 + EDIST(A[1 .. (i − 1)] , B[1 .. (j − 1)]) return min(m 1 , m 2 , m 3 ) Call EDIST(A[1 .. m] , B[1 .. n]) Chandra & Ruta (UIUC) CS473 18 Fall 2016 18 / 57
Memoizing the Recursive Algorithm int M[0 .. m][0 .. n] Initialize all entries of M[i][j] to ∞ return EDIST(A[1 .. m] , B[1 .. n]) EDIST(A[1 .. i] , B[1 .. j]) If (M[i][j] < ∞ ) return M[i][j] (* return stored value *) If ( i = 0 ) M[i][j] = j ElseIf ( j = 0 ) M[i][j] = i Else m 1 = 1 + EDIST(A[1 .. (i − 1)] , B[1 .. j]) m 2 = 1 + EDIST(A[1 .. i] , B[1 .. (j − 1)])) If (A[i] = B[j]) m 3 = EDIST(A[1 .. (i − 1)] , B[1 .. (j − 1)]) Else m 3 = 1 + EDIST(A[1 .. (i − 1)] , B[1 .. (j − 1)]) M[i][j] = min(m 1 , m 2 , m 3 ) return M[i][j] Chandra & Ruta (UIUC) CS473 19 Fall 2016 19 / 57
Removing Recursion to obtain Iterative Algorithm EDIST(A[1 .. m] , B[1 .. n]) int M[0 .. m][0 .. n] for i = 0 to m do M[i , 0] = i for j = 0 to n do M[0 , j] = j for i = 1 to m do for j = 1 to n do [x i � = y j ] + M[i − 1][j − 1] , M[i][j] = min 1 + M[i − 1][j] , 1 + M[i][j − 1] Chandra & Ruta (UIUC) CS473 20 Fall 2016 20 / 57
Recommend
More recommend
