CPSC 490 DP Part 2: Max-IS on Arrays, LCS, Recovery, and Binary - - PowerPoint PPT Presentation
CPSC 490 DP Part 2: Max-IS on Arrays, LCS, Recovery, and Binary Exponentiation Lucca Siaudzionis and Jack Spalding-Jamieson 2020/01/23 University of British Columbia Announcements Again, assignment 1 is due this Saturday!!!!!!!!
CPSC 490 DP
Part 2: Max-IS on Arrays, LCS, Recovery, and Binary Exponentiation
Lucca Siaudzionis and Jack Spalding-Jamieson 2020/01/23
University of British Columbia
Announcements
1
What DP is About
Source: Non deterministic memes for NP complete teens
2
Maximum Independent-Set on a Line
Input: An array of n positive integers. Output: The maximum sum of chosen integers, such that no two chosen values are adjacent in the array.
3
Maximum Independent-Set on a Line
Input: An array of n positive integers. Output: The maximum sum of chosen integers, such that no two chosen values are adjacent in the array.
4
Maximum Independent-Set on a Line: Formulating a Recurrence
The first thing we need for any DP problem is a recurrence. For each element of the array, we need to make a choice: Include it or exclude it. Let’s consider everything back to front! OPTn = max (arrayn + OPTn−2, OPTn−1) We are defining the optimal value at the end using smaller optimal values. The time to compute the new optimal value is constant. Base cases: OPT1 = array1 OPT2 = max(array1, array2).
5
Maximum Independent-Set on a Line: Implementing our Recurrence (1)
Like last class, this leads to a simple naive implementation:
1
fun max_is(arr ,n):
2
if n == 0:
3
return 0
4
if n == 1:
5
return arr [0]
6
return max(arr[n -1]+ max_is(arr ,n-2),max_is(arr ,n-1)) Runtime: O(2n)
6
Maximum Independent-Set on a Line: Implementing our Recurrence (2)
Like last class again, we can memoize the simple naive implementation:
1
memo: hashmap (or array) from n -> max_is(arr ,n); Initially empty
2
fun max_is(arr ,n):
3
if n == 0:
4
return 0
5
if n == 1:
6
return arr [0]
7
if n in memo:
8
return memo[n]
9
memo[n] = max(arr[n -1]+ max_is(arr ,n-2),max_is(arr ,n-1))
10
return memo[n] Runtime: O(n) The function only recurses once for each value up to n.
7
Maximum Independent-Set on a Line: Bottom-Up Implementation
Often, it is much faster to create an iterative implementation of our recurrence. This is called a ”Bottom-up” implementation.
1
fun max_is(arr):
2
memo: hashmap (or array) from n -> max_is(arr ,n); Initially empty
3
memo [0] = 0
4
memo [1] = 1
5
for n from 2 to arr.size:
6
memo[n] = max(arr[n -1]+ memo[n-2], memo[n -1])
7
return memo[arr.size] Runtime: O(n)
8
Longest Common Subsequence: Problem Statement
Input: Two strings s and t. Output: The length of the longest common subsequence of s and t.
9
Longest Common Subsequence: Intuition & Recurrence
Since we have two pieces of data (two strings!), we can no longer just reduce by removing the last character/element. Instead, we can consider removing at least one of the two last characters!This gives us a two-dimensional recurrence on the first n characters of s and the first m characters of t: OPT[n, m] =
s[n − 1] = t[n − 1] max(OPT[n − 1, m], OPT[n, m − 1])
We have a 2D table of computations. Time complexity: Each entry in our table takes constant time to compute, given the previous values, so in total we have O(|s||t|) time.
10
Longest Common Subsequence: Example (1)
11
Longest Common Subsequence: Example (2)
The arrows tell us where an entry in the table came from.
12
Longest Common Subsequence: Simple Bottom-Up Implementation (C++)
1
int lcs(const string &s, const string &t) {
2
int n = s.size(), m = t.size();
3
vector<vector<int>> memo(n+1,vector<int>(m+1));
4
for (int is = 1; is <= n; ++is)
5
for (int it = 1; it <= m; ++it) {
6
if (s[is-1] == t[it-1])
7
memo[is][it] = 1+memo[is-1][it-1];
8
else
9
memo[is][it] = max(memo[is][it-1],memo[is-1][it]);
10
}
11
return memo[n][m];
12
} This code is really short! Most of the difficulty in DP problems is in finding the recurrence.
13
Longest Common Subsequence: Recovering the Solution
We now have the length of the LCS. What is the LCS though? Can we get the actual string? Let’s look at those arrows again:
We can actually store these arrows!
14
Longest Common Subsequence: Recovery Implementation
1
string lcs_recovery(const string &s, const string &t) {
2
int n = s.size(), m = t.size();
3
// [... LCS Code here ...]
4
string str; int is = n, it = m;
5
while (is > 0 && it > 0) {
6
if (s[is-1] == t[it-1]) {
7
str += s[is-1]; --is; --it;
8
}
9
else if (memo[is][it] == memo[is][it-1]) --it;
10
else --is;
11
}
12
reverse(str.begin(),str.end());
13
return str;
14
}
15
Discussion Problem: Levenschtein Distance
Input: Two strings s and t. Output: The minimum number of ”edits” to transform s into t, where an edit is one of:
16
Discussion Problem: Levenschtein Distance – Insight
This problem seems very similar to the longest common subsequence problem. We can do something very similar: OPT(i, j) = max(i, j) i = 0 ∨ j = 0 OPT(i − 1, j − 1) si = tj min(OPT(i − 1, j − 1) + 1, OPT(i − 1, j) + 1, OPT(i, j − 1) + 1)
Compared to LCS, the maximum is swapped for a minimum, and the cases are subtly different.
17
Fibonacci Numbers: Even Faster
Input: A positive number n ≤ 1018. Output: The nth Fibonacci number Fn, modulo 109 + 7. Why modulo 109 + 7? We cannot represent Fibonacci numbers too large in memory.
18
Fibonacci Numbers: A Matrix
The nth Fibonacci Number is defined solely on the n − 1th and n − 2nd Fibonacci numbers: Fn = Fn−1 + Fn−2 We can represent this as a vector addition:
Fn−1
Fn−1
Fn−1
1 1 Fn−1 Fn−2
Fibonacci Numbers: Matrix Exponentiation
We now have a recurrence based on a matrix, equivalent to our recurrence from before:
Fn−1
1 1 Fn−1 Fn−2
Fn−1
1 1 2 Fn−2 Fn−3
Fn−1
1 1 n−2 F2 F1
Fibonacci Numbers: A First Algorithm with Matrix Exponentiation
Using our general recurrence, all we really need to do are n − 2 matrix-multiplications to compute
1 1 n−2 = Mn−2. If n − 2 was a power of two, we could do something much faster: Say n − 2 = 2k. We want to find M2k. Equivalently, we want to find (M2)2k−1. Repeating, we get (((((M2)2)2)2) . . .2). We halve the number of multiplications needed each time. We can actually compute every power of 2 separately up to ⌊log2 n⌋ in O(log n) time.
21
A Faster Algorithm for Matrix Exponentiation
Suppose that in binary, n − 2 = 10012. Then, n − 2 = 23 + 20. So Mn−2 = M23+20 = M23M20. In general, Mn−2 is equal to the product of the powers of 2 which correspond to a 1 in the binary representation. This gives us an O(log n) solution to Fibonacci numbers!
22
A Faster Algorithm for Matrix Exponentiation: Pseudocode
The following algorithm returns the exponentiated matrix Mn:
1
fun mat_exp(M,n):
2
res: matrix; Initialized to identity matrix
3
while n > 0:
4
if n mod 2 == 1:
5
res = res * M
6
M = M * M
7
n = n / 2
8
return res
23
Discussion Problem: General Fibonacci Queries
Input: A number q ≤ 105 of queries, followed by q lines of three numbers each. On each query line, there will be three values f1, f2, n ≤ 1018. Output: For each query, output the nth Fibonacci number, modulo 109 + 7, where the first and second Fibonacci number are redefined to be f1 and f2 (instead of being 1 and 1).
24
Discussion Problem: General Fibonacci Queries – Insight
Even though the initial values of the numbers may change, the matrices we can use do not. The matrix for each query can still be completed in logarithmic time, and then used in constant time. To save some time, we could also pre-compute all the power of two matrices (although this may not be necessary if your implementation is fast enough already).
25
Discussion Problem - Longest Increasing Subsequence (LIS)
Find the length of the longest increasing subsequence (LIS) in an array of N ≤ 100, 000 integers. Example: [10, 11, 1, 5, 3, 9, -1, 7, 25]
26
Discussion Problem - Longest Increasing Subsequence (LIS) – Insight (1)
With a good enough implementation, we might be able to ’squeeze’ an O(n2) solution. Try the recurrence OPT[j] = 1 + maxi<j,a[i]<a[j] OPT[i]. Computing this will take O(n2) time, but will have a very small constant. By storing some additional data, we can compute this max function more quickly.
27
Discussion Problem - Longest Increasing Subsequence (LIS) – Insight (2)
We will again iterate through the array. As we iterate, make best[k] be the smallest element so far that is the end of a LIS with size k, and make m be the size of your maximum LIS so far. Note that the array best will be increasing. Initially, m = 0 and best[0] = −∞. We process our sequence in order. When we are processing a number x, we check:
x and replace it with x. Implementation detail: In C++, while binary searching on the array, we could use std::lower bound.
28
Fun Things to do This Weekend
Jack’s recommendation: Go skiing
29
Fun Things to do This Weekend
Lucca’s recommendation: Blade Runner 2049
30