CPSC 490 DP and Range Queries Part 5: Binary Jumping, LCA; Part 1: - PowerPoint PPT Presentation

CPSC 490 DP and Range Queries Part 5: Binary Jumping, LCA; Part 1: Intro, Prefix Sums, Fenwick Trees Lucca Siaudzionis and Jack Spalding-Jamieson 2020/02/04 University of British Columbia

Announcements • Reminder: The upsolver for A1 is up. It is worth 25% of its original marks. • Reminder (for Jack’s section): Office hours are after class from 2-4! • A2 is due on Sunday! 1

An interesting problem Given an array A of N ≤ 10 6 positive numbers, answer Q ≤ 10 6 queries of the form “What is the minimum number in the subarray A [ l .. r ]?” 2

An interesting problem – Insight Notice that you can “stack mins up.” • That is, the min( a , b , c , d , e ) = min(min( a , b , c ) , min( c , d , e )). If you know the answer to two halves of the sub-array, you can combine them to get the answer to your query. • This suggests some sort of divide-and-conquer strategy. 3

Binary Jumping Let f ( i , p ) = the minimum element in the range [ i , i + 2 p ). Initialize the dp table to − 1, and compute values from smaller p to larger p . • f ( i , 0) = A [ i ] the base case for interval [ i , i + 1). • f ( i , p ) = min { f ( i , p − 1) , f ( i + 2 p , p − 1) } if i + 2 p < n . • f ( i , p ) = f ( i , p − 1) otherwise. To handle a query of the form [ ℓ, r ], let q be the largest power of 2 less than or equal to r − ℓ + 1. Then our answer for the minimum value is min { f ( ℓ, q ) , f ( r − 2 q + 1 , q ) } . Initializing our table takes O ( N log N ), every query takes O (1)! 4

What is a way to use this? Remember this situation? [1, 2, 3, 3, 2, 4, 4, 2, 1, 5, 6, 7, 7, 6, 8, 8, 6, 5, 1, 9, 10, 10, 9, 1] 5

Lowest Common Ancestor (LCA) Given a tree of N ≤ 100 , 000 nodes with a well defined root at node 0, answer Q ≤ 100 , 000 queries of: What is the lowest common ancestor of u and v ? 6

LCA – Insights This is an example graph and root. 7

LCA – Insights This is an example of two nodes’ LCA. 8

LCA – Insights We can observe “layers” in the graph. 9

LCA – Insights These layers are the depth of each node, which is the distance to the root (in number of edges). You can find this with a DFS or BFS. d = 0 d = 1 d = 2 d = 3 d = 4 10

LCA DP – Filling the DP Table DP State • depth[u] = “depth of u ” = number of nodes from the root • par[u][k] = the 2 k -th ancestor of u How do we fill the par array efficiently? • Fill out the par array for all ancestors first (base case = root) • par[u][0] = parent of u • par[u][k] = par[par[u][k-1]][k-1] Time complexity: O ( N log N ) 11

LCA DP – Computing the LCA DP State • depth[u] = “depth of u ” = number of nodes from the root • par[u][k] = the 2 k -th ancestor of u Algorithm Outline • Move up the deeper node: say u is deeper, then iteratively replace u = par[u][k] where k is the largest integer such that depth[u] − 2 k ≥ depth[v] • Move both nodes up to their LCA: iteratively replace u = par[u][k], v = par[v][k] where k is the largest integer such that par[u][k] != par[v][k] Time complexity: O (log N ) 12

LCA DP – Computing the LCA LCA(u, v): 1 if (depth[u] < depth[v]): 2 swap(u, v) 3 for k = log N to 0: 4 if depth[u] - 2^k >= depth[v]: 5 u = par[u][k] 6 for k = log N to 0: 7 if par[u][k] != par[v][k]: 8 u = par[u][k], v = par[v][k] 9 if u != v: 10 u = par[u][0], v = par[v][0] 11 return u 12 13

Discussion Problem Given an unweighted tree with N ≤ 10 5 nodes, answer Q ≤ 10 5 queries asking for the length of the shortest path between two nodes. 14

Discussion Problem – Insight • Let d = LCA ( u , v ). • The path from u to v must go through d . • Hence, length ( u → v ) = length ( u → d ) + length ( d → v ). • Notice that the length of path between a node and one of its ancestors is just the different in their depths. • And d is an ancestor of both u and v . 15

End of Dynamic Programming • This is the “end” of the DP unit in our class • However, DP is such a broad topic, that we’ll be able to pair it with many topics we’ll study in the next weeks • There will never be any shortage of DP problems :) 16

Range Queries 16

Standard Range Query Problem • Start with an array • Make updates on it • Make queries on it 17

Updates • Change the value of one or more consecutive elements (let’s focus on just one for now) • For example: add 5 to v[4] 18

Queries • Extract some information from one or more consecutive elements • For example: what is the value of the sum v[1] + v[2] + v[3] + v[4] ? 19

Standard Solution • Division of responsibility: • Have sums of some consecutive values stored • When v [ u ] is updated, all you need to do is update all the values responsible for u 20

Standard Solution 21

Standard Solution - Update 22

Standard Solution - Query 23

Prefix Sums • For every x , store prefix[x] = v[0] + v[1] + ... + v[x] • (v[a] + ... + v[b]) = (prefix[b] - prefix[a-1]) 24

Prefix Sums • Queries in O (1) • Updates in O ( N ) • We can do better 25

Binary Indexed Tree (BIT) or Fenwick Tree 26

BIT - Division of Responsibility • Make everything 1-indexed • Let idx be an index of the BIT • Let r be the position of least significant bit of idx in binary (i.e., the rightmost bit which is equal to 1) • For example, idx = 4 = 100 2 , r = 2 • For example, idx = 6 = 110 2 , r = 1 • bit[idx] stores the sum of the range [idx − 2 r + 1 , idx] 27

BIT - Initialization const int MAXN = 100000; 1 int bit[MAXN+1]; 2 28

BIT - Division of Responsibility • For example, bit[4] stores the sum for [4 − 2 2 + 1 , 4] = [1 , 4] • For example, bit[6] stores the sum for [6 − 2 1 + 1 , 6] = [5 , 6] 29

BIT - Updates • Update’s complexity is bounded by the number of bits in a number ⇒ O ( lg N ) 30

BIT - Update Visualized Perform an update on position 11 = 1011 2 on an array of size 100 • Update at 1011 2 = 11 • Update at 11 + 2 lsb (11) = 1011 2 + 1 2 = 1100 2 = 12 • Update at 12 + 2 lsb (12) = 1100 2 + 100 2 = 10000 2 = 16 • Update at 16 + 2 lsb (16) = 10000 2 + 10000 2 = 100000 2 = 32 • Update at 32 + 2 lsb (32) = 100000 2 + 100000 2 = 1000000 2 = 64 • Next position would be 64 + 2 lsb (64) = 128 > 100 31

BIT - Update Implemented void update(int x, int v) { // increase position x by v 1 while (x <= MAXN) { 2 bit[x] += v; 3 x += (x & -x); // (x & -x) == 2^lsb(x) 4 } 5 } 6 32

BIT - Queries • Finding the value of sum(a, b) may be hard for arbitrary a and b • But how about finding the value of sum(1, x) ? 33

BIT - Queries • sum(1, x) can be found by constantly ”erasing” the lsb of x and adding these values up ⇒ O ( lg N ) • sum(a, b) can then be found by doing sum(1, b) - sum(1, a) 34

BIT - Query Visualized Perform a query to find sum(1, 11) , where 11 = 1011 2 • Query at 1011 2 = 11 • Query at 11 − 2 lsb (11) = 1011 2 − 1 2 = 1010 2 = 10 • Query at 10 − 2 lsb (10) = 1010 2 − 10 2 = 1000 2 = 8 • Next position would be 8 − 2 lsb (8) = 0 < 1 • Note: The amount of positions we had to query is exactly the same as the number of bits 1 in 11 35

BIT - Query Implemented int query(int x) { // find sum(1, x) 1 int sum = 0; 2 while (x > 0) { 3 sum += bit[x]; 4 x -= (x & -x); // (x & -x) == 2^lsb(x) 5 } 6 return sum; 7 } 8 36

Discussion Problem: Inversion Counting Given an array v[1], v[2], ..., v[n] of distinct number, find the number of pairs ( i , j ) such that i < j and v[i] > v[j] 37

Discussion Problem: Inversion Counting – Insight The actual values in the array do not matter. We can first apply any injective increasing function, and the solution will be the same. We can change the array v to only include values from 1 to n using a sort. We can iterate through the array from left to right, and keep a BIT (of size n ) of the counts of every number in the array seen so far. We then do a quick query on the number of values seen greater than our current value. Overall runtime: O ( n log n ) 38

BIT - Pros and Cons Pros: • Incredibly fast and easy to code • May be O ( log N ), but it has a very small constant • Uses very little memory (often possible to story many BITs!) Cons: • Only point updates • Difficult to find anything other than range sums • For example, we may need max/min/product of an interval • Why can’t we use them to compute the product of an interval (in general)? • Very formally, BITs are nice for operations on groups, when the base operations are easy to compute and the elements are easy to store • The important piece of information that this implies is that BITs are quite limited in scope for range queries. That’s why it’s nice to have alternative solutions • Which we’ll see next class :) 39