CPSC 490 DP and Range Queries Part 5: Binary Jumping, LCA; Part 1: - - PowerPoint PPT Presentation

CPSC 490 DP and Range Queries

Part 5: Binary Jumping, LCA; Part 1: Intro, Prefix Sums, Fenwick Trees

Lucca Siaudzionis and Jack Spalding-Jamieson 2020/02/04

University of British Columbia

Announcements

• Reminder: The upsolver for A1 is up. It is worth 25% of its original marks.
• Reminder (for Jack’s section): Office hours are after class from 2-4!
• A2 is due on Sunday!

1

An interesting problem

Given an array A of N ≤ 106 positive numbers, answer Q ≤ 106 queries of the form “What is the minimum number in the subarray A[l..r]?”

2

An interesting problem – Insight

Notice that you can “stack mins up.”

• That is, the min(a, b, c, d, e) = min(min(a, b, c), min(c, d, e)).

If you know the answer to two halves of the sub-array, you can combine them to get the answer to your query.

• This suggests some sort of divide-and-conquer strategy.

3

Binary Jumping

Let f (i, p) = the minimum element in the range [i, i + 2p). Initialize the dp table to −1, and compute values from smaller p to larger p.

• f (i, 0) = A[i] the base case for interval [i, i + 1).
• f (i, p) = min{f (i, p − 1), f (i + 2p, p − 1)} if i + 2p < n.
• f (i, p) = f (i, p − 1) otherwise.

To handle a query of the form [ℓ, r], let q be the largest power of 2 less than or equal to r − ℓ + 1. Then our answer for the minimum value is min{f (ℓ, q), f (r − 2q + 1, q)}. Initializing our table takes O(N log N), every query takes O(1)!

4

What is a way to use this?

Remember this situation?

[1, 2, 3, 3, 2, 4, 4, 2, 1, 5, 6, 7, 7, 6, 8, 8, 6, 5, 1, 9, 10, 10, 9, 1]

5

Lowest Common Ancestor (LCA)

Given a tree of N ≤ 100, 000 nodes with a well defined root at node 0, answer Q ≤ 100, 000 queries of: What is the lowest common ancestor of u and v?

6

LCA – Insights

This is an example graph and root.

7

LCA – Insights

This is an example of two nodes’ LCA.

8

LCA – Insights

We can observe “layers” in the graph.

9

LCA – Insights

These layers are the depth of each node, which is the distance to the root (in number of edges). You can find this with a DFS or BFS.

d = 0 d = 1 d = 2 d = 3 d = 4

10

LCA DP – Filling the DP Table

DP State

• depth[u] = “depth of u” = number of nodes from the root
• par[u][k] = the 2k-th ancestor of u

How do we fill the par array efficiently?

• Fill out the par array for all ancestors first (base case = root)
• par[u][0] = parent of u
• par[u][k] = par[par[u][k-1]][k-1]

Time complexity: O(N log N)

11

LCA DP – Computing the LCA

DP State

• depth[u] = “depth of u” = number of nodes from the root
• par[u][k] = the 2k-th ancestor of u

Algorithm Outline

• Move up the deeper node: say u is deeper, then iteratively replace u = par[u][k] where

k is the largest integer such that depth[u] − 2k ≥ depth[v]

• Move both nodes up to their LCA: iteratively replace u = par[u][k], v = par[v][k]

where k is the largest integer such that par[u][k] != par[v][k] Time complexity: O(log N)

12

LCA DP – Computing the LCA

1

LCA(u, v):

2

if (depth[u] < depth[v]):

3

swap(u, v)

4

for k = log N to 0:

5

if depth[u] - 2^k >= depth[v]:

6

u = par[u][k]

7

for k = log N to 0:

8

if par[u][k] != par[v][k]:

9

u = par[u][k], v = par[v][k]

10

if u != v:

11

u = par[u][0], v = par[v][0]

12

return u

13

Discussion Problem

Given an unweighted tree with N ≤ 105 nodes, answer Q ≤ 105 queries asking for the length of the shortest path between two nodes.

14

Discussion Problem – Insight

• Let d = LCA(u, v).
• The path from u to v must go through d.
• Hence, length(u → v) = length(u → d) + length(d → v).
• Notice that the length of path between a node and one of its ancestors is just the different

in their depths.

• And d is an ancestor of both u and v.

15

End of Dynamic Programming

• This is the “end” of the DP unit in our class
• However, DP is such a broad topic, that we’ll be able to pair it with many topics we’ll

study in the next weeks

• There will never be any shortage of DP problems :)

16

Range Queries

16

Standard Range Query Problem

• Start with an array
• Make updates on it
• Make queries on it

17

Updates

• Change the value of one or more consecutive elements (let’s focus on just one for now)
• For example: add 5 to v[4]

18

Queries

• Extract some information from one or more consecutive elements
• For example: what is the value of the sum v[1] + v[2] + v[3] + v[4]?

19

Standard Solution

• Division of responsibility:
• Have sums of some consecutive values stored
• When v[u] is updated, all you need to do is update all the values responsible for u

20

Standard Solution

21

Standard Solution - Update

22

Standard Solution - Query

23

Prefix Sums

• For every x, store prefix[x] = v[0] + v[1] + ...

+ v[x]

• (v[a] + ...

+ v[b]) = (prefix[b] - prefix[a-1])

24

Prefix Sums

• Queries in O(1)
• Updates in O(N)
• We can do better

25

Binary Indexed Tree (BIT) or Fenwick Tree

26

BIT - Division of Responsibility

• Make everything 1-indexed
• Let idx be an index of the BIT
• Let r be the position of least significant bit of idx in binary (i.e., the rightmost bit which

is equal to 1)

• For example, idx = 4 = 1002, r = 2
• For example, idx = 6 = 1102, r = 1
• bit[idx] stores the sum of the range [idx−2r + 1, idx]

27

BIT - Initialization

1

const int MAXN = 100000;

2

int bit[MAXN+1];

28

BIT - Division of Responsibility

• For example, bit[4] stores the sum for [4 − 22 + 1, 4] = [1, 4]
• For example, bit[6] stores the sum for [6 − 21 + 1, 6] = [5, 6]

29

BIT - Updates

• Update’s complexity is bounded by the number of bits in a number ⇒ O(lg N)

30

BIT - Update Visualized

Perform an update on position 11 = 10112 on an array of size 100

• Update at 10112 = 11
• Update at 11 + 2lsb(11) = 10112 + 12 = 11002 = 12
• Update at 12 + 2lsb(12) = 11002 + 1002 = 100002 = 16
• Update at 16 + 2lsb(16) = 100002 + 100002 = 1000002 = 32
• Update at 32 + 2lsb(32) = 1000002 + 1000002 = 10000002 = 64
• Next position would be 64 + 2lsb(64) = 128 > 100

31

BIT - Update Implemented

1

void update(int x, int v) { // increase position x by v

2

while (x <= MAXN) {

3

bit[x] += v;

4

x += (x & -x); // (x & -x) == 2^lsb(x)

5

}

6

}

32

BIT - Queries

• Finding the value of sum(a, b) may be hard for arbitrary a and b
• But how about finding the value of sum(1, x)?

33

BIT - Queries

• sum(1, x) can be found by constantly ”erasing” the lsb of x and adding these values up

⇒ O(lg N)

• sum(a, b) can then be found by doing sum(1, b) - sum(1, a)

34

BIT - Query Visualized

Perform a query to find sum(1, 11), where 11 = 10112

• Query at 10112 = 11
• Query at 11 − 2lsb(11) = 10112 − 12 = 10102 = 10
• Query at 10 − 2lsb(10) = 10102 − 102 = 10002 = 8
• Next position would be 8 − 2lsb(8) = 0 < 1
• Note: The amount of positions we had to query is exactly the same as the number of bits

1 in 11

35

BIT - Query Implemented

1

int query(int x) { // find sum(1, x)

2

int sum = 0;

3

while (x > 0) {

4

sum += bit[x];

5

x -= (x & -x); // (x & -x) == 2^lsb(x)

6

}

7

return sum;

8

}

36

Discussion Problem: Inversion Counting

Given an array v[1], v[2], ..., v[n] of distinct number, find the number of pairs (i, j) such that i < j and v[i] > v[j]

37

Discussion Problem: Inversion Counting – Insight

The actual values in the array do not matter. We can first apply any injective increasing function, and the solution will be the same. We can change the array v to only include values from 1 to n using a sort. We can iterate through the array from left to right, and keep a BIT (of size n) of the counts of every number in the array seen so far. We then do a quick query on the number of values seen greater than our current value. Overall runtime: O(n log n)

38

BIT - Pros and Cons

Pros:

• Incredibly fast and easy to code
• May be O(log N), but it has a very small constant
• Uses very little memory (often possible to story many BITs!)

Cons:

• Only point updates
• Difficult to find anything other than range sums
• For example, we may need max/min/product of an interval
• Why can’t we use them to compute the product of an interval (in general)?
• Very formally, BITs are nice for operations on groups, when the base operations are easy to

compute and the elements are easy to store

• The important piece of information that this implies is that BITs are quite limited in scope for

range queries.

That’s why it’s nice to have alternative solutions

• Which we’ll see next class :)

39