Divide-Conquer-Glue Algorithms 5. D IVIDE AND C ONQUER Closest Pair - - PowerPoint PPT Presentation

Divide-Conquer-Glue Algorithms

Closest Pair Tyler Moore

CS 2123, The University of Tulsa

Some slides created by or adapted from Dr. Kevin Wayne. For more information see http://www.cs.princeton.edu/~wayne/kleinberg-tardos. Some code reused from Python Algorithms by Magnus Lie Hetland.

SECTION 5.4

• 5. DIVIDE AND CONQUER
• mergesort
• counting inversions
• closest pair of points
• randomized quicksort
• median and selection

2 / 32

21

Closest pair of points

Closest pair problem. Given n points in the plane, find a pair of points with the smallest Euclidean distance between them. Fundamental geometric primitive.

・Graphics, computer vision, geographic information systems,

molecular modeling, air traffic control.

・Special case of nearest neighbor, Euclidean MST

, Voronoi.

fast closest pair inspired fast algorithms for these problems

3 / 32

22

Closest pair of points

Closest pair problem. Given n points in the plane, find a pair of points with the smallest Euclidean distance between them. Brute force. Check all pairs with Θ(n2) distance calculations. 1d version. Easy O(n log n) algorithm if points are on a line. Nondegeneracy assumption. No two points have the same x-coordinate.

4 / 32

Closest-pair problem in one dimension

In the closest-pair problem, you are to select the pair of points (p1, p2) from a set S that are closest to each other.

dmin l1 l2 l3 r3 r1 r2

Sl = {l1, l2, l3} Sr = {r1, r2, r3}

m

dmin dmin

5 / 32

Closest-pair problem has divide-and-conquer solution

A divide-and-conquer algorithm works as follows.

1 Base Case: If the list contains two points, then they must be the

closest pair.

2 Divide: Divide the set into two halves (e.g., Sl and Sr in the figure

above). Put all points less than the midpoint m in Sl and all points greater than or equal to the midpoint in Sr.

3 Conquer: Find the closest-pair for each half ((l1, l2) for Sl and

(r1, r2) for Sr).

4 Glue: To find the closest pair in the entire set, select from 3 options: 1

closest pair in the left half ((l1, l2));

2

closest pair in the right half ((r1, r2));

3

a pair with one point each from Sl and Sr.

6 / 32

1D Glue procedure

A divide-and-conquer algorithm works as follows. Glue: To find the closest pair in the entire set, select from 3 options:

1

closest pair in the left half ((l1, l2));

2

closest pair in the right half ((r1, r2));

3

a pair with one point each from Sl and Sr.

For the closest pair to take a point from both sets, each point must be within distance dmin of the midpoint m between the two sets (here dmin = min(distance(l1, l2), distance(r1, r2))). Only the largest point in the left set lmax and the smallest point in the right set rmin could be closer than dmin. Compute distance(lmax,rmin) and update the closest pair if less than dmin.

7 / 32

Sorting solution.

・Sort by x-coordinate and consider nearby points. ・Sort by y-coordinate and consider nearby points.

23

Closest pair of points: first attempt

8 / 32

Sorting solution.

・Sort by x-coordinate and consider nearby points. ・Sort by y-coordinate and consider nearby points.

24

Closest pair of points: first attempt

8 9 / 32

25

Closest pair of points: second attempt

• Divide. Subdivide region into 4 quadrants.

10 / 32

26

Closest pair of points: second attempt

• Divide. Subdivide region into 4 quadrants.
• Obstacle. Impossible to ensure n / 4 points in each piece.

11 / 32

27

Closest pair of points: divide-and-conquer algorithm

・Divide: draw vertical line L so that n / 2 points on each side. ・Conquer: find closest pair in each side recursively. ・Combine: find closest pair with one point in each side. ・Return best of 3 solutions.

seems like Θ(N2)

L

12 21 8

12 / 32

Find closest pair with one point in each side, assuming that distance < δ.

・Observation: only need to consider points within δ of line L.

28

How to find closest pair with one point in each side?

L δ δ = min(12, 21)

12 21

13 / 32

29

How to find closest pair with one point in each side?

Find closest pair with one point in each side, assuming that distance < δ.

・Observation: only need to consider points within δ of line L. ・Sort points in 2 δ-strip by their y-coordinate. ・Only check distances of those within 11 positions in sorted list!

L δ

2 3 4 5 6 7

why 11?

1

δ = min(12, 21)

21 12

14 / 32

30

How to find closest pair with one point in each side?

• Def. Let si be the point in the 2 δ-strip, with the ith smallest y-coordinate.
• Claim. If | i – j | ≥ 12, then the distance

between si and sj is at least δ. Pf.

・No two points lie in same ½ δ-by-½ δ box. ・Two points at least 2 rows apart

have distance ≥ 2 (½ δ). ▪

• Fact. Claim remains true if we replace 12 with 7.

δ

27 29 30 31 28 26 25

δ

2 rows

½δ ½δ ½δ

39

i j

⋮ ⋮

15 / 32

31

Closest pair of points: divide-and-conquer algorithm

O(n log n) 2 T(n / 2) O(n) O(n log n) O(n)

CLOSEST-PAIR (p1, p2, …, pn)

Compute separation line L such that half the points are on each side of the line. δ1 ← CLOSEST-PAIR (points in left half). δ2 ← CLOSEST-PAIR (points in right half). δ ← min { δ1 , δ2 }. Delete all points further than δ from line L. Sort remaining points by y-coordinate. Scan points in y-order and compare distance between each point and next 11 neighbors. If any of these distances is less than δ, update δ. RETURN δ.

16 / 32

32

Closest pair of points: analysis

• Theorem. The divide-and-conquer algorithm for finding the closest pair of

points in the plane can be implemented in O(n log2 n) time. Lower bound. In quadratic decision tree model, any algorithm for closest pair (even in 1D) requires Ω(n log n) quadratic tests.

(x1 - x2) 2 + (y1 - y2) 2

Θ(1) if n = 1 T ( ⎡n / 2⎤ ) + T ( ⎣n / 2⎦ ) + O(n log n)

• therwise

T(n) =

17 / 32

33

Improved closest pair algorithm

• Q. How to improve to O(n log n) ?
• A. Yes. Don't sort points in strip from scratch each time.

・Each recursive returns two lists: all points sorted by x-coordinate,

and all points sorted by y-coordinate.

・Sort by merging two pre-sorted lists.

• Theorem. [Shamos 1975] The divide-and-conquer algorithm for finding the

closest pair of points in the plane can be implemented in O(n log n) time. Pf.

• Note. See SECTION 13.7 for a randomized O(n) time algorithm.

Θ(1) if n = 1 T ( ⎡n / 2⎤ ) + T ( ⎣n / 2⎦ ) + Θ(n)

• therwise

T(n) =

not subject to lower bound since it uses the floor function

18 / 32

2D Divide-Conquer-Glue

(1, 1) (2, 6) (3, 2) (5, 5) (6, 1) (6, 5) (9, 4)

Step 0: Get Sorted Lists seq = [(5,5),(1,1),(6,5),(3,2),(6,1),(9,4),(2,6)] seqx = [(1,1),(2,6),(3,2),(5,5),(6,1),(6,5),(9,4)] seqy = [(1,1),(6,1),(3,2),(9,4),(6,5),(5,5),(2,6)]

19 / 32

2D Divide-Conquer-Glue

(1, 1) (2, 6) (3, 2) (5, 5) (6, 1) (6, 5) (9, 4)

Divide

m = 5

lftx = [(1,1),(2,6),(3,2),(5,5)], rgtx=[(6,1),(6,5),(9,4)] lfty = [(1,1),(3,2),(5,5),(2,6)], rgty=[(6,1),(9,4),(6,5)] Conquer: invoke cpp helper recursively with left and right halves

19 / 32

2D Divide-Conquer-Glue

(1, 1) (2, 6) (3, 2) (5, 5) (6, 1) (6, 5) (9, 4)

Glue

L(dmin) R(dmin)

lfty = [(1,1),(3,2),(5,5),(2,6)], rgty=[(6,1),(9,4),(6,5)]

m = 5 dmin (3, 2) (5, 5) (6, 1) (6, 5)

ymin = [(6,1),(3,2),(6,5),(5,5)] newy = [(1,1),(6,1),(3,2),(9,4),(6,5),(5,5),(2,6)] Check 6 neighbors of every point in ymin for distance < dmin

d′

min < dmin

19 / 32

SECTION 5.5

DIVIDE AND CONQUER II

• master theorem
• integer multiplication
• matrix multiplication
• convolution and FFT

22 / 32

13

• Addition. Given two n-bit integers a and b, compute a + b.
• Subtraction. Given two n-bit integers a and b, compute a – b.

Grade-school algorithm. Θ(n) bit operations.

• Remark. Grade-school addition and subtraction algorithms are

asymptotically optimal.

Integer addition

1 1 1 1 1 1 1

1 1 1 1 1

+

1 1 1 1 1 1 1 1 1 1 23 / 32

• Multiplication. Given two n-bit integers a and b, compute a × b.

Grade-school algorithm. Θ(n2) bit operations.

• Conjecture. [Kolmogorov 1952] Grade-school algorithm is optimal.
• Theorem. [Karatsuba 1960] Conjecture is wrong.

14

Integer multiplication

1 1 1 1 1

×

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 24 / 32

15

To multiply two n-bit integers x and y:

・Divide x and y into low- and high-order bits. ・Multiply four ½n-bit integers, recursively. ・Add and shift to obtain result.

Divide-and-conquer multiplication

1

(2m a + b) (2m c + d) = 22m ac + 2m (bc + ad) + bd

2 3 4

c = ⎣ y / 2m⎦ d = y mod 2m m = ⎡ n / 2 ⎤

• Ex. x = 1 0 0 0 1 1 0 1 y = 1 1 1 0 0 0 0 1

a b c d

use bit shifting to compute 4 terms

a = ⎣ x / 2m⎦ b = x mod 2m

25 / 32

16

Divide-and-conquer multiplication

MULTIPLY(x, y, n)

IF (n = 1) RETURN x 𐄃 y. ELSE

m ← ⎡ n / 2 ⎤. a ← ⎣ x / 2m⎦; b ← x mod 2m. c ← ⎣ y / 2m⎦; d ← y mod 2m. e ← MULTIPLY(a, c, m). f ← MULTIPLY(b, d, m). g ← MULTIPLY(b, c, m). h ← MULTIPLY(a, d, m).

RETURN 22m e + 2m (g + h) + f.

26 / 32

17

• Proposition. The divide-and-conquer multiplication algorithm requires

Θ(n2) bit operations to multiply two n-bit integers.

• Pf. Apply case 1 of the master theorem to the recurrence:

Divide-and-conquer multiplication analysis

T(n) = 4T n/2

( )

recursive calls

     + Θ(n)

add, shift

   ⇒ T(n) = Θ(n2)

27 / 32

18

To compute middle term bc + ad, use identity: Bottom line. Only three multiplication of n / 2-bit integers.

Karatsuba trick

bc + ad = ac + bd – (a – b) (c – d) a = ⎣ x / 2m⎦ b = x mod 2m c = ⎣ y / 2m⎦ d = y mod 2m m = ⎡ n / 2 ⎤ = 22m ac + 2m (ac + bd – (a – b)(c – d)) + bd

1 2 3 1 3

(2m a + b) (2m c + d) = 22m ac + 2m (bc + ad ) + bd

middle term

28 / 32

19

Karatsuba multiplication

KARATSUBA-MULTIPLY(x, y, n)

IF (n = 1) RETURN x 𐄃 y. ELSE

m ← ⎡ n / 2 ⎤. a ← ⎣ x / 2m⎦; b ← x mod 2m. c ← ⎣ y / 2m⎦; d ← y mod 2m. e ← KARATSUBA-MULTIPLY(a, c, m). f ← KARATSUBA-MULTIPLY(b, d, m). g ← KARATSUBA-MULTIPLY(a – b, c – d, m).

RETURN 22m e + 2m (e + f – g) + f.

29 / 32

20

• Proposition. Karatsuba's algorithm requires O(n1.585) bit operations to

multiply two n-bit integers.

• Pf. Apply case 1 of the master theorem to the recurrence:
• Practice. Faster than grade-school algorithm for about 320-640 bits.

Karatsuba analysis

T(n) = 3 T(n / 2) + Θ(n) ⇒ T(n) = Θ(nlg 3) = O(n1.585).

30 / 32

21

Integer arithmetic reductions

Integer multiplication. Given two n-bit integers, compute their product.

problem arithmetic running time integer multiplication

a × b Θ(M(n))

integer division

a / b, a mod b Θ(M(n))

integer square

a 2 Θ(M(n))

integer square root

⎣√a ⎦ Θ(M(n))

integer arithmetic problems with the same complexity as integer multiplication

31 / 32

22

History of asymptotic complexity of integer multiplication

• Remark. GNU Multiple Precision Library uses one of five

different algorithm depending on size of operands.

year algorithm

• rder of growth

? brute force

Θ(n2)

1962 Karatsuba-Ofman

Θ(n1.585)

1963 Toom-3, Toom-4

Θ(n1.465), Θ(n1.404)

1966 Toom-Cook

Θ(n1 + ε)

1971 Schönhage–Strassen

Θ(n log n log log n)

2007 Fürer

n log n 2 O(log*n)

? ?

Θ(n)

number of bit operations to multiply two n-bit integers used in Maple, Mathematica, gcc, cryptography, ...

32 / 32