[PPT] - ADVANCED ALGORITHMS 2 LECTURE 4 ANNOUNCEMENTS Homework 0 grading PowerPoint Presentation

SLIDE 1

ADVANCED ALGORITHMS

LECTURE 4: D&C, DYNAMIC PROGRAMMING

1

SLIDE 2

LECTURE 4

ANNOUNCEMENTS

▸ Homework 0 grading (close to done) ▸ Homework 1 posted; due on Monday Sep 10, 11:59 PM ▸ Start early! (Post comments/questions on canvas…)

▸

Unless immediate, argue why algorithm is correct

▸

Three steps — algorithm description, proof of correctness, complexity analysis 2

SLIDE 3

LECTURE 4

LAST CLASS

3

▸ Divide and conquer

▸

Break instance into smaller pieces, solve pieces separately, combine solutions

▸

Instances disjoint in examples we saw — not necessary (today)

▸

Merge-sort algorithm

▸ Recurrences

▸

Guess and prove by induction

▸

Plug and chug

▸

Recurrence tree

link on

iMaster theorem

homepage

Akira Bazzi Theorem

SLIDE 4

LECTURE 4

DIVIDE AND CONQUER

4

PROBLEM INSTANCE

Divide step Solve sub-problems

FULL SOLUTION

Combine/“conquer”

SLIDE 5

LECTURE 4

TODAY’S PLAN

5

▸ More examples — divide and conquer

▸

Continue integer multiplication

▸

Median finding

▸ Introduction — dynamic programming

SLIDE 6

LECTURE 4

EXAMPLE 2: MULTIPLYING INTEGERS

6

Question: given two n digit numbers, how efficiently can we multiply them?

▸ Elementary school algorithm: takes time O(n2) ▸ Basic divide and conquer — recap

SLIDE 7

LECTURE 4

DIVIDE AND CONQUER

7 A = B = X Y Z W

A = 10n/2X + Y

Some observations:

B = 10n/2Z + W AB = 10nXZ + 10n/2(YZ + XW) + YW

▸ Divide and conquer alg: compute XZ, YZ, XW, YW, compute sum above ▸ Gives recurrence:

T(n) = 4T(n/2) + O(n)

n

Mz

X

OO

O

Mz

El

D

7L'm

azn digits

SLIDE 8

LECTURE 4

SOLVING RECURRENCE

8

Tht

4TH

Oln

1

Tfn

042

f

E g mastertheorem

Corl plug

a chug

Can

we

do better than elementary

school Arg

SLIDE 9

LECTURE 4

CAN WE DO BETTER?

9

Reason to hope: we only need to find three quantities: XZ, (XW + YZ), YW

SLIDE 10

LECTURE 4

CAN WE DO BETTER?

10

Reason to hope: we only need to find three quantities: XZ, (XW + YZ), YW Kartsuba’s observation:

XW + YZ = (X + Y)(W + Z) − XZ − YW

r

a

we needonly THREE multiplications

TIM

3 TIZI

Oln

SLIDE 11

LECTURE 4

OVERALL RUNNING TIME

11

TIM 3M

c

n

g

Cin 131C I 1347

log

C n 1 z

cn 1

32T ng

N

I

cnfltZztfzIt.r.JCatEHHcn.fzfzre

aani.s.s 37171

SLIDE 12

LECTURE 4

EVEN BETTER?

12

▸ Can we do linear time? ▸ HW1 — will see something approaching linear I 35

N

m

K

ite

2

N

a

Fast

Fourier Transform

Ofmlog n

time alg

42

Open problem if

we can do better

SLIDE 13

LECTURE 4

COMPUTING THE MEDIAN

13

Question: given an unsorted list A[0], A[1], …, A[n-1], find the (n/2)th smallest element Divide and conquer?

Sort

thearray

utput Ithsmallest O n log n

0cal

SLIDE 14

A

n

medians of the pieces don't give much meaningful

information

i

l

SLIDE 15

LECTURE 4

KEY INSIGHT: “APPROXIMATE MEDIAN” HELPS!

14

AH

Aln i

tf

Af AIDAN

TAIND Ifs

Fddi

12 elements

t

213

13

SLIDE 16

Observations 1

we

can now restrict search to a

2.13

sized array

een

2

needtobe careful

answer is

not median of sub array but f

1h

Smallest element

i l

a

qb6n

Given

an array

some

k find the Kth

smallest element Select A

K

1

a

select

A Z Select

Bg

E

all elements

in

Eu

1

n

T Iz

Oh

time takento

findsub array

Ent E

t C

n t

B

n

l t Zzt

E

t Ept

c'n

SLIDE 17

B

t

n

21

3

Select Ez Z

How does recursive call work

B

24313 B

x

Let

a

be

njrd smallest element of B

myth smallest element of B

is the smallest elementof B

select

B

Z F

Rida

select

A

K

wilt

a

find

approx median

select B k

tidesThat

y

cursive call10h

a

givesdetails

new array

B sat

IBILF IAI

SLIDE 18

LECTURE 4

MEDIAN OF MEDIANS IS APPROXIMATE MEDIAN!

15

Trick

AIA

Afi

AIn

Tfn

1 Fo 101Mt Tff

SLIDE 19

LECTURE 4

DIVIDE AND CONQUER — OVERVIEW

16

▸ Problem “cleanly” divides into not-too-many sub-problems ▸ Solutions easily (efficiently) combined ▸ Leads to simple, efficient solutions ▸ Analysis by induction, complexity by recurrences

SLIDE 20

LECTURE 4

OVERLAP IN RECURSIVE CALLS

17

▸ Divide & conquer: usually sub-problems don’t interact “much” ▸ What if they do? Can run into wastefulness — solving same sub-problem

many times…

▸ Motivation for next topic — dynamic programming

SLIDE 21

LECTURE 4

NEW PROBLEM: SUBSET SUM

18

Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.

is

Afol t Akid t AEs

B

all subsets

find if any

sum to S

2

n

time

SLIDE 22

LECTURE 4

DIVIDE AND CONQUER

19

▸ Try 1: divide into two halves

I

4

Alo

f

AlnD

T

T T

s

is.tl

1

n

Sti TIE

ntogs

s

1

logan

SLIDE 23

LECTURE 4

RUNNING TIME

20

log

n

SLIDE 24

LECTURE 4

TRY 2

21

▸ See if first element is picked or not

Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.

I

lookfords in AID

Aln B

look

for a sumof

a sumof

s Afo

in AID

Takes time 2h

AlnD

SLIDE 25

LECTURE 4

TRY 2

22

▸ See if first element is picked or not ▸ Is this really divide and conquer?

Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.

SLIDE 26

LECTURE 4

RUNNING TIME

23

SLIDE 27

LECTURE 4

LOOKING MORE CLOSELY — EXAMPLE

24 A[] = {1, 2, 3, 5, 7, 9, 10, 11}; S = 20

SLIDE 28

LECTURE 4

AVOIDING MULTIPLE SOLVES ON “SAME” INSTANCE

25

▸ Store answers!

▸

How much are we storing?

▸

Running time