ADVANCED ALGORITHMS 2 LECTURE 9 ANNOUNCEMENTS a bit lengthy Cask - - PowerPoint PPT Presentation

ADVANCED ALGORITHMS

LECTURE 9: LOCAL SEARCH, SHORTEST PATH

1

LECTURE 9

ANNOUNCEMENTS

▸ Homework 2 out — due next Friday ▸ Contacting the TAs: adv-algorithms-ta-fall18@googlegroups.com

2 a bit lengthy

Cask for clarifications

LECTURE 9

LAST CLASS

3

▸ Greedy algorithm has a non-trivial approximation guarantee ▸ Analysis the tricky part of greedy algorithms ▸ Local search — start with any feasible solution, improve it

Approximation algorithm

O 63

times

• pt

LECTURE 9

EXAMPLE — MATCHING

4

Matching: suppose we have n children, n gifts and “happiness” values Hij. Assign gifts to children to maximize “total happiness”

▸ Local search algorithm: ▸ Start with any assignment ▸ For each pair of children, see if “swapping gifts” improves total

cost

i

LECTURE 9

NO IMPROVEMENT => INEQUALITY

5

▸ Denote “locally optimal” solution by assignment p1, p2, …, pn

Hi,pi + Hj,pj ≥ Hi,pj + Hj,pi

H

tate n

a perm

• f

l

n

Vij

because otherwise

we would have swapped

gifts of children i j

LECTURE 9

LOCALLY OPTIMUM => APPROXIMATELY OPTIMAL

6

• Theorem. Consider any solution that is “locally optimum”. Its cost is at

least (1/2) * OPT

▸ Denote “locally optimal” solution by assignment q1, q2, …, qn

vahyof

Hiq

value of solution

we found

Hi pi

LECTURE 9

APPROXIMATION

7

q

Let j be the index

Ep

that

was assigned

in

• ur solution

ti

I

e

Pj

hi

E

O

Hi.it

iai

5 Hi.aitHj pi3 HIIi

11

111

valueofoPT valostanour

Hj Qi

solution

111

2x

val

• f our sobs

LECTURE 9

CLASSIC EXAMPLE — GRADIENT DESCENT

8 Warning: multivariate calculus coming up

• Problem. Given a convex function f over a domain D, find

D ⊆ ℝn

argminx∈D f(x)

I

f

IR 7112

f

n

• fonnex

IF

fh.tn7 f.tHt

tnIm

1

affulth Hfly

V

A

Telo

Fact

If

f

satisfies fluty

a f beltfly

2

then the

min of f

• ver

a

convex domain D

is unique

is

LECTURE 9

LOCAL SEARCH

9

• Problem. Given a convex function f over a domain D, find

argminx∈D f(x)

start

with any

ft

lookat all

inbis

e It

if one ofthem

ftp.o

has lower f17

value

moveto

that nbr continue

fi IR

IR

Start with

any

in D

centerraps

• D

Bank

an

Algorithm

if

a yE Ballingyfp't

n

fly

flu

move

The figanalocally optimal ptf y

repeat

Claim This

will always find the

min

value

for f

y

Xn

I

a n't

fly E Xfln 11 a

If

flat

sik flat

then fly c flu

LECTURE 9

LOCAL SEARCH

10

• Problem. Given a convex function f over a domain D, find

argminx∈D f(x)

▸ What is a good direction to move? ▸ Need domain D to be convex

this

is

an easy

step

just

move

in the direction

• pposite the

gradient

LECTURE 9

COMMENTS ON GRADIENT DESCENT

11

▸ How many steps to take? ▸ What is the step size? (should it be fixed?)

forinash

Perhaps the most used

algorithm in ML

LECTURE 9

LOCAL SEARCH SUMMARY

12

▸ Start with solution, move to “neighboring” solution by changing

current solution slightly

▸ How neighbors are chosen determines complexity ▸ Not always optimal ▸ Number of steps can be LARGE if not done carefully — standard trick:

always make sure you improve by significant amount

in Hw

2

LECTURE 9

GRAPH ALGORITHMS

13

LECTURE 9

GRAPH BASICS

14

▸ Common data structures — adjacency lists, adjacency matrix ▸ Basic procedures: Breadth first search, depth first search ▸ TODAY: shortest paths — one of the first polynomial time

algorithms!

t

ANTI

1950

s

LECTURE 9

SHORTEST PATH PROBLEM

15

Shortest path: given a (possibly directed) graph G = (V, E), and two vertices u, v, find the length of the shortest path from u to v

▸ What if the graph is not weighted? ▸ Negative weight edges?

luv

u

n

even

If

tons

BFSfindsthe shortestpath

7

us

edge weights

1

Edgeof wt 3

LECTURE 9

BFS + DISTANCE UPDATES?

16

U

uol.ae

H v E G

maintain

• distfu

initialized using BFS

W for

an edge ij

see if

distfj

distlittlij

d

we know that distfj

is incorrect

distlj

distfilthy

LECTURE 9

DISTANCE UPDATES — FULL ALGORITHM

17

LECTURE 9

CORRECTNESS

18

LECTURE 9

DIJKSTRA’S ALGORITHM — CAN WE AVOID MULTIPLE VISITS?

19

LECTURE 9

DIJKSTRA’S ALGORITHM

20