Randomization Algorithm Theory WS 2012/13 Fabian Kuhn Last Lecture - - PowerPoint PPT Presentation

Chapter 6

Randomization

Algorithm Theory WS 2012/13 Fabian Kuhn

Algorithm Theory, WS 2012/13 Fabian Kuhn 2

Last Lecture

Due to technical problems, we have no recordings  We discussed 2 problems

• Contention resolution:

We will put the copy of a book chapter on the web

(beginning of Ch. 13 of book Algorithm Design by Kleinberg & Tardos)

• Miller‐Rabin primality test:

Use slides, colleagues, wikipedia page

– For the exam, the number‐theoretic details are not relevant

Algorithm Theory, WS 2012/13 Fabian Kuhn 3

Randomized Quicksort

Quicksort:

function Quick (: sequence): sequence; {returns the sorted sequence } begin if # 1 then return else { choose pivot element in ; partition into ℓ with elements , and with elements return end;

• ℓ
• Quick(ℓ) Quick()

Algorithm Theory, WS 2012/13 Fabian Kuhn 4

Randomized Quicksort Analysis

Randomized Quicksort: pick uniform random element as pivot Running Time of sorting elements:

• Let’s just count the number of comparisons
• In the partitioning step, all 1 non‐pivot elements have to be

compared to the pivot

• Number of comparisons:

#

• If rank of pivot is :

recursive calls with and elements

Algorithm Theory, WS 2012/13 Fabian Kuhn 5

Randomized Quicksort Analysis

Random variables:

• : total number of comparisons (for a given array of length )
• : rank of first pivot
• ℓ, : number of comparisons for the 2 recursive calls

1 ℓ Law of Total Expectation: ℙ ⋅ |

• ℙ ⋅ 1 ℓ |

Algorithm Theory, WS 2012/13 Fabian Kuhn 6

Randomized Quicksort Analysis

We have seen that: ℙ ⋅ 1 ℓ |

• Define:
• : expected number of comparisons when sorting elements

ℓ 1 Recursion: ⋅

Algorithm Theory, WS 2012/13 Fabian Kuhn 7

Randomized Quicksort Analysis

Theorem: The expected number of comparisons when sorting elements using randomized quicksort is 2 ln . Proof: 1 ⋅ 1 1

• ,

0 0

Algorithm Theory, WS 2012/13 Fabian Kuhn 8

Randomized Quicksort Analysis

Theorem: The expected number of comparisons when sorting elements using randomized quicksort is 2 ln . Proof: 1 4 ⋅ ln

• ln ln

2 4 ln ln 2 4

Algorithm Theory, WS 2012/13 Fabian Kuhn 9

Alternative Analysis

Array to sort: [ 7 , 3 , 1 , 10 , 14 , 8 , 12 , 9 , 4 , 6 , 5 , 15 , 2 , 13 , 11 ] Viewing quicksort run as a tree:

Algorithm Theory, WS 2012/13 Fabian Kuhn 10

Comparisons

• Comparisons are only between pivot and non‐pivot elements
• Every element can only be the pivot once:

 every 2 elements can only be compared once!

• W.l.o.g., assume that the elements to sort are 1,2, … ,
• Elements and are compared if and only if either or is a

pivot before any element : is chosen as pivot

– i.e., iff is an ancestor of or is an ancestor of

ℙ comparison betw. and 2 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 11

Counting Comparisons

Random variable for every pair of elements , : 1, if there is a comparison between and 0,

• therwise

Number of comparisons:

• What is ?

Algorithm Theory, WS 2012/13 Fabian Kuhn 12

Randomized Quicksort Analysis

Theorem: The expected number of comparisons when sorting elements using randomized quicksort is 2 ln . Proof:

• Linearity of expectation:

For all random variables , … , and all , … , ∈ ,

• .

Algorithm Theory, WS 2012/13 Fabian Kuhn 13

Randomized Quicksort Analysis

Theorem: The expected number of comparisons when sorting elements using randomized quicksort is 2 ln . Proof: 2 1 1

• 2 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 14

Types of Randomized Algorithms

Las Vegas Algorithm:

• always a correct solution
• running time is a random variable
• Example: randomized quicksort, contention resolution

Monte Carlo Algorithm:

• probabilistic correctness guarantee (mostly correct)
• fixed (deterministic) running time
• Example: primality test

Algorithm Theory, WS 2012/13 Fabian Kuhn 15

Minimum Cut

Reminder: Given a graph , , a cut is a partition ,

• f such that ∪ , ∩ ∅, , ∅

Size of the cut , : # of edges crossing the cut

• For weighted graphs, total edge weight crossing the cut

Goal: Find a cut of minimal size (i.e., of size ) Maximum‐flow based algorithm:

• Fix , compute min ‐‐cut for all
• ⋅

per ‐ cut

• Gives an O

‐algorithm Best‐known deterministic algorithm: log

Algorithm Theory, WS 2012/13 Fabian Kuhn 16

Edge Contractions

• In the following, we consider multi‐graphs that can have

multiple edges (but no self‐loops) Contracting edge , :

• Replace nodes , by new node
• For all edges , and , , add an edge ,
• Remove self‐loops created at node

not ok

• k
• ,

contract ,

Algorithm Theory, WS 2012/13 Fabian Kuhn 17

Properties of Edge Contractions

Nodes:

• After contracting , , the new node represents and
• After a series of contractions, each node represents a subset of

the original nodes Cuts:

• Assume in the contracted graph, represents nodes ⊂
• The edges of a node in a contracted graph are in a one‐to‐one

correspondence with the edges crossing the cut , ∖

• ,

,

• ,

, , ,

• ,

, , ,

, , , , , , ,

Algorithm Theory, WS 2012/13 Fabian Kuhn 18

Randomized Contraction Algorithm

Algorithm: while there are 2 nodes do contract a uniformly random edge return cut induced by the last two remaining nodes

(cut defined by the original node sets represented by the last 2 nodes)

Theorem: The random contraction algorithm returns a minimum cut with probability at least 1 ⁄ .

• We will show this next.

Theorem: The random contraction algorithm can be implemented in time .

• There are 2 contractions, each can be done in time .
• You will show this in the exercises.

Algorithm Theory, WS 2012/13 Fabian Kuhn 19

Contractions and Cuts

Lemma: If two original nodes , ∈ are merged into the same node of the contracted graph, there is a path connecting and in the original graph s.t. all edges on the path are contracted. Proof:

• Contracting an edge , merges the node sets represented by

and and does not change any of the other node sets.

• The claim the follows by induction on the number of edge

contractions.

Algorithm Theory, WS 2012/13 Fabian Kuhn 20

Contractions and Cuts

Lemma: During the contraction algorithm, the edge connectivity (i.e., the size of the min. cut) cannot get smaller. Proof:

• All cuts in a (partially) contracted graph correspond to cuts of

the same size in the original graph as follows:

– For a node of the contracted graph, let be the set of original nodes that have been merged into (the nodes that represents) – Consider a cut , of the contracted graph – , with ≔

∈

, ≔

∈

is a cut of . – The edges crossing cut , are in one‐to‐one correspondence with the edges crossing cut , .

Algorithm Theory, WS 2012/13 Fabian Kuhn 21

Contraction and Cuts

Lemma: The contraction algorithm outputs a cut , of the input graph if and only if it never contracts an edge crossing , . Proof:

• 1. If an edge crossing , is contracted, a pair of nodes ∈ ,

∈ is merged into the same node and the algorithm outputs a cut different from , .

• 2. If no edge of , is contracted, no two nodes ∈ , ∈

end up in the same contracted node because every path connecting and in contains some edge crossing , In the end there are only 2 sets  output is ,

Algorithm Theory, WS 2012/13 Fabian Kuhn 22

Getting The Min Cut

Theorem: The probability that the algorithm outputs a minimum cut is at least 2 1 ⁄ . To prove the theorem, we need the following claim: Claim: If the minimum cut size of a multigraph (no self‐loops) is , has at least 2 ⁄ edges. Proof:

• Min cut has size ⟹ all nodes have degree

– A node of degree gives a cut , ∖

• f size
• Number of edges

⁄ ⋅ ∑ deg

Algorithm Theory, WS 2012/13 Fabian Kuhn 23

Getting The Min Cut

Theorem: The probability that the algorithm outputs a minimum cut is at least 2 1 ⁄ . Proof:

• Consider a fixed min cut , , assume , has size
• The algorithm outputs , iff none of the edges crossing

, gets contracted.

• Before contraction , there are 1 nodes

 and thus 1 2 ⁄ edges

• If no edge crossing , is contracted before, the probability to

contract an edge crossing , in step is at most

• 1

2

• 2

1 .

Algorithm Theory, WS 2012/13 Fabian Kuhn 24

Getting The Min Cut

Theorem: The probability that the algorithm outputs a minimum cut is at least 2 1 ⁄ . Proof:

• If no edge crossing , is contracted before, the probability to

contract an edge crossing , in step is at most ⁄ .

• Event : edge contracted in step is not crossing ,

Algorithm Theory, WS 2012/13 Fabian Kuhn 25

Getting The Min Cut

Theorem: The probability that the algorithm outputs a minimum cut is at least 2 1 ⁄ . Proof:

• ℙ | ∩ ⋯ ∩

⁄

• No edge crossing , contracted: event ⋂

Algorithm Theory, WS 2012/13 Fabian Kuhn 26

Randomized Min Cut Algorithm

Theorem: If the contraction algorithm is independently repeated log times, one instance returns a minimum cut w.h.p. Proof:

• Probability to not get a minimum cut in ⋅

2 ⋅ ln iterations: 1 1

• 2

⋅ ⋅

1