Randomness in Computing L ECTURE 19 Last time Finding Hamiltonian - - PowerPoint PPT Presentation

4/7/2020

Randomness in Computing

LECTURE 19

Last time

• Finding Hamiltonian cycles in

random graphs

Today

• Probabilistic method

Sofya Raskhodnikova;Randomness in Computing

The probabilistic method

To prove that an object with required properties exists:

• 1. Define a distribution on objects.
• 2. Sample an object.
• 3. Prove that a sampled object has required properties with

positive probability.

• Sometimes proof of existence can be converted into efficient

randomized constructions.

• Sometimes they can be converted into deterministic

constructions (derandomization).

4/7/2020

Sofya Raskhodnikova; Randomness in Computing

Method 1: The counting argument

• 𝐿𝑜 = complete graph on 𝑜 vertices (𝑜-clique)

Proof: Define a random experiment:

Color each edge of 𝑳𝒐 independently and uniformly blue or red.

• Fix an ordering of the 𝑜

𝑙 different 𝑙-cliques.

• Let 𝑁𝑗 be the event that clique 𝑗 is monochromatic, for 𝑗 = 1, … , 𝑜

𝑙 Pr 𝑁𝑗 = 2 ⋅ 2− 𝑙

2

• Pr ⋃𝑗=1

𝑜 𝑙 𝑁𝑗 ≤ σ𝑗=1 𝑜 𝑙 Pr[𝑁𝑗] = 𝑜

𝑙 ⋅ 2− 𝑙

2 +1 < 1

• Probability of a coloring with no monochromatic 𝑙-clique is > 0.

4/7/2020

Sofya Raskhodnikova; Randomness in Computing Image by Richtom80 at English Wikipedia, CC BY-SA 3.0

Theorem

If 𝑜 𝑙 ⋅ 2− 𝑙

2 +1 < 1 then it is possible to color the edges

• f 𝐿𝑜 with two colors so that no 𝐿𝑙 is monochromatic.

Union Bound

Converting an existence proof into an efficient randomized construction

• Can we efficiently sample a coloring?
• How many samples do we need to generate

a coloring with no monochromatic 𝒍-clique?

– Probability of success is 𝑞 = 1 − 𝑜 𝑙 ⋅ 2− 𝑙

2 +1

– # of samples ∼Geom(𝑞), expectation: 1/𝑞 – Want: 1/𝑞 to be polynomial in the problem size – If 𝑞 = 1 − 𝑝(1), we get a Monte Carlo construction algorithm that errs w. p. 𝑝(1).

• To get a Las Vegas algorithm (always correct answers),

we need a poly-time procedure for checking if the coloring is monochromatic.

– If 𝑙 is constant, we can check that all 𝑜 𝑙 cliques are not monochromatic.

4/7/2020

Sofya Raskhodnikova; Randomness in Computing

Yes

Method 2: The expectation argument

• It can’t be that everybody is better (or worse) than the average.

Proof: Suppose to the contrary that Pr 𝑌 ≥ 𝜈 = 0. Then 𝜈 = 𝔽 𝑌 = ෍

𝑦

𝑦 Pr 𝑌 = 𝑦 < ෍

𝑦

𝜈 Pr 𝑌 = 𝑦 = 𝜈 ෍

𝑦

Pr 𝑌 = 𝑦 = 𝜈, a contradiction.

4/7/2020

Sofya Raskhodnikova; Randomness in Computing

Claim Let 𝑌 be a R.V. with 𝔽 𝑌 = 𝜈. Then Pr 𝑌 ≥ 𝜈 > 0 and Pr 𝑌 ≤ 𝜈 > 0.

≤ >

Example: Finding a large cut

Recall:

• A cut in a graph 𝐻 = (𝑊, 𝐹) is a partition of 𝑊 into two nonempty sets.
• The size of the cut is the number of edges that cross it.
• Finding a max cut is NP-hard.

4/7/2020

Sofya Raskhodnikova; Randomness in Computing

Theorem

Let 𝐻 be an undirected graph with 𝑛 edges. Then 𝐻 has a cut of size ≥ 𝑛/2.

Example: Existence of a large cut

Proof: Construct sets 𝐵 and 𝐶 of vertices by assigning each vertex to 𝐵 or 𝐶 uniformly and independently at random.

• For each edge 𝑓, let 𝑌𝑓 = ቊ1 if edge connects 𝐵 to 𝐶

0 otherwise 𝔽 𝑌𝑓 =1/2

• Let 𝑌 = # of edges crossing the cut.

𝔽[𝑌] = 𝔽 σ𝑓∈𝐹 𝑌𝑓 = σ𝑓∈𝐹 𝔽 𝑌𝑓 = 𝑛 ⋅

1 2 = 𝑛 2

There exists a cut (𝐵, 𝐶) of size 𝑛/2.

4/7/2020

Sofya Raskhodnikova; Randomness in Computing

Theorem

Let 𝐻 be an undirected graph with 𝑛 edges. Then 𝐻 has a cut of size ≥ 𝑛/2.

Example: Finding a large cut

• It is easy to choose a random cut
• Probability of success: 𝑞 = Pr 𝑌 ≥

𝑛 2

• An upper bound on 𝑌?

𝑛 2 = 𝔽 𝑌 = ෍

𝑗<𝑛/2

𝑗 ⋅ Pr[𝑌 = 𝑗] + ෍

𝑗≥𝑛/2

𝑗 ⋅ Pr[𝑌 = 𝑗] ≤ 𝑛 − 1 2 ⋅ 1 − 𝑞 + 𝑛 ⋅ 𝑞 𝑛 ≤ 𝑛 − 1 − 𝑛 − 1 ⋅ 𝑞 + 2𝑛 ⋅ 𝑞 𝑞 ≥ 1 𝑛 + 1

• Expected # of samples to find a large cut:
• Can test if a cut has ≥

𝑛 2 edges by counting edges crossing the cut (poly time)

4/7/2020

Sofya Raskhodnikova; Randomness in Computing

X ≤ 𝑛 Las Vegas ≤ 𝑛 + 1

Derandomization: conditional expectations

Finding a large cut

Idea: Place each vertex deterministically, ensuring that 𝔽 𝑌| placement so far ≥ 𝔽 𝑌 ≥ 𝑛 2

• R.V. 𝑍

𝑗 is 𝐵 or 𝐶, indicating which set vertex 𝑗 is placed in, ∀𝑗 ∈ [𝑜]

Base case: 𝔽 𝑌|𝑍

1 = 𝐵 = 𝔽 𝑌|𝑍 1 = 𝐶 = 𝔽 𝑌

Inductive step: Let 𝑧1, … , 𝑧𝑙 be placements so far (each is 𝐵 or 𝐶) and suppose 𝔽 𝑌|𝑍

1 = 𝑧1, … , 𝑍 𝑙 = 𝑧𝑙 ≥ 𝔽 𝑌 .

𝔽 𝑌|𝑍

1 = 𝑧1, … , 𝑍 𝑙 = 𝑧𝑙 = 1 2 𝔽 𝑌|𝑍 1 = 𝑧1, … , 𝑍 𝑙 = 𝑧𝑙, 𝑍 𝑙+1 = 𝐵

+ 1 2 𝔽 𝑌|𝑍

1 = 𝑧1, … , 𝑍 𝑙 = 𝑧𝑙, 𝑍 𝑙+1 = 𝐶

Then 𝔽 𝑌|𝑍

1 = 𝑧1, … , 𝑍 𝑙+1 = 𝑧𝑙+1 ≥ 𝔽 𝑌|𝑍 1 = 𝑧1, … , 𝑍 𝑙 = 𝑧𝑙 ≥ 𝔽 𝑌

4/7/2020

Sofya Raskhodnikova; Randomness in Computing

By symmetry (it doesn’t matter where the first node is) By Law of Total Expectation Pick 𝒛𝒍+𝟐 to maximize conditional probability Pick 𝒛𝒍+𝟐 to maximize conditional probability

Finding a large cut: derandomization

When the dust settles

• Place vertex 𝑙 + 1 on the side with fewer neighbors,

breaking ties arbitrarily

4/7/2020

Sofya Raskhodnikova; Randomness in Computing

𝐵 𝐶 undecided

1 1 1/2

𝒍 + 𝟐

Example 2: Maximum satisfiability (MAX-SAT)

Logical formulas

• Boolean variables: variables that can take on values T/F (or 1/0)
• Boolean operations: ∨, ∧, and ¬
• Boolean formula: expression with Boolean variables and ops

SAT (deciding if a given formula has a satisfying assignment) is NP-complete

• Literal:

A Boolean variable or its negation.

• Clause:

OR of literals.

• Conjunctive normal form (CNF): AND of clauses.

MAX-SAT: Given a CNF formula, find an assignment satisfying as many clauses as possible.

• Assume no clause contains 𝑦 and ҧ

𝑦 (o.w., it is always satisfied).

4/7/2020

Sofya Raskhodnikova; Randomness in Computing

Ex: x1 = 1, x2 = 1, x3 = 0 satisfies the formula.

฀ x1  x2  x3

  

x1  x2  x3

  

x2  x3

  

x1  x2  x3

 

𝑦𝑗 or ഥ 𝑦𝑗 𝐷1 = 𝑦1 ∨ 𝑦2 ∨ 𝑦3 𝐷1 ∧ 𝐷2 ∧ 𝐷3 ∧ 𝐷4

Example 2: MAX-SAT

Proof: Assign values 0 or 1 uniformly and independently to each variable.

• 𝑌𝑗 = indicator R.V. for clause 𝑗 being satisfied.
• 𝑌 = # of satisfied clauses = σ𝑗∈[𝑛] 𝑌𝑗
• Pr 𝑌𝑗 = 1 = 1 − 2−𝑙𝑗

𝔽[𝑌] = ෍

𝑗∈[𝑛]

𝔽 𝑌𝑗 = ෍

𝑗∈[𝑛]

(1 − 2−𝑙𝑗) ≥ 𝑛(1 − 2−𝑙)

• There exists an assignment satisfying that many clauses.

4/7/2020

Sofya Raskhodnikova; Randomness in Computing

Theorem

Given 𝑛 clauses, let 𝑙𝑗 = # literals in clause 𝑗, for 𝑗 ∈ [𝑛]. Let 𝑙 = min

𝑗∈[𝑛] 𝑙𝑗. There is a truth assignment that satisfies at least

෍

𝑗∈[𝑛]

1 − 2−𝑙𝑗 ≥ 𝑛 1 − 2−𝑙 clauses.

Example 3: Large sum-free subset

• Given a set 𝑩 of positive integers, a sum-free subset 𝑻 ⊆ 𝑩

contains no three elements 𝒋, 𝒌,𝒍 ∈ 𝑻 satisfying 𝒋 + 𝒌 = 𝒍.

• Goal: find as large as 𝑻 as possible.
• Examples: A = {2, 3, 4, 5, 6, 8, 10}

A = {1, 2, 3, 4, 5, 6, 8, 9, 10, 18}

4/7/2020

Sofya Raskhodnikova; Randomness in Computing

Theorem

Every set 𝑩 of 𝒐 positive integers contains a sum-free subset of size greater than 𝒐/3.

Finding a large sum-free subset

A randomized algorithm

• 1. Let 𝒒 > max element of 𝑩 be a prime, where 𝒒 = 𝟒𝒍 + 𝟑.

//The other choice, 𝟒𝒍 + 𝟐, would also work.

• 2. Select a number 𝒓 uniformly at random from {1, 𝒒 − 𝟐}.
• 3. Map each element 𝒖 ∈ 𝑩 to 𝒖𝒓 mod 𝒒.

4. 𝑻  all elements of 𝑩 that got mapped to {𝒍 + 𝟐, … , 𝟑𝒍 + 𝟐}.

• 5. Return 𝑻.

Need to prove:

• 𝑻 is sum-free
• The expected number of elements from 𝑩 that are mapped to

{𝒍 + 𝟐, … , 𝟑𝒍 + 𝟐} is > 𝒐/𝟒.

4/7/2020

Sofya Raskhodnikova; Randomness in Computing; based on slides by Surender Baswana

Showing that 𝑻 is sum-free

16

• Let 𝒋 and 𝒌 be any two elements in 𝑻.
• Say 𝒋 is mapped to 𝜷; 𝒌 is mapped to 𝜸; 𝜷, 𝜸 ∈ 𝒍 + 𝟐, 2𝒍 + 𝟐
• Then 𝜷 = 𝒋𝒓 𝐧𝐩𝐞 𝒒

and 𝜸 = 𝒌𝒓 𝐧𝐩𝐞 𝒒

• We need to show that 𝒋 + 𝒌, if present in 𝑩, is not mapped to

[𝒍 + 𝟐, 2𝒍 + 𝟐].

• 𝒋 + 𝒌 is mapped to ??

Argue that

• (𝜷 + 𝜸) must be greater than 2𝒍 + 𝟐.
• If (𝜷 + 𝜸) > 𝒒, then (𝜷 + 𝜸) 𝐧𝐩𝐞 𝒒 is at most 𝒍.

1 2 … 𝒍 𝒍 + 𝟐 … … … 𝟑𝒍 + 𝟐 … 3𝒍 + 𝟐 𝜷 𝜸 (𝜷 + 𝜸) 𝐧𝐩𝐞 𝒒

Sofya Raskhodnikova; Randomness in Computing; based on slides by Surender Baswana

The expected size of 𝑻

• 1. Let 𝒒 > max element of 𝑩 be a prime, where 𝒒 = 𝟒𝒍 + 𝟑.
• 2. Select a number 𝒓 uniformly at random from {1, 𝒒 − 𝟐}.
• 3. Map each element 𝒖 ∈ 𝑩 to 𝒖𝒓 mod 𝒒.

4. 𝑻  all elements of 𝑩 that got mapped to {𝒍 + 𝟐, … , 𝟑𝒍 + 𝟐}. Main idea: Every element 𝒖 ∈ 𝑩 gets mapped to 𝒖𝒓 mod 𝒒, which is a uniformly random element of {𝟐, … , 𝟒𝒍 + 𝟐}. Pr 𝒖 is selected to be in 𝑻 = |{𝒍 + 𝟐, … , 𝟑𝒍 + 𝟐}| |{𝟐, … , 𝟒𝒍 + 𝟐}| > 1/3

4/7/2020

Sofya Raskhodnikova; Randomness in Computing; based on slides by Surender Baswana

Example 3: Large sum-free subset

• Given a set 𝑩 of positive integers, a sum-free subset 𝑻 ⊆ 𝑩

contains no three elements 𝒋, 𝒌,𝒍 ∈ 𝑻 satisfying 𝒋 + 𝒌 = 𝒍.

• Goal: find as large as 𝑻 as possible.
• Examples: A = {2, 3, 4, 5, 6, 8, 10}

A = {1, 2, 3, 4, 5, 6, 8, 9, 10, 18}

4/7/2020

Sofya Raskhodnikova; Randomness in Computing

Theorem

Every set 𝑩 of 𝒐 positive integers contains a sum-free subset of size greater than 𝒐/3.