Randomness in Computing L ECTURE 1 Randomness in Computing Course - - PowerPoint PPT Presentation

1/21/2020

Sofya Raskhodnikova

Randomness in Computing

LECTURE 1

Randomness in Computing

• Course information
• Verifying polynomial identities
• Probability Amplification

Probability Review

Sofya Raskhodnikova;Randomness in Computing

Course information

• 1. Course staff
• 2. Course website(s)
• 3. Piazza bonus
• 4. Prerequisites
• 5. Textbook(s)

6. Syllabus 7. Homework logistics 8. Collaboration policy 9. Exams and grading

Sofya Raskhodnikova; Randomness in Computing

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Tips for the course

• Concepts in this course take some time to sink in: be

careful not to fall behind.

• Do the assigned reading on each topic before the

corresponding lecture.

• Attend the lectures: most of the material will be presented
• n the blackboard (and some of it is not in the book).
• Attend the discussions: practice problem solving.
• Take advantage of office hours.
• Be active in lectures and on piazza.
• Allocate lots of time for the course: comparable to a

project course, but spread more evenly.

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Tips for the course: HW

• Start working on HW early.
• Spread your HW time over multiple days.
• You can work in groups (up to 4 people), but

spend 1-2 hours thinking about it on your own before your group meeting.

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Tips: learning problem solving

To learn problem solving, you have to do it:

• Try to think how you would solve any presented

problem before you read/hear the answer.

• Do exercises in addition to HW.

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Tips: how to read a math text

• Not like reading a mystery novel.
• The goal is not to get the answers, but to learn

the techniques.

• Always try to foresee what is coming next.
• Always think how you would approach a

problem before reading the solution.

• This applies to things that are not explicitly

labeled as problems.

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Skills we will work on

• Mathematical reasoning
• Expressing your ideas

– abstractly (suppress inessential details) – precisely (rigorously)

• Probabilistic thinking
• Algorithmic thinking
• Problem solving
• Having FUN with all of the above!!!

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Uses of Randomness in Computing

• To speed up algorithms.
• To enable new applications:

– Symmetry breaking in distributed algorithms, cryptography, privacy, online games and gambling.

• To simulate real world events in physical systems:

model them as happening randomly.

• To analyze algorithms when data is generated from

some distribution:

– learning theory, data compression.

• To analyze algorithms when errors happen randomly

– error-correcting codes.

• Analyzing statistics from sampling.

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

§1.1 (MU) Verifying Polynomial Identities

• 𝑦 + 1

𝑦 − 2 𝑦 + 3 𝑦 − 4 𝑦 + 5 𝑦 − 6 ≡? 𝑦6 − 7𝑦 + 37 Task: Given two polynomials 𝐺 𝑦 and 𝐻 𝑦 , verify if 𝐺 𝑦 ≡ 𝐻 𝑦 . Idea 1 (deterministic): Convert both polynomials to canonical form 𝑗=0

𝑒

𝑑𝑗𝑦𝑗 .

• If 𝐺 𝑦 is given as Π𝑗=1

𝑒

𝑦 − 𝑏𝑗 , conversion by consecutively multiplying monomials requires Θ(𝑒2) multiplications of coefficients.

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Faster with Fourier Transform

§1.1 (MU) Verifying Polynomial Identities Task: Given two polynomials 𝐺 𝑦 and 𝐻 𝑦 , verify if 𝐺 𝑦 ≡ 𝐻 𝑦 . Idea 2 (randomized): Evaluate the polynomials on random integers. Error Analysis: Probability of accepting incorrectly Pr error = Pr 𝑠 is a root of 𝐺 𝑦 − 𝐻 𝑦 ≤ 𝑒 100𝑒 = 1 100

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

1. Pick 𝑠 uniformly from 1, … , 100𝑒 . 2. Compute 𝐺(𝑠) and 𝐻(𝑠). 3. 3. 𝐬𝐟𝐤𝐟𝐝𝐮 if 𝐺 𝑠 ≠ 𝐻 𝑠 ; o. w. 𝐛𝐝𝐝𝐟𝐪𝐮.

Let 𝑒 = max degree of 𝐺(𝑦) and 𝐻(𝑦)

O(d) ops for product form

Fundamental Theorem of Algebra

A polynomial of degree d has at most d roots.

1-sided error

Review: Axioms of Probability

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Probability space has three components

• Sample space Ω
• Family of allowable events 𝐹 ⊆ Ω
• A probability function Pr that maps events E to ℝ such that
• Pr 𝐹 ∈ [0,1] for any event 𝐹;
• Pr Ω = 1;
• For any finite or countable sequence of pairwise disjoint

events 𝐹1, 𝐹2, … , Pr

𝑗≥1

𝐹𝑗 =

𝑗≥1

Pr 𝐹𝑗 .

Review: Inclusion-Exclusion Principle

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

For any two events 𝐹1 and 𝐹2, Pr 𝐹1 ∪ 𝐹2 = Pr(𝐹1) + Pr 𝐹2 − Pr 𝐹1 ∩ 𝐹2 . For any 𝑜 events 𝐹1, 𝐹2, … , 𝐹𝑜, Pr

𝑗∈[𝑜]

𝐹𝑗 =

𝑗∈[𝑜]

Pr 𝐹𝑗 −

𝑗<𝑘

Pr 𝐹𝑗 ∩ 𝐹

𝑘

+

𝑗<𝑘<𝑙

Pr 𝐹𝑗 ∩ 𝐹

𝑘 ∩ 𝐹𝑙 − ⋯ + −1 𝑜+1 Pr 𝑗∈ 𝑜

𝐹𝑗 .

For any three events 𝐹1, 𝐹2 and 𝐹3,

Pr 𝐹1 ∪ 𝐹2 ∪ 𝐹3 = Pr(𝐹1) + Pr 𝐹2 + Pr 𝐹3 − Pr 𝐹1 ∩ 𝐹2 − Pr 𝐹1 ∩ 𝐹3 − Pr 𝐹2 ∩ 𝐹3 + Pr 𝐹1 ∩ 𝐹2 ∩ 𝐹3

Review: Union Bound

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Union Bound

For finite or countable sequence of events 𝐹1, 𝐹2, … , Pr

𝑗≥1

𝐹𝑗 ≤

𝑗≥1

Pr 𝐹𝑗 .

Probability Amplification

• Our algorithm for verifying polynomial identities accepts

incorrectly with probability ≤

1 100

Idea: Repeat the algorithm and accept if all iterations accept.

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Pr error in all 𝑙 iterations ≤ 1 100

𝑙

Review: Independence

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Independent events

• Two events 𝐹1 and 𝐹2 are independent if

Pr 𝐹1 ∩ 𝐹2 = Pr 𝐹1 ⋅ Pr 𝐹2 .

• Events 𝐹1, … , 𝐹𝑜 are mutually independent if,

for every subset 𝐽 ∈ 𝑜 , Pr

𝑗∈𝐽

𝐹𝑗 =

𝑗∈𝐽

Pr 𝐹𝑗 .

• If all pairs of events among 𝐹1, … , 𝐹𝑜 are independent then

𝐹1, … , 𝐹𝑜 are pairwise independent.

• Pairwise independence does not necessarily imply mutual

independence!

[𝒐] denotes {𝟐, 𝟑, … , 𝒐}

Independence: Example

• We toss a fair coin twice.
• Let A be the event that the 1st flip is HEADS.
• Let B be the event that the 2nd flip is HEADS.
• Let C be the event that both flips are the same.

Are events A,B,C pairwise independent? Are they mutually independent?

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Review: Conditional Probability

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Conditional Probability

The conditional probability of event 𝐹 given event 𝐺 is Pr 𝐹 𝐺 = Pr(E ∩ 𝐺) Pr(𝐺) ⋅

• When 𝐹 and 𝐺 are independent,

Pr 𝐹 𝐺 = Pr(E ∩ 𝐺) Pr(𝐺) = Pr E ⋅ Pr(𝐺) Pr(𝐺) = Pr 𝐹 .

Well defined only if Pr 𝑮 ≠ 𝟏

Review question

• Event E: the numbers on two dice sum to 8.
• Event F: the numbers on two dice are both even.
• What is the probability of E given that F occurred,

Pr(𝐹|𝐺)?

• A. Less than 1/3
• B. 1/3
• C. Greater than 1/3, but less than 2/3
• D. 2/3
• E. Greater than 2/3

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Review question

Card dealing We deal two cards. What is the probability that the second card is an ace, given that the first is an ace?

• A. 3/52
• B. 3/51
• C. 4/52
• D. 5/52
• E. None of the answers above are correct.

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Product rule

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

For any two events 𝐹1 and 𝐹2,

Pr 𝐹1 ∩ 𝐹2 = Pr(𝐹1) ⋅ Pr 𝐹2|𝐹1 .

For all events 𝐹1, … , 𝐹𝑜,

Pr(∩𝑗=1

𝑜

𝐹𝑗) = Pr 𝐹1 ⋅ Pr 𝐹2 𝐹1 ⋅ … ⋅ (𝐹𝑜| ∩𝑗=1

𝑜−1 𝐹𝑗)

Sampling without replacement

• Let 𝐹𝑗 be the event that we choose a root in iteration 𝑗
• It is 0 if 𝑙 > 𝑒.
• If 𝑙 ≤ 𝑒, then

1/21/2020

Sofya Raskhodnikova; Randomness in Computing

Pr 𝐹

𝑘 𝐹1 ∩ ⋯ ∩ 𝐹 𝑘−1 =

𝑒 − (𝑘 − 1) 100𝑒 − (𝑘 − 1) Pr error in all 𝑙 iterations = Pr 𝐹1 ∩ ⋯ ∩ 𝐹𝑙 = Pr 𝐹1 ⋅ Pr 𝐹2 𝐹1 ⋅ … ⋅ Pr[𝐹𝑙|𝐹1 ∩ ⋯ ∩ 𝐹𝑙−1] Pr error in all 𝑙 iterations ≤ 1 100

𝑙