[PPT] - Foundations of Computing II Lecture 5: Introduction to probability PowerPoint Presentation

SLIDE 1

CSE 312

Foundations of Computing II

Lecture 5: Introduction to probability

Stefano Tessaro

tessaro@cs.washington.edu

1

SLIDE 2

Organization

HW1 – Gradescope info soon
Note office hours change

– My own are on Monday 3-4pm right now. – Few more have moved – check homepage!

https://forms.gle/QA2ASR4LzepGEVKT7

– Slides online

2

SLIDE 3

Today

Combinatorics wrap-up: Pigeonhole principle
Introduction to probability

3

SLIDE 4

Pigeonhole Principle – Goal Mostly showing that a set ! has at least a certain size.

4

SLIDE 5

Pigeonhole Principle

5

If there are " pigeons in # < " holes, then one hole must contain at least two pigeons!

SLIDE 6

Pigeonhole Principle – Refined version

6

If there are " pigeons in # < " holes, then one hole must contain at least

% & pigeons!

SLIDE 7

Pigeonhole Principle – Refined version

7

If there are " pigeons in # < " holes, then one hole must contain at least

% & pigeons!

Proof. Assume there are < "/# pigeons per hole.

Then, there are < #×

% & = " pigeons overall.

A contradiction!

SLIDE 8

Pigeonhole Principle – Even stronger version

8

If there are " pigeons in # < " holes, then one hole must contain at least

% & pigeons!

Here: * = first integer ≥ * (verbalized “ceiling of *”) e.g., 2.731 = 3, 3 = 3

Reason: There can only be an integer number of pigeons in a hole …

SLIDE 9

Pigeonhole Principle – Example

9

In a room with 367 people, there are at least two with the same birthday.

Solution:

367 pigeons = people
366 holes = possible birthdays
Person goes into hole corresponding to own birthday

SLIDE 10

Pigeonhole Principle – Example (Surprising?)

10

In every set ! of 38 numbers, there are two whose difference is a multiple of 37.

Solution:

38 pigeons = numbers " ∈ !
37 Holes = Numbers {0, … , 36}
A number " ∈ ! goes into hole " mod 37

PHP → there are distinct "9, ": ∈ ! s.t. "9 mod 37 = ": mod 37 → "9 − ": multiple of 37

SLIDE 11

Next – Probability

We want to model a non-deterministic process.

– i.e., outcome not determined a-priori – E.g. throwing dice, flipping a coin, … – We want to numerically measure likelihood of outcomes = probability. – We want to make complex statements about these likelihoods.

We will not argue why a certain physical process realizes the

probabilistic model we study

– Why is the outcome of the coin flip really “random”?

First part of class: “Discrete” probability theory

– Experiment with finite / discrete set of outcomes.

11

SLIDE 12

Example – Coin Tossing Imagine we toss coins – each one can be heads or tails.

12

SLIDE 13

Probability space

Definition. A (discrete) probability space

is a pair (Ω, ℙ) where:

Ω is a set called the sample space.
ℙ is the probability measure, a function

ℙ: Ω → ℝ such that:

– ℙ * ≥ 0 for all * ∈ Ω – ∑D∈E ℙ * = 1

13

Set of possible elementary

utcomes

Likelihood of each elementary

utcome

Either finite or infinite countable (e.g., integers)

SLIDE 14

Example – Coin Tossing Imagine we toss one coin – outcome can be heads or tails.

14

Ω = {H, T} ℙ? Depends! What do we want to model?!

Fair coin toss ℙ H = ℙ T = 1 2 = 0.5

SLIDE 15

Example – Unfair Coin Toss Imagine we toss an unfair coin – outcome can be heads or tails.

15

Ω = {H, T}

ℙ H = I

I 1 − I

ℙ T = 1 − I

I can be determined by tossing coin several times and observing frequency of heads (“frequentist interpretation”)

SLIDE 16

Example – Two Coin Tosses Imagine we toss two fair coins

16

Ω = {HH, HT, TH, TT}

ℙ HH = ℙ HH = ℙ HH = ℙ HH = 1

4 = 0.25

25% 25% 25% 25%

SLIDE 17

Example – Glued Coin Tosses Imagine we toss two coins, glued to always show opposite faces.

17

Ω = {HT, TH}

ℙ HT = ℙ TH = 0.5

50% 50%

SLIDE 18

Example – Fair Dice We throw two fair dice.

18

Ω = K, L K, L ∈ [6]} ℙ K, L = 1 36 .

SLIDE 19

Uniform Probability Space

Definition. A uniform probability space is a pair

(Ω, ℙ) such that ℙ * = 1 Ω for all * ∈ Ω.

19

All of the above are uniform, except the unfair coin!

SLIDE 20

Summary – Probability spaces

The probability space describes only a single experiment,

sampling a single outcome.

Two-toss experiment ≠ 2 x one-toss experiment

– We need to explicitly explain how the two coin tosses are related.

20

SLIDE 21

Next – Events Typical questions we would like to answer in a random experiment.

21

Assume that we flip three fair coins, then:

What is the probability that all tosses give us tails?
What is the probability that we get heads at least once?
What is the probability that we get an even number of

heads? These are not basic outcomes!

SLIDE 22

Events

Definition. An event in a probability space

(Ω, ℙ) is a subset O ⊆ Ω. Its probability is ℙ O = Q

R∈O

ℙ(S)

22

Convenient abuse of notation: ℙ is extended to be defined over sets

ℙ S = ℙ( S )

Ω

O

SLIDE 23

Events - Examples

Definition. An event in a probability space (Ω, ℙ) is a

subset O ⊆ Ω. Its probability is ℙ O = Q

R∈O

ℙ(S)

23

“all tosses give us tails” O = {TTT} “we get heads at least once” ℬ = {TTH, THT, THH, HTT, HTH, HHT, HHH} “we get an even number

f heads”

U = {TTT, THH, HHT, HTH} ℙ HHH = ℙ HHT = ⋯ = ℙ TTT = 1/8

Ω = {HHH, HHT, … , TTT} ℙ O = ℙ TTT = 1 8 ℙ ℬ = 7× 1 8 = 7 8 ℙ U = 4× 1 8 = 4 8 = 1 2

SLIDE 24

Events – AND

24

“we get heads at least once and we get an even number of heads” “we get heads at least once” “we get an even number

f heads”

ℬ = {TTH, THT, THH, HTT, HTH, HHT, HHH} U = {TTT, THH, HHT, HTH} ℬ ∩ U = {THH, HHT, HTH} ℙ ℬ ∩ U = 3 8

ℬ U

SLIDE 25

Events – OR

25

“all tosses give us tails or all tosses give us heads” “all tosses give us tails” “all tosses give us heads” O = {TTT} Y = {HHH}

O ∪ Y = {TTT, HHH} ℙ O ∪ Y = 2 8 = 1 4

O Y

SLIDE 26

Events – NOT

26

“not all tosses are tails” “all tosses give us tails” O = {TTT}

O[ = {TTH, THT, THH, HTT, HTH, HHT, HHH} ℙ O[ = 7 8

Ω

O

SLIDE 27

Properties of Probability

27

ℙ O = Q

R∈O

ℙ(S)

For all events O, ℬ ℙ O ∪ ℬ = ℙ O + ℙ ℬ − ℙ(O ∩ ℬ) 0 ≤ ℙ O ≤ 1 ℙ O[ = 1 − ℙ(O) ℙ ∅ = 0 ℙ Ω = 1 If O ⊆ ℬ then ℙ ℬ ∖ O = ℙ ℬ − ℙ(O)

SLIDE 28

Example – Fair Dice We throw two fair dice.

28

Ω = K, L K, L ∈ [6]} ℙ K, L = 1 36 .

What is the probability the two dice add up to 7? What is the probability the two dice do not add up to 7?

SLIDE 29

Example – Fair Dice

29

CSE 312

Foundations of Computing II

Lecture 5: Introduction to probability

Stefano Tessaro

tessaro@cs.washington.edu

Organization

Today

Pigeonhole Principle – Goal Mostly showing that a set ! has at least a certain size.

Pigeonhole Principle

If there are " pigeons in # < " holes, then one hole must contain at least two pigeons!

Pigeonhole Principle – Refined version

If there are " pigeons in # < " holes, then one hole must contain at least

Pigeonhole Principle – Refined version

If there are " pigeons in # < " holes, then one hole must contain at least

Pigeonhole Principle – Even stronger version

If there are " pigeons in # < " holes, then one hole must contain at least

Reason: There can only be an integer number of pigeons in a hole …

Pigeonhole Principle – Example

In a room with 367 people, there are at least two with the same birthday.

Solution:

Pigeonhole Principle – Example (Surprising?)

In every set ! of 38 numbers, there are two whose difference is a multiple of 37.

Solution:

PHP → there are distinct "9, ": ∈ ! s.t. "9 mod 37 = ": mod 37 → "9 − ": multiple of 37

Next – Probability

probabilistic model we study

Example – Coin Tossing Imagine we toss coins – each one can be heads or tails.

Probability space

is a pair (Ω, ℙ) where:

ℙ: Ω → ℝ such that:

Example – Coin Tossing Imagine we toss one coin – outcome can be heads or tails.

Ω = {H, T} ℙ? Depends! What do we want to model?!

Fair coin toss ℙ H = ℙ T = 1 2 = 0.5

Example – Unfair Coin Toss Imagine we toss an unfair coin – outcome can be heads or tails.

Ω = {H, T}

ℙ H = I

I 1 − I

ℙ T = 1 − I

Example – Two Coin Tosses Imagine we toss two fair coins

Ω = {HH, HT, TH, TT}

ℙ HH = ℙ HH = ℙ HH = ℙ HH = 1

4 = 0.25

Example – Glued Coin Tosses Imagine we toss two coins, glued to always show opposite faces.

Ω = {HT, TH}

ℙ HT = ℙ TH = 0.5

Example – Fair Dice We throw two fair dice.

Ω = K, L K, L ∈ [6]} ℙ K, L = 1 36 .

Uniform Probability Space

(Ω, ℙ) such that ℙ * = 1 Ω for all * ∈ Ω.

All of the above are uniform, except the unfair coin!

Summary – Probability spaces

sampling a single outcome.

Next – Events Typical questions we would like to answer in a random experiment.

Events

(Ω, ℙ) is a subset O ⊆ Ω. Its probability is ℙ O = Q

ℙ(S)

ℙ S = ℙ( S )

Ω

Events - Examples

Events – AND

ℬ U

Events – OR

O ∪ Y = {TTT, HHH} ℙ O ∪ Y = 2 8 = 1 4

O Y

Events – NOT

O[ = {TTH, THT, THH, HTT, HTH, HHT, HHH} ℙ O[ = 7 8

Ω

O

Properties of Probability

For all events O, ℬ ℙ O ∪ ℬ = ℙ O + ℙ ℬ − ℙ(O ∩ ℬ) 0 ≤ ℙ O ≤ 1 ℙ O[ = 1 − ℙ(O) ℙ ∅ = 0 ℙ Ω = 1 If O ⊆ ℬ then ℙ ℬ ∖ O = ℙ ℬ − ℙ(O)

Example – Fair Dice We throw two fair dice.

Ω = K, L K, L ∈ [6]} ℙ K, L = 1 36 .

What is the probability the two dice add up to 7? What is the probability the two dice do not add up to 7?

Example – Fair Dice

Ω = K, L K, L ∈ [6]} ℙ K, L = 1 36 .

What is the probability the two dice add up to 7? What is the probability the two dice do not add up to 7?

O = { 1,6 , 6,1 , 2,5 , 5,2 , 3,4 , 4,3 }

ℙ O = 6 36 = 1 6 ℙ O[ = 1 − 1 6 = 5 6