Probability and Statistics for Computer Science A major use of - - PowerPoint PPT Presentation

ì

Probability and Statistics for Computer Science

“A major use of probability in sta4s4cal inference is the upda4ng of probabili4es when certain events are

• bserved” – Prof. M.H.

DeGroot

Last time

Probability

More probability

calculation

• Conditional probability

* Bayes rule

✓

* Independence

Objectives

Condi4onal Probability

PC AIB) = PCANBSP

* Product

rule

• f

joint prob.

* Bayes rule * Independence

Counting: how many ways?

to

put

7

hats

chats

are

indistinguishable )

• n

• f

lo

#

dourly ?

A

€-1

:)

Warm up: which is larger?

PC An B )

• r

PC Al B)

A )

PC An B )

B )

PC Al B)

c)

unsure

Conditional Probability

The probability of A given B

P(A|B) = P(A ∩ B) P(B)

P(B) = 0

The line-crossed area is the new sample space for condi4onal P(A| B)

Joint Probability Calculation

⇒ P(A ∩ B) = P(A|B)P(B)

P(soup ∩ meat) = P(meat|soup)P(soup) = 0.5 × 0.8 = 0.4

Pc Anb )

pl Al B)

• . 8×0.5
• PCB )
• I]
• disjoint

• II

Bayes rule

Given the defini4on of condi4onal

probability and the symmetry of joint probability, we have: And it leads to the famous Bayes rule:

P(A|B)P(B) = P(A ∩ B) = P(B ∩ A) = P(B|A)P(A)

P(A|B) = P(B|A)P(A) P(B)

Plants

PCB)

Total probability

A1 A2 A3

B

Pc B) =p ( Bn A) t Pc Bn Ac )

=p

• i. e .

A = It ,

r

Total probability general form

A1 A2 A3

B

PL B) = I PCBN Aj )

j

=

5g PCB l Aj > PCAJ )

tiny

,

*

Total probability:

Pcsoupnneat

parent

Isoup,

p ( meat ) = ?

• p(soup'
• P

disjoint

+ pc meatI juice ) P ljuice)

• mile)

p c soup I meat )

P"

• = p e meat n som '

E-

Bayes rule using total prob.

PCBI Aj ) PCAJ )

PCAJIB )

= #

p"

= PlBlAj)pcAj#

pub)

I

PCBI Aj

j

Ajntti

it iej

Bayes rule: rare disease test

P(D|T) = P(T|D)P(D) P(T) = P(T|D)P(D) P(T|D)P(D) + P(T|Dc)P(Dc) P(D|T)

There is a blood test for a rare disease. The

frequency of the disease is 1/100,000. If one has it, the test confirms it with probability 0.95. If one doesn't have, the test gives false posi4ve with probability 0.001. What is , the probability

• f having disease given a posi4ve test result?

Using total prob.

Bayes rule: rare disease test

P(D|T)

There is a blood test for a rare disease. The

frequency of the disease is 1/100,000. If one has it, the test confirms it with probability 0.95. If one doesn't have, the test gives false posi4ve with probability 0.001. What is , the probability

• f having disease given a posi4ve test result?

P(D|T) = P(T|D)P(D) P(T|D)P(D) + P(T|Dc)P(Dc)

• ①
• .gfX

, ,g

what

about

could

test ?

Suppose

freq

et

Cour'd

=/

test

accuracy

=

95%

• false

positive

= T

• ol
• PIPIT) = ?

P

• TIPI . .gs P' " = ' -2%

pctlp

• T

Z

PCTLD) PCD)

pipa)

=#

• 21%

PCTID> PCDHPCTIDC) pep)

99

.

Independence

One defini4on:

Whether A happened doesn’t change

the probability of B and vice versa

P(A|B) = P(A) or P(B|A) = P(B)

• -

Independence: example

Suppose that we have a fair coin and it is

tossed twice. let A be the event “the first toss is a head” and B the event “the two outcomes are the same.”

These two events are independent!

A1€

'

IIE

.r:÷÷÷÷÷=÷¥

'"

I ✓

Independence

Alterna4ve defini4on

P(A|B) = P(A) ⇒ P(A ∩ B) P(B) = P(A)

⇒ P(A ∩ B) = P(A)P(B)

LHS by defini4on

12HnB)=ppcBgyg#**

Testing Independence:

Suppose you draw one card from a

standard deck of cards. E1 is the event that the card is a King, Queen or Jack. E2 is the event the card is a Heart. Are E1 and E2 independent?

• #nM=pcH) PIB)

pi Ein Er) = #

• Pp"i÷'¥÷=÷s¥¥= 's

Independence vs Disjoint

• Q. Two disjoint events that have

probability> 0 are certainly dependent to each other.

• A. True
• B. False

⇒

• l

E)

pc Anny

= o

pas)

pimp

• p can b)f p Cbs PCB)
• =#

Independence of empty event

• Q. Any event is independent of

empty event B.

• A. True
• B. False

D

a- ol

p ( 0/1=0

Ion H

P

plot )

=

p

in

= o

Pairwise independence is not mutual independence in larger context

A1 A2 A4 A3

P(A1) = P(A2) = P(A3) = P(A4) = 1/4 A = A1 ∪ A2; P(A) = 1 2 B = A1 ∪ A3; P(B) = 1 2 C = A1 ∪ A4; P(C) = 1 2

P(ABC) is the shorthand for P(A ∩ B ∩ C)

*

(÷ AV.AZ#3UAy=zggpcAnB)=pCAspcBjv

• ①- =L

PCB?

.cl?-pcb3pccgv.C--tu.C%cann4--PUtpY..--pFmp.adEP

Mutual independence

Mutual independence of a collec4on

• f events is :

It’s very strong independence!

j, k, ...p = i A1, A2, A3...An

it

⇒ ④

|pcAilAjn Ann

• Ap ) =p (Ai)

/

p ( A1 Bnc )

BCA)

⇒

P can Bnc)

• =p LA,
• penny I

⇒ PLANE

.IT#pco3n4pcA1pcBjpiy

Probability using the property of Independence: Airline overbooking (1)

An airline has a flight with 6 seats. They

always sell 7 4ckets for this flight. If 4cket holders show up independently with probability p, what is the probability that the flight is overbooked ?

we

(7)

=

.

7 y

P

"

p

• Ail

=p CA. ) pcAu)

• p (Az)

=p_p.r--p#

Probability using the property of Independence: Airline overbooking (1)

An airline has a flight with 6 seats. They

always sell 7 4ckets for this flight. If 4cket holders show up independently with probability p, what is the probability that the flight is overbooked ?

P( 7 passengers showed up)

Probability using the property of Independence: Airline overbooking (2)

An airline has a flight with 6 seats. They

always sell 8 4ckets for this flight. If 4cket holders show up independently with probability p, what is the probability that exactly 6 people showed up?

P(6 people showed up) =

• o

• p6 ftp.IP.P-rp.p.p

Probability using the property of Independence: Airline overbooking (3)

An airline has a flight with 6 seats. They

always sell 8 4ckets for this flight. If 4cket holders show up independently with probability p, what is the probability that the flight is overbooked ?

P( overbooked) =

Probability using the property of Independence: Airline overbooking (4)

An airline has a flight with s seats. They

always sell t (t>s) 4ckets for this flight. If 4cket holders show up independently with probability p, what is the probability that exactly u people showed up?

P( exactly u people showed up)

O

O

(E)

p

E- a

Probability using the property of Independence: Airline overbooking (5)

An airline has a flight with s seats. They

always sell t (t>s) 4ckets for this flight. If 4cket holders show up independently with probability p, what is the probability that the flight is overbooked ?

P( overbooked)

G-

FT

s

¥

lil r;

a -pit

• u
• ¥4

Condition may affect Independence

Assume event A and B are pairwise

independent A B C Given C, A and B are not independent any more because they become disjoint

r

Conditional Independence

Event A and B are condi4onal

independent given event C if the following is true.

P(A ∩ B|C) = P(A|C)P(B|C)

Assignments

HW3 Finish Chapter 3 of the textbook Next 4me: Random variable

Additional References

Charles M. Grinstead and J. Laurie Snell

"Introduc4on to Probability”

Morris H. Degroot and Mark J. Schervish

"Probability and Sta4s4cs”

Another counting problem

There are several (>10) freshmen,

sophomores, juniors and seniors in a

• dormitory. In how many ways can a team of 10

students be chosen to represent the dorm? There are no dis4nc4on to make between each individual student other than their year in school.

See you next time

See You!