Probability*and*Statistics* ! ! for*Computer*Science** - - PowerPoint PPT Presentation
Probability*and*Statistics* ! ! for*Computer*Science** I!have!now!used!each!of!the! terms!mean,!variance,! covariance!and!standard! devia5on!in!two!slightly! different!ways.!<<<Prof.! Forsythe!! ! Credit:!wikipedia!
!!
“I!have!now!used!each!of!the! terms!mean,!variance,! covariance!and!standard! devia5on!in!two!slightly! different!ways.”!<<<Prof.! Forsythe!! !
Hongye!Liu,!Teaching!Assistant!Prof,!CS361,!UIUC,!2.13.2020! Credit:!wikipedia!
Random!Variable!!
The!Defini5on!of!Random!Variable! Probability*distribu.on!of!random!
variable!
Joint*&*Condi.onal*probability*
distribu.on!of!random!variable!
Random!Variable!!
Expected*value* Variance*&*covariance! Markov’s*inequality!
Random!variables!have!
probability*func/ons*
Random!variables!can!be!
condi/oned!on!events!or!other! random!variables!
Random!variables!have!averages*
Random!Variable!!
Expected(value( Variance*&*covariance! Markov’s*inequality!
The!expected*value!(or!expecta/on)!
The!expected!value!is!a!weighted!sum!
!
E[X] =
xP(x)
pc X=k)
←
The!expected*value!of!a!random!
variable!X!is!
The!expected!value!is!a!weighted!sum!
!
E[X] =
xP(x)
<=!1! Ep # =L
N
A"company"has"a"project"that"has"p"
probability"of"earning"10"million"and"1#p" probability"of"losing"10"million."
Let"X"be"the"return"of"the"project."
"
E- EX) =
to Pt f -lo) ( t - P )
=
cop
> o
P S
,
0.5
co profit
A!company!has!a!project!that!has!p!
probability!of!earning!10!million!and!1.p! probability!of!losing!10!million.!
Let!X!be!the!return!of!the!project.!
!
E[X] = 10p + (−10)(1 − p)
= 20p − 10
For E[X] > 0 we need p > 0.5
wrhies
5
ME TE FE ¥7
Mean
= ? I
$1 each
92 each
A) random
draw
I
Expected
value = ?
0.75×11-0.25×2
= Ig
D) random
draw
1 twice
with
replacement
indeputy
( if
the
two
draws
are
the
game .
You
3
get
the prize, w -4 %
$1
Expected
value
=
? w 3-It
is
4
4
16
g
,
TE
= It
eh 4h
To → $2
Suppose!we!have!a!data!set!{xi}!of!N!data!
points.!Let’s!define!a!random!variable!X! taking!on!each!of!the!data!points!with! equal!probability!1/N.!!
The!expected!value!is!also!called!the!
mean.!
!
E[X] =
xiP(xi) = 1
N
xi = mean({xi})
For!random!variables!X!and!Y!
and!constants!k,c!
Scaling!property!
!
Addi5vity! And!!
E[X + Y ] = E[X] + E[Y ]
E[kX] = kE[X]
E[kX + c] = kE[X] + c
Proof!of!the!addi5ve!property! !
!
E[X + Y ] = E[X] + E[Y ]
E-Cx t'll = I 5g
Ht 2) PHo=xo=y)
next
pg
E Ext Y) = I -2g
(key) P CX
= I -2g
cuty, pox. y )
=
TZ Fy apex,y ) t § I y poky,
y
=
Ee k Ey
Pl x. y l t Ey §
y pi x.y,
= Iaz
t Ey 2Epcx
d p ck)
ptcy,
= Ty xp ex) t Tg y pay, =
E-Cx) t ECT)
Proof!of!the!addi5ve!property! !
!
E[X + Y ] = E[X] + E[Y ]
E[X + Y ] =
(x + y)P(x, y)
P({X = x} ∩ {Y = y})
!
!
E[X + Y ] =
(x + y)P(x, y) =
xP(x, y) +
yP(x, y) =
xP(x, y) +
yP(x, y) =
x
P(x, y) +
y
P(x, y) =
xP(x) +
yP(y)
!
!
E[X + Y ] =
(x + y)P(x, y) =
xP(x, y) +
yP(x, y) =
xP(x, y) +
yP(x, y) =
x
P(x, y) +
y
P(x, y) =
xP(x) +
yP(y)
= E[X] + E[Y ]
Q.*What’s*the*value?*
What!is!E[E[X]+1]?!
!!!!!A.!E[X]+1!!!!!!!B.!1!!!!!!!!C.!0!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Expected*value*of*a*function*of*X
If!f!is!a!func5on!of!a!random!
variable!X!,!then!Y!=!f!(X)!is!a! random!variable!too!
The!expected!value!of!Y!=!f!(X)!is! !
!
Eff Gi ) = E y
Expected*value*of*a*function*of*X
If!f!is!a!func5on!of!a!random!
variable!X!,!then!Y!=!f!(X)!is!a! random!variable!too!
The!expected!value!of!Y!=!f!(X)!is! !
!
E[Y ] = E[f(X)] =
f(x)P(x)
A!cat!moves!with!random!constant!
speed!V,!either!5mile/hr!or!20mile/hr! with!equal!probability,!what’s!the! expected!5me!for!it!to!travel!50!miles?!
!
!
EW)
= WE
ECT ) -_ El
I
=
. plV=Y) tupcv=Vu,=
Z
A!cat!moves!with!random!constant!
speed!V,!either!5mile/hr!or!20mile/hr! with!equal!probability,!what’s!the! expected!5me!for!it!to!travel!50!miles?!
!
!
t(V ) = 50 V
E[t(V )] =
50 v P(v) = 50 5 × 1 2 + 50 20 × 1 2 = 6.25 hours
Q:*Is*this*statement*true?**
If!there!exists!a!constant!such!that! P(X!≥!a)!=!!1,!then!E[X]!≥!a!.!It!is:! !
I
E Cx)
= Esa xp
+ Ea E
'?FE'aa
Q:*Is*this*statement*true?**
If!there!exists!a!constant!such!that! P(X!≥!a)!=!!1,!then!E[X]!≥!a!.!It!is:! !
E[X] =
x<a
xP(X = x) +
∞
xP(X = x)
0!
≥a!
Random!Variable!!
Expected*value* Variance(&(covariance* Towards*the*week*law*of*large*
numbers!
Variance*and*standard*deviation
The!variance!of!a!random!
variable!X!is!
The!standard!devia5on!of!a!
random!variable!X!is!
!
std[X] =
var[X] = E[(X − E[X])2]
var (fXi) > = I
l Xi- meanCl ki 35
ga
For!random!variable!X!and!
constant!k!
!
!
!
var[kX] = k2var[X] var[X] ≥ 0
non - negative !
E
p
scaling
with
squared
K
A*neater*expression*for*variance
!
!
var[X] = E[X2] − E[X]2
Variance!of!Random!Variable!X!is!
defined!as:!!
It’s!the!same!as:!
A*neater*expression*for*variance
!
!
var[X] = E[(X − E[X])2]
n -_ Efx]
= El XII MX
tu ' ]
=
Efx'T - 2E -4×1 + Elm')
= ECI ) - 2h ECXJTNZ
=
ECI ) - ZEEXIETXITETX ]
=
ECW)
A*neater*expression*for*variance
!
!
var[X] = E[(X − µ)2] where µ = E[X] = E[X2 − 2Xµ + µ2] = E[X2] − 2µE[X] + µ2 var[X] = E[(X − E[X])2] By!linearity!of!expecta5on!
A*neater*expression*for*variance
!
!
var[X] = E[(X − µ)2] where µ = E[X] = E[X2 − 2Xµ + µ2] = E[X2] − 2µE[X] + µ2 = E[X2] − 2E[X]E[X] + (E[X])2 var[X] = E[(X − E[X])2]
A*neater*expression*for*variance
!
!
var[X] = E[(X − µ)2] where µ = E[X] = E[X2 − 2Xµ + µ2] = E[X2] − 2µE[X] + µ2 = E[X2] − 2E[X]E[X] + (E[X])2 = E[X2] − E[X]2 var[X] = E[(X − E[X])2]
For!the!profit!example,!what!is!the!
variance!of!the!return?!We!know!E[X]=! 20p<10! var[X] = E[X2] − (E[X])2 = (102p + (−10)2(1 − p)) − (20p − 10)2 = 100 − (400p2 − 400p + 100) = 400p(1 − p)
! ! ! ! ! !
x' to
Solve infgroup
y
'TP
x
ktpck )
Cup
2C
10? -
p +
C-lo)'ll-:p)
For!the!profit!example,!what!is!the!
variance!of!net!return?!We!know!E[X]=! 20p<10! var[X] = E[X2] − (E[X])2 = (102p + (−10)2(1 − p)) − (20p − 10)2 = 100 − (400p2 − 400p + 100) = 400p(1 − p)
! ! ! ! ! !
For!the!profit!example,!what!is!the!
variance!of!net!return?!We!know!E[X]=! 20p<10! var[X] = E[X2] − (E[X])2 = (102p + (−10)2(1 − p)) − (20p − 10)2 = 100 − (400p2 − 400p + 100) = 400p(1 − p)
! ! ! ! !
For!the!profit!example,!what!is!the!
variance!of!net!return?!We!know!E[X]=! 20p<10! var[X] = E[X2] − (E[X])2 = (102p + (−10)2(1 − p)) − (20p − 10)2 = 100 − (400p2 − 400p + 100) = 400p(1 − p)
Study!the!rela5onship!between!
random!variables*
Note!that!it’s!the!un<normalized!
correla5on!
Applica5ons!include!the!fire!control!
presence!of!noise.!!
The*covariance!of!random!
variables!X!and!Y!is!
Note!that!
cov(X, Y ) = E[(X − E[X])(Y − E[Y ])]
cov(X, X) = E[(X − E[X])2] = var[X]
a- . *Iii
'
A!neater!expression!for*
covariance!(similar!deriva5on!as! for!variance)!
cov(X, Y ) = E[XY ] − E[X]E[Y ]
Correlation*coefficient*is* normalized**covariance*
The!correla5on!coefficient!is!
*
When!X, Y!takes!on!values!with!equal!
probability!to!generate!data!sets! {(x,y)},!the!correla5on!coefficient!will! be!as!seen!in!Chapter!2.!
corr(X, Y ) = cov(X, Y ) σXσY
Correlation*coefficient*is* normalized**covariance*
The!correla5on!coefficient!can!also!be!
wriken!as:!
* corr(X, Y ) = E[XY ] − E[X]E[Y ]
σXσY
Cov CX, Y
)
L
Correlation*seen*from*scatter*plots*
Posi5ve!! correla5on! ! Nega5ve!! correla5on! Zero!! Correla5on! !
Credit:! Prof.Forsyth!
Covariance*seen*from*scatter*plots*
Posi5ve!! Covariance! ! Nega5ve!! Covariance! Zero!! Covariance! !
Credit:! Prof.Forsyth!
When*correlation*coefficient*or* covariance*is*zero**
The!covariance!is!0!! That!is:! This!is!a!necessary!property!of!
independence!of!random!variables!*!(not! equal!to!independence)!
E[XY ] − E[X]E[Y ] = 0 E[XY ] = E[X]E[Y ]
→-
Variance*of*the*sum*of*two*random* variables*
var[X + Y ] = var[X] + var[Y ] + 2cov(X, Y )
Properties*of*independence*in* terms*of*expectations*
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
E[XY ] = E[X]E[Y ]
"' ECXYI-E-zxyplk-x.tk#.:.dp+
=D
my pix) ply ,
← x. y
n pix) j ypay)
=
Efx ) ECT]
→CoV ( x ,2) = Efx's - Efx ) El Y) = o
Proof*of*independence*in*terms*
!
E[XY ] = E[X]E[Y ]
E[XY ] =
xyP(X = x ∩ Y = y) =
(xyP(x, y)) =
(xyP(x)P(y)) =
xP(x)
yP(y) = (
xP(x))(
yP(y)) = E[X]E[Y ]
Properties*of*independence*in* terms*of*expectations*
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
E[XY ] = E[X]E[Y ]
cov(X, Y ) = 0
var[X + Y ] = var[X] + var[Y ]
We!toss!two!iden5cal!coins!A!&!B!
independently!for!three!5mes!and!4!5mes! respec5vely,!for!each!head!we!earn!$1,!we! define!X!is!the!earning!from!A!and!Y!is!the! earning!from!B.!What!is!E(XY)?! !!!!!A.!$2!!!!!!!B.!$3!!!!!!!!C.!$4!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
fair
R
a
Efx:3
# X 's
= 's
Group*Discussion*
If!two!random!variables!are!
uncorrelated,!does!this!mean!they!are! independent?!Inves5gate!the!case!X! takes!<1,!0,!1!with!equal!probability! and!Y=X2.!
+
EM
ELY )
Ecxyy
EIXY )
" x } teeny
= EN )
=O
=u
= Ixtztoxtztlxtz
Y
* I
*
:
' ie
.E V
'
It’s!an!underlying!concept!in!principal! component!analysis!in!Chapter!10!!!!!!!!!!!!!!!!!!
Assignments)
Finish&Chapter&4&of&the&textbook& Next&4me:&Proof&of&Markov&inequality,&
Chebyshev&inequality&&&Weak&law&of& large&numbers,&Con4nuous&random& variable&
&
Additional*References*
Charles!M.!Grinstead!and!J.!Laurie!Snell!
"Introduc5on!to!Probability”!!
Morris!H.!Degroot!and!Mark!J.!Schervish!
"Probability!and!Sta5s5cs”!
See*you*next*time*