Probability Theory & Uncertainty Read Chapter 13 of textbook - - PDF document

AI: 15-780 / 16-731 Mar 1, 2007

Probability Theory & Uncertainty

Read Chapter 13 of textbook

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

What you will learn today

• fundamental role of uncertainty in AI
• probability theory can be applied to many of these problems
• probability as uncertainty
• probability theory is the calculus of reasoning with uncertainty
• probability and uncertainty in different contexts
• review of basis probabilistic concepts
• discrete and continuous probability
• joint and marginal probability
• calculating probability
• next probability lecture: the process of probabilistic inference

2

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007 3

What is the role of probability and inference in AI?

• Many algorithms are designed as if knowledge is perfect, but it rarely is.
• There are almost always things that are unknown, or not precisely known.
• Examples:
• bus schedule
• quickest way to the airport
• sensors
• joint positions
• finding an H-bomb
• An agent making optimal decisions must take into account uncertainty.

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Probability as frequency: k out of n possibilities

• Suppose we’re drawing cards from a standard deck:
• P(card is the Jack ♥ | standard deck) = 1/52
• P(card is a ♣ | standard deck) = 13/52 = 1/4
• What’s the probability of a drawing a pair in 5-card poker?
• P(hand contains pair | standard deck) =

# of hands with pairs _______________ total # of hands

• Counting can be tricky (take a course in combinatorics)
• Other ways to solve the problem?
• General probability of event given some conditions:

P(event | conditions)

4

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007 5

Making rational decisions when faced with uncertainty

• Probability

the precise representation of knowledge and uncertainty

• Probability theory

how to optimally update your knowledge based on new information

• Decision theory: probability theory + utility theory

how to use this information to achieve maximum expected utility

• Consider again the bus schedule. What’s the utility function?
• Suppose the schedule says the bus comes at 8:05.
• Situation A:

You have a class at 8:30.

• Situation B:

You have a class at 8:30, and it’s cold and raining.

• Situation C:

You have a final exam at 8:30.

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Probability of uncountable events

• How do we calculate probability that it will rain tomorrow?
• Look at historical trends?
• Assume it generalizes?
• What’s the probability that there was life on Mars?
• What was the probability the sea level will rise 1 meter within the century?
• What’s the probability that candidate X will win the election?

6

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

The Iowa Electronic Markets: placing probabilities on single events

• http://www.biz.uiowa.edu/iem/
• “The Iowa Electronic Markets are real-money futures markets in which contract

payoffs depend on economic and political events such as elections.”

• Typical bet: predict vote share of candidate X - “a vote share market”

7 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Political futures market predicted vs actual outcomes

8

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

John Craven and the missing H-Bomb

• In Jan. 1966, used Bayesian probability and subjective odds to

locate H-bomb missing in the Mediterranean ocean.

9 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Probabilistic Methodology

10

0, 1, or 2 parachutes open? type of collision prevailing wind direction

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Probabilistic assessment of dangerous climate change

11

from Forrest et al (2001) from Mastrandrea and Schneider (2004)

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Factoring in Risk Using Decision Theory

12

P(“DAI” = 55.8%) P(“DAI” = 27.4% Carbon Tax 2050 = $174/T

• n

Dangerous Climate Change

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Uncertainty in vision: What are these?

13 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Uncertainty in vision

14

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Edges are not as obvious they seem

15 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

An example from Antonio Torralba

16

What’s this?

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

We constantly use other information to resolve uncertainty

17 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Image interpretation is heavily context dependent

18

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

This phenomenon is even more prevalent in speech perception

19

• It is very difficult to recognize phonemes from naturally spoken speech when they

are presented in isolation.

• All modern speech recognition systems rely heavily on context (as do we).
• HMMs model this contextual dependence explicitly.
• This allows the recognition of words, even if there is a great deal of uncertainty in

each of the individual parts.

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

De Finetti’s definition of probability

• Was there life on Mars?
• You promise to pay $1 if there is, and $0 if there is not.
• Suppose NASA will give us the answer tomorrow.
• Suppose you have an oppenent
• You set the odds (or the “subjective probability”) of the outcome
• But your oppenent decides which side of the bet will be yours
• de Finetti showed that the price you set has to obey the axioms of probability or

you face certain loss, i.e. you’ll lose every time.

20

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007 21

Axioms of probability

• Axioms (Kolmogorov):

0 P(A) 1 P(true) = 1 P(false) = 0 P(A or B) = P(A) + P(B) P(A and B)

• Corollaries:
• A single random variable must sum to 1:
• The joint probability of a set of variables must also sum to 1.
• If A and B are mutually exclusive:

P(A or B) = P(A) + P(B)

n

• i=1

P(D = di) = 1

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Rules of probability

• conditional probability
• corollary (Bayes’ rule)

22

Pr(A|B) = Pr(A and B) Pr(B) , Pr(B) > 0 Pr(B|A)Pr(A) = Pr(A and B) = Pr(A|B)Pr(B) ⇒ Pr(B|A) = Pr(A|B)Pr(B) Pr(A)

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Discrete probability distributions

• discrete probability distribution
• joint probability distribution
• marginal probability distribution
• Bayes’ rule
• independence

23 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007 24

A*3,B$(1+,C(<+'(9D+($1

E3=(%3,F$',G:H(1),:,I$(1+,2(<+'(9D+($1, $F,7,J:'(:9;3<> "6 7:H3,:,+'D+*,+:9;3,;(<+(1),:;;, =$G9(1:+($1<,$F,J:;D3<,$F,&$D', J:'(:9;3<,K(F,+*3'3,:'3,7,L$$;3:1, J:'(:9;3<,+*31,+*3,+:9;3,4(;;,*:J3, !7,'$4<M6 !6 N$',3:=*,=$G9(1:+($1,$F,J:;D3</, <:&,*$4,%'$9:9;3,(+,(<6 O6 PF,&$D,<D9<='(93,+$,+*3,:Q($G<,$F, %'$9:9(;(+&/,+*$<3,1DG93'<,GD<+, <DG,+$,"6 RQ:G%;3>,L$$;3:1, J:'(:9;3<,0/,L/,#

.6". " " " .6!S . " " .6". " . " .6.S . . " .6.S " " . .6". . " . .6.S " . . .6O. . . .

!"#$ % & '

L #

.6.S .6!S .6". .6.S .6.S .6". .6". .6O.

All the nice looking slides like this one from now on are from Andrew Moore.

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007 25

A*3,B$(1+,C(<+'(9D+($1

E3=(%3,F$',G:H(1),:,I$(1+,2(<+'(9D+($1, $F,7,J:'(:9;3<> "6 7:H3,:,+'D+*,+:9;3,;(<+(1),:;;, =$G9(1:+($1<,$F,J:;D3<,$F,&$D', J:'(:9;3<,K(F,+*3'3,:'3,7,L$$;3:1, J:'(:9;3<,+*31,+*3,+:9;3,4(;;,*:J3, !7,'$4<M6 !6 N$',3:=*,=$G9(1:+($1,$F,J:;D3</, <:&,*$4,%'$9:9;3,(+,(<6 O6 PF,&$D,<D9<='(93,+$,+*3,:Q($G<,$F, %'$9:9(;(+&/,+*$<3,1DG93'<,GD<+, <DG,+$,"6 RQ:G%;3>,L$$;3:1, J:'(:9;3<,0/,L/,#

.6". " " " .6!S . " " .6". " . " .6.S . . " .6.S " " . .6". . " . .6.S " . . .6O. . . .

!"#$ % & '

L #

.6.S .6!S .6". .6.S .6.S .6". .6". .6O.

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007 26

A*3,B$(1+,C(<+'(9D+($1

E3=(%3,F$',G:H(1),:,I$(1+,2(<+'(9D+($1, $F,7,J:'(:9;3<> "6 7:H3,:,+'D+*,+:9;3,;(<+(1),:;;, =$G9(1:+($1<,$F,J:;D3<,$F,&$D', J:'(:9;3<,K(F,+*3'3,:'3,7,L$$;3:1, J:'(:9;3<,+*31,+*3,+:9;3,4(;;,*:J3, !7,'$4<M6 !6 N$',3:=*,=$G9(1:+($1,$F,J:;D3</, <:&,*$4,%'$9:9;3,(+,(<6 O6 PF,&$D,<D9<='(93,+$,+*3,:Q($G<,$F, %'$9:9(;(+&/,+*$<3,1DG93'<,GD<+, <DG,+$,"6 RQ:G%;3>,L$$;3:1, J:'(:9;3<,0/,L/,#

.6". " " " .6!S . " " .6". " . " .6.S . . " .6.S " " . .6". . " . .6.S " . . .6O. . . .

!"#$ % & '

L #

.6.S .6!S .6". .6.S .6.S .6". .6". .6O.

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

T<(1),+*3, B$(1+

U13,&$D,*:J3,+*3,BC,&$D,=:1, :<H,F$',+*3,%'$9:9(;(+&,$F,:1&, ;$)(=:;,3Q%'3<<($1,(1J$;J(1), &$D',:++'(9D+3

!

"

!

" ! "

! "#$%&'() ! *+,-

. *+, / . /

27 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

T<(1),+*3, B$(1+

U13,&$D,*:J3,+*3,BC,&$D,=:1, :<H,F$',+*3,%'$9:9(;(+&,$F,:1&, ;$)(=:;,3Q%'3<<($1,(1J$;J(1), &$D',:++'(9D+3

!

"

!

" ! "

! "#$%&'() ! *+,-

. *+, / . /

28

8D8##&+7:;3E+F+-6@GH@

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

T<(1),+*3, B$(1+

U13,&$D,*:J3,+*3,BC,&$D,=:1, :<H,F$',+*3,%'$9:9(;(+&,$F,:1&, ;$)(=:;,3Q%'3<<($1,(1J$;J(1), &$D',:++'(9D+3

!

"

!

" ! "

! "#$%&'() ! *+,-

. *+, / . /

29

8D8##&E+F+-6IG-@

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

T<(1),+*3, B$(1+

U13,&$D,*:J3,+*3,BC,&$D,=:1, :<H,F$',+*3,%'$9:9(;(+&,$F,:1&, ;$)(=:;,3Q%'3<<($1,(1J$;J(1), &$D',:++'(9D+3

!

"

!

" ! "

! "#$%&'() ! *+,-

. *+, / . /

30

C2D4'42>4, 5(+*,+*4, E$(2+

! !

" # "

! " ! " #

" $%&'()*+ " ,-./ " %*0 " $%&'()*+ " ,-./ ! ! # ! #

1 ,-. 2 1 ,-. 2 1 2 1 2 1 3 2

! ! !

" " ! " ! ! " ! ! "

9G8;<4 H,9$$'I,J,.7AFBA,K,.7LF.A,J,.7F/!,

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Continuous probability distributions

• probability density function (pdf)
• joint probability density
• marginal probability
• calculating probabilities using the pdf
• Bayes’ rule

31 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007 32

1+9=B+#C+1D4&'E;2+1(4>+'2+-...

more of Andrew’s nice slides

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

1+9=B+#C+1D4&'E;2+1(4>+'2+-...

G4*+H+:4+;+E#2*'2I#I>+&;23#D+ J;&';:<47 KC+$LMN+'>+;+9&#:;:'<'*%+=42>'*%+ BI2E*'#2+C#&+H+*)42O

! "

#

$

$ % &

! " #

$# # % ! & " ' ! "

! "

#

$

$ % &

#$ %$ &'(

&'( ! &'( " #$ )'( %$ $ % '

P+.7!F

33 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007 34

What does p(x) mean?

• It does not mean a probability!
• First of all, it’s not a value between 0 and 1.
• It’s just a value, and an arbitrary one at that.
• The likelihood of p(a) can only be compared relatively to other values p(b)
• It indicates the relative probability of the integrated density over a small delta:

OJ+ *)42+

! ! " # " " ! " ! " ! $ ! % $ ! & $ # % $ # &

$

" # $ $ % # $ $ %

&

! " ! " #$%

&

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007 35

"#$%&'()*+,+-../0+123&45+67+8##&4 9&#:;:'<'*%+=42>'*'4>?+@<'34+/A

BC$4D*;*'#2>

BEFG+H+*)4+4C$4D*43+I;<J4+#K+ &;23#L+I;&';:<4+F H+*)4+;I4&;(4+I;<J4+54M3+>44+ 'K+54+*##N+;+I4&%+<;&(4+2JL:4&+ #K+&;23#L+>;L$<4>+#K+F

!

" #" $

$

!

"! ! # ! ! "

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

"#$%&'()*+,+-../0+123&45+67+8##&4 9&#:;:'<'*%+=42>'*'4>?+@<'34+/O

BC$4D*;*'#2>

BEFG+H+*)4+4C$4D*43+I;<J4+#K+ &;23#L+I;&';:<4+F H+*)4+;I4&;(4+I;<J4+54M3+>44+ 'K+54+*##N+;+I4&%+<;&(4+2JL:4&+ #K+&;23#L+>;L$<4>+#K+F

!

" #" $

$

!

"! ! # ! ! "

H+*)4+K'&>*+L#L42*+#K+*)4+ >);$4+K#&L43+:%+*)4+;C4>+;23+ *)4+:<J4+DJ&I4 H+*)4+:4>*+I;<J4+*#+D)##>4+'K+ %#J+LJ>*+(J4>>+;2+J2N2#52+ $4&>#2M>+;(4+;23+%#JM<<+:4+ K'243+*)4+>PJ;&4+#K+%#J&+4&&#& BE;(4GHQR7O!A

36

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

#$%&'()*+,-,.""!/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,!@

AB%3C+:+($1,$D,:,DE1C+($1

!FAGDHIJK,F,+*3,3B%3C+32, L:;E3,$D,DHBJ,4*3'3,B,(=,2':41, D'$M,IN=,2(=+'(9E+($16, F,+*3,:L3':)3,L:;E3,43N2,=33, (D,43,+$$O,:,L3'&,;:')3,1EM93', $D,':12$M,=:M%;3=,$D,DHIJ

"

# $# %

%

!

"! ! # ! $ ! " ! " !

P$+3,+*:+,(1,)313':;>

#! $ " !# " $ % & $ ! $ & & %& ' ()*% # +,- $

. %

& %. ' (.** #! +,- $ "

. %

&

37 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

#$%&'()*+,-,.""!/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,."

Q:'(:1C3

'. F,Q:'GIK,F,+*3, 3B%3C+32,=RE:'32, 2(DD3'31C3,93+4331, B,:12,AGIK,

"

# $# %

$ %

!

"! ! # ! ! " ! "

. .

! '

F,:M$E1+,&$EN2,3B%3C+,+$,;$=3, (D,&$E,ME=+,)E3==,:1,E1O1$41, %3'=$1N=,:)3,:12,&$EN;;,93, D(132,+*3,=RE:'3,$D,&$E',3''$'/, :12,:==EM(1),&$E,%;:&, $%+(M:;;&

/. ' &0* # +,- $ 1+2 %

38

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

?*:12:&2+<3@':*'#1

!- A+B:&CDE+A+*)3+ 3F$3G*32+=HI:&32+ 2'JJ3&31G3+93*4331+ F+:12+KCDE+

"

# $# %

$ %

!

"! ! # ! ! " ! "

# #

& !

A+:L#I1*+%#IM2+3F$3G*+*#+;#=3+ 'J+%#I+LI=*+(I3==+:1+I1N1#41+ $3&=#1M=+:(3+:12+%#IM;;+93+ J'132+*)3+=HI:&3+#J+%#I&+3&&#&/+ :12+:==IL'1(+%#I+$;:%+ #$*'L:;;% ! A+?*:12:&2+<3@':*'#1+A+ O*%$'G:;P+23@':*'#1+#J+D+J&#L+ '*=+L3:1

$# % &'( ) *+,

• .*/

% )

• .*/ $

% ! 0# % ## % !

39 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

Q1+-+ 2'L31='#1=

$RF/%S+A+$&#9:9';'*%+231='*%+#J+ &:12#L+@:&':9;3=+RD/TS+:*+ ;#G:*'#1+RF/%S

40

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

#$%&'()*+,-,"..!/,012'34,56,7$$'3 8'$9:9(;(+&,<31=(+(3=>,?;(23,"@

A1,", 2(B31=($1=

C3+,D/E, 93,:,%:(',$F,G$1+(1H$H=,':12$B, I:'(:9;3=/,:12,;3+,J,93,=$B3,'3)($1,$F,KD/EL, =%:G3M

!!

"

# "

! " #

$"$# " # % ! & ' (

! " #

! " # ! ! " ##

41 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

A1,", 2(B31=($1=

C3+,D/E, 93,:,%:(',$F,G$1+(1H$H=,':12$B, I:'(:9;3=/,:12,;3+,J,93,=$B3,'3)($1,$F,KD/EL, =%:G3M

!!

"

# "

! " #

$"$# " # % ! & ' (

! " #

! " # ! ! " ##

8K,".OB%)O@.,:12 "P..O43()*+O@...L,Q :'3:,H123',+*3,"R2,=H'F:G3,4(+*(1, +*3,'32,'3G+:1);3

42

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,.A

B2,., 3(C42>($2>

D4+,E0F, :4,;,%;(',$G,H$2+(2I$I>,';23$C, J;'(;:<4>0,;23,<4+,K,:4,>$C4,'4)($2,$G,LE0FM, >%;H4N

!!

"

# "

! " #

$"$# " # % ! & ' (

! " #

! " # ! ! " ##

9L,OLC%)P.AMQ!/R. S, OL54()*+P""//MQ!A//R. T,!,M,U ;'4;,I234',+*4,.P3,>I'G;H4,5(+*(2, +*4,'43,$J;<

43 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

#$%&'()*+,-,.//!0,123'45,67,8$$'4 9'$:;:(<(+&,=42>(+(4>?,@<(34,.V

B2,., 3(C42>($2>

D4+,E0F, :4,;,%;(',$G,H$2+(2I$I>,';23$C, J;'(;:<4>0,;23,<4+,K,:4,>$C4,'4)($2,$G,LE0FM, >%;H4N

!!

"

# "

! " #

$"$# " # % ! & ' (

! " #

! " # ! ! " ##

W;X4,+*4,>%4H(;<,H;>4,$G,'4)($2,K,U,Y4J4'&5*4'4Z7 K4C4C:4',+*;+,5(+*,%'$:;:(<(+&,!0,LE0FM,5(<<,:4,3';52,G'$C, Y>$C45*4'4Z7, @$77

! !

$ %$ # $ %$ #

#

# "

$"$# " # % $ ! " #

44

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

B2,C, 3(C42>($2>

D4+,LE!0E.0NECM :4,;2,2P+I%<4 $G,H$2+(2I$I>, ';23$C,J;'(;:<4>0,;23,<4+,K,:4,>$C4,'4)($2, $G,!C N

# " ! ! "***" " ##

$ +

! ' ' ' (

*

!! !

"! # # # * *

*

$# $# $# # # # %

! "***" " # + $ $ +

$ +

" "*** " ! "***" " # ***

45 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

B234%42342C4

BD,E,;23,F,;'4,(234%42342+, +*42,G2$5(2),+*4,H;<I4,$D,E, 3$4>,2$+,*4<%,%'43(C+,+*4, H;<I4,$D,F

! " ! " ! # " $ % & '# $ ()) $ ! " # " ! # " $ % ! " #

J%)054()*+,KLM, (234%42342+

46

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

B234%42342C4

BD,E,;23,F,;'4,(234%42342+, +*42,G2$5(2),+*4,H;<I4,$D,E, 3$4>,2$+,*4<%,%'43(C+,+*4, H;<I4,$D,F

! " ! " ! # " $ % & '# $ ()) $ ! " # " ! # " $ % ! " #

+*4,C$2+$I'>,>;&,+*;+, ;CC4<4';+($2,;23,54()*+,;'4, (234%42342+

47 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

8B<+(C;'(;+4,DE%4F+;+($2 ! ! ! " #"

!

" " ! " # ! " # $

DGH%)054()*+I,J K.L7M0."//N O*4,F42+'$(3,$P,+*4, F<$B3

48

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

8B<+(C;'(;+4,DE%4F+;+($2 ! ! ! "

!

" ! " $ $ # ! " ! " !# " $

49 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

B4>+,&$C',C234'>+;23(2)

! " # $ # $ # $ " %&'( " ')'*+ " ,-. "/0'1" 2 34'(5-&1 ! " # " ! # " ! " !

D1<<,+*4,+(E4F DG2<&,5*42,H,;23,I,;'4,(234%42342+F DJ+,K;2,L;(<,4M42,(L,H,;23,I,;'4,(234%42342+F

50

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

O(M;'(;+4,PQ%4K+;+($2

#

" " $%$& % & ' & ! # ( " & % & ( + 6 , +# 6 , $ "50'1" + 6 , "

• .

#

" $%$& % & ' % & ( % & ( " + 6 , + 6 , +# 6 , $

#

" " $%$& % & ' % ! # ( " % % & ( + 6 , +# 6 , $ "50'1" + 6 , "

• .

#

! " ! " $%$& % & ' % & ! # ( " % & % & ( + 6 , + , +# 6 , $ "50'1" + 6 , "

• .

# $ # $ # $ " ! " # " ! # " ! " !

51 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

C(D;'(;+4,#$D;'(;2E4

!" !# $# " % &'($

! " "!

# $ % # $ ! ! " # # $ $ " ! $# " $ " % &'($

) ) " " ""

$ % $ &'( $ $ ! " " # $ $ $ $ " ! $# " $ " % &'($

) ) ! ! !!

# % # &'( # # ! " " # $ $ $ $

52

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

C(D;'(;+4,#$D;'(;2E4

!" !# $# " % &'($

! " "!

# $ % # $ ! ! " # # $ $ " ! $# " $ " % &'($

) ) " " ""

$ % $ &'( $ $ ! " " # $ $ $ $ " ! $# " $ " % &'($

) ) ! ! !!

# % # &'( # # ! " " # $ $ $ $

*+,-. % * * /01+- % % & ' ( ( ) * $ # $ ) ) !

& ' ( ( ) * $ $ # # $

! "! "! " * " "

) + +, , % )

) )

" $ " * $ " " " " ! " ! " ! ! #$%

53 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

#$C;'(;2D4,E2+F(+($2

GHI%)054()*+J,K L.M7N0.O//P

!

"#$ !

" !

"#$ !

" %&&

'()$*+ !

" %&&

'()$*+ !

"

54

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

#$C;'(;2D4,E2+F(+($2

GHI%)054()*+J,K L.M7N0.O//P

!

"#$ !

" !

"#$ !

" %&&

'()$*+ !

" %&&

'()$*+ !

"

9'(2D(%;< G()42C4D+$' $R,#

55 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

#$B:'(:1C3,DE1,D:C+=

! ! " # $ $ % & ' ' ( ( '

! "! "! " # " "

$ % %& & ' $

! !

" # " $ # ) ) ) ) ! ! " ! " " #$%

FG'E3,$',D:;=3>,HI,)J& K,",+*31,L,:12,M,:'3, (123%31231+ FG'E3,$',D:;=3>,HI,L,:12,M,:'3,(123%31231+, +*31,)J& K," FG'E3,$',D:;=3>,HI,)J& K,)J )& +*31,L,:12,M,:'3, 23+3'N(1(=+(C:;;&,'3;:+32 FG'E3,$',D:;=3>,HI,L,:12,M,:'3,23+3'N(1(=+(C:;;&, '3;:+32,+*31,)J& K,)J )&

O$4,C$E;2, &$E,%'$B3, $',2(=%'$B3, +*3=3P

56

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

R313':;,#$B:'(:1C3

! ! " ! " " #$% ' ( ( ' $ % %& & ' $

# " "

" # " $ #

S3+,! K,TL./L!/,U,LVW 93,:,B3C+$',$I,V C$1+(1E$E=,':12$N,B:'(:9;3=

( )"

" ( ) )(

* * +,- ) ' ' " % # !

?,(=,:,V,J,V,=&NN3+'(C,1$1X13):+(B3,23I(1(+3,N:+'(J HI,:;;,2(=+'(9E+($1=,:'3,;(13:';&,(123%31231+,(+,(=,%$=(+(B3,23I(1(+3 HI,+*3,2(=+'(9E+($1=,:'3,;(13:';&,23%31231+,(+,*:=,23+3'N(1:1+,Y3'$

57 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

7:')(1:;,<(=+'(9B+($1=

#

$ %$ "

"

&

'& & ( ) ( ) + 6 , + ,

58

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

"#12'*'#1:;+ <'=*&'9B*'#1=

! " # ! $ % ! ! !"#$%! ! &' ! ()*)') + ,

• +

./00 "$12#3 , 4(2

• !

% + 5600 "$12#3 , 4(2

• !

% + 6000 "$12#3 , 4(2

• !

%

59 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

"#12'*'#1:;+ <'=*&'9B*'#1=

! " # ! $ % ! ! !"#$%! ! &' ! ()*)') + ,

• +

./00 "$12#3 , 4(2

• !

%

+

• +

7

• +

,

• !

% ! $ % ! $ % !

5)%C

60

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

C234%42342D4,E4F(>(+43

C+G>,4;>&,+$,%'$F4,+*;+,+*4>4,>+;+4H42+>,;'4,4IJ(F;<42+K

! " ! " ! # " $ % & '# $ ()) $ ! " # " ! # " $ % ! " #

! " ! * " $ % & '# ! " ! * " $ % & '# ! " ! " ! # " $ % & '# ! " # ! " # " ! # " ! " # " ! # " ! " $ ! " $ ! "

61 Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007

8$'4,J>4MJ<,>+JMM

N;&4> EJ<4 OP*4>4,D;2,;<<,:4, %'$F43,M'$H, 34M(2(+($2>,$2, %'4F($J>,><(34>Q

+ ! * " !

%

& '& ! #

&# ! # " ! * " ! * # " ! # * " ' ! " ' ! # " ' ! # " ! ! " ! " ! * " ! * " ! " # " # ! " ! # " !

62

Michael S. Lewicki Carnegie Mellon AI: Probability Theory Mar 1, 2007 63

Next time: The process of probabilistic inference

• 1. define model of problem
• 2. derive posterior distributions and estimators
• 3. estimate parameters from data
• 4. evaluate model accuracy