CS 4100: Artificial Intelligence Bayes Nets Jan-Willem van de - PDF document

CS 4100: Artificial Intelligence Bayes’ Nets Jan-Willem van de Meent, Northeastern University [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials are available at http://ai.berkeley.edu.] Probabilistic Models • Models describe how (a portion of) the world works ks • Mo Model els ar are e al alway ays simplifi ficat cations • May not account for every variable • May not account for all interactions between variables • “All models are wrong; but some are useful.” – George E. P. Box • Wha What do do we do do with h pr proba babi bilistic mode dels? • We (or our agents) need to reason about unknown variables, given evidence ple: explanation (diagnostic reasoning) • Ex Exampl • Ex Exampl ple: prediction (causal reasoning) • Ex Exampl ple: value of information

Independence Independence • Tw Two o variables are in independent if if: • This says that their joint distribution factors into a product two simpler distributions • Another form: • We write: • In Indep epen enden ence ce is a a simplifying model eling as assumption • Em Empi pirical jo join int dis istrib ibutio ions: at best “close” to independent • What could we assume for {W {Weather, Traffic, Cavity, Toothache} ?

Example: Independence? T P hot 0.5 cold 0.5 T W P T W P hot sun 0.4 hot sun 0.3 hot rain 0.1 hot rain 0.2 cold sun 0.2 cold sun 0.3 cold rain 0.3 cold rain 0.2 W P sun 0.6 rain 0.4 Example: Independence • N fa fair ir, in independent coin oin flip flips: H 0.5 H 0.5 H 0.5 T 0.5 T 0.5 T 0.5

Conditional Independence • Jo Joint distribution: P(T (Toothache, Cavit ity, Catch) If If I h I have a a cavi cavity, t , the p probability t that t the p probe cat catch ches es in in it it • do doesn't de depe pend d on whether I have a to tooth thache: • P(+ P(+cat catch ch | +toothach ache, e, +cavi cavity) y) = P(+ P(+cat catch ch | +cavi cavity) The sa The same ind ndepend ndenc nce hol holds s if I don on’t t have a cavi cavity: • P(+cat P(+ catch ch | +toothach ache, e, -cavi cavity) ) = P(+ P(+cat catch ch| -cavi cavity) • Catch is Ca is condit itio ionally lly in independent of Toot Tootha hache he gi given n Ca Cavit ity: • • P(C P(Cat atch ch | Toothach ache, e, Cavi avity) y) = P(C P(Cat atch ch | Cavi avity) y) Equi Equivalent nt statement nts: • • P(Toothach P(T ache e | Cat atch ch , Cavi avity) y) = P(T P(Toothach ache e | Cavi avity) y) P(Toothach P(T ache, e, Cat atch ch | Cavi avity) y) = P(T P(Toothach ache e | Cavi avity) y) P(C P(Cat atch ch | Cavi avity) y) • One can be derived from the other easily • Conditional Independence • Un Uncondit itio ional l (a (abs bsolute) ) inde depe pende dence very rare (wh (why?) ?) • Co Condi nditiona nal in independence is is our r most basic ic and ro robust form of kn knowledge about uncertain en environmen ents. • X X is is co conditional ally indep epen enden ent of of Y gi given Z if if an and only if : or or, e , equivalently, i , if a and o only i if

Conditional Independence • Wha What ab about this domai ain: • Tr Traffic Traffic ⫫ Um Tr Umbrella lla • Um Umbrella lla • Ra Raining Tr Traffic ⫫ Um Umbrella lla | Rain inin ing Conditional Independence • Wha What ab about this domai ain: • Fi Fire • Smoke ke • Al Alarm Al Alarm ⫫ Fi Fire | Smoke ke

Conditional Independence and the Chain Rule • Cha Chain n rul ule: • Tr Trivial decom ompos ositi tion on: • Wi With h assum umpt ption n of condi nditiona nal inde ndepe pende ndenc nce: • Ba Bayes’ ne nets / / graphical mo models he help us us express cond nditiona nal ind ndepend ndenc nce assum umptions ns Ghostbusters Chain Rule Ea Each h sens nsor de depe pends nds onl nly • P(T,B,G) = P(G) P(T|G) P(B|G) on on whe here the he ghost ghost is T B G P(T,B,G) • That means, Tha ns, the he two o se sensor nsors s ar are e co condition onal ally y indep epen enden ent, , +t +b +g 0.16 gi given n th the ghost t positi tion +t +b -g 0.16 T: Top square is re • red +t -b +g 0.24 B: Bottom square is re B: red +t -b -g 0.04 G: G: Ghost is in the top -t +b +g 0.04 Givens (assu Giv assumption ons): ): • -t +b -g 0.24 P( +g +g ) = = 0. 0.5 P( ( -g g ) = = 0.5 -t -b +g 0.06 P( +t +t | +g +g ) = = 0.8 -t -b -g 0.06 P( +t +t | -g g ) = = 0.4 P( +b +b | +g +g ) = = 0.4 P( +b +b | -g g ) = = 0.8

Bayes’ Nets: Big Picture Bayes’ Nets: Big Picture • Tw Two o prob oblems with th using g fu full jo join int dis istrib ributio ion table les as as ou our pr proba babi bilistic mode dels: • Unless there are only a few variables, the joint is WAY too big to represent explicitly • Hard to learn (estimate) anything empirically about more than a few variables at a time • Ba Bayes’ ne nets: De Describ ribe comple lex jo join int dis istrib ributio ions (mo models) ) us using ng simple, local distribut utions ns (co conditional al probab abilities es) • More properly called gr graphi hical mod odels • We describe how var variab ables es local cally y inter eract act • Local interactions chain together to give global, in indir irect in interactio ions • For about 10 min, we’ll be vague about how these interactions are specified

Example Bayes’ Net: Insurance Example Bayes’ Net: Car

Graphical Model Notation • No Node des: v : variables ( s (with d domains) s) • Can be assigned ( ob obse served ) or unassigned ( unob unobse served ) • Ar Arcs: interactions • Similar to CSP constraints • Indicate “di direct influence” between variables • For Formally: encode conditional independence (more later) • Fo For no now: im imagin ine that arro rrows mean dire irect cau causat ation (in gen ener eral al, they ey don’t! t!) Example: Coin Flips • N ind independ ndent nt coin oin flip flips X 1 X 2 X n • No No int interactions ions between n varia iable les: abs absolute te ind independ ndenc nce

Example: Traffic • Va Variables: • R: R: It It r rains • T: The There is s traffic • Mo Model el 1: in independence • Mo Model el 2: ra rain in causes tra raffic ic R R T T • Why Why is is an agent usin ing mo model 2 be better? Example: Traffic II • Le Let’s bui uild a caus usal graphi hical model! • Va Variables • T: T: Tr Traffic • R: R: It It r rains • L: L: Low Low pressur ssure • D: D: Ro Roof drips • B: B: Ba Ballgame • C: C: Ca Cavit ity L R B C D T

Example: Alarm Network • Va Variables • B: B: Bu Burglary • A: A: Al Alarm goes off • M: M: Ma Mary ry calls • J: J: Jo John calls • E: E: Earthquake ke! E B A J M Bayes’ Net Semantics