Learning From Observat ions I n w hich w e describe agent s t hat - PowerPoint PPT Presentation

Learning From Observat ions “I n w hich w e describe agent s t hat can improve t heir behavior t hrough diligent st udy of t heir ow n experiences.” - Art if icial I nt elligence: A M odern Approach Prepared by: San Chua, Natalie Weber, Henry Kwong

Out line • Learning agent s • I nduct ive learning • Learning decision t rees – Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance • Learning general logical descript ions – Current - best hypot hesis search algorit hm – Version space learning algorit hm • Comput at ional learning t heory • Summary

Learning Agent • Four Component s – Perf ormance Element : collect ion of know ledge and procedures t o decide on t he next act ion. E.g. w alking, t urning, draw ing, et c. – Learning Element : t akes in f eedback f rom t he crit ic and modif ies t he perf ormance element accordingly.

Learning Agent (con’t ) - Crit ic: provides t he learning element w it h inf ormat ion on how w ell t he agent is doing based on a f ixed perf ormance st andard. E.g. t he audience - Problem Generat or: provides t he perf ormance element w it h suggest ions on new act ions t o t ake.

Designing a Learning Element • Depends on t he design of t he perf ormance element • Four maj or issues – Which component s of t he perf ormance element t o improve – The represent at ion of t hose component s – Available f eedback – Prior know ledge

Component s of t he Perf ormance Element • A direct mapping f rom condit ions on t he current st at e t o act ions • I nf ormat ion about t he w ay t he w orld evolves • I nf ormat ion about t he result s of possible act ions t he agent can t ake • Ut ilit y inf ormat ion indicat ing t he desirabilit y of w orld st at es

Represent at ion • A component may be represent ed using dif f erent represent at ion schemes • Det ails of t he learning algorit hm w ill dif f er depending on t he represent at ion, but t he general idea is t he same • Funct ions are used t o describe a component

Feedback & Prior Know ledge • Supervised learning: input s and out put s available • Reinf orcement learning: evaluat ion of act ion • Unsupervised learning: no hint of correct out come • Background know ledge is a t remendous help in learning

Out line • Learning agent s • I nduct ive learning • Learning decision t rees – Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance • Learning general logical descript ions – Current - best hypot hesis search algorit hm – Version space learning algorit hm • Comput at ional learning t heory • Summary

I nduct ive Learning • Ke y ide a : – To use specif ic examples t o reach general conclusions • Given a set of examples, t he syst em t ries t o approximat e t he evaluat ion f unct ion. • Also called Pure I nduct ive I nf erence

Recogniz ing Handw rit t en Digit s Learning Agent Training Examples

Recogniz ing Handw rit t en Digit s Different variations of handwritten 3’s

Bias • Bias: any pref erence f or one hypot hesis over anot her, beyond mere consist ency w it h t he examples. • Since t here are almost alw ays a large number of possible consist ent hypot heses, all learning algorit hms exhibit some sort of bias.

Example of Bias Is this a 7 or a 1? Some may be more biased toward 7 and others more biased toward 1.

Formal Def init ions • Example: a pair (x, f (x)), w here – x is t he input , – f (x) is t he out put of t he f unct ion applied t o x. • hypot hesis: a f unct ion h t hat approximat es f , given a set of examples.

Task of I nduct ion • The t ask of induct ion: Given a set of examples, f ind a f unct ion h t hat approximat es t he t rue evaluat ion f unct ion f .

Out line • Learning agent s • I nduct ive learning • Learning decision t rees – Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance • Learning general logical descript ions – Current - best hypot hesis search algorit hm – Version space learning algorit hm • Comput at ional learning t heory • Summary

Decision Tree Example Goal Predicate: Patrons? Will wait for a table? none full some No Yes WaitEst? >60 0-10 30-60 10-30 No Alternate? Hungry? Yes yes no yes no No Yes Reservation? Fri/Sat? yes no yes no No Yes No Yes http://www.cs.washington.edu/education/courses/473/99wi/

Logical Represent at ion of a Pat h Patrons? none full some WaitEst? 0-10 >60 30-60 10-30 Hungry? yes no Yes r [Patrons(r, full) Wait_Estimate(r, 10-30) Hungry(r, yes)] Will_Wait(r)

Expressiveness of Decision Trees • Any Boolean f unct ion can be w rit t en as a decision t ree • Limit at ions – Can only describe one obj ect at a t ime. – Some f unct ions require an exponent ially large decision t ree. • E.g. Parit y f unct ion, maj orit y f unct ion • Decision t rees are good f or some kinds of f unct ions, and bad f or ot hers. • There is no one ef f icient represent at ion f or all kinds of f unct ions.

Principle Behind t he Decision- Tree- Learning Algorit hm • Uses a general principle of induct ive learning of t en called Ock ha m ’s r a z or : “ The most likely hypot hesis is t he simplest one t hat is consist ent w it h all observat ions .”

Out line • Learning agent s • I nduct ive learning • Learning decision t rees – Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance • Learning general logical descript ions – Current - best hypot hesis search algorit hm – Version space learning algorit hm • Comput at ional learning t heory • Summary

Decision- Tree- Learning Algorit hm • Goa l: Find a relat ively small decision t ree t hat is consist ent w it h all t raining examples, and w ill correct ly classif y new examples. • Not e t hat f inding t he smallest decision t ree is an int ract able problem. So t he Decision- Tree- Algorit hm uses some simple heurist ics t o f ind a “smallish” one.

Get t ing St art ed • Come up w it h a set of at t ribut es t o describe t he obj ect or sit uat ion. • Collect a complet e set of examples (t raining set ) f rom w hich t he decision t ree can derive a hypot hesis t o def ine (answ er) t he goal predicat e.

The Rest aurant Domain G A b u e s o a t t r t i l W W m E H P P R R E a p e F u n a e a n e T p e a x c s y s r r l i t i i t i l l i t m N N X Y S Y F Y o e s o e $ $ $ o e s e n c h 0 1 0 e s r - 1 X N Y N N N o e F u $ o o T h a 3 0 6 0 o s - l l i 2 m N N N N X S B Y o o o e $ o o u g e 0 1 0 e s r r - 3 X Y Y N N Y e e F u $ o o T h a 1 0 3 0 e s s s - l l i 4 X Y N N Y N e o F u $ $ $ o e F e n h > 6 0 o s s c r l l 5 m X N Y S Y Y Y o e o e $ $ e e a a n 0 1 0 e s s s s I t - l i 6 X N N N Y N B N o o o n e $ e o u g e 0 1 0 o s r r - 7 m X N Y S Y Y Y o e o e $ $ e e T h a 0 1 0 e s s s s - i 8 X Y N Y N B N e o F u $ e o u g e > 6 0 o s s r r l l 9 X Y Y F N Y N e s e s u $ $ $ o e s a a n 1 0 3 0 o I t - l l l i 1 0 X N N N N N N o o o n e $ o o T h a 0 1 0 o - i 1 1 X Y Y F N N B Y e s e s u $ o o u g e 3 0 6 0 e s r r - l l 1 2 Will we wait, or not? http://www.cs.washington.edu/education/courses/473/99wi/

Split t ing Examples by Test ing on At t ribut es + X1, X3, X4, X6, X8, X12 (Positive examples) - X2, X5, X7, X9, X10, X11 (Negative examples)

Split t ing Examples by Test ing on At t ribut es (con’t ) + X1, X3, X4, X6, X8, X12 (Positive examples) - X2, X5, X7, X9, X10, X11 (Negative examples) Patrons? none full some + +X1, X3, X6, X8 +X4, X12 - X7, X11 - - X2, X5, X9, X10

Split t ing Examples by Test ing on At t ribut es (con’t ) + X1, X3, X4, X6, X8, X12 (Positive examples) - X2, X5, X7, X9, X10, X11 (Negative examples) Patrons? none full some + +X1, X3, X6, X8 +X4, X12 - X7, X11 - - X2, X5, X9, X10 No Yes

Split t ing Examples by Test ing on At t ribut es (con’t ) + X1, X3, X4, X6, X8, X12 (Positive examples) - X2, X5, X7, X9, X10, X11 (Negative examples) Patrons? none full some + +X1, X3, X6, X8 +X4, X12 - X7, X11 - - X2, X5, X9, X10 No Yes Hungry? no yes + X4, X12 + - X2, X10 - X5, X9

What M akes a Good At t ribut e? Better Patrons? Attribute none full some + +X1, X3, X6, X8 +X4, X12 - X7, X11 - - X2, X5, X9, X10 Not As Good Type? An Attribute French Burger Italian Thai + X1 +X6 + X4,X8 +X3, X12 - X5 - X10 - X2, X11 - X7, X9 http://www.cs.washington.edu/education/courses/473/99wi/

Final Decision Tree Patrons? none some full Hungry? No Yes No Yes No Type? French burger Italian Thai Yes Fri/Sat? Yes No no yes No Yes http://www.cs.washington.edu/education/courses/473/99wi/