Learning From Observat ions I n w hich w e describe agent s t hat - - PowerPoint PPT Presentation

▶
learning from observat ions
SMART_READER_LITE
LIVE PREVIEW

Learning From Observat ions I n w hich w e describe agent s t hat - - PowerPoint PPT Presentation

May 25, 2023 424 likes •1.09k views

Learning From Observat ions I n w hich w e describe agent s t hat can improve t heir behavior t hrough diligent st udy of t heir ow n experiences. - Art if icial I nt elligence: A M odern Approach Prepared by: San Chua, Natalie Weber,

slide-1
SLIDE 1

Learning From Observat ions

“I n w hich w e describe agent s t hat can improve t heir behavior t hrough diligent st udy of t heir ow n experiences.”

  • Art if icial I nt elligence: A M odern Approach

Prepared by: San Chua, Natalie Weber, Henry Kwong

slide-2
SLIDE 2

Out line

  • Learning agent s
  • I nduct ive learning
  • Learning decision t rees

– Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance

  • Learning general logical descript ions

– Current - best hypot hesis search algorit hm – Version space learning algorit hm

  • Comput at ional learning t heory
  • Summary
slide-3
SLIDE 3

Learning Agent

  • Four Component s

– Perf ormance Element : collect ion of know ledge and procedures t o decide

  • n t he next act ion.

E.g. w alking, t urning, draw ing, et c. – Learning Element : t akes in f eedback f rom t he crit ic and modif ies t he perf ormance element accordingly.

slide-4
SLIDE 4

Learning Agent (con’t )

  • Crit ic: provides t he learning element

w it h inf ormat ion on how w ell t he agent is doing based on a f ixed perf ormance st andard. E.g. t he audience

  • Problem Generat or: provides t he

perf ormance element w it h suggest ions

  • n new act ions t o t ake.
slide-5
SLIDE 5

Designing a Learning Element

  • Depends on t he design of t he

perf ormance element

  • Four maj or issues

– Which component s of t he perf ormance element t o improve – The represent at ion of t hose component s – Available f eedback – Prior know ledge

slide-6
SLIDE 6

Component s of t he Perf ormance Element

  • A direct mapping f rom condit ions on t he

current st at e t o act ions

  • I nf ormat ion about t he w ay t he w orld

evolves

  • I nf ormat ion about t he result s of possible

act ions t he agent can t ake

  • Ut ilit y inf ormat ion indicat ing t he

desirabilit y of w orld st at es

slide-7
SLIDE 7

Represent at ion

  • A component may be represent ed using

dif f erent represent at ion schemes

  • Det ails of t he learning algorit hm w ill dif f er

depending on t he represent at ion, but t he general idea is t he same

  • Funct ions are used t o describe a

component

slide-8
SLIDE 8

Feedback & Prior Know ledge

  • Supervised learning: input s and out put s

available

  • Reinf orcement learning: evaluat ion of

act ion

  • Unsupervised learning: no hint of correct
  • ut come
  • Background know ledge is a t remendous

help in learning

slide-9
SLIDE 9

Out line

  • Learning agent s
  • I nduct ive learning
  • Learning decision t rees

– Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance

  • Learning general logical descript ions

– Current - best hypot hesis search algorit hm – Version space learning algorit hm

  • Comput at ional learning t heory
  • Summary
slide-10
SLIDE 10

I nduct ive Learning

  • Ke y ide a :

– To use specif ic examples t o reach general conclusions

  • Given a set of examples, t he syst em t ries

t o approximat e t he evaluat ion f unct ion.

  • Also called Pure I nduct ive I nf erence
slide-11
SLIDE 11

Recogniz ing Handw rit t en Digit s

Learning Agent Training Examples

slide-12
SLIDE 12

Recogniz ing Handw rit t en Digit s

Different variations of handwritten 3’s

slide-13
SLIDE 13

Bias

  • Bias: any pref erence f or one hypot hesis
  • ver anot her, beyond mere consist ency

w it h t he examples.

  • Since t here are almost alw ays a large

number of possible consist ent hypot heses, all learning algorit hms exhibit some sort

  • f bias.
slide-14
SLIDE 14

Example of Bias

Is this a 7 or a 1? Some may be more biased toward 7 and others more biased toward 1.

slide-15
SLIDE 15

Formal Def init ions

  • Example: a pair (x, f (x)), w here

– x is t he input , – f (x) is t he out put of t he f unct ion applied t o x.

  • hypot hesis: a f unct ion h t hat approximat es

f , given a set of examples.

slide-16
SLIDE 16

Task of I nduct ion

  • The t ask of induct ion: Given a set of

examples, f ind a f unct ion h t hat approximat es t he t rue evaluat ion f unct ion f .

slide-17
SLIDE 17

Out line

  • Learning agent s
  • I nduct ive learning
  • Learning decision t rees

– Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance

  • Learning general logical descript ions

– Current - best hypot hesis search algorit hm – Version space learning algorit hm

  • Comput at ional learning t heory
  • Summary
slide-18
SLIDE 18

Patrons? No Yes WaitEst? No Alternate? Hungry? Yes Reservation? Fri/Sat? No Yes Yes none some full >60 30-60 10-30 0-10 no yes no yes no yes

Decision Tree Example

Goal Predicate: Will wait for a table?

No No Yes no yes

http://www.cs.washington.edu/education/courses/473/99wi/

slide-19
SLIDE 19

Patrons? WaitEst? Hungry? Yes none some full >60 30-60 10-30 0-10 no yes

Logical Represent at ion of a Pat h

r [Patrons(r, full) Wait_Estimate(r, 10-30) Hungry(r, yes)] Will_Wait(r)

slide-20
SLIDE 20

Expressiveness of Decision Trees

  • Any Boolean f unct ion can be w rit t en as a decision

t ree

  • Limit at ions

– Can only describe one obj ect at a t ime. – Some f unct ions require an exponent ially large decision t ree.

  • E.g. Parit y f unct ion, maj orit y f unct ion
  • Decision t rees are good f or some kinds of

f unct ions, and bad f or ot hers.

  • There is no one ef f icient represent at ion f or all

kinds of f unct ions.

slide-21
SLIDE 21

Principle Behind t he Decision- Tree- Learning Algorit hm

  • Uses a general principle of induct ive

learning of t en called Ock ha m ’s r a z or : “The most likely hypot hesis is t he simplest one t hat is consist ent w it h all

  • bservat ions.”
slide-22
SLIDE 22

Out line

  • Learning agent s
  • I nduct ive learning
  • Learning decision t rees

– Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance

  • Learning general logical descript ions

– Current - best hypot hesis search algorit hm – Version space learning algorit hm

  • Comput at ional learning t heory
  • Summary
slide-23
SLIDE 23
  • Goa l: Find a relat ively small decision t ree

t hat is consist ent w it h all t raining examples, and w ill correct ly classif y new examples.

  • Not e t hat f inding t he smallest decision

t ree is an int ract able problem. So t he Decision- Tree- Algorit hm uses some simple heurist ics t o f ind a “smallish” one.

Decision- Tree- Learning Algorit hm

slide-24
SLIDE 24

Get t ing St art ed

  • Come up w it h a set of at t ribut es t o

describe t he obj ect or sit uat ion.

  • Collect a complet e set of examples

(t raining set ) f rom w hich t he decision t ree can derive a hypot hesis t o def ine (answ er) t he goal predicat e.

slide-25
SLIDE 25

A t t r i b u t e s G

  • a

l E x a m p l e F r i H u n P a t P r i c e R a i n R e s T y p e E s t W i l l W a i t X

1

N

  • Y

e s S

  • m

e $ $ $ N

  • Y

e s F r e n c h

  • 1

Y e s X

2

N

  • Y

e s F u l l $ N

  • N
  • T

h a i 3

  • 6

N

  • X

3

N

  • N
  • S
  • m

e $ N

  • N
  • B

u r g e r

  • 1

Y e s X

4

Y e s Y e s F u l l $ N

  • N
  • T

h a i 1

  • 3

Y e s X

5

Y e s N

  • F

u l l $ $ $ N

  • Y

e s F r e n c h > 6 N

  • X

6

N

  • Y

e s S

  • m

e $ $ Y e s Y e s I t a l i a n

  • 1

Y e s X

7

N

  • N
  • N
  • n

e $ Y e s N

  • B

u r g e r

  • 1

N

  • X

8

N

  • Y

e s S

  • m

e $ $ Y e s Y e s T h a i

  • 1

Y e s X

9

Y e s N

  • F

u l l $ Y e s N

  • B

u r g e r > 6 N

  • X

1

Y e s Y e s F u l l $ $ $ N

  • Y

e s I t a l i a n 1

  • 3

N

  • X

1 1

N

  • N
  • N
  • n

e $ N

  • N
  • T

h a i

  • 1

N

  • X

1 2

Y e s Y e s F u l l $ N

  • N
  • B

u r g e r 3

  • 6

Y e s

Will we wait, or not?

The Rest aurant Domain

http://www.cs.washington.edu/education/courses/473/99wi/

slide-26
SLIDE 26

Split t ing Examples by Test ing

  • n At t ribut es

+ X1, X3, X4, X6, X8, X12 (Positive examples)

  • X2, X5, X7, X9, X10, X11 (Negative examples)
slide-27
SLIDE 27

Split t ing Examples by Test ing

  • n At t ribut es (con’t )

+ X1, X3, X4, X6, X8, X12 (Positive examples)

  • X2, X5, X7, X9, X10, X11 (Negative examples)

Patrons? +

  • X7, X11

none some full +X1, X3, X6, X8

  • +X4, X12
  • X2, X5, X9, X10
slide-28
SLIDE 28

Split t ing Examples by Test ing

  • n At t ribut es (con’t )

+ X1, X3, X4, X6, X8, X12 (Positive examples)

  • X2, X5, X7, X9, X10, X11 (Negative examples)

Patrons? +

  • X7, X11

none some full +X1, X3, X6, X8

  • +X4, X12
  • X2, X5, X9, X10

No Yes

slide-29
SLIDE 29

Split t ing Examples by Test ing on At t ribut es (con’t )

+ X1, X3, X4, X6, X8, X12 (Positive examples)

  • X2, X5, X7, X9, X10, X11 (Negative examples)

Patrons? +

  • X7, X11

none some full +X1, X3, X6, X8

  • +X4, X12
  • X2, X5, X9, X10

Hungry? + X4, X12

  • X2, X10

+

  • X5, X9

no yes No Yes

slide-30
SLIDE 30

Patrons? +

  • X7, X11

none some full +X1, X3, X6, X8

  • +X4, X12
  • X2, X5, X9, X10

Type? + X1

  • X5

French Italian Thai +X6

  • X10

+X3, X12

  • X7, X9

+ X4,X8

  • X2, X11

Burger

What M akes a Good At t ribut e?

Better Attribute

Not As Good An Attribute

http://www.cs.washington.edu/education/courses/473/99wi/

slide-31
SLIDE 31

Patrons? No Yes Hungry? Type? Fri/Sat? No Yes Yes Yes No No none some full No Yes French Italian no yes Thai burger

Final Decision Tree

http://www.cs.washington.edu/education/courses/473/99wi/

slide-32
SLIDE 32

Patrons? No Yes WaitEst? No Alternate? Hungry? Yes Reservation? Fri/Sat? No Yes Yes none some full >60 30-60 10-30 0-10 no yes no yes no yes

Original Decision Tree Example

Goal Predicate: Will wait for a table?

No No Yes no yes

http://www.cs.washington.edu/education/courses/473/99wi/

slide-33
SLIDE 33

Out line

  • Learning agent s
  • I nduct ive learning
  • Learning decision t rees

– Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance

  • Learning general logical descript ions

– Current - best hypot hesis search algorit hm – Version space learning algorit hm

  • Comput at ional learning t heory
  • Summary
slide-34
SLIDE 34

Assessing t he Perf ormance of t he Learning Algorit hm

  • A learning algorit hm is good if it produces

hypot heses t hat do a good j ob of predicat ing t he classif icat ions of unseen examples

  • Test t he algorit hm’s predict ion

perf ormance on a set of new examples, called a t est set .

slide-35
SLIDE 35

M et hodology in Accessing Perf ormance

  • 1. Collect a large set of examples.
  • 2. Divide it int o 2 disj oint set : t he t raining set and

t he t est set . I t is very import ant t hat t hese 2 set s are separat e so t hat t he algorit hm doesn’t cheat . Usually t his division of examples is done randomly.

  • 3. Use t he learning algorit hm w it h t he t raining set as

examples t o generat e a hypot hesis H.

slide-36
SLIDE 36

M et hodology (con’t )

  • 4. M easure t he percent age of examples in

t he t est set t hat are correct ly classif ied by H.

  • 5. Repeat st eps 1 t o 4 f or dif f erent siz es of

t raining set s and dif f erent randomly select ed t raining set s of each siz e.

slide-37
SLIDE 37

Analyz ing t he Result s

0 20 40 60 80 100

1.0 0.9 0.8 0.7 0.6 0.5 0.4

% correct

  • n test set

Training set size Learning Curve for the Decision Tree Algorithm (On examples in the restaurant domain) Happy Graph

slide-38
SLIDE 38

Overf it t ing

  • Overf it t ing is w hat happens w hen a learning

algorit hm f inds meaningless “regularit y” in t he dat a.

  • Caused by irrelevant at t ribut es.
  • Solut ion: decision t ree pruning.

– Result ing decision t ree is.

  • Smaller.
  • M ore t olerant t o noise.
  • M ore accurat e in it s predict ions.
slide-39
SLIDE 39

Pract ical Uses of Decision Tree Learning

  • Designing oil plat f orm equipment .
  • Learning t o f ly a plane.
  • Diagnosing heart at t acks.
slide-40
SLIDE 40

Out line

  • Learning agent s
  • I nduct ive learning
  • Learning decision t rees

– Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance

  • Learning general logical descript ions

– Current - best hypot hesis search algorit hm – Version space learning algorit hm

  • Comput at ional learning t heory
  • Summary
slide-41
SLIDE 41

Learning General Logical Descript ion

  • Ke y ide a :

– Look at induct ive learning generally – Find a logical descript ion t hat is equivalent t o t he (unknow n) evaluat ion f unct ion

  • M ake our hypot hesis more or less specif ic

t o mat ch t he evaluat ion f unct ion.

slide-42
SLIDE 42

Out line

  • Learning agent s
  • I nduct ive learning
  • Learning decision t rees

– Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance

  • Learning general logical descript ions

– Current - best hypot hesis search algorit hm – Version space learning algorit hm

  • Comput at ional learning t heory
  • Summary
slide-43
SLIDE 43

Current - best - hypot hesis Search

  • Ke y ide a :

– M aint ain a single hypot hesis t hroughout . – Updat e t he hypot hesis t o maint ain consist ency as a new example comes in.

slide-44
SLIDE 44

Def init ions

  • Posit ive example: an inst ance of t he

hypot hesis

  • Negat ive example: not an inst ance of t he

hypot hesis

  • False negat ive example: t he hypot hesis

predict s it should be a negat ive example but it is in f act posit ive

  • False posit ive example: should be posit ive

but it is act ually negat ive.

slide-45
SLIDE 45

Current - best - hypot hesis Search Algorit hm

  • 1. Pick a random example t o def ine t he

init ial hypot hesis

  • 2. For each example,

– I n case of a f alse negat ive:

  • Generaliz e t he hypot hesis t o include it

– I n case of a f alse posit ive:

  • Specializ e t he hypot hesis t o exclude it
  • 3. Ret urn t he hypot hesis
slide-46
SLIDE 46

How t o Generaliz e

  • Replacing Const ant s w it h Variables:

Obj ect (Animal,Bird) Obj ect (X,Bird)

  • Dropping Conj unct s:

Obj ect (Animal,Bird) & Feat ure(Animal,Wings) Obj ect (Animal,Bird)

  • Adding Disj unct s:

Feat ure(Animal,Feat hers) Feat ure(Animal,Feat hers) v Feat ure(Animal,Fly)

  • Generaliz ing Terms:

Feat ure(Bird,Wings) Feat ure(Bird,Primary- Feat ure)

ht t p:/ / w w w .pit t .edu/ ~sut hers/ inf sci1054/ 8.ht ml

slide-47
SLIDE 47

How t o Specializ e

  • Replacing Variables w it h Const ant s:

Obj ect (X, Bird) Obj ect (Animal, Bird)

  • Adding Conj unct s:

Obj ect (Animal,Bird) Obj ect (Animal,Bird) & Feat ure(Animal,Wings)

  • Dropping Disj unct s:

Feat ure(Animal,Feat hers) v Feat ure(Animal,Fly) Feat ure(Animal,Fly)

  • Specializ ing Terms:

Feat ure(Bird,Primary- Feat ure) Feat ure(Bird,Wings)

ht t p:/ / w w w .pit t .edu/ ~sut hers/ inf sci1054/ 8.ht ml

slide-48
SLIDE 48

What do all t hese mean?

  • Let ’s look at some examples...
slide-49
SLIDE 49

Generaliz e and Specializ e

  • M ust be consist ent w it h all ot her

examples

  • Non- det erminist ic

– At any point t here may be several possible specializ at ions or generaliz at ions t hat can be applied.

slide-50
SLIDE 50

Pot ent ial Problem of Current - best - hypot hesis Search

  • Ext ension made not necessarily lead t o t he

simplest hypot hesis.

  • M ay lead t o an unrecoverable sit uat ion

w here no simple modif icat ion of t he hypot hesis is consist ent w it h all of t he examples.

  • The program must backt rack t o a previous

choice point .

slide-51
SLIDE 51

Problem of Backt racking

  • Require large space t o st ore all examples
  • Need t o check all previous inst ances af t er

each modif icat ion of t he hypot hesis.

  • Search and check all t hese previous

inst ances over again af t er each modif icat ion is very expensive

slide-52
SLIDE 52

Out line

  • Learning agent s
  • I nduct ive learning
  • Learning decision t rees

– Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance

  • Learning general logical descript ions

– Current - best hypot hesis search algorit hm – Version space learning algorit hm

  • Comput at ional learning t heory
  • Summary
slide-53
SLIDE 53

Version Space Learning Algorit hm

  • Least - Commit ment Search
  • No backt racking
  • Ke y ide a :

– M aint ain t he most general and specif ic hypot heses at any point in learning. Updat e t hem as new examples come in.

slide-54
SLIDE 54

Def init ions

  • Version space: a set of all hypot heses

consist ent w it h examples seen so f ar

  • Boundary set s: set s of hypot heses

def ining boundary on w hich hypot heses are consist ent w it h examples – M ost general (G- set ) and most specif ic (S- set ) boundary set s

slide-55
SLIDE 55

Requirement

  • Usually requires an enormous number of

hypot heses t o record.

  • An assumpt ion: a part ial ordering (more-

specif ic- t han ordering) exist s on all of t he hypot heses in t he space – hierarchical

  • Boundary set s circumscribing t he space of

possible hypot heses. – G- set (t he most general boundary) – S- set (t he most specif ic boundary)

slide-56
SLIDE 56

Version Space Learning Algorit hm

  • 1. I nit ially, t he G- set is True, and t he S- set is

False

  • 2. For each new example, t here are 6

possible cases: a) f alse posit ive f or Si in S

  • Si is t oo general - no consist ent

specializ at ions.

  • Throw it out .

b) f alse negat ive f or Si in S

  • Si is t oo specif ic.
  • Replace it w it h it s generaliz at ions.
slide-57
SLIDE 57

Version Space Learning Algorit hm (con’t )

c) f alse posit ive f or G

i in G

  • G

i is t oo general.

  • Replace it w it h it s specializ at ions.

d) f alse negat ive f or G

i in G

  • G

i is t oo specif ic - no consist ent

generaliz at ions.

  • Throw it out .

e) Si more general t han some ot her hypot hesis in S or G

  • Throw it out .

f ) G

i more specif ic t han some ot her

hypot hesis in S or G

  • Throw it out .
slide-58
SLIDE 58

Version Space Learning Algorit hm (con’t )

  • 3. Repeat t he process unt il one of t hree

t hings happens: a) Only one hypot hesis lef t in t he version space.

  • This is t he answ er w e w ant .

b) The version space collapses, i.e. eit her G or S becomes empt y.

  • This means t here are no consist ent

hypot heses.

c) We run out of examples w hile t he version space st ill has several hypot heses.

  • Use t heir collect ive evaluat ion (breaking

disagreement s w it h maj orit y vot e).

slide-59
SLIDE 59

Advant ages of t he Algorit hm

  • Never f avor one possible hypot hesis over

anot her; all remaining hypot heses are consist ent

  • Never require backt racking
slide-60
SLIDE 60

Pot ent ial Problems

  • Does not deal w it h noise

– Not very pract ical in real- w orld learning problem

  • Unlimit ed disj unct ions in t he version space leads t o

– The S- set has a single most specif ic hypot hesis – The G- set has a most general hypot hesis

slide-61
SLIDE 61

Out line

  • Learning agent s
  • I nduct ive learning
  • Learning decision t rees

– Example of a decision t ree – Decision- t ree- learning algorit hm – Accessing t he perf ormance

  • Learning general logical descript ions

– Current - best hypot hesis search algorit hm – Version space learning algorit hm

  • Comput at ional learning t heory
  • Summary
slide-62
SLIDE 62

Why Learning Works

  • Problem: How can you know if a t heory

w ill accurat ely predict t he f ut ure? OR

How can you know t hat a hypot hesis is close t o t he t arget f unct ion if you don’t know w hat t he t arget f unct ion is?

  • Answ ers provided by Comput at ional

Learning Theory

slide-63
SLIDE 63

Comput at ional Learning Theory

  • M ain principle: “any hypot hesis t hat is

seriously w rong w ill almost cert ainly be ‘f ound out ’ w it h high probabilit y af t er a small number of examples, because it w ill make an incorrect predict ion.”

  • Assumes t hat t he t raining and t est set s

are draw n randomly

slide-64
SLIDE 64

Summary

  • Learning agent s
  • I nduct ive learning
  • Learning decision t rees
  • Learning general logical descript ions

– Current - best hypot hesis search algorit hm – Version space learning algorit hm

  • Comput at ional learning t heory
slide-65
SLIDE 65

Ref erences

  • Russel, S. and P. Norvig (1995). Art if icial

I nt elligence - A M odern Approach. Upper Saddle River, NJ, Prent ice Hall.

  • ht t p:/ / w w w .pit t .edu/ ~sut hers/ inf sci1054/ 8.ht ml
  • ht t p:/ / enuxsa.eas.asu.edu/ ~cse471/ 4- 20