D etecti ng and Correcti ng Errors of O m i ssi on A fter - - PowerPoint PPT Presentation

SLIDE 1

D etecti ng and Correcti ng Errors

• f

O m i ssi

• n

A fter Expl anati

• n-based

Learni ng

M i chael J. Pazzani Depar tm ent

• f

I nf

• r

m ati

• n

and Com puter Sci ence Uni versi ty

• f

Cal i forni a, I rvi ne, CA 92717 pazzani @ i cs. uci . edu

1 I JCA I

• 89

M onday, A ugust 14, 1989

SLIDE 2

O utl i ne

I .Probl em : A . D etecti ng er r

• r

s i n gener al i zati

• n

pr

• duced

by EBL

• B. A ssi

gni ng bl am e to r ul es i n dom ai n theor y

• C. Cor

r ecti ng dom ai n theor y I I . Types

• f

Errors A . Errors

• f

O m i ssi

• n-

Fai l to m ake cor r ect pr edi cti

• n
• B. Errors
• f

Com m i ssi

• n -

M ake i ncorrect predi cti

• n

I I I . I ndexi ng gener al i zati

• ns

i n M em or y A . Expl anatory- Gi ven resul t, Predi ct cause

• B. Predi

cti ve- Gi ven acti

• n,

Pr edi ct r esul t I V . Unsuper vi sed detecti

• n
• f

er r

• r

s

• f
• m i

ssi

• n

V . Bl am e A ssi gnm ent & r evi si ng dom ai n theor y V I . Experi m ental Resul ts

2 I JCA I

• 89

M onday, A ugust 14, 1989

SLIDE 3

Backgr

• und:

O CCA M

• Perform ance

task

• Pr

edi ct

• utcom e
• f

econom i c sancti

• n

i nci dents

• O utcom e

i nferred by hi erarchi cal cl assi fi cati

• n
• Er

r

• r
• f
• m i

ssi

• n
• ccur

s i f i nci dent cannot be cl assi fi ed

• Error
• f

com m i ssi

• n
• ccurs

i f i nci dent i s cl assi fi ed i ncorrectl y

• Lear

ni ng M ethod:

• Com bi

nes em pi r i cal and expl anati

• n-

based l ear ni ng

• em pi

r i cal techni ques l ear n the dom ai n theor y used by EBL

• Probl

em : i ncor r ect dom ai n theor y → i ncorrect general i zati

• ns

3 I JCA I

• 89

M onday, A ugust 14, 1989

coerce s1 s2 s3 s5 s6 s7 s8

SLIDE 4

Term i nol

• gy

Types

• f

exam pl es: Foundati

• nal

: exam pl es f r

• m

w hi ch the dom ai n theor y i s l ear ned E. g. , Par ent hel pi ng chi l d Perform ance: exam pl es

• f

the per f

• r

m ance task E. g. , Ki dnappi ng exam pl es Foundati

• nal

exam pl es ar e subpr

• bl

em s

• f

the per f

• r

m ance task. Per f

• r

m ance exam pl es ar e exam pl es

• f

the per f

• r

m ance task. D i sti ncti

• n

i s r el ati ve to task. I f task i s pr edi cti ng w hom to sel l r ansom i nsur ance to, the ki dnappi ng exam pl es ar e f

• undati
• nal

. Types

• f

r ul es (or schem ata) Dom ai n: used by EBL to expl ai n per f

• r

m ance exam pl e Com pi l ed: cr eated by EBL to be used i n per f

• r

m ance task.

4 I JCA I

• 89

M onday, A ugust 14, 1989

SLIDE 5

Pr

• bl

em Statem ent:

• Rul

e i n dom ai n theor y ar e l ear ned (or hand- coded) Par ents have a goal

• f

pr eser vi ng thei r chi l dr en’ s heal th

• Cr

eate com pi l ed r ul e w i th EBL O ne pl an to

• btai

n m oney i s to thr eaten to ki l l the chi l d

• f

a ri ch person.

• D etect

er r

• r
• f
• m i

ssi

• n

i n com pi l ed r ul e fr

• m

per for m ance exam pl e A ki dnapper

• btai

ns m oney f r

• m

gr andpar ent

• f

hostage.

• A ssi

gn Bl am e for error

• n

rul e i n dom ai n theory

• Revi

se r ul e i n dom ai n theor y M em ber s

• f

the sam e f am i l y have a goal

• f

pr eser vi ng each

• ther

’ s heal th

• Revi

se Rul e i n com pi l ed theor y

5 I JCA I

• 89

M onday, A ugust 14, 1989

SLIDE 6

W hy Errors

• f

O m i ssi

• n?

O nl y type

• f

error created by

• ne-

si ded l earni ng al gori thm s

• I

ncr em ental , hi l l

• cl

i m bi ng al gor i thm s (Langl ey et al . , 1987)

.

• A ccounts

f

• r

som e hum an l ear ni ng

• Gram m ar:

(Berw i ck, 1986)

• Concept

A cqui si ti

• n

(Br uner , et al , 1956)

• Subj

ect

• f

theor eti cal anal ysi s (V al i ant, 84; H aussl er , 87)

• H ypothesi

s never m ore general than “true” hypothesi s. I ncr em ental bl am e assi gnm ent and r evi si

• n

possi bl e: Error

• f

O m i ssi

• n:

O ne

• f

the dom ai n theor y r ul es used to cr eate a com pi l ed r ul e needs to be gener al i zed by dr

• ppi

ng a condi ti

• n

that i s not pr esent i n a per f

• r

m ance exam pl e Error

• f

com m i ssi

• n:

O ne

• f

the dom ai n theor y r ul es used to cr eate a com pi l ed r ul e needs to be speci al i zed by addi ng a condi ti

• n

that i s pr esent i n a new per f

• r

m ance exam pl e.

6 I JCA I

• 89

M onday, A ugust 14, 1989

SLIDE 7

Unsuper vi sed detecti

• n
• f

er r

• r
• f
• m i

ssi

• n

Probl em : Fi nd a schem a i n m em or y that w oul d pr edi ct the

• utcom e
• f

a new per f

• r

m ance exam pl e, i f the schem a w er e m or e gener al . A ppr

• ach:

Di sti ngui sh betw een tw o uses

• f

schem ata Predi cti ve: What would happen if the United States refused to sell computers to South Korea unless South Korea stopped exporting automobiles to Canada? Expl anatory: What could cause the price of oil to rise?

7 I JCA I

• 89

M onday, A ugust 14, 1989

coerce s1 s2 s3 s5 s6 s7 s8

SLIDE 8

D er i vati

• n

and use

• f

I ndi ces D eri ved anal yti cal l y: expl anator y: Featur es f r

• m

consequent

• f

dom ai n r ul es. pr edi cti ve: Featur es f r

• m

antecedent

• f

dom ai n r ul es. Use duri ng retri eval : expl anator y: Fi ndi ng a schem a to expl ai n the cause

• f

an

• utcom e

predi cti ve: Fi ndi ng a schem a to pr edi ct the

• utcom e
• f

an event. D eter m i ni ng i ndi ces: A n exam pl e A ustr al i a and Fr ance, 1983 I n 1983, A ustr al i a r efused to sel l ur ani um to Fr ance, unl ess Fr ance ceased nucl ear testi ng i n the South Paci f i c. Fr ance pai d a hi gher pr i ce to buy ur ani um f r

• m

South A f r i ca and conti nued nucl ear testi ng. Pr

• cessi

ng goal : Expl ai n r esul t (Fr ance buys ur ani um el sew her e)

8 I JCA I

• 89

M onday, A ugust 14, 1989

SLIDE 9

D eter m i ni ng i ndi ces: A n exam pl e 1. Thr eat

• >

I ncr eased dem and 2. I ncr eased dem and

• >

W i l l i ngness to pay hi gher pr i ce 3. Purchase

• >

Possess

9 I JCA I

• 89

M onday, A ugust 14, 1989

(ACT TYPE (SELL) ACTOR (POLITY EXPORTS ?Y ECONOMY (FREE)) TO ?X:(POLITY IMPORTS ?Y ECONOMY (FREE)) OBJECT ?Y:(COMMODITY) MODE (NEG)) (STATE TYPE (DEMAND-INCREASE) ACTOR ?X OBJECT ?Y) result

SLIDE 10

I ndexi ng by pr edi cti ve and expl anator y f eatur es

10 I JCA I

• 89

M onday, A ugust 14, 1989

coerce (coerce actor (polity exports =OBJECT economy (free)) target (polity economic-health (strong) economy (free) imports =OBJECT) ... response (act type (sell) actor (polity bus-rel =TARGET exports =OBJECT)

• bject =OBJECT

price (money value (>market)) to =TARGET)

• utcome (goal-outcome type (failure))

actor response target

• utcome

...

A B C D

SLIDE 11

D etecti ng an er r

• r
• f
• m i

ssi

• n

i nput: N ew per f

• r

m ance exam pl e Hi er ar chy

• f

schem a (dom ai n & com pi l ed theor y) Retr i eve schem a by fol l

• w i

ng pr edi cti ve i ndi ces I f schem a has

• utcom e ,

Then I f

• utcom e
• f

exam pl e and schem a agr ee Then EXI T El se “Error

• f

com m i ssi

• n”

El se I f exam pl e i s expl ai nabl e Then EBL(exam pl e) El se I f r etr i eve schem a by expl anator y i ndi ces Then “attem pt bl am e assi gnm ent” El se TDL(exam pl e)

• r

SBL(exam pl e) Fi nd a schem a suffi ci entl y cl

• se

to the per f

• r

m ance exam pl e that w oul d expl ai n the exam pl e i f gener al i zed.

11 I JCA I

• 89

M onday, A ugust 14, 1989

SLIDE 12

Bl am e assi gnm ent

(defi ni ng “suffi ci entl y cl

• se”)

1. Reexpl ai n the schem a w i th EBL that m ai ntai ns dependenci es betw een constr ai nts i n the com pi l ed r ul e and r ul es i n the dom ai n theor y. (Tr ade-

• f

f betw een stor i ng & r ecom puti ng): (coerce target (polity economic-health (strong) <- Rule.013 economy (free) <- Rule.012 imports =OBJECT) <- {Rule.01 Rule.13} 2. Fi nd di f f er ences betw een the schem a and the new event. 3. Col l ect i nference rul es responsi bl e for di fferences. 4. I f

• ne

i nference rul e i s responsi bl e for al l the di fferences, then assi gn bl am e to thi s i nf er ence r ul e.

12 I JCA I

• 89

M onday, A ugust 14, 1989

SLIDE 13

Bl am e assi gnm ent: A n exam pl e US and USSR, 1980 I n 1980, the US r ef used to sel l gr ai n to the Sovi et Uni

• n

i f the Sovi et Uni

• n

di d not w i thdr aw i ts tr

• ops

f r

• m

A f ghani stan. The Sovi et Uni

• n

pai d a hi gher pr i ce to buy gr ai n f r

• m

A r genti na and di d not w i thdr aw fr

• m

A fghani stan. 2. Fi nd di ffer ences: (coerce target (polity economy (free))) 3. Col l ect rul es: {rul e. 12}

13 I JCA I

• 89

M onday, A ugust 14, 1989

SLIDE 14

Cor r ecti ng the dom ai n theor y Conver t per f

• r

m ance exam pl e to f

• undati
• nal

exam pl e: (COERCE ACTOR (POLITY NAME (US) ECONOMY (FREE) ...) TARGET (POLITY NAME (USSR) ECONOMY (CONTROLLED)...) THREAT (ACT TYPE (SELL) ACTOR =ACTOR TO =TARGET MODE (NEG)) (ACT TYPE (SELL) ACTOR (POLITY TYPE (COUNTRY) NAME (US) ECONOMY (FREE) ...) TO (POLITY TYPE (COUNTRY) NAME (USSR) ECONOMY (CONTROLLED)...) OBJECT (COMMODITY AVAILABILITY (COMMON) TYPE (GRAIN)) MODE (NEG)) RESULT (STATE TYPE (DEMAND-INCREASE) ACTOR (POLITY NAME (USSR) ...) OBJECT (COMMODITY TYPE (GRAIN) ...)

14 I JCA I

• 89

M onday, A ugust 14, 1989

SLIDE 15

Cor r ecti ng the dom ai n theor y (cont’ d) Gener al i ze antecedent

• f

r ul e to accom m odate new exam pl e: Old: (ACT TYPE (SELL) ACTOR (POLITY EXPORTS ?Y ECONOMY (FREE)) TO ?X:(POLITY IMPORTS ?Y ECONOMY (FREE)) OBJECT ?Y:(COMMODITY) MODE (NEG)) New: (ACT TYPE (SELL) ACTOR (POLITY EXPORTS ?Y ECONOMY (FREE)) TO ?X:(POLITY IMPORTS ?Y) OBJECT ?Y:(COMMODITY) MODE (NEG)) M axi m al l y speci fi c conj uncti ve gener al i zati

• n

assum es no noi se.

15 I JCA I

• 89

M onday, A ugust 14, 1989

SLIDE 16

Exper i m ental D esi gn

• A l

l r ul es that i ndi cate effect

• f

thr eat w er e m odi fi ed by conj

• i

ni ng al l pr econdi ti

• ns.

(cf . Kol

• dner

, 1984)

• Run
• n

econom i c sancti

• n

database (15, actual , 5 hypo)

• A ccuracy

m easured after 10 and 15 exam pl es for 10 tr i al s. Resul ts

O CCA M

w i th error correcti

• n

m ore accurate than

O CCA M

p<.005, t(18)=3.16

O CCA M

w i th cor r ect know l edge base m or e accur ate (f ew exam pl es)

16 I JCA I

• 89

M onday, A ugust 14, 1989 10 15 0. 0. 2 0. 4 0. 6 0. 8 1.

• ccam
• ccam +errors
• ccam +errors+

Number of examples Accuracy

correcti

• n

SLIDE 17

Rel ated W or k

• Recover

i ng f r

• m

i ncor r ect know l edge i n SO A R (Lai r d, 1988)

• “patches”

com pi l ed theor y to avoi d i ncor r ect know l edge

• doesn’

t update dom ai n theor y

• m ay

r equi r e patchi ng f

• r

each use

• M L-

SM A RT (Ber gadano et al . , 1988)

• Batch

system that w or ks

• n

com pl ete set

• f

exam pl es

• can

handl e er r

• r

s

• f

com m i ssi

• n
• super

vi sed

• Theor

y Revi si

• n

(Gi nsber g, 1988)

• Batch

system

• Can

handl e cl assi f i cati

• n

noi se

• super

vi sed

• can

handl e er r

• r

s

• f

com m i ssi

• n

17 I JCA I

• 89

M onday, A ugust 14, 1989

SLIDE 18

Concl usi

• n

I . Unsuper vi sed detecti

• n
• f

er r

• r

s

• f
• m i

ssi

• n

r equi r es di sti ngui shi ng betw een expl anator y and pr edi cti ve uses

• f

schem ata. I I . A ssi gni ng bl am e

• n

r ul es i n the dom ai n theor y f

• r

er r

• r

s i n com pi l ed theor y can be accom pl i shed by m ai ntai ni ng dependenci es betw een condi ti

• ns
• f

com pi l ed r ul e and a r ul e i n the dom ai n theor y

18 I JCA I

• 89

M onday, A ugust 14, 1989