Eli lici citin ing Fu Fuzz zzy Kn Knowl wledg dge e fr from - - PowerPoint PPT Presentation

Eli lici citin ing Fu Fuzz zzy Kn Knowl wledg dge e fr from th the PI PIMA MA Dataset

Antonio d’Acierno ISA A – CNR daci cier erno.a@ no.a@is isa.cnr a.cnr.it .it Giuseppe eppe De Pietro ro, , Massimo mo Esposit sito ICAR R – CNR giuseppe.depiet eppe.depietro@ ro@na. na.icar.c icar.cnr nr.it .it massimo imo.espo .esposito sito@na.i @na.icar.cnr.it car.cnr.it

2

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Our wor

• rk

 Rece

cent ntly, ly, we prop

• pose
• sed

d a six-steps eps a data driven en metho hodolo dology y to automa mati ticall cally y build fuzzy inferen ence ce systems ems [6].

– The methodo

• dology

logy produ duce ces s FIS with h an user define ned number ber of rules. s.

– Each

ch step can be approa roach ched ed using g several al strategi egies es

 In this paper

er, , we use an implement ementati ation

• n of our metho

hodo dolo logy gy to elici cit t knowledge ledge from m the PIMA dataset et

 We obtai

tain: n:

– an interesting

sting perfor

• rmance

mance in terms of correct ect class ssifi ificat cation ion rate

– linguis

guistic tic varia iables les are likely ly to be easily ily understoo tood from m huma man beings. ngs.

[6]

• A. d’Acierno, G. De Pietro, M. Esposito. Data Driven Generation of Fuzzy Systems: An Application to Breast Cancer

Detecti ction,

• n, 7th Internationa
• nal Meeting

g on Computa putationa

• nal Intelligen

ence ce Method

• ds for Bioinfor
• rma

matics cs and Biosta tatis tistics (CIBB BB 2010), ), Palermo mo (Ita (Italy), ), 16-18 8 Septe temb mber r 2010.

3

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Intr trod

• ductio

uction

 To increas

ase e the chan ange of su success ssful ul treat atments, ts, ear arly y de detectio ion n of al almost st an any di dise seas ase is a s a ke key fac acto tor.

 The de

detectio ion can an be be often formula late ted d as as a bi a binar ary de decisi sion mak aking probl blem: m:

– unce

cert rtaint nty y in form of informat rmation ion inco complet mpleten eness ess, , impre reci cisene seness ss, , fragmen gmentar tarines ness, s, not fully reliabil bilit ity, y, vagu guen eness ess and co contr tradi adictor ctorine iness ss often en affects cts these e probl

• blems.

ems.

 Compute

uterize rized d di diag agnost stic c tools s to su support physi sicia cians ns in interpretin eting g da data a hav ave be been thus s de developed

– Diagnos

gnosti tic c Deci cisio ion n suppor

• rt

t Systems ems (DDSS) S)

4

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

DD DDSS



A diagnos nosti tic c tool l should posses sess s [1] three ee (often en in co cont ntras rast) t) ch character acteristic stics: s:

–

it must attain ain the best t possib ible le perfor

• rmance

mance in terms of correct ct clas assi sification fication rate.

–

It would ld be desirable able the system em not only provide des s a diagnosis nosis but also so a numerica rical l value ue represe senting nting the degree to which ich the system em is is confid nfiden ent t in the solution. tion.

–

It would ld be also

• useful

ul if the phys ysici ician n is not face ced d with h a blac ack k box that at simply ly output puts s answe wers rs but the system em should uld provid ide some insig ight into how the solution ution has been derived (inter erpre preta tabil ility). ity).

[1]

• C. A. Pena-Reyes

Reyes and M.

• M. Sipper.

. A fuzzy-genet etic approach

• ach to breast

st cancer er diagnosis.

• sis.

Artificial cial Intelligenc ence e in Medicine, ne, 17(2):13 131–155, 55, 1999.

5

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

DD DDSS

 Di

Diag agnost stic ic tools, s, however, , typically ally hav ave unequal al clas assi sificatio fication error cost sts s so so th that at st strai aight ht CR can annot be be as assu sumed d as as a ca a careful ul meas asure of the goodn dness ss of the clas assi sifie fier. r.

 A Receiv

iver er Op Operat atin ing Char aracte acteris ristic tic (ROC OC) grap aph has as be been sh showed d to be be a m a more ac accurate ate techniqu ique e for se select cting ing clas assi sifie fiers s ba base sed d on th their performance ance.

 We guess

ss that at al also so the confide dence nce  can an be be use sed f d for se select cting ing clas assi sifie fier si since a g a good d clas assi sifie fier sh should d be be highly hly confide dent nt with correctly tly clas assi sifi fied d exa xamples es while it should be “doubtful” with misclassified data points. s.

6

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

FI FIS

 A Fuzzy Infere

renc nce e System em (FIS) ) is a system m that (tries es to) solve ve a (typical ically ly co compl plex ex and nonline inear) ar) problem blem by utilizing zing fuzzy logic ic metho hodo dologi logies es and d it is co composed posed of

1. 1.

a fuzzifi ifier r (transl nslate tes s real- value lued d inputs ts into fuzzy y value lues) s)

2. 2.

an inference ence engine e (applie plies s a fuzzy y reason soning ing mech chani anism sm to obtain tain a fuzzy zy output), put),

3. 3.

a defuzzif ifie ier r (transl nslates es this s latter r output ut into a crisp sp value), ue),

4. 4.

• f a knowl

wledge dge base e (containing

• ntaining both

th rules and memb mber ership ship funct ctio ions). s).

7

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

FI FIS

 The inference

nce process ess is performed rmed by the engine ne using g the rules s conta taine ined in the rule base se if antece cede dent nt then consequent equent

 The antece

cedent ent is a fuzzy-logic logic express ssion ion composed posed of one or more simple ple fuzzy y expressi sions s connect ected ed by fuzzy y operators ators (the fuzzy zy equivale ivalent nt of the class ssical ical and, or and not), ,

 In Mamdani

dani systems ems, the consequent equent is an express ssion ion that t assig igns ns fuzzy zy values ues to the output: put: if service ce is good then tip is average age

 In Takag

kagi-Sugeno(TS) Sugeno(TS) syste tems, ms, the conse sequ quent nt express sses s output ut variab iable les s as a function tion that t maps s the input space ce into the output ut space: ce: if service ce is good then tip = f(serv rvice) ice) wher ere e f is (typically pically) a first t order linear ear function ction that t becomes

• mes a

consta stant nt in zero-order rder TS systems. tems.

8

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Fu Fuzz zzy y mo modeling

 Fuzzy model

eling ing is the task of identify ifying ng the paramet meter ers of a FIS so that a desired ed behavior vior is attaine ined. .

 Knowl

• wledge

edge dr driven en appro roach ch:

– When

en the availab ilable le knowled ledge ge is complete lete and the problem lem space ce is not very large e the system tem can be constru tructe cted d directly ly using ing knowl

• wledge

dge elicited ted from m huma man experts. s.

 Alter

ernati natively vely, , data driven n fuzzy modeli ling ng ca can be used:

– Avail

ailab able le data a and AI technologies nologies are used to build ld the rules and/or

• r

memb mber ership ship funct ctio ions. s.

 A probl

blem: m: the knowledg dge e ba base se generat ated d au automati atically cally from da data a may ay not be be fully y interpre pretable. table.

9

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Inte terpr pretab etabili ility ty



Three e co condit itio ions ns ca can be defined ed to obtain ain an interpr rpretabl etable e fuzzy y model l [5]:

1. 1.

the fuzzy sets ca can be interpr rpreted eted as linguistic stic labels ls (low, , medium, um, high, , medium-lo low, w, etc) c);

2. 2.

the set of rules must be as small as possible ible; ;

3. 3.

the if-part rt of the rules should be derived ed from m a subset t of indepe ependen ndent t variables ables rather er than from m the full set.



Inter erpre retabi tabilit ity y is a key feature ure in a DDSS. S.

5. 5.

Serge ge Guillaume.

• e. Designi

ning ng fuzzy inferen ence ce systems from data: : An interpre retabi ability-

• rient

nted ed review. . IEEE Transac sactions

• ns on Fuzzy Systems,

s, 9(3):426 26–44 443, , 2001.

10

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Th The 6-ste teps ps me meth thod

• dol
• log
• gy

 We ext

xtrac act crisp sp rules. s.

 If the cas

ase, we se select R use seful rules. s.

 If the cas

ase, we redu duce the rules. s.

 Usi

sing the fuzzyfie yfier, we bu build d fuzzy y rules. s.

 We generat

ate the FIS (TS sy syst stems s ar are use sed) d).

 We ad

adap apt membe bership ship functions ions.

FIS Generator Adapter Extractor Selector Fuzzyfier Reducer Data Knowledge Base

R

11

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Th The imp mpleme mente nted d me meth thod

• dol
• log
• gy



To extract ct crisp p rules s we use a full decisi sion

• n tree (without

thout pruning) ing) with h a Gini’s diversity index as split criterion and a split minimum factor equal ual to 1.



Given en a user defined ed numbe ber r of rules R (assu sume med d to be even), ), we select, ct, for each ch class ss, , the R/2 most coverin ring g leaf nodes. .



Each ch rule is simplif lified ied so that t its anteced ceden ent t contains tains each h feature ure at most st one time; three operator ator (atmosT mosT9 are conside idere red: d:

–

greate ater r than n a threshold shold ( high) h)

–

less s than n a thresh shold

• ld

( low)

–

betwe tween en two thresh sholds

• lds

( medium um)



ANFIS FIS [7] ] is used to adapt pt membe bersh rship ip functions tions

[7]

• J. S. R. Jang. Anfis: Adaptive

ve network based d fuzzy inferenc nce e system. Systems, s, Man and Cyberneti etics cs, , IEEE Transac sactions

• ns on, 23(3):665

65–68 685, 5, May/June une 1993.

12

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

The PIMA Data tase set



A (comple mplex) x) collection ection of medica cal l diagnosi gnosis s reports ts of 768 example mples s from m a populat lation ion living ng near Phoen enix, ix, Arizona na, , USA



Patie tients nts here are femal ales s at least st 21 years s old of Pima ma Indian an heritag tage. e.



Binary nary-va value lued d variable iable invest stigat igated d is whether ther the patient ent shows ws signs s

• f diabe

betes tes accor cording ing to World d Health th Organization ization criteria; ria;



There re are 500 negative tive examples ples and 268 posit itive ive ones and for each h patient ient there are 8 independ ndent ent variables ables reporte ted: d:

1. 1.

number er of times pregnant; ant;

2. 2.

plasma ma glucos cose e concen entrat tration ion a 2 h hours in an oral gluco cose se toleran ance ce test;

3. 3.

diastolic tolic blood pressure;

4. 4.

triceps ceps skin fold thick ckness ss;

5. 5.

2-Hour r serum insulin in;

6. 6.

body y mass index;

7. 7.

diabetes betes pedigree ree function ion;

8. 8.

age.

13

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Exp xperimen ments ts

 The prop

• pose
• sed

d appro roach ach has been n implement emented ed using funct ctio ions ns availab able e in the standard ard versio ion n of the R2009a a 64 64-bit t version ion of MATLAB. AB.

 We use a ten-fol

fold d cr cross validati ation

• n that is repeat

eated ed 10 times. s.

 We use TS systems.

ms.

 We use a thres

eshol hold d to cl classify y the sampl ple: e:

– We choose

• se the thresh

shold

• ld that maximi

imizes es the class ssif ificat icatio ion rate on the learning rning set.

 We measure

ure: :

– the average

age CR on the learning ning set (LS LS) ) and on the test set (TS) ) for both th untrai ained ed FISs s and adapt pted ed ones.

14

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Prelimina minary y results ts

100% on the learning set, 70% on the test set

15

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Prelimina minary y results ts

 An intere

restin sting g perfo forman rmance ce is obtaine ained d using just two rules.

 The perfo

forman rmance ce incr creas eases es using more e rules.

 In

In [6] 6] it is obtai tained ned a 77.65 65% % CR R using 125 rules in a single le run of a five fold cr cross s validati ation

• n.

[8]S ]S. . N. Ghazav avi and T. W. Liao.

• . Medi

dical cal data mini ning ng by fuzzy model deling ng with h selecte ected d featu tures.

• es. Artifi

ficial cial Inte telligence ence in Medici cine, ne, 43(3):1 3):195–206, 6, 2008.

Rules LS TS LS TS 2 63,70% 63,50% 75,30% 73,80% 4 63,90% 63,90% 76,10% 74,20% 6 64,30% 64,30% 76,60% 74,10% 8 64,80% 64,80% 76,90% 74,40% 10 64,80% 64,70% 77,20% 74,50% 12 64,80% 64,60% 77,60% 74,90% 14 65,10% 65,00% 77,60% 75,00% 16 65,10% 65,10% 77,80% 75,00% 18 65,10% 65,00% 77,90% 74,90% 20 65,30% 65,10% 77,90% 74,90% AFIS UFIS

16

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

2-Rules ules Sys yste tem



Using ng the whole e data set:

1. 1.

If (Pregn egnant ant is low) and (Glucose cose co conce cent ntra ratio tion n is low) and d (Body dy mass index is low) and (Diabete abetes s PF is low) and (Age ge is low) then n (Ou Output put is 2) (weigh ght t 0.7932) 2)

2. 2.

If If (Glucose cose co conce cent ntra ratio tion n is high) and d (2 (2-Ho Hour ur serum insulin is low) and (Body dy mass index is high) ) and (Diabete abetes s PF is high) and (Age ge is low) then n (Ou Output ut is 4) (weight ht 1.5376) 6)

17

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

2-Rules ules Sys yste tem

18

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

2-Rules ules Sys yste tem

 THR=2.

2.97, 97, AUC=0. 0.819 19, , CR=77.08 .08

P N P 454 130 N 46 138 Actual Predicted

19

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

14 14-Rul ules S es Syste ystem



Using ng the whole le data set, is class ss1 1 if:

1. 1.

If (Pregnant ant is low) and (Gluco cose se concen entr trat ation is low) and (Body mass index x is low) and (Diabetes betes PF i is low) and (Age is low)

2. 2.

If (Gluc ucose

• se conc

ncen entra tration tion is low) and nd (Diastolic astolic is high) ) and nd (2-Hour ur serum insulin in is high) h) and (Body mass index is medium) ) and (Diabetes betes PF is low) and (Age e is low)

3. 3.

If (Glucos cose e concen entrat tration ion is low) and (Body y mass index is medium) m) and (Age ge is high) h)

4. 4.

If (Pregnant ant is low) and (Gluco cose se concen entr trat ation is medium) m) and (Body mass index is high) ) and (Diabetes betes PF is low) and (Age is high)

5. 5.

If (Pregnant ant is low) and (Gluco cose se concen entr trat ation is low) and (Body mass index x is low) and (Diabetes betes PF i is high) h) and (Age e is low)

6. 6.

If (Glucos cose e concen entrat tration ion is low) and (Dias astolic tolic is medium) m) and (2-Hour r serum insulin in is low) and (Body y mass index is medium) m) and (Diabetes betes PF is low) and nd (Age e is low)

7. 7.

If (Pregnant ant is low) and (Gluco cose se concen entr trat ation is medium) m) and (Diast stoli lic is high) h) and (Body mass index is medium) m) and (Diabetes betes PF is low) and (Age ge is low)

20

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

14 14-Rul ules S es Syste ystem



Using ng the whole le data set, is class ss2 2 if:

8. 8.

If (Glucos cose e concen entrat tration ion is high) h) and (2-Hour r serum insulin in is low) and (Body y mass index is high) h) and (Diabe abetes tes PF is high) ) and (Age e is low)

9. 9.

If (Gluc ucose

• se conc

ncen entra tration tion is medium) um) and nd (2-Hour ur serum um ins nsul ulin is low) and (Body y mass index is medium) m) and (Diabetes betes PF is high) ) and (Age is high) gh)

10.

• 0. If (Pregnant

ant is high) ) and (Glucos cose e concentration ration is medium) m) and (Body mass index is high) ) and (Diabetes betes PF is high) h) and (Age e is high) )

• 11. If (Glucos

cose e concen entrat tration ion is high) h) and (2-Hour r serum insulin in is low) and (Body y mass index is high) h) and (Diabe abetes tes PF is low) and (Age e is medium) m)

12.

• 2. If (Glucos

cose e concen entrat tration ion is high) h) and (Body mass index is low) and (Age e is medium) m)

13.

• 3. If (Glucos

cose e concen entrat tration ion is medium) m) and (Diastolic tolic is low) and (Body mass index is high) ) and (Age e is low)

• 14. If (Pr

Pregnan nant t is high) ) and nd (Gluc ucose

• se conc

ncentrat ration ion is high) ) and nd (2-Hour ur serum insulin in is low) and (Body y mass index is high) ) and (Diabetes betes PF is high) gh) and (Age e is high) h)

21

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

14 14-Rul ules S es Syste ystem

22

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

14 14-Rul ules S es Syste ystem

 THR=3.033

.033, , AUC=0. =0.836 836, , CR=77.99 .99

P N P 439 108 N 61 160 Actual Predicted

23

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Prelimina minary y results ts

 We de

define the confide dence nce as as =ab abs( s(O-)/ )/

– O is

O is the output ut of the FIS IS

–  is a normal

malizi izing ng fact ctor

• r so that

t [0,1] 0,1]

 A good c

d clas assi sifie fier r sh should d be be highly ly confide dent nt with correctly ctly clas assi sifie fied d exa xample les s while e it sh should d be be do doubt btful ul with misc sclas assi sifie ied d da data a points. s.

 Star

artin ting g from the confusi sion mat atrix x on th the test st se sets, s, we meas asure the cas ases s correctly tly clas assi sifie fied d (NT NTP an and d NT NTN) N) with  > 0. 0.7 an and t d the numbe ber of inst stan ances s incorre rectly ctly clas assi sifie fied d (NF NFP an and N d NFN) N) with  < 0. 0.3.

24

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Prelimina minary y results ts

23,13% 54,62% 46,24% 50,00% 58,70% 21,01% 72,13% 1,25% 2 RULES 14 Rules

25

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Fu Futur ture e directi tions

• ns

 Ou

Our metho hodology dology needs to be teste ted using: :

– othe

her r data sets;

– differ

ferent ent strategi egies s for extrac acting ing rules.

 Differ

eren ent t tech chni niqu ques es to determ rmine ne the co corre rect ct thresholds holds for the FISs co could impro rove ve the whole e perfo forman rmance: ce:

– the one that

t minimi imizes es the mean squar are e error;

– the one that

t minimize mizes s a cost function ction that t takes s into account count the unequa qual l class ssifi ification ation error costs. s.

 Weight

hted rules co could be also used.

 It is in worth

h testing ing the possibil ibilit ity y of using in parall llel el FISs with differe rent nt number ers of rules:

– their

ir predictions tions could ld be combined mbined using g several al strategi tegies es based sed on the confid fidence ence showed wed by each ch system. em.

26

Joint NETTAB 2010 and BBCC 2010, Naples (Italy), November 29 – December 1, 2010, Antonio d’Acierno, dacierno.a@isa.cnr.it

Qu Questio stions? ns?