Improvements to the COR Methodology by means of Weighted Fuzzy Rules - - PowerPoint PPT Presentation

SLIDE 1

7th Online World Conference on Soft Computing In Industrial Applications

Improvements to the COR Methodology by means of Weighted Fuzzy Rules

• R. Alcalá, J. Casillas, O. Cordón, F. Herrera

{ alcala,casillas,ocordon,herrera} @decsai.ugr.es

Department of Computer Science and Artificial Intelligence University of Granada, Spain

WSC7, September 23th, 2002

SLIDE 2

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

1

Summary

• 1. Introduction: Fuzzy Modeling
• 2. COR (COoperative Rules)

2.1. Characteristics 2.2. Algorithm 2.3. A simple example

• 3. Weighted Linguistic Fuzzy Rules
• 4. WCOR (Weighted COR)

2.1. Description 2.2. A specific genetic algorithm (GA)

• 5. Experiments

5.1. Learning methods 5.2. Problem description 5.3. Results 5.4. Fuzzy models obtained

• 6. Concluding Remarks

SLIDE 3

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

2

• 1. Introduction: Fuzzy Modeling

! Fuzzy modeling (FM): system modeling with fuzzy rule-

based systems

! Two opposite requirements:

! Interpretability ! Accuracy

! Two approaches:

! Linguistic FM: interpretability as main objective ! Precise FM: accuracy as main objective

! Interpretable models have no sense if they are not

accurate enough

! A good trade-off between them is needed to perform a

useful fuzzy modeling

• 1. I ntroduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 4

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

3

• 1. Introduction: Fuzzy Modeling

! Two possibilities to find the desired balance: improve the

accuracy in linguistic FM or improve the interpretability in precise FM

! This paper is focused on the first approach proposing a

mechanism to improve the accuracy preserving good interpretability

1 2

Linguistic Fuzzy Modeling Precise Fuzzy Modeling

Accuracy improvement Interpretability improvement (interpretability as main objective) (accuracy as main objective)

Good trade-off

• 1. I ntroduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 5

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

4

• 2. COR: Characteristics

!

COR is a fuzzy system learning methodology that exclusively designs the linguistic fuzzy rule set

!

COR obtains the highest interpretability thanks to keeping the membership functions and the model structure unaltered, as well as making fuzzy rule set reduction

!

The accuracy is achieved by developing a smart search space reduction and by inducing cooperation among the fuzzy rules

!

COR consists of two stages:

1. Search space construction: A set of fuzzy input subspaces and a set of candidate rules for each are defined 2. Selection of the most cooperative fuzzy rule set: A combinatorial search is performed to select a fuzzy rule for each subspace (from the candidate rule sets)

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 6

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

5

• 2. COR: Algorithm

1. Search space construction:

a) Define the positive example set for each fuzzy input subspace b) Select only those subspaces containing positive examples c) Define the candidate consequent set for each subspace d) Define the candidate rule set for each subspace e) Add the “don’t care” symbol to each candidate rule set to allow fuzzy rule set reduction

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 7

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

6

• 2. COR: Algorithm

2. Selection of the most cooperative fuzzy rule set: A combinatorial search algorithm is used to look for the combination: with the best accuracy The mean square error (that measure the global cooperation of the rules) is considered to evaluate the quality of each solution:

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 8

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

7

(b)

P M G P M G X 2 X 1

e1 e5 e2 e3 e4 e6

B B Y B

1 2 3

2

e = (0,2 , 1,0 , 0,3)

1

e = (0,4 , 0,8 , 1,5)

2

e = (0,7 , 0,0 , 0,4)

3

e = (1,0 , 1,2 , 1,6)

4

e = (1,2 , 0,6 , 1,1)

5

e = (1,8 , 1,8 , 2,0)

6 l 1 2

e = ( ) , , x x y

l l l

Data Set

(-0,35 , 0 , 0,65)

1

P = B ( 0,35 , 1 , 1,65) M = B2 ( 1,35 , 2 , 2,65) G = B3

Data Base

P M G 2 P M G 2 B1 2 B3 B2 Y X1 X2

(a)

S 1 S 2 S 3 S 4

B

1 B 2

B

1 B 2 B 3

B

2 B 3

B

3

There are not examples There are not examples No hay not exampes There are Not exampesn There are not examples

P M G P M G

X

2

X 1

(c)

S1 S2 S3 S4

B1 B2 B2 B3 P M G P M G

X 2 X 1

(e)

X

1 is

THEN Y is IF R1 =

1

X

2 is

y

P M B

X1 is THEN Y is IF R2 =

2

X2 is y

M P B

X1 is THEN Y is IF R3 =

2

X2 is y

M M B

X1 is THEN Y is IF R4 =

3

X2 is y

G G B (f) Rule base

S 1 S 2 S 3 S 4 B 1 B 1 B 1 B 1 B 1 B 2 B 2 B 2 B 2 B 2 B 2 B 2 B 1 B 1 B 2 B 2 B 3 B 3 B 1 B 1 B 2 B 2 B 3 B 3 B 2 B 3 B 2 B 3 B 2 B 3 B 2 B 3 B 2 B 3 B 2 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3

Combinatorial Search

(d) e = (0,2 , 1,0 , 0,3)

1

e = (0,4 , 0,8 , 1,5)

2

e = (0,7 , 0,0 , 0,4)

3

e = (1,0 , 1,2 , 1,6)

4

e = (1,2 , 0,6 , 1,1)

5

e = (1,8 , 1,8 , 2,0)

6 l 1 2

e = ( ) , , x x y

l l l

Data Set

(-0,35 , 0 , 0,65)

1

P = B ( 0,35 , 1 , 1,65) M = B2 ( 1,35 , 2 , 2,65) G = B3

Data Base

P M G 2 P M G 2 B1 2 B3 B2 Y X1 X2

(a) (b)

P M G P M G X 2 X 1 e1 e5 e2 e3 e4 e6 B B Y B

1 2 3

2

Step 1: Candidate consequents generation Step 2: Combinatorial search inducing cooperation Inputs

(b) The examples are located in four different subspaces (a) Data set and DB previously defined (d) Combinatorial search in the solution space (c) Candidate consequent sets for the four rules (e) Decision table of the four linguistc rules obtained (f) RB generated from the third combination

• 2. COR: A simple example

(b)

P M G P M G X 2 X 1

e1 e5 e2 e3 e4 e6

B B Y B

1 2 3

2

S1 S2 S3 S4

B

1 B 2

B

1 B 2 B 3

B

2 B 3

B

3

There are not examples There are not examples There are not examples There are not examples There are not examples

P M G P M G

X 2 X 1

(c)

S1 S2 S3 S4

B

1 B 2

B

1 B 2 B 3

B

2 B 3

B

3

There are not examples There are not examples There are not examples There are not examples There are not examples

P M G P M G

X 2 X 1

(c)

S 1 S 2 S 3 S 4 B 1 B 1 B 1 B 1 B 1 B 2 B 2 B 2 B 2 B 2 B 2 B 2 B 1 B 1 B 2 B 2 B 3 B 3 B 1 B 1 B 2 B 2 B 3 B 3 B 2 B 3 B 2 B 3 B 2 B 3 B 2 B 3 B 2 B 3 B 2 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3

Combinatorial Search

(d)

S 1 S 2 S 3 S 4 B 1 B 1 B 1 B 1 B 1 B 2 B 2 B 2 B 2 B 2 B 2 B 2 B 1 B 1 B 2 B 2 B 3 B 3 B 1 B 1 B 2 B 2 B 3 B 3 B 2 B 3 B 2 B 3 B 2 B 3 B 2 B 3 B 2 B 3 B 2 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3 B 3

Combinatorial Search

(d)

S 1 S 2 S 3 S 4

B 1 B 2 B 2 B 3 P M G P M G

X 2 X 1

(e)

S 1 S 2 S 3 S 4

B 1 B 2 B 2 B 3 P M G P M G

X 2 X 1

(e)

X1 is THEN Y is IF R1 =

1

X2 is and

P M B

X1 is THEN Y is IF R2 =

2

X2 is and

M P B

X1 is THEN Y is IF R3 =

2

X2 is and

M M B

X1 is THEN Y is IF R4 =

3

X2 is and

G G B (f) Rule Base

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 9

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

8

• 3. Weighted Linguistic Fuzzy Rules

! A way to improve the fuzzy model accuracy involves

the use of a weight for each fuzzy rule

! A weight is a parameter that indicates the importance

degree of its associated rule in the inference process

! These weights modulate the firing strength of each

rule

! They can describe how a rule interacts with its

neighbor ones

! Therefore, weighted linguistic fuzzy rules represent a

good framework to improve the accuracy preserving interpretability

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 10

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

9

• 3. Weighted Linguistic Fuzzy Rules

! Weighted linguistic fuzzy rule structure: ! With this structure, the fuzzy reasoning must be extended:

Center of gravity weighted by the matching degree and the rule weight

] 1 , [ ], [

1 1

∈ w w with B is Y THEN A is X and and A is X IF

n n

K

∑ ∑

⋅ ⋅ ⋅ =

i i i i i i i

w m P w m y0

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 11

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

10

• 4. WCOR: Description

!

The COR methodology can be easily extended to consider the use of weighted linguistic fuzzy rules

!

To do so, a second vector that encodes the weight associated to each fuzzy input subspace is optimized during the second stage of the COR methodology:

2. Selection of the most cooperative weighted fuzzy rule set:

Combinatorial (rules) and real-coded (associated weights)

• ptimization are simultaneously performed

with being the number of input subspaces with positive examples

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 12

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

11

• 4. WCOR: a specific GA

! A GA is proposed in this paper to develop the WCOR

methodology

! Representation: two vectors with integer (rules) and real-

valued (weights) coding schemes are used

! Initial pool: for the rule part, values at random; for the

weight part, an individual with weights 1.0 and the remaining ones at random

! Genetic operators:

! Standard two-point crossover in the integer part and max-min-

arithmetical crossover in the real-valued part

! Mutation operator: once a gene (subspace) is randomly

selected, the current rule is replace by other candidate one at random and a new weight in [0,1] is generated

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 13

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

12

• 5. Experiments: Learning Methods

! Experimental comparison among different learning

methods

COR methodolgy developed with a GA COR A GA that add weights to a previously designed linguistic fuzzy rule set WRL The proposed process that learn rules and weights simultaneously WCOR Well-known Wang-Mendel method

Description

WM

Method

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 14

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

13

• 5. Experiments: Problem Description

Electrical Application Problem

! Estimation of the total length of low voltage line

installed in a town

! Two input variables: number of inhabitants and radius

• f the population

! Output variable: employed line length ! Sample data with 495 towns ! Random division into 396 and 99 data for training and

test, respectively (80-20%)

! Seven labels for each fuzzy partition

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 15

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

14

• 5. Experiments: Results

2nd stage: WRL

• 175,358

252,483

MSEtst

• 198,128

242,680

MSEtra

196,399 218,675 11 COR 161,511 161,414 12 WCOR 282,029

MSEtst

298,450 13 WM

MSEtra # R Method

A tuning of the weights in a second stage slightly improves the accuracy When the weight tuning is made

• ver a cooperative rule set, best

accuracy results are obtained The best accuracy results are

• btained with WCOR, where the

tight relation between rules and weights is properly addressed

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 16

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

15

• 5. Experiments: Fuzzy Models Obtained

Indirectly covered region Uncovered region

MSE = 218,675 MSE = 196,399

tra tst

L1 L2 L3 L4 L5 x2 L1 L4 L3 L5 L2 #R x1 11 L1 L2 L2 L3 L1 L2 L3 L3 L2 L5 L3

COR

L1 L2 L3 L4 L5 x2 L1 L4 L3 L5 L2 #R x1 12 L1 - 0.3 L1 - 0.1 L1 - 0.1 L2 - 0.0 L1 - 0.3 L2 - 0.7 L3 - 0.5 L3 - 0.0 L3 - 0.4 L2 - 0.6 L5 - 0.5 L3 - 0.5 MSE = 161,414 MSE = 161,511

tra tst

WCOR

Weights

L1-L1. w = 0.2465

1

L1-L2. w = 0.1132

2

L1-L3. w = 0.0676

3

L1-L4. w = 0.0195

4

L1-L5. w = 0.4440

5

L2-L1. w = 0.2762

6

L2-L2. w = 0.6664

7

L2-L3. w = 0.5357

8

L2-L4. w = 0.0001

9

L3-L2. w = 0.6429

10

L3-L3. w = 0.4662

11

L5-L3. w = 0.5215

12

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks

CONTENTS

SLIDE 17

Improvements to the COR Methodology by means of Weighted Fuzzy Rules WSC7, September 23th, 2002

16

• 6. Concluding Remarks

! The use of weights seems to be a good tool to improve the

fuzzy model accuracy preserving good interpretability:

! More flexibility to the fuzzy model is given, thus providing a

better approximation of the real system

! Significant rules can be identified by studying their weights,

thus helping us to interpret the model behavior

! The integration of weight learning within COR methodology

performs better than a sequential procedure, being an effective and simple fuzzy rule learning method

• 1. Introduction
• 2. COR
• 3. Weighted

linguistic fuzzy rules

• 4. WCOR
• 5. Experiments
• 6. Concluding

remarks CONTENTS