Fuzzy Self-Organizing Map based on Regularized Fuzzy c-means - - PowerPoint PPT Presentation

Fuzzy Self-Organizing Map based on Regularized Fuzzy c-means Clustering

Sándor Migály, János Abonyi and Ferenc Szeifert University of Veszprém, Department of Process Engineering

www.fmt.vein.hu/ softcomp

2/10

Overview

Steps and tasks of data mining Concept of Self-Organizing Maps Smoothed fuzzy c-means clustering Illustrative examples Summary

3/10

Steps of Data Mining

How soft-computing can help ???

www.fmt.vein.hu/softcomp

Database Data warehouse Data Mining Model

4/10

Tasks of Data Mining

Classification Change and Deviation Detection Dependency Modelling

Clustering

(prototypes, codebook, signatures,

• prob. density estimation )

Summation

(inc. Visualisation, Feature extraction)

Regression and time-series analysis

5/10

Clustering

Detect groups of data

Hierarchical (dendograms) or not

Prototypes (signatures)

are based on a

similarity measure

(distance) (semi)-supervised or

unsupervised Can be fuzzy !!!

∑ ∑

= ∈

− =

C i Q x i

i

E

1 2

v x

x1 x2

6/10

Feature Extraction

(Nonlinear) mapping of the input space into a lower dimensional one Reduction of the number of inputs Useful for visualisation

Non-parametric

(Sammon projection)

• r Model-based

(principal curves, NN, Gaussian mixtures, SOM)

7/10

Concept of the SOM I.

[ ]

im i i

v v ,...,

1

= v

Input space Input layer Reduced feature space Map layer

[ ]

in i i

r r r ,...,

1

= m n <<

s1 s2 x1 x2 x3

Clustering and ordering of the cluster centers

in a two dimensional grid

Cluster centers (code vectors) Place of these code vectors in the reduced space

8/10

Concept of the SOM II.

{ }

i i c

' min ' v x v x − ′ = − ′

x1 x2 x3 x4 x5

Known inputs Unknown inputs

u = [u1, u2, u3] y = [y1, y2]

y = f ( u )

mc

mc1 mc2 mc3 mc4 mc5

We can use it for regression

Best Matching Unit:

We can use it for visualization We can use it for clustering

mc :

9/10

Smoothed Fuzzy c-means

x1

x2

v9 v8 v7 v6 v5 v4 v3 v2 v1

( )

∑∑ ∑

= = =

        ∂ ∂ + =

c i N k c i i k i m ik

D J

1 1 1 2 2 2 ,

) ( x v V U, Z, ϑ µ

∫ ∂

∂ = x x v d S

2 2

The smoothness can be measured as The new cost-function:

10/10

Fuzzy line-trace application

• 1
• 0.5

0.5 1 1.5

• 1
• 0.5

0.5 1 1.5 0.5 1 1.5

• 1
• 0.5

0.5 1 1.5

• 1
• 0.5

0.5 1 1.5 0.5 1 1.5

Detected clusters and the

• btained ordering

when standard FCM algorithm is used Detected clusters and the

• btained ordering when the

proposed method is used

ϑ =2,

• 1
• 0.5

0.5 1 1.5

• 1
• 0.5

0.5 1 1.5 0.5 1 1.5

trace a part of a spiral in 3D. For this purpose 300 points are available with noise with 0 mean and variance 0.2. The aim of the clustering is to detect seven ordered clusters that can be lined up to detect the 3D curvature.

11/10

Fuzzy surface-trace application

• 1
• 0.5

0.5 1

• 1
• 0.5

0.5 1 0.2 0.4 0.6 0.8 1

• 1
• 0.5

0.5 1

• 1
• 0.5

0.5 1 0.2 0.4 0.6 0.8 1

Detected clusters and the

• btained ordering

when proposed regularized FCM algorithm is used Detected clusters and the

• btained ordering

when standard FCM algorithm is used

We folded a 6x6 grid on a half sphere. 900 points were taken and noise with zero mean and 0.1 variance was added

12/10

Conclusions

New regularized fuzzy c-means clustering algorithm for the visualization of high- dimensional data. The cluster centers are arranged on a grid defined on a small dimensional space that can be easily visualized. Comparison to the existing modifications of the fuzzy c-means algorithm was given and the application examples showed good performance in two geometrical case studies.