[PPT] - Capabilities and Prospects of I nductive Modeling Volodymyr PowerPoint Presentation

SLIDE 1

1

Capabilities and Prospects

f I nductive Modeling

Volodymyr STEPASHKO

Prof., Dr. Sci., Head of Department INFORMATION TECHNOLOGIES FOR INDUCTIVE MODELING International Research and Training Centre

f the Academy of Sciences of Ukraine

SLIDE 2

2

Layout

1. Historical aspects of IM 2. International events on IM 3. Attempt to define IM: what is it? 4. IM destination: what is this for? 5. IM explanation: basic algorithms and tools 6. Basic Theoretical Results 7. IM compared to ANN and CI 8. Real-world applications of IM 9. Main centers of IM research

10. IM development prospects

SLIDE 3

3

1. Historical aspects of I M

1968 First publication on GMDH:

Iвахненко O.Г. Метод групового урахування аргументів – конкурент методу стохастичної апроксимації // Автоматика. – 1968. – № 3. – С. 58-72.

Terminology evolution:

heuristic self-organization of models (1970s) inductive method of model building (1980s) inductive learning algorithms for modeling (1992) inductive modeling (1998)

GMDH: Group Method of Data Handling MGUA: Method of Group Using of Arguments

SLIDE 4

4

A.G.I vakhnenko: GMDH originator

SLIDE 5

5

Main scientific results in inductive modelling theory:

Foundations of cybernetic forecasting device construction Theory of models self-organization by experimental data Group method of data handling (GMDH) for automatic construction (self-organization) of model for complex systems Method of control with optimization of forecast Principles of noise-immunity modelling from noisy data Principles of polynomial networks construction Principle of neural networks construction with active neurons

SLIDE 6

6

Academician Ivakhnenko

Originator of the scientific school of inductive modelling
Author of 44 monographs and numerous articls
Prepared more than 200 Cand. Sci (Ph.D.) and 27 Doct. Sci

Academician Ivakhnenko

Originator of the scientific school of inductive modelling
Author of 44 monographs and numerous articls
Prepared more than 200 Cand. Sci (Ph.D.) and 27 Doct. Sci

SLIDE 7

7

2. I nternational events on I M

2002 Lviv, Ukraine 1st International Conference on Inductive Modelling ICIM’2002 2005 Kyiv, Ukraine 1st International Workshop on Inductive Modelling IWIM’2002 2007 Prague, Czech Republic 2nd International Workshop on Inductive Modelling IWIM’2007 2008 Kyiv, Ukraine 2nd International Conference on Inductive Modelling ICIM’2008 2009 Krynica, Poland 3rd International Workshop on Inductive Modelling IWIM’2009 2010 Yevpatoria, Crimea, Ukraine 3rd International Conference on Inductive Modelling ICIM’2009 Zhukyn (near Kyiv, Ukraine) Annual International Summer School on Inductive Modelling

SLIDE 8

8

3. Attempt to define I M: what is it?

IM is MGUA / GMDH IM is a technique for model self-organization IM is a technology for building models from noisy data IM is the technology of inductive transition from data to models under uncertainty conditions:

small volume of noisy data unknown character and level of noise inexact composition of relevant arguments (factors) unknown structure of relationships in an object

SLIDE 9

9

4. I M destination: what is this for?

IM is used for solving the following problems:

Modelling from experimental data Forecasting of complex processes Structure and parametric identification Classification and pattern recognition Data clasterization Machine learning Data Mining Knowledge Discovery

SLIDE 10

10

Given: data sample of n observations after m input x1, x2,…, xm and output y variables Find: model y = f(x1, x2,…, xm ,θ) with minimum variance of prediction error GMDH Task:

models

f

set criterion quality model ) ( ), ( min arg

, −

ℑ − =

ℑ ∈ ∗

f C f C f

f

Basic principles of the GMDH as an inductive method:

1. generation of variants of the gradually complicated structures of models
2. successive selection of the best variants using the freedom of decisions choice
3. external addition (due to the sample division) as the selection criterion

Part А Generation of models

ℑ ∈ f

Calculation of criterion С( f ) C→ min Sample Part В f *

5. I M explanation: algorithms and tools

Basic Principles of GMDH as an Inductive Method

SLIDE 11

11

D

D A A T T A A ( (s s a a m m p p l le e , , a a p p r r i io

r

r y y i in n f fo

r

rm m a a t ti io

n

n ) ) C C h h o

i

ic c e e

f

f a a m m o

d

d e e l l c c l la a s s s s S S t tr ru u c c t tu u r re e g g e e n n e e r r a a t ti io

n

n P P a a r ra a m m e e t te e r r e e s s t ti im m a a t ti io

n

n

C

C r r i it te e r r i io

n

n m m i in n i im m i iz z a a t ti io

n

n

A

A d d e e q q u u a a c c y y a a n n a a l ly y s s i is s

F

F i in n i is s h h i in n g g t th h e e p p r r o

c

c e e s s s s

Main stages of the modeling process

SLIDE 12

12

GMDH features

Model Classes: linear, polynomial, autoregressive, difference (dynamic), nonlinear of network type etc. Parameter estimation: Least Squares Method (LSM) Model structure generators:

GMDH Generators Sorting-out Iterative

Exhaustive search Directed search Multilayered Relaxative

SLIDE 13

13

Main generators of models structures

1. Combinatorial:

1 1

, 1 , , , 1 , ) | (

− −

= = =

s s j s i s l s

F l i m s x X y θ ) )

2. Combinatorial-selective:
3. Selective (multilayered iterative):

2 2 5 2 4 3 2 1 1

, 1 ; , 1 , ,...; 1 , , ) ( ) (

F r j r i r j r i l r j l r i l r l

C l F j i r y y y y y y y = = = + + + + =

+

ϑ ϑ ϑ ϑ ϑ

) ,..., , ( ; 2 ..., , 1 ,

2 1 m m v v v

d d d d v X y = = = θ )

SLIDE 14

14

External Selection Criteria

Given sample: W = (X |y), X [nxm], y [nx1] Division into two subsamples:

n n n y y y X X X W W W

B A B A B A B A

= + ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = ; ; ; , , , , ) (

1

W B A G y X X X

G T G G T G G

= =

−

θ )

Parameter estimation for a model y=Xθ: Regularity criterion:

2 B W A W

X X CB θ θ ) ) − =

Unbiasedness criterion:

2 A B B B

X y AR θ ) − =

SLIDE 15

15

I M tools Information Technology ASTRID (Kyiv) KnowledgeMiner (Frank Lemke, Berlin) FAKE GAME (Pavel Kordik at al., Prague) GMDHshell (Oleksiy Koshulko, Kyiv)

SLIDE 16

16

6. Basic Theoretical Results

). ( min arg

*

f C f

F f∈

=

F – set of model structures С – criterion of a model quality Structure of a model:

)

Q – criterion of the quality of model parameters estimation

) , (

f f

X f y θ ) =

Estimation of parameters:

). ( min arg

f R f

Q

m f

θ θ

θ ∈

= )

SLIDE 17

17

Main concept: Self-organizing evolution of the model of

ptimal complexity under uncertainty

conditions Main result: Complexity of the optimum forecasting model depends on the level of uncertainty in the data: the higher it is, the simpler (more robust) there must be the optimum model Main conclusion: GMDH is the method for construction of models with minimum variance of forecasting error

SLIDE 18

18

1 2 3 4 5 6 1 2 3 4 s

J

b(s)=

=J(s|0) σ

2 = 1,0

σ

2 = 0

σ

2 = 0,5

σ

2 = 1,5

σ

2 = 2,0

J(s|σ

2)

1 2 3 4 5 6 0,5 1 1,5 2 2,5

s= 0 s= 1 σ

2

s= 2 s= 3 s= 4 J(σ

2|s)

σ

2 кр(2,3)

σ

2 кр(1,2)

σ

2 кр(0,1)

Illustration to the GMDH theory

SLIDE 19

19

1

x

2

x

3

x

4

x m

1

f

2

f

3

f

4

f F Cm ⇒

2 1

g

3

g

4

g F CF ⇒

2 2

g

∗

f

7. I M compared to ANN and CI

Selective (multilayered) GMDH algorithm:

SLIDE 20

20

1

x

2

x

3

x

4

x m

1

f

3

f

4

f F Cm ⇒

2 4

g F CF ⇒

2 2

g

∗

f

Optimal structure of the multilayered net

SLIDE 21

21

8. Real-world applications of I M

1. Prediction of tax revenues and inflation

2. Modelling of ecological processes

activity of microorganisms in soil under influence of heavy metals irrigation of trees by processed wastewaters water ecology

3. System prediction of power indicators
4. Integral evaluation of the state of the complex

multidimensional systems

economic safety investment activity ecological state of water reservoirs power safety

5. Technology of informative-analytical support of
perative management decisions

SLIDE 22

22

9. Main centers of I M research

IRTC ITS NANU, Kyiv, Ukraine NTUU “KPI”, Kyiv, Ukraine KnowledgeMiner, Berlin, Germany CTU in Prague, Czech Sichuan University, Chengdu, China

SLIDE 23

23

10. I M development prospects

The most promising directions:

1. Theoretical investigations
2. Integration of best developments of

IM, NN and CI

3. Paralleling
4. Preprocessing
5. Ensembling
6. Intellectual interface
7. Case studes

SLIDE 24

24

THANK YOU!

Volodymyr STEPASHKO

Address: Prof. Volodymyr Stepashko, International Centre of ITS, Akademik Glushkov Prospekt 40, Kyiv, MSP, 03680, Ukraine. Phone: +38 (044) 526-30-28 Fax: +38 (044) 526-15-70 E-mail: stepashko@irtc.org.ua Web: www.mgua.irtc.org.ua