Calibration: an essential Calibration: an essential inverse problem - - PowerPoint PPT Presentation

Calibration: an essential Calibration: an essential inverse problem inverse problem

3º Colóquio de Matemática da Região Sul – Florianópolis (SC), Brasil

Haroldo Fraga de Campos Velho

• Lab. Associado de Computação e Matemática Aplicada

E-mail: haroldo@lac.inpe.br http://www.lac.inpe.br/~haroldo

Presentation outline

Inverse problem classification Model calibration (or: parameter identification) Model calibration: an optimization problem Model calibration: non-linear mapping Applications

Hydrological model Fault diagnosis Ocean colour (two approaches) Data assimilaton

Final remarks

Classification of Inverse Problems

• 1. Regarding to mathematical nature of the method: Explicit

Implicit

• 2. Regarding to statistical nature of the method: Deterministic

Stochastic

• 3. Regarding to nature of the estimated property: Initial condition

Boundary condition Source/sink term Properties of the system

• 4. Regarding to nature of the solution (Beck): Parameter estimation

Function estimation

• 5. Silva Neto / Moura Neto: Type-1 (DP-f e IP-f)

Type-2 (DP-∞ e IP-f) Type-3 (DP-∞ e IP-∞) Type-4 (DP-f and IP-∞) – does not apply

“Solve an Inverse Problem is to determine unknown CAUSES from the observed or desired EFFECTS” - H.W. Engl (1996).

M D causes effects

ℑ

1 −

ℑ

M space of parameters or models D space of data or observations forward model inverse model

ℑ 1

−

ℑ

Model calibration (or: parameter identification)

Model calibration: an optimization problem

Inverse problem formulated as na optimization

problem

) ( ) ( ) (

2 2 Mod Exp

f f T T f J Ω + − = α

α

• perator

tion regulariza : ) ( parameter tion regulariza : f Ω α

Model calibration: non-linear mapping

Non-linear mapping by artificial neural network

x Parametrer y Data FORWARD MODEL K INVERSE MODEL K-1

Model calibration: non-linear mapping

Non-linear mapping by artificial neural network

( )

mapping) linear

• (non

, f

f n

• n

NN a n

x x x =

Training phase: determination

• f the connection weights, bias

Activation phase: generating analized data.

Applications: Hydrological model

Inverse problem classification

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum A, B, C, Ef, Ff = known matrices Ѳf = [fa fp fs] = faults

fa = parameter vector from actuator(s) fp = parameter vector from the process fs = parameter vector from sensor(s)

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum Ns = number of samples (in time)

• = output vector measured by sensors
• = output vector computed by the model

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

One more equation introduced:

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

Problem structure (biadjacentcy matrix)

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

Problem structure (biadjacentcy matrix) Dulmage-Meldelsohn decomposition

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

Dulmage-Meldelsohn decomposition:

Allow partioning the model M on three parts (graph): Under determined (M0) Just determined (UjMj) Over determined (M+)

The D-M decomposition gives the order between

connected components of the graph.

Measures on any variable x1, x2, ..., x5 make the fault

structurally detectable.

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

ACO: Ant Colony Optimization

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

ACO: Ant Colony Optimization

Best ant = min J(x) Pheromone deposit (bolt dotted curve)

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

ACO: Ant Colony Optimization

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

Fuzzy-ACO

Applications: Fault diagnosis

Fault diagnosis: inverted-pendulum

ACO vs Fuzzy-ACO: results

Application: Ocean colour (function estimation)

Hydrologic optics Hydrologic optics Radiative Radiative transfer process transfer process Inverse problem Inverse problem Ant colony optimization Ant colony optimization

Applications: Ocean colour

Internal sources Absorption coefficient

Scattering coefficient

Scattering angle Scattering phase function

medium Light beam Polar angle azimuthal angle Boundary Conditions Optical depth Boundary Conditions

Applications: Ocean colour

where Internal source Radiance :a measure of amount of light beam energy

Applications: Ocean colour

Multi-spectral estimation

[ ]

[ ]

∑∑

= =

Ω + − =

λ ξ

α

N k N l l k l k

L L J

1 1 2 Mod , Exp ,

) ( ) ( p p p

Applications: Ocean colour

Multi-spectral estimation

Applications: Ocean colour

Absortion and scattering coefficients:

[ ][

]

[ ]

) ( 30 . 550 2 . 1 ) ( 06 .

65 . , ) 440 ( 014 . 65 . ,

z C b e z C a a a

g g r c g w g g r

g

λ

λ

= + + =

− −

Chorophyll concentration: Phase function:

( ) [ ]

( )

2 3 2 2 / 2 1

cos 2 1 1 4 1 ) cos ( 2 ) (

2 max

Θ − + − = Θ + =

− −

f f f p e s h C z C

s z z

π π

Applications: Ocean colour

Multispectral reconstruction:different seeds an avearge value for ACS

Applications: data assimilation

Data assimilation: the problem

Applications: data assimilation

Data assimilation: Methods

Newtonian relaxation (nudging) Statistical (“optimal”) interpolation Kalman filter Variational method: 3D and 4D New methods for data assimilation:

Ensemble Kalman filter Particle filter Artificial neural networks

Applications: data assimilation

Data assimilation: essencial issue

Hybrid Methods in Engineering: (2000) 2(3): 291-310

Applications: data assimilation

Data assimilation: Kalman filter

[ ]

n n n t t n n n n

t O F t F

n

x E x x F x x ≈ ∆ + ∂ ∂ + ≈ =

= +

) ( ,

2 1

?

Applications: data assimilation

Data assimilation: adaptive Kalman filter

q a n q f n n f n , , 1 1

: in time Advance 1. P P q q = =

+ +

( ) [ ]

f n f n f n q n f n a n 1 1 1 1 1 1

estimation Update 3.

+ + + + + +

− + + = q ν q G q q

[ ]

1 , , 1 1

gain Kalman . 2

− + +

+ =

q f n

• n

q f n q n

P W P G

[ ]

q f n q n q a n , 1 1 , 1

covariance error Update . 4

+ + +

− = P G I P

• Adaptive KF
• Fokker-Planck
• (L)EKF

Applications: data assimilation

Data assimilation: Neural Networks

Applications: data assimilation

Data assimilation: Neural Networks

Applications: data assimilation

Data assimilation: Non-extensive Particle Filter

Applications: data assimilation

Data assimilation: Non-extensive Particle Filter

Applications: data assimilation

Data assimilation: Non-extensive Particle Filter

Helaine C. Morais Furtado Haroldo F. de Campos Velho Elbert E. Macau

Applications: data assimilation

Error: [Kalman, Particle, Variational] x Neural Network

Applications: data assimilation

Data assimilation: Ocean (shallow water 2D)

Applications: data assimilation

Data assimilation: Space weather

Interaction Sun-Earth: Solar Propagation Impact on Activity magnetosphere ionosphere

Applications: data assimilation

Data assimilation: Space weather

Applications: data assimilation

How can we find a good architecture for an ANN?

• Standard approach: employing an empirical process:

Some preliminaries topologies are defined and tested, changing the ANN parameters, and the process is re-started.

• Disadvantages:

Requeriment of a continuos effort from an expert This can require a long time

Applications: data assimilation

How can we find a good architecture for an ANN?

• Alternative approach:

Formulating the problem as an optimization problem.

• The goal:

Determine the best set of parameters for the ANN to

• ptimize an objetive function.

Applications: data assimilation

Neural network: self-configuring

Best configuration for multi-layer perceptron neural network

(MLP-NN):

Multi-Particle Collision Algorithm (MPCA)

The MLP network was configured to identify the NN applied

to data assimilation.

Two data set used for training: NN emulating Kalman filter for

data assimilation.

Applications: data assimilation

Neural network: self-configuring PCA Algorithm (Introduced by prof. Wagner F. Sacco, UFOP) Algorithm inspired from particle traveling inside

• f a nuclear reactor:

Absorption Scattering

Final position Final position Initial position Initial position

Applications: data assimilation

NN self-configuring: Absorption

Final position Final position Initial position Initial position

Applications: data assimilation

NN self-configuring: Scattering

Applications: data assimilation

NN self-configuring: PCA

Initial guess: Old_Config Best_Fitness = Fitness(Old_Config) For n=0 up to # iterations Perturbation() IF Fitness(New_Config) > Fitness(Old_Config) IF Fitness(New_Config) > Best_Fitness Best_Fitness := Fitness(New_Config) End-IF Old_Config = New_Config Exploration() ELSE Scattering() End-IF End-Stop

Applications: data assimilation

NN self-configuring: MPCA

Initial guess: Old_Config Best_Fitness = Fitness(Old_Config) For n=0 up to # iterations Perturbation() IF Fitness(New_Config) > Fitness(Old_Config) IF Fitness(New_Config) > Best_Fitness Best_Fitness := Fitness(New_Config) End-IF Old_Config = New_Config Exploration() ELSE Scattering() End-IF End-Stop

Applications: data assimilation

Parameter otimization for MLP-NN using MPCA

2 1 2 1*

ρ ρ ρ ρ + ∗ ∗ ) E + E ( penalty = F

gen trein

• bj

∑

+ =

− + − =

M N k k k gen

d N M E

1 2

) s ( ) 1 ( 1

∑

=

− =

N k k k trein

d N E

1 2

) s ( 1

( )

( ) (

) 1

) (# * *

2

• factor

complexity 2 1

• factor

complexity 2 # 1

+ × = 4 4 3 4 4 2 1 4 4 3 4 4 2 1 epoch c e c penality

neuron

Applications: data assimilation

Parameter otimization for MLP-NN using MPCA

MPCA solution

# hidden layers # neurons layer-1 # neurons layer-2 # neurons layer-3 Activation function Momentum ratio Learning ratio

Parameters Vallue Number of hidden layers |1| |2| |3| Number of neurons for each layer |1| ... |32| Learning ratio |0| ... |1| Momentum |0| ... |0.9| Activation function |Tanh| |Log| |Gauss|

Applications: data assimilation

Parameter otimization for MLP-NN using MPCA

Parameter Values Penalty 1 MPCA 6 particules MPCA stopping Maximum number of objetive function evaluation 500 Learning stopping 10000 epoch OR error < 0.00001 Assimilation cycle Each 10 time-steps Experiments First guess ANN – 1 Weights and bias: all values equal = 0.5 ANN – 2 Weights and bias: random

Applications: data assimilation

Parameter otimization for MLP-NN using MPCA

Evaluation: wave equation 1D

) , ( t x F x c t = ∂ ∂ + ∂ ∂ η η

η c F x t,

Distance Wave speed External forcing Time, space

• 1. Inital condition

(Furtado, Campos Velho, Macau 2011)

[ ]

∆ − = / ) ( cosh 1 ) , (

2

v x x η η

• 2. Periodic boundary conditions

Applications: NN self-configuring by MPCA

MPCA

Empirical ANN MPCA ANN-1 MPCA ANN-2

(Furtado, Campos Velho, Macau 2011)

Applications: NN self-configuring by MPCA

Some remarks

Difficult problem for solving:

data representation!

Can ANNs be the ultimate solution for data assimilation? I do not know. But I am convinced that neural networks must

be investigated as a new method for data assimilation.

ANN interesting features:

They are intrinsicly parallel ANN can be implemented on hardware device.

Applications: data assimilation by NN

Neural network on hardware device (FPGA)

Applications: data assimilation by NN

Neural network on hardware device (FPGA)

Cray XD1: 12 processors, 6 FPGA

Applications: data assimilation by NN

Neural network on hardware device (FPGA)

Applications: data assimilation by NN

Neural network on hardware device (FPGA)

Perceptron-NN for the Cray XD1

MLP-NN: Combining neurons

Inputs connected by one bus Ready to receive new data Results to Lookup Table (LUT): the pipeline

Neuron Uses of MAC MAC: Multiplier /accumulator

Input x weight Storaged on ACC or bias (depending on fc signal

Applications: data assimilation

Parameter otimization for MLP-NN using MPCA

Evaluation: wave equation 1D

) , ( t x F x c t = ∂ ∂ + ∂ ∂ η η

η c F x t,

Distance Wave speed External forcing Time, space

• 1. Inital condition

(Furtado, Campos Velho, Macau 2011)

[ ]

∆ − = / ) ( cosh 1 ) , (

2

v x x η η

• 2. Periodic boundary conditions

Applications: data assimilation by NN

Neural network on hardware device (FPGA)

Evaluation: wave equation 1D

Difference: Max error = 5.2E-05 variance = 1.8E-08

Software FPGA

Final remarks

Calibration problem is an extreme relevant inverse

problem: application every where.

Calibration process can be used for configuring the

Inverse-solver.

Neuro-computer as a hardware inversion tool. FPGA is the best processing component with focus

• n green-computing.

Best combination for green-computing:

ARM processor + FPGA