Asok Asok Ray Ray The Pennsylvania State University University - - PowerPoint PPT Presentation

Seminar Seminar at at Nat National Inst

• nal Institut

itute of S e of Standar andards and Technology ds and Technology

ANOMALY DETECTION ANOMALY DETECTION AND FAILURE MITIGATION ND FAILURE MITIGATION IN COMPLEX DYNAMICAL SYSTEMS IN COMPLEX DYNAMICAL SYSTEMS

Supported by

Army Research Office Grant No. DAAD19-01-1-0646

Asok Asok Ray Ray

The Pennsylvania State University University Park, PA 16802

Email: axr2@psu.edu Tel: (814) 865-6377

November 19, 2004

1 8 5 5

Outline of the Presentation

Anomaly Detection in Complex Dynamical Systems

Microstate Information Based on Macroscopic Observables

Thermodynamic Formalism of Multi-time-scale Nonlinearities Symbolic Time Series Analysis of Macroscopic Observables Pattern Discovery via Information-theoretic Analysis

Real-time Experimental Validation on Laboratory Apparatuses

Active Electronic Circuits and Three-phase Electric Induction Motors Multi-Degree-of-Freedom Mechanical Vibration and Chaotic Systems Fatigue Damage Testing in Polycrystalline Alloys

Discrete Event Supervisory Control for Failure Mitigation

Quantitative Measure for Language-based Decision and Control Real-time Identification of Language Measure Parameters Robust and Optimal Control in Language-theoretic Setting

1 8 5 5

Future Collaborative Research in Complex Microstructures

Modeling and Control of Hidden Anomalies and their Propagation Experimentation on Real-time Detection and Mitigation of Malignant Anomalies on a Hardware-in-the-loop Simulation Test Bed

Intelligent Health Management and Failure Mitigating Control

• f Aerial Vehicle Systems

Other Information Life Extending Control System (including Feedforward Control) Avionic and Structural Health and Usage Monitoring System (HUMS) Robust Wide range Gain Scheduling Feedback Control System Aircraft Flight and Structural Dynamics Conventional and Special-Purpose Sensor Systems Actuator Dynamics Analytical Measures (including real-time NDA)

• f Damage States and

Damage State Derivatives Anomaly Detection Information Fusion (e.g., FDI, calibration, data fusion, and redundancy management)

. .

Mission/Vehicle Management Level Mission/Vehicle Management Level (Discrete Even (Discrete Event t Decision Making) Decision Making) Flight Management Level Flight Management Level

(Continuous/Discrete Event Decision Making) (Continuous/Discrete Event Decision Making)

Avionics and Fli Avionics and Flight Control ght Control Level Level

1 8 5 5

Anomaly Detection and Classification: Symbolic Time Series Analysis Symbolic Time Series Analysis

1 8 5 5

Multi-Time-Scale Nonlinear Dynamics Multi-Time-Scale Nonlinear Dynamics

Slow T

• w Time Scale: A

me Scale: Anomaly Pr

• maly Propagat
• pagation (
• n (Non-st
• n-stat

ationary St ionary Stat atist istics) cs) Fast Fast Time Scale: Process R Time Scale: Process Response (St sponse (Stationar

• nary Statist

y Statistics) cs)

Model-based Statistical Model-based Statistical Methods Methods

Modeling w it Modeling w ith Nonlinear Nonlinear St Stocha

• chast

stic D ic Different fferential Equat ial Equations

• ns

Ito Equation: Fokker Planck Equation: Uncertainties in Model Identification and Loss of Robustness Stat atistical Mechanical istical Mechanical Modeli Modeling ( ng (Canonical anonical Ensemble Ensemble Approach) proach) Symbolic Time Series Analysis

Small perturbation stimulus Self-excited oscillations

Thermodynamic Formalism and Information Theory Hidden Markov Modeling (HMM) and Shift Spaces

Notion of Symbolic Dynamics Notion of Symbolic Dynamics

Discretization of the Dynamical System in Space and Time Representation of Trajectories as Sequences of Symbols ……φ χ γ η δ α δ χ…… Symbol Sequence Finite State Machine

1

φ

2

χ, ε γ

3

η δ α β

• 1
• 0.5

0.5 1

• 1
• 0.5

0.5 1 2 4 6 8 10

η α β γ δ χ φ ε

Phase Trajectory

1 8 5 5

State Machine Construction

D-Markov Machine

Ν ∈

Ν ∈ U } {

1 1 0 (

)

k D i D i i

w A

− − − =

∑

k D k k k

w w w W

1 1

...

−

=

k D k k

w w w

1 1

..........

−

1 1 1 1 1

........

+ − + + k D k k

w w w

1 1 + − k D

w

k

w0

1 1 1 1 1 1

...

+ − + + + = k D k k k

w w w W

( )

1 1 1 k k D k D

W w A A w

− + −

− +

In In Out

State to state transitions from a symbol sequence alphabet size = |A| window size = D kth word (state)= kth word value = (k+1)th word = (k+1)th word value =

p00 1-p00 0 0 0 0 p01 1-p01 p10 1-p10 0 0 0 0 p11 1-p11

00 10 01 11 1 1 1 1

|A|=2; D=2; AD = 4

Example

1-D Ising (Spin-1/2) Model Nearest Neighbor Interactions

1 8 5 5

Computationally efficient for anomaly measure

• Fixed depth D and alphabet size A
• Only the state transition probabilities to be determined

based on symbol strings derived from time series data

• r wavelet-transformed data

States represented by an equivalence class of strings whose last D strings are identical

State Space Construction via D-Markov Machine

1 8 5 5

Anomaly Measure

Based on the D-Markov Machine Based on the D-Markov Machine State Transition State Transition Matrix Construction Matrix Construction

Banded structure Separation into irreducible subsystems Stationary state probability vector Information on the dynamical system characteristics Chaotic motion, period doubling, and bifurcation

State Probability Vector State Probability Vector

Reference Point: Nominal Condition p(το ) Epochs {τk} of Slow Time Scale {p(τk)}

Anomaly Measure at Slow -Time Epochs Anomaly Measure at Slow -Time Epochs

M(τk; το) = d(p(τk), p(το))

1 8 5 5

Comparison of Epsilon Machine and D-Markov Machine Epsilon Machine [Santa Fe Institute]

A priori unknown machine structure Optimal prediction of the symbol process Maximization of mutual information (i.e., minimization of conditional entropy) I[X;Y] = H[X] – H[X|Y] Analogous to the class of Sofic Shifts in Shift Spaces

D-Markov Machine

A priori known machine structure (Fixed order fixed structure with given |A| and D) Excess states yielding redundant reducible matrices (Perron-Frobenius Theorem) Suboptimal prediction of the symbol process Analogous to the class of Finite-type Shifts in Shift Spaces

1 8 5 5

Anomaly Detection Procedure

Forward (Analysis) Problem:

Characterization of system dynamical behavior

Parametric and non-parametric anomalies

Evolution of the grammar in the system dynamics

Representation of dynamical behavior as formal languages Thermodynamic formalism of anomaly measure

Inverse (Synthesis) Problem

Estimation of feasible ranges of anomalies

Fusion of information generated from responses under several stimuli chosen in the forward problem

1 8 5 5

00011000110101…

Signal Conditioning Time Series Data Current, Voltage or

• ther Signals

Sampling and Quantization; Denoising, and Decimation Wavelet Transform 00 01 11 10 1 1 1 1 Partitioning of Wavelet Coefficients D-Markov Machine HMM Construction Real-time Analytical Prediction Dynamical System Pre- Processing Symbol Generation Pattern Representation Anomaly Detection

Summary of Anomaly Detection Procedure

1 8 5 5

Anomaly

• maly D

Detect tection and C ion and Classificat assification

• n

Signal Conditioning and Decimation Denoising Embedding Symbol Sequence Generation Phase space partitioning

Wavelet space partitioning

Markov Modeling of Symbol Dynamics Epsilon machine (sofic shift) D-Markov machine (finite type shift)

Thermodynamic Formalism of Generated Information

Externally Stimulated Duffing Equation

w ith a single slow ly varying parametric anomaly

1 8 5 5

Slowly Varying Damping Nonlinear Spring Mass Externally Applied Force T

t y t y )] ( ) ( [ & ) (ωt Cos A

Random Initial Conditions

) ( )] ( ) ( [

δ

B t y t y

T

• ∈

&

s

t t fast time slow time

Governing Equations:

) , [ ∞ ∈ o t t ) , [ ∞ ∈ o t t ) ( ) ( ) ( ) (

3 ) ( ) (

2 2

ω α θ

dt t dy s dt t y d

t Cos A t y t y t = + + + Parameters:

1 ; 0.01 ; 22; 5.0 A α δ ω = = = =

Externally Stimulated Duffing Equation

Electromechanical Systems Laboratory Anomaly Detection Apparatus for Hybrid Electronic Circuits

Computer and Data Acquisition System A Sin(ωt)

1 8 5 5

Phase-plane Plots under Nominal and Anomalous Conditions

• 4.0
• 3.0
• 2.0
• 1.0

0.0 1.0 2.0 3.0 4.0

Phase Variable dy/dt

• 2
• 1.5
• 1
• 0.5

0.0 0.5 1 1.5 2 2.5

Phase Variable y θ= 0.10

• 4.0
• 3.0
• 2.0
• 1.0

0.0 1.0 2.0 3.0 4.0

Phase Variable dy/dt

• 2
• 1.5
• 1
• 0.5

0.0 0.5 1 1.5 2 2.5

Phase Variable y θ= 0.27 Phase Variable dy/dt Phase Variable y

• 4.0
• 3.0
• 2.0
• 1.0

0.0 1.0 2.0 3.0 4.0

• 2
• 1.5
• 1
• 0.5

0.0 0.5 1 1.5 2 2.5

θ= 0.28

Phase Variable y

• 2.5
• 2.0
• 1.5
• 1.0
• 0.5

0.0 0.5 1.0 1.5 2.0 2.5

Phase Variable dy/dt

• 1
• 0.5

0.0 0.5 1 1.5

θ=0.29

Time

200 400 600 800 1000 1200 1400 1600 1800 2000

• 2.0
• 1.5
• 1.0
• 0.5

0.0 0.5 1.0 1.5 2.0 2.5

Green

θ = 0.27 Red θ = 0.28 Black θ = 0.29

Phase Variable

Blue

θ = 0.10

Electromechanical Systems Laboratory

1 8 5 5

Construction of Finite State Machines

0.7079 0.0030 0.0035 0.2856

θ = 0.28

00 01 10 11

0/0.9986 1/0.0014 0/0.251 1/0.749 0/0.5 1/0.5 0/0.0070 1/0.9930

S=0.6397

Interpretation using Thermodynamic Formalism Thermodynamic Formalism

|A|=2; D=2; AD = 4

0/0.0048

00 01 10 11

0/0.9987 1/0.0013 0/0.5 1/0.5 0/0.5 1/0.5 1/0.9952 0.7889 0.0020 0.0020 0.2071

θ = 0.10

S=0.5380

0/0.0070

θ = 0.27

0.7659 0.0025 0.0025 0.2291 1/0.9930

00 01 10 11

0/0.9987 1/0.0013 0/0.33

1/0.67 0/0.50

1/0.50

S=0.5718

11

1/1.0

θ> = 0.29

0.0 0.0 0.0 1.0

S=0.0

Ideal Monatomic Gas Constant V, N Energy E Entropy S X X X X X

Emin

? ? θ−1 ~ partial

derivative of S wrt E

1 8 5 5

Electronic Circuit Apparatus

0.10 0.15 0.20 0.25 0.30 0.35 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Parameter θ Normalized Anomaly Measure

MLPNN RBFNN D-Markov SFNN D-Markov WS PCA MLPNN RBFNN D-Markov SFNN D-Markov WS PCA

Sensitivity of the Detection Algorithm to the Anomalous Parameter θ

1 8 5 5

Electromechanical Systems Laboratory

Anomaly Detection Apparatus for Mechanical Vibration Systems

4 8 12 16 20 24 28 32 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Wavelet Space (WS) Partitioning Principal Componen t Analysis (PCA) Symbolic False Nearest Neighbor (SFNN) Partitioning Radial Basis Function Neural Network (RBFNN) Multilayer Perceptro n Neural Network (MLPNN)

Normalized Anomaly Measure

Time in minutes

1 8 5 5

1 2 3 4 5 6 7 8 x 10

4

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Number of Fatigue Cycles

Normalized Anomaly Measure D-Markov WS D-Markov SFNN MLPNN RBFNN PCA

First Crack Detection on Microscope

Electromechanical Systems Laboratory

Fatigue Test Apparatus for Fatigue Test Apparatus for Damage Sensing Damage Sensing in Ductile Alloys uctile Alloys

1 8 5 5

Anomaly Detection

Symbolic Time Series Analysis Advantages Advantages

Foundations on fundamental principles of physics and mathematics Quantitative measure as opposed to qualitative measure Robustness to measurement noise and spurious signal distortion Sensitivity to signal distortion due to nonlinearity and nonstationarity Adaptability to low-resolution sensing Applicability to real-time anomaly detection

(Near-term) Disadvantages (Near-term) Disadvantages

Requirement for advanced knowledge to understand the basics Need for much theoretical and experimental research Seemingly counter-intuitive to inadequately trained technical personnel

1 8 5 5

• A. Ray, “Symbolic Dynamic Analysis of Complex Systems for Anomaly Detection,”

Signal Processing, Vol. 84, No. 7, July 2004, pp. 1115-1130.

Quantitative Measure for Quantitative Measure for Discrete Event Discrete Event Supervisory Control Supervisory Control

1 8 5 5

Discrete Event Supervisory (DES) Control of Continuous Plants

External Commands Treated as Uncontrollable Events

Real-Time Real-Time Continuously Continuously Varying Varying Plant Plant

Event Generator (C/D) Discrete Event Model

• f the

Continuously Varying Plant (Treated as State Estimator) Other information

Supervisor

Command Generator (D/C) Sensor data Disabling Controllable events

Observable Events Generated Events

1 8 5 5

Modeling of Discrete Event Supervisory Control Systems

f(x, u) f(x, u)

∫ ∫

g(x, u) g(x, u) h(x, u) h(x, u)

f(x, u) f(x, u)

∫ ∫

g(x, u) g(x, u) h(x, u) h(x, u) fi(x, u) fi(x, u)

∫ ∫

g(x, u) g(x, u) h(x, u) h(x, u)

Continuously Varying Systems

Discrete Event Systems Hybrid Systems

f(x, u) fi(x, u) Decoupling: continuous evolutions and discrete transitions Coupling: continuous evolutions and discrete transitions

1 8 5 5

Behavior-based Robotic System DES Control Architecture

Precision Intelligence

DES control

Event Detection

Continuous-time

• perational control

Reference Generation

Continuous time Discrete Event

Pioneer 2 AT Stage Simulator

interchangeable

1 8 5 5

Signed Real Measure of Regular Languages Signed Real Measure of Regular Languages Salient Features Salient Features

Language Metric |µ|

Total variation of the signed real measure (Real positive) distance between two languages

Applications of the Language Measure for Failure Mitigation

Robust and optimal control of discrete-event systems

Anomaly quantification, classification, and mitigation

Vector Space of Formal Languages

Infinite-dimensional space Galois field GF(2) Vector addition operator - Exclusive-OR

Quantitative Measure of *-Languages

Finite alphabet Σ Σ∗ cardinality N0

Language Measure µ: 2Σ* R

1 8 5 5

State-Based Partitioning State-Based Partitioning

The set Qm of marked states is partitioned as: By using Myhill-Nerode Theorem Hahn Decomposition Theorem where is the set of good marked states (positive measure) is the set of bad marked states (negative measure) ∅ = =

− + − + m m m m m

Q Q ; Q Q Q I U

+ m

Q

− m

Q

1 8 5 5

Lemma:

The regular expressions can be expressed by the following set of symbolic equations:

( ), { 1 ,2 , , } L LG i I n i i ≡ ∈ ≡ L

I ∈ ∀ + ∑ = i L L

i j j j i i

ϑ σ where ⎩ ⎨ ⎧ ∅ ∈ =

• therwise

Q q if

m i i

ε ϑ

Theorem: The language measure of the regular expressions

is given by the unique solution of the following set of algebraic equations: I ∈ ∀ χ + µ ∑π = µ i

j j j ij i

} n , , 2 , 1 { i , Li L ∈ In vector notation, the system has a unique solution: Χ + µ Π = µ

1

] I [

−

Π −

Remark:

exists and is bounded above by

∞

Π / 1

Main Result:

Signed Real Measure of Regular Languages

µ=[Ι−Π] −1 Χ

• A. Ray, V. V. Phoha and S. Phoha, QUANTITATIVE MEASURE FOR

DISCRETE EVENT SUPERVISORY CONTROL: Theory and Applications, Springer, New York, 2004. ISBN 0-387-02108-6 0-387-02108-6

1 8 5 5

Algorithm for the Unconstrained Optimal Control Law

⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ = Π

nn n n n n 2 1 2 22 21 1 12 11

π π π π π π π π π L M O M M L L

( ) χ

µ µ µ µ

1 2 1 −

Π − = ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎛ = I

n

M

Given the information on the plant model Gi = (Q, Σ , δ ,qi , Qm) along with the state transition cost matrix Π and characteristic vector χ , the unconstrained optimal control maximizes the language measure by deleting some

• f the “bad” strings so that optimality of the supervised plant

sublanguage is achieved

1 8 5 5

Supporting Theorems

Theorem #1 (Monotonicity) : Disabling the controllable events leading to states with negative (positive) performance does decrease (increase) supervised plant performance. Theorem #2 (Monotonicity) : Enabling the controllable event(s) leading to states with non-negative performance does not decrease the performance for any state. Theorem #3 (Global Performance): The controller at the termination of the algorithm is the global optimal controller in terms of supervised plant performance. Theorem #4 (Computational Complexity): The optimal control law is solved in at most n steps and each step requires a solution of n linear algebraic equations where n is the number of states of the plant model. Therefore, the computational complexity for synthesis of the optimal control algorithm of a polynomial order in n.

• A. Ray, J. Fu and C.M. Lagoa, "Optimal Supervisory Control of Finite State Automata,"

International Journal of Control, Vol. 77, No. 12, August 2004, pp. 1083-1100.

1 8 5 5

Language-Measure-Theoretic Discrete-Event Supervisory Control Advantages Advantages

Foundations on principles of automata theory and functional analysis Quantitative measure as opposed to qualitative measure Robustness to measurement noise and spurious signal distortion Capability for emulation of human reasoning in a quantitative way Adaptability to low-resolution sensing Applicability to real-time decision-making at multiple time scales

Disadvantages Disadvantages

Potential source of instability under switching actions Need for much theoretical and experimental research Requirement for advanced knowledge to understand the basics

1 8 5 5

• A. Ray, V. V. Phoha and S. Phoha, QUANTITATIVE MEASURE FOR

DISCRETE EVENT SUPERVISORY CONTROL: Theory and Applications, Springer, New York, 2004. IS

ISBN 0-387-02108-6 0-387-02108-6

1 8 5 5

Future Collaborative Research Future Collaborative Research in Complex Microstructures in Complex Microstructures

1 8 5 5

Modeling and Control of Hidden Modeling and Control of Hidden Anomalies and their Propagation Anomalies and their Propagation

Problem Definition Problem Definition

Let the time series of (macroscopic) measurement(s) θ be available. The first problem is to estimate the unobservable performance parameter(s) β

(e.g., damage states and derivatives).

The second problem is to control the microstates via manipulation of macroscopically

controllable inputs u

u to satisfy desired performance specifications p. Proposed Solution Proposed Solution

Construction of a canonical-ensemble model with the state probability vector π(θ, u) of the unobservable phenomena that are macroscopically controlled by inputs u

u through

usage of the time series data and a microstructural model, such as the OOF

OOF of NIST. Formulation of constitutive equations for the unobservable parameters β(π, u) that are

indicator(s) of the internal microstates and control laws u(βd , πd, p) to satisfy desired performance specifications (e.g., remaining service life and reliability)

Experimentation for Real-time Detection Experimentation for Real-time Detection and Mitigation of Malignant Anomalies and Mitigation of Malignant Anomalies

1 8 5 5

Experimental Validation of the Novel Constitutive Relations Experimental Validation of the Novel Constitutive Relations

Special-purpose Fatigue Testing Machine at Penn State Object Oriented Finite-element (OOF OOF) Modeling Package at NIST

Experimental Validation of the Supervisory Control Concept Experimental Validation of the Supervisory Control Concept

Special-purpose Multi-degree-of-freedom Machine at Penn State Control of the unobservable parameters, such as plastic zone size, that are indicator(s) of the internal microstates Discrete Event Supervisory Control at the Upper level Derived parameter(s) β (e.g., damage states and their derivatives) to provide

the input event sequence to the supervisory control module at the upper level Supervisor command(s) to provide the control inputs u, such as shaft torque, to the test apparatus to satisfy the desired performance specifications p

p

1 8 5 5

Questions & Suggestions