fo for r th the co e coupled upled el elec ectric tric dri - PowerPoint PPT Presentation

Ke Kern rnel el man anifold ifold re regress gression ion fo for r th the co e coupled upled el elec ectric tric dri rives ves dat ataset aset Mirko ko Mazzolen leni Matteo Sca cande della lla Fabi bio Previd vidi mirko.mazzoleni@unibg.it 11 April 2018 matteo.scandella@unibg.it fabio.previdi@unibg.it

Ou Outline line 1. Introduction and motivation 2. A new framework for nonparametric system identification 3. Application to the coupled electric drive dataset 4. Conclusions and future developments 2/32

Ou Outline line 1. 1. Introd oduc uctio ion n and mo d motiva vation ion 2. A new framework for nonparametric system identification 3. Application to the coupled electric drive dataset 4. Conclusions and future developments 3/32

In Introduction oduction an and d mot otiv ivation ation Data Distance Model 4/32

In Introduction oduction an and d mot otiv ivation ation Data Distance Mod odel el 5/32

In Introduction oduction an and d mot otiv ivation ation Mo Model del definition inition Consider the NARX RX system tem: 𝒯: 𝑧 𝑢 + 1 = 𝑕 𝑦 𝑣 𝑢 , 𝑦 𝑧 (𝑢) + 𝑓 𝑢 , where: •  𝑧 𝑢 ∈ ℝ is the system output  𝑕(𝑢) is a nonlinear inear function ction 𝑈 ∈ ℝ 𝑞×1 is a regressor vector of past 𝑞 inputs  𝑦 𝑣 𝑢 = 𝑣 𝑢 , … , 𝑣 𝑢 − 𝑞 + 1 𝑈 ∈ ℝ 𝑟×1 is a regressor vector of past 𝑟 outputs  𝑦 𝑧 𝑢 = 𝑧 𝑢 , … , 𝑧 𝑢 − 𝑟 + 1 𝑈 ∈ ℝ 𝑞+𝑟×1  𝑦 𝑢 = 𝑦 𝑧 𝑢 , 𝑦 𝑧 𝑢  𝑓 𝑢 ∈ ℝ is an additive white noise 6/32

Le Lear arning ning from om dat ata Repr Re producing oducing Kernel rnel Hilbert bert Spaces aces (R (RKHS) KHS) An RKHS is a Hilber bert space ce ℋ such that: • a. Its elements are functions 𝑣: Ω → ℝ , where Ω is a generic set b. ∀𝑦 ∈ Ω, 𝑀 𝑦 : ℋ → ℝ is a continuous linear functional 𝑣 → 𝑣(𝑦) 𝑦 ∈ ℋ s.t. 𝑀 𝑦 𝑣 = 𝑣 𝑦 = 𝑣, 𝑠 Riesz’s repr pres esentation entation theorem em  ∃ 𝑠 • 𝑦 The function 𝑠 𝑦 (⋅) is called the repr pres esenter enter of evaluatio aluation in 𝑦 • 7/32

Lear Le arning ning from om dat ata Repr Re producing oducing Kernel rnel Hilbert bert Spaces aces (R (RKHS) KHS) The reproducing kernel is defined as: 𝐿 𝑦, 𝑨 = • 𝑠 𝑦 , 𝑠 𝑨 , 𝐿 = Ω × Ω → ℝ a. Symmetric: 𝐿 𝑦, 𝑨 = 𝐿 𝑨, 𝑦 b. Semidefinite positive σ 𝑗,𝑘=1 𝑜 𝑑 𝑗 𝑑 𝑘 𝐿(𝑦 𝑗 , 𝑦 𝑘 ) ≥ 0 ∀𝑜, 𝑑 𝑗 ∈ ℝ, ∀𝑦 𝑗 ∈ Ω Mo Moor ore-Ar Aronsza onszajn jn theo theorem  A RKHS KHS defines a corresponding repr eproducing oducing • kernel rnel. Conversely, a reproducing oducing kernel rnel defines a unique RKH KHS 8/32

Lear Le arning ning from om dat ata Exa xampl mples of kerne nels ls • Constant tant kernel: rnel: 𝐿 𝑦, 𝑨 = 1 • Polyn ynomial mial kernel el: : 𝐿 𝑦, 𝑨 = 𝑦 ⋅ 𝑨 + 1 𝑒 rnel: 𝐿 𝑦, 𝑨 = 𝑓 − 𝑦−𝑨 2 • Linear ear kernel: rnel: 𝐿 𝑦, 𝑨 = 𝑦 ⋅ 𝑨 • Gauss ssian an kernel: 2𝜏2 Kernel rnel comp mposi osition tion th theorem eorem: • A linear combination of valid kernel functions is a valid kernel function • The space induced by this kernel is the span of the spaces induced by the single ones 𝐼 =⊕ 𝑗 𝐼 𝑗 9/32

A new ew fram amewo ework rk for or sy syst stem em iden entifi tification cation Kernel rnel me method thods in syste tem identi entific fication ation Stab able spline ine kernel el 𝑡 Represe resenters ters Pillonetto , Gianluigi and Giuseppe De Nicolao. “A new kernel - based approach for linear system identification.” Automatica 46 (2010): 81-93. • Pillonetto, Gianluigi et al. “A New Kernel-Based Approach for NonlinearSystem Identification .” IEEE Transactions on Automatic Control 56 (2011): 2825-2840. • 10/32 10

In Introduction oduction an and d mot otiv ivation ation Data Di Dist stan ance ce Model 11 11/3 /32

Lear Le arning ning from om dat ata No Nonpa parametric rametric learning rning Consider the variational riational formulation: • 𝑂 2 + 𝜇 𝑈 ⋅ 𝑕 ℋ 2 𝑕 = arg min ො ෍ 𝑧 𝑗 − 𝑕 𝑦 𝑗 𝑧 𝑗 = 𝑧 𝑢 𝑗 ; 𝑦 𝑗 = 𝑦 𝑢 𝑗 𝑕∈ℋ 𝑗=1 Tikhonov regularization: 𝑕 2 ℋ penalizes the norm of the fitted function • The minimization problem is on the RKHS space ℋ  infinite number of parameters! • 12/32 12

Lear Le arning ning from om dat ata Re Representer presenter th theorem orem The minimizer imizer of the variational problem is given by: • 𝑂 𝑂 For some 𝑂 -tuple 𝑕(𝑦) = ෍ ො 𝑑 𝑗 𝐿 𝑦, 𝑦 𝑗 = ෍ 𝑑 𝑗 𝑠 𝑦 𝑗 (𝑦) 𝑑 = 𝑑 1 , 𝑑 2 , … , 𝑑 𝑂 𝑈 ∈ ℝ 𝑂×1 𝑗=1 𝑗=1 Linear comb mbination ination of the representer esenters of the training points 𝑦 𝑗 , evaluated in the point 𝑦 • The solution is expressed as combination of «basis functions» which properties are • determined by ℋ 13/32 13

Ƹ Le Lear arning ning from om dat ata No Nonpa parametric rametric learning rning – Solution ution Using the representer theorem it possible to express the variational problem as: • • 𝑍 ∈ ℝ 𝑂×1 : vector of observations 2 + 𝜇 𝑈 ⋅ 𝑑 𝑈 𝒧𝑑 𝑑 = arg min 𝑍 − 𝒧𝑑 2 • 𝒧 ∈ ℝ 𝑂𝑦𝑂 : semidefinite positive and 𝑑∈ℝ 𝑂 symmetric matrix, s.t. 𝒧 𝑗𝑘 = 𝐿(𝑦 𝑗 , 𝑦 𝑘 ) Since the expression is quadratic in 𝑑 we have the closed osed-fo form solution ution: • 𝒧 + 𝜇 𝑈 ⋅ 𝐽 𝑂 ⋅ Ƹ 𝑑 = 𝑍 14 14/3 /32

Outline Ou line 1. Introduction and motivation 2. A new fr 2. frame mework work fo for nonpa parame rametric ric sy syst stem m ide dentif ific icati ation on 3. Application to the coupled electric drive dataset 4. Conclusions and future developments 15 15/32

Introduction In oduction an and d mot otiv ivation ation Manifol Ma ifold d learning rning (s (sta tatic tic system tems) s) Suppose that the regressors’ belong to a • manif ifold old in the regressors’ space The position ition of the regressors adds prio ior r • information mation How to incorporate this information in a • learning ning framework? amework? 16/32 16

Introduction In oduction an and d mot otiv ivation ation Manifol Ma ifold d learning rning (s (sta tatic tic system tems) s) Suppose that the regressors’ belong to a • manif ifold old in the regressors’ space The position ition of the regressors adds prio ior r • information mation How to incorporate this information in a • learning ning framework? amework? 17/32 17

In Introduction oduction an and d mot otiv ivation ation Incorpor orporating ating th the ma manifol fold d infor formation mation Semi Semi-su super pervi vise sed d smoothne thness ss assumption ption If two regressors 𝑦 𝑗 and 𝑦 𝑘 in a high-density region are close, then so should be their corresponding outputs 𝑧 𝑗 and 𝑧 𝑘 18/32 18

In Introduction oduction an and d mot otiv ivation ation Link Li k to to dynamic namical al syste tems ms In dynamical systems, regressors can • be stron ongly gly corr rrelated elated It is meaningful to think that they lie lie on • a a manifold fold of the regressors’ space PCA reveals how 91% of variance riance • explained by one component 19 19/32

Introduction In oduction an and d mot otiv ivation ation Ma Manifol ifold regular ularizati ization on One way to enforce the smoothness ness assumption ption is to minimize the quantity: • 𝛼 ⋅ 𝑕 2 𝑒𝑞 𝑦 = න 𝑇 𝑕 = න 𝑕 ⋅ Δ ⋅ 𝑕 𝑒𝑞 𝑦 𝒣 𝒣 𝑞 𝑦 : probability distribution of the regressors, 𝛼 : Gradient, Δ : Laplacian-Beltrami operators • Minimizing 𝑇 𝑕 means mi minim imizing izing th the gradient dient of the function • The term can rarely be computed since 𝑞 𝑦 and 𝒣 are unkno nown wn • 20/32 20

Introduction In oduction an and d mot otiv ivation ation Ma Manifol ifold regular ularizati ization on The term 𝑇 𝑕 can be modeled led using the reg egres esso sor r graph ph, encoding connections and • distasnces between points: 𝜏 𝑓 is a tuning parameter • Higher value  similar •  with the regressors as its vertices regressors 2 − 𝑦 𝑗 −𝑦 𝑘 2  the weights on the edges are defined as: 2𝜏 𝑓 𝑥 𝑗,𝑘 = 𝑓 Considering the Laplacian matrix 𝑀 = 𝐸 − 𝑋 ∈ ℝ 𝑂×𝑂 , where: •  𝐸 ∈ ℝ 𝑂×𝑂 is a diagonal matrix 𝐸 𝑗𝑗 = σ 𝑘=1 𝑥 𝑗,𝑘 ; 𝑋 ∈ ℝ 𝑂×𝑂 contains the 𝑥 𝑗,𝑘 𝑂 21/32 21