The harmonic analysis of kernel functions Mattia Zorzi Department - - PowerPoint PPT Presentation

▶
the harmonic analysis of kernel functions
SMART_READER_LITE
LIVE PREVIEW

The harmonic analysis of kernel functions Mattia Zorzi Department - - PowerPoint PPT Presentation

Feb 16, 2024 230 likes •760 views

The harmonic analysis of kernel functions Mattia Zorzi Department of Information Engineering University of Padova Cison di Valmarino, September 26th, 2016 Joint work with: A. Chiuso (University of Padova) Kernels in system identification

slide-1
SLIDE 1

The harmonic analysis of kernel functions

Mattia Zorzi

Department of Information Engineering University of Padova

Cison di Valmarino, September 26th, 2016

Joint work with:

  • A. Chiuso (University of Padova)
slide-2
SLIDE 2

Kernels in system identification

Model class (e.g. OE models) yt =

∞

  • s=1

gsut−s + et gt impulse response Gaussian linear regression model yN = Φθ + eN θ ∼ N(0, K), K kernel function

y N :=    y1 . . . yN    , θ :=    g1 g2 . . .   

gt is modeled as Gaussian process with zero mean and covariance function Cov[gtgs] = K(t, s)

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 2 / 15

slide-3
SLIDE 3

Kernels in system identification

Model class (e.g. OE models) yt =

∞

  • s=1

gsut−s + et gt impulse response

ut yt et

Gaussian linear regression model yN = Φθ + eN θ ∼ N(0, K), K kernel function

y N :=    y1 . . . yN    , θ :=    g1 g2 . . .   

gt is modeled as Gaussian process with zero mean and covariance function Cov[gtgs] = K(t, s)

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 2 / 15

slide-4
SLIDE 4

Kernels in system identification

Model class (e.g. OE models) yt =

∞

  • s=1

gsut−s + et gt impulse response

ut yt et

Gaussian linear regression model yN = Φθ + eN θ ∼ N(0, K), K kernel function

y N :=    y1 . . . yN    , θ :=    g1 g2 . . .   

gt is modeled as Gaussian process with zero mean and covariance function Cov[gtgs] = K(t, s)

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 2 / 15

slide-5
SLIDE 5

Kernels in system identification

Model class (e.g. OE models) yt =

∞

  • s=1

gsut−s + et gt impulse response

ut yt et

Gaussian linear regression model yN = Φθ + eN θ ∼ N(0, K), K kernel function

y N :=    y1 . . . yN    , θ :=    g1 g2 . . .   

gt is modeled as Gaussian process with zero mean and covariance function Cov[gtgs] = K(t, s)

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 2 / 15

slide-6
SLIDE 6

Kernels in system identification (cont’d)

K encodes the a priori information on gt Our a priori information on the impulse response: BIBO stable Frequency content Question How to embed this information in the kernel function?

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 3 / 15

slide-7
SLIDE 7

Kernels in system identification (cont’d)

K encodes the a priori information on gt Our a priori information on the impulse response: BIBO stable Frequency content Question How to embed this information in the kernel function?

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 3 / 15

slide-8
SLIDE 8

Kernels in system identification (cont’d)

K encodes the a priori information on gt Our a priori information on the impulse response: BIBO stable Frequency content

10 20 30 40 50 60 70 80 90 100 −0.8 −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8 1

t gt

−40 −20 20 Magnitude (dB) 10

−1

10 10

1

−360 −315 −270 −225 −180 Phase (deg)

Bode Diagram

Frequency (rad/s)

Question How to embed this information in the kernel function?

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 3 / 15

slide-9
SLIDE 9

Kernels in system identification (cont’d)

K encodes the a priori information on gt Our a priori information on the impulse response: BIBO stable Frequency content

10 20 30 40 50 60 70 80 90 100 −0.8 −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8 1

t gt

−40 −20 20 Magnitude (dB) 10

−1

10 10

1

−360 −315 −270 −225 −180 Phase (deg)

Bode Diagram

Frequency (rad/s)

Question How to embed this information in the kernel function?

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 3 / 15

slide-10
SLIDE 10

Modeling the impulse response

How? gt as a sum of damped sinusoids gt =

M

  • k=1

|ck|e− αk

2 t cos(ωkt + ∠ck)

ck complex Gaussian random variable such that:

◮ ck is zero mean ◮ Cov(ck, ¯

cj) = pkδk−j

◮ Cov(ck, cj) = 0

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 4 / 15

slide-11
SLIDE 11

Modeling the impulse response

How? gt as a sum of damped sinusoids gt =

M

  • k=1

|ck|e− αk

2 t cos(ωkt + ∠ck)

10 20 30 40 50 60 70 80 90 100 −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8 1

t gt e−

αk 2 t

ωk

ck complex Gaussian random variable such that:

◮ ck is zero mean ◮ Cov(ck, ¯

cj) = pkδk−j

◮ Cov(ck, cj) = 0

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 4 / 15

slide-12
SLIDE 12

Modeling the impulse response

How? gt as a sum of damped sinusoids gt =

M

  • k=1

|ck|e− αk

2 t cos(ωkt + ∠ck)

10 20 30 40 50 60 70 80 90 100 −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8 1

t gt e−

αk 2 t

ωk

ck complex Gaussian random variable such that:

◮ ck is zero mean ◮ Cov(ck, ¯

cj) = pkδk−j

◮ Cov(ck, cj) = 0

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 4 / 15

slide-13
SLIDE 13

Modeling the impulse response (cont’d)

Sum of damped sinusoids in a grid gt =

  • i
  • j

|cij|e−

αj 2 t cos(ωit + ∠cij)

“Infinite dense sum” of damped sinusoids gt = ∞ ∞

−∞

|c(α, ω)|e− α

2 t cos(ωt + ∠c(α, ω))dωdα

c(α, ω) generalized Fourier transform of gt

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 5 / 15

slide-14
SLIDE 14

Modeling the impulse response (cont’d)

Sum of damped sinusoids in a grid gt =

  • i
  • j

|cij|e−

αj 2 t cos(ωit + ∠cij)

α1 α2 !1 !2

k = 1 k = 2

“Infinite dense sum” of damped sinusoids gt = ∞ ∞

−∞

|c(α, ω)|e− α

2 t cos(ωt + ∠c(α, ω))dωdα

c(α, ω) generalized Fourier transform of gt

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 5 / 15

slide-15
SLIDE 15

Modeling the impulse response (cont’d)

Sum of damped sinusoids in a grid gt =

  • i
  • j

|cij|e−

αj 2 t cos(ωit + ∠cij)

α1 α2 !1 !2

k = 1 k = 2

“Infinite dense sum” of damped sinusoids gt = ∞ ∞

−∞

|c(α, ω)|e− α

2 t cos(ωt + ∠c(α, ω))dωdα

c(α, ω) generalized Fourier transform of gt

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 5 / 15

slide-16
SLIDE 16

Modeling the impulse response (cont’d)

Sum of damped sinusoids in a grid gt =

  • i
  • j

|cij|e−

αj 2 t cos(ωit + ∠cij)

α1 α2 !1 !2

k = 1 k = 2

“Infinite dense sum” of damped sinusoids gt = ∞ ∞

−∞

|c(α, ω)|e− α

2 t cos(ωt + ∠c(α, ω))dωdα

c(α, ω) generalized Fourier transform of gt

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 5 / 15

slide-17
SLIDE 17

Harmonic analysis

Let gt = ∞ ∞

−∞

|c(α, ω)|e− α

2 t cos(ωt + ∠c(α, ω))dωdα

Harmonic representation of the kernel function K K(t, s) = 1 2 ∞ ∞

−∞

p(α, ω)e−α t+s

2 cos(ω(t − s))dωdα

p(α, ω) generalized power spectral density

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 6 / 15

slide-18
SLIDE 18

Harmonic analysis

Let gt = ∞ ∞

−∞

|c(α, ω)|e− α

2 t cos(ωt + ∠c(α, ω))dωdα

Harmonic representation of the kernel function K K(t, s) = 1 2 ∞ ∞

−∞

p(α, ω)e−α t+s

2 cos(ω(t − s))dωdα

p(α, ω) generalized power spectral density

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 6 / 15

slide-19
SLIDE 19

Harmonic analysis

Let gt = ∞ ∞

−∞

|c(α, ω)|e− α

2 t cos(ωt + ∠c(α, ω))dωdα

Harmonic representation of the kernel function K K(t, s) = 1 2 ∞ ∞

−∞

p(α, ω)e−α t+s

2 cos(ω(t − s))dωdα

p(α, ω) generalized power spectral density

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 6 / 15

slide-20
SLIDE 20

Harmonic analysis

Let gt = ∞ ∞

−∞

|c(α, ω)|e− α

2 t cos(ωt + ∠c(α, ω))dωdα

Harmonic representation of the kernel function K K(t, s) = 1 2 ∞ ∞

−∞

p(α, ω)e−α t+s

2 cos(ω(t − s))dωdα

p(α, ω) generalized power spectral density

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 6 / 15

slide-21
SLIDE 21

Harmonic analysis

Let gt = ∞ ∞

−∞

|c(α, ω)|e− α

2 t cos(ωt + ∠c(α, ω))dωdα

Harmonic representation of the kernel function K K(t, s) = 1 2 ∞ ∞

−∞

p(α, ω)e−α t+s

2 cos(ω(t − s))dωdα

p(α, ω) generalized power spectral density

! α p(α; !)

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 6 / 15

slide-22
SLIDE 22

Example 1: Stationary kernels

Choice: p(α, ω) = δ(α)q(ω) Stationary kernel K(t − s) = 1 2 ∞

−∞

q(ω) cos(ω(t − s))dω q(ω) power spectral density Stationary process (“infinite dense sum” of sinusoids) gt = ∞

−∞

|c(ω)| cos(ωt + ∠c(ω))dω c(ω) Fourier transform

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 7 / 15

slide-23
SLIDE 23

Example 1: Stationary kernels

Choice: p(α, ω) = δ(α)q(ω)

! α p(α; !)

Stationary kernel K(t − s) = 1 2 ∞

−∞

q(ω) cos(ω(t − s))dω q(ω) power spectral density Stationary process (“infinite dense sum” of sinusoids) gt = ∞

−∞

|c(ω)| cos(ωt + ∠c(ω))dω c(ω) Fourier transform

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 7 / 15

slide-24
SLIDE 24

Example 1: Stationary kernels

Choice: p(α, ω) = δ(α)q(ω)

! α p(α; !)

Stationary kernel K(t − s) = 1 2 ∞

−∞

q(ω) cos(ω(t − s))dω q(ω) power spectral density Stationary process (“infinite dense sum” of sinusoids) gt = ∞

−∞

|c(ω)| cos(ωt + ∠c(ω))dω c(ω) Fourier transform

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 7 / 15

slide-25
SLIDE 25

Example 1: Stationary kernels

Choice: p(α, ω) = δ(α)q(ω)

! α p(α; !)

Stationary kernel K(t − s) = 1 2 ∞

−∞

q(ω) cos(ω(t − s))dω q(ω) power spectral density Stationary process (“infinite dense sum” of sinusoids) gt = ∞

−∞

|c(ω)| cos(ωt + ∠c(ω))dω c(ω) Fourier transform

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 7 / 15

slide-26
SLIDE 26

Example 1: Stationary kernels (cont’d)

The power spectral density is even: q(ω) = ˜ q(ω) + ˜ q(−ω) 2 Shape ˜ q(ω) Kernel Laplacian

˜ qL(ω) =

β/2 π[(ω−ω0)2+(β/2)2]

KL(t − s) = e− β

2 |t−s| cos(ω0(t − s))

Cauchy

˜ qC (ω) =

1 2β e − |ω−ω0| β

KC (t − s) =

1 1+β2(t−s)2 cos(ω0(t − s))

Gaussian

˜ qG (ω) =

1

  • 2πβ2 e

− (ω−ω0)2 2β2

KG (t − s) = e− β2(t−s)2

2

cos(ω0(t − s))

ω0 center frequency β bandwidth

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 8 / 15

slide-27
SLIDE 27

Example 1: Stationary kernels (cont’d)

The power spectral density is even: q(ω) = ˜ q(ω) + ˜ q(−ω) 2 Shape ˜ q(ω) Kernel Laplacian

˜ qL(ω) =

β/2 π[(ω−ω0)2+(β/2)2]

KL(t − s) = e− β

2 |t−s| cos(ω0(t − s))

Cauchy

˜ qC (ω) =

1 2β e − |ω−ω0| β

KC (t − s) =

1 1+β2(t−s)2 cos(ω0(t − s))

Gaussian

˜ qG (ω) =

1

  • 2πβ2 e

− (ω−ω0)2 2β2

KG (t − s) = e− β2(t−s)2

2

cos(ω0(t − s))

ω0 center frequency β bandwidth

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 8 / 15

slide-28
SLIDE 28

Example 1: Stationary kernels (cont’d)

The power spectral density is even: q(ω) = ˜ q(ω) + ˜ q(−ω) 2 Shape ˜ q(ω) Kernel Laplacian

˜ qL(ω) =

β/2 π[(ω−ω0)2+(β/2)2]

KL(t − s) = e− β

2 |t−s| cos(ω0(t − s))

Cauchy

˜ qC (ω) =

1 2β e − |ω−ω0| β

KC (t − s) =

1 1+β2(t−s)2 cos(ω0(t − s))

Gaussian

˜ qG (ω) =

1

  • 2πβ2 e

− (ω−ω0)2 2β2

KG (t − s) = e− β2(t−s)2

2

cos(ω0(t − s))

ω0 center frequency β bandwidth

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 8 / 15

slide-29
SLIDE 29

Example 1: Stationary kernels (cont’d)

The power spectral density is even: q(ω) = ˜ q(ω) + ˜ q(−ω) 2 Shape ˜ q(ω) Kernel Laplacian

˜ qL(ω) =

β/2 π[(ω−ω0)2+(β/2)2]

KL(t − s) = e− β

2 |t−s| cos(ω0(t − s))

Cauchy

˜ qC (ω) =

1 2β e − |ω−ω0| β

KC (t − s) =

1 1+β2(t−s)2 cos(ω0(t − s))

Gaussian

˜ qG (ω) =

1

  • 2πβ2 e

− (ω−ω0)2 2β2

KG (t − s) = e− β2(t−s)2

2

cos(ω0(t − s))

14 16 18 20 22 24 26 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

ω0

˜ qL ˜ qC ˜ qG

ω0 center frequency β bandwidth

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 8 / 15

slide-30
SLIDE 30

Example 2: Locally stationary kernels

Choice: p(α, ω) = q1(α)q2(ω) Locally stationary kernel (Silverman, 1957) K(t, s) = K1 t + s 2

  • K2(t − s)

stationary kernel

Choice: q1(α) = δ(α − ¯ α) ECLS kernel K(t, s) = e−¯

α t+s

2 K2(t − s)

Examples of ECLS kernels (Chen-Ljung, 2015): stable-spline, dynamic-correlated, tuned-correlated...

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 9 / 15

slide-31
SLIDE 31

Example 2: Locally stationary kernels

Choice: p(α, ω) = q1(α)q2(ω) Locally stationary kernel (Silverman, 1957) K(t, s) = K1 t + s 2

  • K2(t − s)

stationary kernel

Choice: q1(α) = δ(α − ¯ α) ECLS kernel K(t, s) = e−¯

α t+s

2 K2(t − s)

Examples of ECLS kernels (Chen-Ljung, 2015): stable-spline, dynamic-correlated, tuned-correlated...

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 9 / 15

slide-32
SLIDE 32

Example 2: Locally stationary kernels

Choice: p(α, ω) = q1(α)q2(ω) Locally stationary kernel (Silverman, 1957) K(t, s) = K1 t + s 2

  • K2(t − s)

stationary kernel

Choice: q1(α) = δ(α − ¯ α) ECLS kernel K(t, s) = e−¯

α t+s

2 K2(t − s)

Examples of ECLS kernels (Chen-Ljung, 2015): stable-spline, dynamic-correlated, tuned-correlated...

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 9 / 15

slide-33
SLIDE 33

Example 2: Locally stationary kernels

Choice: p(α, ω) = q1(α)q2(ω) Locally stationary kernel (Silverman, 1957) K(t, s) = K1 t + s 2

  • K2(t − s)

stationary kernel

Choice: q1(α) = δ(α − ¯ α) ECLS kernel K(t, s) = e−¯

α t+s

2 K2(t − s)

! α p(α; !) ¯ α

Examples of ECLS kernels (Chen-Ljung, 2015): stable-spline, dynamic-correlated, tuned-correlated...

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 9 / 15

slide-34
SLIDE 34

Example 2: Locally stationary kernels

Choice: p(α, ω) = q1(α)q2(ω) Locally stationary kernel (Silverman, 1957) K(t, s) = K1 t + s 2

  • K2(t − s)

stationary kernel

Choice: q1(α) = δ(α − ¯ α) ECLS kernel K(t, s) = e−¯

α t+s

2 K2(t − s)

! α p(α; !) ¯ α

Examples of ECLS kernels (Chen-Ljung, 2015): stable-spline, dynamic-correlated, tuned-correlated...

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 9 / 15

slide-35
SLIDE 35

Example 2: Locally stationary kernels

Choice: p(α, ω) = q1(α)q2(ω) Locally stationary kernel (Silverman, 1957) K(t, s) = K1 t + s 2

  • K2(t − s)

stationary kernel

Choice: q1(α) = δ(α − ¯ α) ECLS kernel K(t, s) = e−¯

α t+s

2 K2(t − s)

! α p(α; !) ¯ α

Examples of ECLS kernels (Chen-Ljung, 2015): stable-spline, dynamic-correlated, tuned-correlated...

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 9 / 15

slide-36
SLIDE 36

Example 2: Locally stationary kernels (cont’d)

Probability density function of a 2nd order stable model

Transfer function G(z) = 1 (z − ρ)(z − ¯ ρ) =

∞

  • k=1

gkz−k |ρ| < 1 ρ = xr + jxi pole of the model gk process with kernel K → gk with pdf p(xr, xi)

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 10 / 15

slide-37
SLIDE 37

Example 2: Locally stationary kernels (cont’d)

Probability density function of a 2nd order stable model

Transfer function G(z) = 1 (z − ρ)(z − ¯ ρ) =

∞

  • k=1

gkz−k |ρ| < 1 ρ = xr + jxi pole of the model gk process with kernel K → gk with pdf p(xr, xi)

−1 −0.8 −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

xr xi p(xr, xi)

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 10 / 15

slide-38
SLIDE 38

Example 2: Locally stationary kernels (cont’d)

−1 −0.5 0.5 1 0.2 0.4 0.6 0.8 1

Filtered

real imag −1 −0.5 0.5 1 0.2 0.4 0.6 0.8 1 real

Laplacian

imag −1 −0.5 0.5 1 0.2 0.4 0.6 0.8 1

Cauchy

real imag −1 −0.5 0.5 1 0.2 0.4 0.6 0.8 1 real

Gaussian

imag

Figure: pdf of a 2nd order stable model with ECLS kernel

¯ α = 0.9 ω0 = 3

4π

β small

14 16 18 20 22 24 26 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

ω0

˜ qL ˜ qC ˜ qG

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 11 / 15

slide-39
SLIDE 39

Example 2: Locally stationary kernels (cont’d)

−1 −0.5 0.5 1 0.2 0.4 0.6 0.8 1

Filtered

real imag −1 −0.5 0.5 1 0.2 0.4 0.6 0.8 1 real

Laplacian

imag −1 −0.5 0.5 1 0.2 0.4 0.6 0.8 1

Cauchy

real imag −1 −0.5 0.5 1 0.2 0.4 0.6 0.8 1 real

Gaussian

imag

Figure: pdf of a 2nd order stable model with ECLS kernel

¯ α = 0.9 ω0 = 3

4π

β large

14 16 18 20 22 24 26 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16

ω0

˜ qL ˜ qC ˜ qG

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 12 / 15

slide-40
SLIDE 40

Example 3: Integrated kernels

Choice: p(α, ω) = 1[αm,αM](α)

α/2 π[ω2+(α/2)2]

Integrated TC kernel (Pillonetto et. al.) K(t, s) = e−αm max{t,s} − e−αM max{t,s} max{t, s} Choice: p(α, ω) = 1[αm,αM](α)q(ω) Integrated kernel K(t, s) = 2e−αm t+s

2 − e−αM t+s 2

t + s K(t − s)

stationary kernel

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 13 / 15

slide-41
SLIDE 41

Example 3: Integrated kernels

Choice: p(α, ω) = 1[αm,αM](α)

α/2 π[ω2+(α/2)2]

Integrated TC kernel (Pillonetto et. al.) K(t, s) = e−αm max{t,s} − e−αM max{t,s} max{t, s} Choice: p(α, ω) = 1[αm,αM](α)q(ω) Integrated kernel K(t, s) = 2e−αm t+s

2 − e−αM t+s 2

t + s K(t − s)

stationary kernel

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 13 / 15

slide-42
SLIDE 42

Example 3: Integrated kernels

Choice: p(α, ω) = 1[αm,αM](α)

α/2 π[ω2+(α/2)2]

Integrated TC kernel (Pillonetto et. al.) K(t, s) = e−αm max{t,s} − e−αM max{t,s} max{t, s} Choice: p(α, ω) = 1[αm,αM](α)q(ω) Integrated kernel K(t, s) = 2e−αm t+s

2 − e−αM t+s 2

t + s K(t − s)

stationary kernel

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 13 / 15

slide-43
SLIDE 43

Example 3: Integrated kernels

Choice: p(α, ω) = 1[αm,αM](α)

α/2 π[ω2+(α/2)2]

Integrated TC kernel (Pillonetto et. al.) K(t, s) = e−αm max{t,s} − e−αM max{t,s} max{t, s} Choice: p(α, ω) = 1[αm,αM](α)q(ω) Integrated kernel K(t, s) = 2e−αm t+s

2 − e−αM t+s 2

t + s K(t − s)

stationary kernel

! α p(α; !) αM

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 13 / 15

slide-44
SLIDE 44

Example 3: Integrated kernels

Choice: p(α, ω) = 1[αm,αM](α)

α/2 π[ω2+(α/2)2]

Integrated TC kernel (Pillonetto et. al.) K(t, s) = e−αm max{t,s} − e−αM max{t,s} max{t, s} Choice: p(α, ω) = 1[αm,αM](α)q(ω) Integrated kernel K(t, s) = 2e−αm t+s

2 − e−αM t+s 2

t + s K(t − s)

stationary kernel

! α p(α; !) αM

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 13 / 15

slide-45
SLIDE 45

Example 3: Integrated kernels (cont’d)

−1 −0.5 0.5 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 real

Laplacian

imag −1 −0.5 0.5 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Cauchy

real imag −1 −0.5 0.5 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 real

Gaussian

imag

Figure: pdf of a 2nd order model with ECLS kernel

−1 −0.5 0.5 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 real

Laplacian

imag −1 −0.5 0.5 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 real

Cauchy

imag −1 −0.5 0.5 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 real

Gaussian

imag

Figure: pdf of a 2nd order model with INTEGRATED kernel

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 14 / 15

slide-46
SLIDE 46

Conclusions

Key points: Harmonic analysis for non-stationary Gaussian processes How the statistical power is distributed? generalized psd Relation between the generalized psd and the pdf of the process? Kernel design through the generalized psd

Thank you!

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 15 / 15

slide-47
SLIDE 47

Conclusions

Key points: Harmonic analysis for non-stationary Gaussian processes How the statistical power is distributed? generalized psd Relation between the generalized psd and the pdf of the process? Kernel design through the generalized psd

Thank you!

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 15 / 15

slide-48
SLIDE 48

Conclusions

Key points: Harmonic analysis for non-stationary Gaussian processes How the statistical power is distributed? generalized psd Relation between the generalized psd and the pdf of the process? Kernel design through the generalized psd

Thank you!

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 15 / 15

slide-49
SLIDE 49

Conclusions

Key points: Harmonic analysis for non-stationary Gaussian processes How the statistical power is distributed? generalized psd Relation between the generalized psd and the pdf of the process? Kernel design through the generalized psd

Thank you!

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 15 / 15

slide-50
SLIDE 50

Conclusions

Key points: Harmonic analysis for non-stationary Gaussian processes How the statistical power is distributed? generalized psd Relation between the generalized psd and the pdf of the process? Kernel design through the generalized psd

Thank you!

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 15 / 15

slide-51
SLIDE 51

Conclusions

Key points: Harmonic analysis for non-stationary Gaussian processes How the statistical power is distributed? generalized psd Relation between the generalized psd and the pdf of the process? Kernel design through the generalized psd

Thank you!

  • M. Zorzi

The harmonic analysis of kernel functions September 26th, 2016 15 / 15