Matrix-correlated random variables: A statistical physics and signal - - PowerPoint PPT Presentation

▶
matrix correlated random variables a statistical physics
SMART_READER_LITE
LIVE PREVIEW

Matrix-correlated random variables: A statistical physics and signal - - PowerPoint PPT Presentation

Mar 31, 2023 122 likes •476 views

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum Matrix-correlated random variables: A statistical physics and signal processing duet Florian Angeletti Work in collaboration with Hugo Touchette,

slide-1
SLIDE 1

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Matrix-correlated random variables: A statistical physics and signal processing duet

Florian Angeletti Work in collaboration with Hugo Touchette, Patrice Abry and Eric Bertin. 4 December 2015

slide-2
SLIDE 2

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Random vectors

Random vectors in signal processing Joint probability density : P (x1, . . . , xn) i.i.d.: P (x1, . . . , xn) = f (x1) . . . f (xn) generalization to non-i.i.d. ?

slide-3
SLIDE 3

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Random vectors

Random vectors in signal processing Joint probability density : P (x1, . . . , xn) i.i.d.: P (x1, . . . , xn) = f (x1) . . . f (xn) generalization to non-i.i.d. ? Out-of-equilibrium physics Asymmetric Simple Exclusion Process model [Derrida et al., J.

  • Phys. A, 1993]

p(x1, . . . , xn) ∝ R(x1) . . . R(xn): f scalar ⇒ R matrix Preserved product structure. Signal processing application?

slide-4
SLIDE 4

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Matrix-correlated random variables

p(x1, . . . , xn) = L (R(x1) . . . R(xn)) L (En) linear form L: L(M) = tr(ATM)

A: d × d positive matrix

R(x): d × d positive matrix function

structure matrix E =

  • R

R(x)dx probability density function matrix Ri,j(x) = Ei,jPi,j(x)

d > 1: Non-commutativity = ⇒ Correlation

slide-5
SLIDE 5

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Objectives

Mathematical model p(x1, . . . , xn) ≈ R(x1) · · · R(xn) Study the statistical properties of theses models Hidden Markov model representation Topology induces correlation Large deviation functions Limit distributions for the sums Limit distributions for the extremes Signal processing applications?

slide-6
SLIDE 6

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Correlation

Product structure: p(x1, . . . , xn) = L(R(x1)...R(xn))

L(En)

Moment matrix: Q (q) =

  • xqR(x)dx

E

  • X p

k

  • = L
  • Ek−1Q (p) En−k

L (En) E [XkXl] = L

  • Ek−1Q (1) El−k−1Q (1) En−l

L (En) E [XkXlXm] = L

  • Ek−1Q (1) El−k−1Q (1) Em−l−1Q (1) En−m

L (En) . . .

slide-7
SLIDE 7

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Stationarity

Translation invariance: p(Xk1 = x1, . . . , Xkl = xl) = p(Xc+k1 = x1, . . . , Xc+kl = xl) Sufficient condition [AT, E] = ATE − EAT = 0 ∀M, L(ME) = L(EM) p(Xk = x) =

L(R(x)En−1) L(En)

p(Xk = x, Xl = y) =

L(R(x)El−k−1R(y)En−|l−k|−1) L(En)

p(Xk = x, Xl = y, Xm = z) =

L(R(x)El−k−1R(y)Em−l−1R(z)En−|m−k|−1) L(En)

slide-8
SLIDE 8

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Numerical generation

p(x1, . . . , xn) = L (R(x1) . . . R(xn)) L (En) How do we generate a random vector X for a given triple (A, E, P)? Expand the matrix product L (En) p(x1, . . . , xn) =

  • Γ∈{1,...,d}n+1

AΓ1,Γn+1EΓ1,Γ2PΓ1,Γ2(x1) . . . EΓn,Γn+1PΓn,Γn+1(xn) p(x1, . . . , xn) =

  • Γ

P(Γ)P(X|Γ) Γ, hidden Markov chain

slide-9
SLIDE 9

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Hidden Markov Model representation

Hidden Markov Chain p(Γ) = AΓ1,Γn+1 L (En)

  • k

EΓk,Γk+1 Conditional pdf (X|Γ) p(Xk = x|Γ) = PΓk,Γk+1(x) E non-stochastic = ⇒ Non-homogeneous markov chain Specific non-homogeneous Hidden Markov model:

Hidden Markov Model Matrix representation

slide-10
SLIDE 10

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Dual representation

Matrix representation Algebraic properties Statistical properties computation Hidden Markov Model 2-layer model: correlated layer + independant layer Efficient synthesis

slide-11
SLIDE 11

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Correlation and Jordan decomposition

E [XkXl] = L

  • Ek−1Q (1) El−k−1Q (1) En−l

L (En) The dependency structure of X depends on the behavior of En λc eigenvalues of E ordered by their real parts ℜ(λ1)ℜ(λ2) > · · · > λr Jm,p Jordan block associated with eigeivalue λm E = B−1    J1,1 ... Jm,p    B, Jm,s =          λm 1 · · · ... ... ... . . . . . . ... ... ... . . . ... ... 1 · · · · · · λm         

slide-12
SLIDE 12

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Dependency structure

Case 1: Short-range correlation λ2 exists: E [XkXl] − E [Xk] E [Xl] ≈ αm λm

λ1 |k−l|

25 50 75 100 k −0.4 0.0 0.4 0.8 1.2 Corr(1, k)

Case 2: Constant correlation More than one block J1,s: Constant correlation term Case 3: Long-range correlation J1,s with size p > 1:

E [XkXl] − E [Xk] E [Xl] ≈ P( k

n, k−l n , l n),

P ∈ R[X, Y , Z]

15 30 45 k 20 40 l

slide-13
SLIDE 13

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Short-range correlation: Ergodic chain

E irreducible ⇐ ⇒ Γ ergodic Irreducible matrix E ⇐ ⇒ G(E) is strongly connected Short-range correlation

slide-14
SLIDE 14

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Constant correlation: Identity E

Disconnected components: E =    1 ... 1    The chain Γ is trapped inside its starting state Constant correlation:

E [XkXl] − E [Xk] E [Xl] =

L(Q(1)2)−L(Q(1))2 L(E)

slide-15
SLIDE 15

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Long-range correlation: Linear irreducible E

Irreversible transitions:

n

n-1

1

E =    1 ǫ ... ... 1    The chain Γ can only stay in its current state or jump to the next All chains with a non-zero probability and the same starting and ending points are equiprobable Polynomial correlation: E [XkXl] ≈

  • r+s+t=d−1

cr,s,t k n r l − k n s 1 − l n t

slide-16
SLIDE 16

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

General shape of E

E =    I1 ∗ Tk,l ... ∗ Ir   

1 2 3 4 5 8 6 7 12 11 17 9 10 26 13 14 15 16 24 18 19 21 20 22 23 25

Irreducible blocks Ik Irreversibles transitions Tk,l Correlation: Mixture of short-range, constant and long-range correlations

slide-17
SLIDE 17

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Summary

Short-range correlation = ⇒ Strongly connected component

  • f size s > 1

More than one weakly connected component = ⇒ Constant correlation Polynomial correlation = ⇒ More than one strongly connected component Necessary but non sufficient conditions

slide-18
SLIDE 18

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Synthesis

p(x1, . . . , xn) = L (R(x1) . . . R(xn)) L (En) How to choose d ? E ? P ? A ?

slide-19
SLIDE 19

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Constraints

Classical constraints Marginal distribution: PS Autocovariance: c1,1 ≡ E [X0Xt] − E [X0] E [Xt] Higher-order dependencies: cq1,q2(t) ≡ E

  • X q1

0 X q2 t

  • − E
  • X q1
  • E
  • X q2

t

  • Limitations: sum of r expoinential time scales θk with

amplitudes β(q1, q2) cq1,q2(t) =

r

  • k=1

Re

  • β(q1, q2)kθt

k

slide-20
SLIDE 20

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Choice of d, A, E, P

A =    1 · · · 1 . . . . . . 1 · · · 1    , Jd =    1 ... ... 1    , E =

  • k

αkJk

d

Stationnarity: [AT, E] = 0 Dependencies: α = F(˜ θ) Objectives ⇒ Free parameters r ⇒ d θ ⇒ α β ⇒ M (q) PS ⇒ P

slide-21
SLIDE 21

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Stationary time series examples

Realisation Marginal Correlation

  • Sq. Corr.
2000 4000 6000 8000 10000 −10 −5 5 10 −6 −4 −2 2 4 6 0.1 0.2 0.3 0.4 0.5 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4

Target

−6 −4 −2 2 4 6 0.1 0.2 0.3 0.4 0.5 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4
slide-22
SLIDE 22

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Stationary time series examples

Realisation Marginal Correlation

  • Sq. Corr.

Target

−6 −4 −2 2 4 6 0.1 0.2 0.3 0.4 0.5 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4 2000 4000 6000 8000 10000 −10 −8 −6 −4 −2 2 4 6 8 −6 −4 −2 2 4 6 0.1 0.2 0.3 0.4 0.5 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4
slide-23
SLIDE 23

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Stationary time series examples

Realisation Marginal Correlation

  • Sq. Corr.
2000 4000 6000 8000 10000 −10 −5 5 10 −6 −4 −2 2 4 6 0.1 0.2 0.3 0.4 0.5 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4

Target

−6 −4 −2 2 4 6 0.1 0.2 0.3 0.4 0.5 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4 2000 4000 6000 8000 10000 −10 −8 −6 −4 −2 2 4 6 8 −6 −4 −2 2 4 6 0.1 0.2 0.3 0.4 0.5 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4 50 100 150 200 250 300 −0.1 0.1 0.2 0.3 0.4
slide-24
SLIDE 24

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Random vector examples

d = 3 Generalization de la structure en produit de matrice p(x1, x2, x3) = L (R1(x1)R2(x2)R3(x3)) L (En) Distribution marginale choisies a priori :

p1 gaussian p2 gamma distribution α = 2 p3 exponential distribution

Inter-covariance E [XiXj] prescribed Distribution partielle bivari´ ee : p1,2 p2,3 p1,3 X

−3 −2 −1 1 2 3 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 −3 −2 −1 1 2 3 1 2 3 4 5 6
slide-25
SLIDE 25

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

p1,2 p2,3 p1,3 X

−3 −2 −1 1 2 3 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 −3 −2 −1 1 2 3 1 2 3 4 5 6
slide-26
SLIDE 26

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

p1,2 p2,3 p1,3 Y

−3 −2 −1 1 2 3 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 −3 −2 −1 1 2 3 1 2 3 4 5 6

X

−3 −2 −1 1 2 3 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 −3 −2 −1 1 2 3 1 2 3 4 5 6

X, Y Same marginal Same correlation

  • Sq. corr. distinct
slide-27
SLIDE 27

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

p1,2 p2,3 p1,3 Y

−3 −2 −1 1 2 3 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 −3 −2 −1 1 2 3 1 2 3 4 5 6

X

−3 −2 −1 1 2 3 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 −3 −2 −1 1 2 3 1 2 3 4 5 6

Z

−3 −2 −1 1 2 3 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 −3 −2 −1 1 2 3 1 2 3 4 5 6

X, Y Same marginal Same correlation

  • Sq. corr. distinct

(joint pdf distinct) X, Z Same marginals Same correlations Same sq. corr. joint pdf distincts

slide-28
SLIDE 28

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Precise control over dependencies structure Synthesis: both stationary time series and random vectors

slide-29
SLIDE 29

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Random vector sum

Sum S(X) = 1 n

n

  • i=1

Xi Correlated random variables Law of large numbers? Central limit theorem? Large deviations? Two paths: Hidden Markov chain representation Matrix representation

slide-30
SLIDE 30

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Hidden Markov path

νi,j fraction of (i → j)-transition: ν = card{k/Γk = i, Γk+1 = j} n

  • i,j

S(X|Γ): sum of sums of i.i.d. random variables: S(X|Γ) =

  • i,j

nνi,j

  • k=1

(Xk|i, j) ≡ S(X|ν) Standard convergence theorem (law of large numbers or central limit theorem )

slide-31
SLIDE 31

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Layer combination

p(S(X) = s) =

  • ν

p(ν)p(S(X|ν) = s) Limit distribution for S(X|ν) p(S(X|ν) = s)

+

Limit distribution for ν p(ν)

⇓

Limit distribution for S(X) p(S(X) = s)

slide-32
SLIDE 32

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Distribution of ν

How ν is distributed? Difficulty: Γ non-homogeneous Markov chain. Three important subclasses: Irreducible E (short-range correlation)

ν converges towards a dirac distribution

Identity E (constant correlation)

ν converges towards a discrete mixture of dirac distributions

Linear irreversible E (long-range polynomial correlation)

ν converges towards a uniform distribution on a d-simplex

slide-33
SLIDE 33

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Limits laws for core examples

Irreducible E (short-range correlations):

Standard limit laws + = ⇒

Identity E (constant correlation):

Discrete mixture of standard limit laws: + = ⇒

Linear irreversible E (long-range correlation):

Continuous mixture of limit laws + = ⇒

slide-34
SLIDE 34

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

General case

1 2 3 4 5 8 6 7 12 11 17 9 10 26 13 14 15 16 24 18 19 21 20 22 23 25

Combinations of three core behaviors

Irreducible blocks: Fast convergence to the stationary state : dirac distribution Separated componentes : discrete mixture Irreversible transitions: continuous mixture

Limit laws :

Discrete mixture of continuous mixture of standard limits distributions

slide-35
SLIDE 35

Introduction Duality Statistical properties Random vectors synthesis Limit laws for the sum

Conclusion

Three kind of correlation:

Exponential short-range correlation Constant correlation Polynomial long-range correlation

Synthesis of stationary time series with controlled correlations. Extension of the law of large numbers and the central limit theorems:

Long-range correlation : Continuous and discrete mixture of standard limit laws

Perspective Extreme statistics Physical model Infinite dimension, higher tensor order