Time series causality inference using the Phase Slope Index. Florin - - PowerPoint PPT Presentation

Popescu NIPS 2009 1

Time series causality inference using the Phase Slope Index.

Florin Popescu Guido Nolte Fraunhofer Institute FIRST, Berlin

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 2

Introduction

 Linear time series analysis techniques can be useful in analyzing data that is

actually generated by nonlin inear ear stochastic processes (i.e. in the real world).

 Linear time series analysis can be conducted in the time domain (e.g.

autoregressive models) or in the frequency domain (e.g. discrete Fourier transform, coherency among spectra) – theoretically both approaches are equivalent but numerically they are not. Causal estimation in time domain (AR): Granger 1973, Kaminski Blinowska 1991, Schreiber 2000, Rosenblum & Pikovsky 2001. Frequency domain method: Phase Slope Index (Nolte et al. 2008, Nolte et al. 2009) . Connection: partially directed coherence (Baccala & Sameshima 1998, 2001).

 Separating correlation

relation from causation sation is hard, even if the data is time-labeled. There can be correlations among non-interacting time-series variables.

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 3

Outline

 Overview of different types of data generating processes (DGPs), which are

stochastic generative models of time series

 Highlight causality assessment challenges in neuroscience and economics.  AR estimation challenges for covariate innovations processes (needed for GC).  PSI - Phase Slope Index  PSI and AR results for bi-variate simulations available on Causality

Workbench.

 Structural causality estimation in multivariate time series.

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 4

DGP: Data Generating Process

Data Generating Process

y(t) y(t-1) y(t-2)

• DGPs are abstractions of real-world

dynamic processes which generate data: not necessarily are they regressive, recursive or stochastic, but are more powerful when they are.

• They can be inferred

ed from data directly

• r by bottom
• m-up

up modeling of the underlying physical /social processes (in neuroscience, economics very hard)

DGP Symbolic representation

Mini-Symposium on Time Series Causality

y(t) u(t)

Popescu NIPS 2009 5

y(t) u(t) y(t-1) y(t-2)

Stochastic DGP

Data Generating Process DGP Symbolic representation

• If the DGP is stochastic and noise in

an input it is generally called innovati tion

• ns

s process cess and it is independently distributed if it is independently distributed.

• If, also then the system

is station

• nary.

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 6

y(t) u(t) y(t-1) y(t-2)

DGP equivalence

Equivalence: DGP Symbolic representation

2 DGPs are output ut equivalent if, for all t : DGPs are stochastic hasticall ally equivalent if, for all t : y1(t) u1 (t)

z Cano nonical nical representation (non-unique)

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 7

DGP variations

Potential DGP ‘upgrades’ DGP Symbolic representation

• covariat

ariate e or mixed innovati tion

• ns
• endogen

enou

• us/

s/exogenous inputs

• cointe

tegration ration u1 (t) u2 (t)

*

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 8

DGP variations

Data Generating Process DGP Symbolic representation

• covariat

ariate e or mixed innovati tion

• ns
• endogen

enou

• us/

s/exogenous inputs some inputs are stochastic but observable le, or non-stochastic, or excluded from potential effects

• co

co-inte ntegratio ration y1(t) u1 (t)

z

d(t) z (t)

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 9

DGP variations

Potential DGP ‘upgrades’ DGP Symbolic representation

• covariat

ariate e or mixed innovati tion

• ns
• endogen

enou

• us/

s/exogenous inputs some inputs are stochastic but observable le, or simply non-stochastic

• co

co-inte ntegratio ration Some states are simple dynamic transformations of i.i.d processes -this can be taken into account y1(t) u1 (t)

z

d(t) z (t)

z

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 10

Structural / G - Causality

G - Causality DGP Symbolic representation z z z

y1(t) u1(t) y2 (t) u2(t) y2 (t) u2,0 (t)

Mini-Symposium on Time Series Causality

Granger causality inference requires derivation of a predictive model (can of worms…)

Popescu NIPS 2009 11

Structural / G - Causality

G-causality DGP Symbolic representation

• G-causality is inferred by comparing conditional

entropy in competing structural models

z z z z

y1(t) u1 (t) u2 (t) y2 (t)

1 2 12 1  2 12

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 12

Structural / G - Causality

Covariate innovations? DGP Symbolic representation

• In many instances it is reasonable to

assume that the innovations process is

• covariate. For example: yearly weather

variability and historical shocks on aggregate indicators.

• Also possible is that other

unobservable factors actually provide root causes for correlations among innovations processes. z z

y1(t) u1 (t) y2 (t) u2 (t)

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 13

Structural / G - Causality

Mixed outputs: EEG DGP Symbolic representation

• In some instances it is the physical process of
• bservation that separates us from the time-series
• f interest. For example cortical sources and

scalp based sensors (the mixing problem).

z

x2 (t)

z

x1(t) y2 (t) y1(t)

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 14

Structural / G - Causality

Stochastic equivalence DGP Symbolic representation

• It is also possible that there is both a non-

diagonal observation matrix and covariate noise but these situations correspond to stochastically equivalent DGPs and cannot be disambiguated without further assumptions

z z

y2 (t) y1(t)

Covariate innovations Mixed output

R is a rotation matrix S is a diagonal (scaling) matrix

*

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 15

Structural / G - Causality

Stochastic equivalence DGP Symbolic representation

• It is also possible that there is both a non-

diagonal observation matrix and covariate noise but these situations correspond to stochastically equivalent DGPs and cannot be disambiguated without further assumptions

z z

y2 (t) y1(t)

Covariate innovations Mixed output

R is a rotation matrix S is a diagonal (scaling) matrix

*

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 16

Structural / G - Causality

Noise covariance estimation DGP Symbolic representation

• Instantaneous mixing / innovations covariance

can be used to establish „source‟ causality (Moneta 2008), (to follow!)

• If a triangular structure is imposed on the

instantaneous „mixing‟ matrix of a linear SVAR the estimate of the equivalent noise covariance is unbiased (Popescu, 2008)

z z

y2 (t) y1(t)

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 17

Structural / G - Causality

Data Generating Process DGP Symbolic representation

,0 , , 1 K n U n U i n i U U U n i

x A x A x b S e

 

   



,0, , U p q

A if q p  

strictly upper diagonal:

Can be solved by standard 2-norm linear regression Strictly upper diagonal means resulting residuals are not correlated.

1 1 1 ,0 , , 1

( )

K U U n U U i n i U U U n i

S I A x S A x S b e

    

   

  

1 1 , , 1 1 1 1 1 1 , , 1 K T U U U n U U i n i U U U n i K T T T T U U U U U U i n i U U U U U U U n i

U V x S A x S b e V x U S A x U S b U e

          

         

 

1 1 1 1 1 , 1 K T T n U U U U i U n i U U U U U n i

y U S A V y U S b e

      

     



T n n

y V x 

Mixed output Zero-lag AR system

z z

y2 (t) y1(t)

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 18

Phase slope index

z z z z Basic principle: mixing does not affect the imaginary part of the complex coherency of a multivariate time series (Nolte 2004)

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 19

Phase slope index

Let us consider the case of a dynamically interacting system with correlated noise observations z z

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 20

Phase slope index

Let us consider the case of a dynamically interacting system with correlated noise observations. Relative influence of covariate noise? z z

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 21

• 1
• 0.5

0.5 1

• 1
• 0.8
• 0.6
• 0.4
• 0.2

0.2 0.4 0.6 0.8 1 re(coherency) im(coherency)

Phase Slope Index

fnyquist/2 fnyqu

fnyquist Fnyquist 0.5fnyquist

Complex coherency Cij

ij is

calculated from the complex spectral density Sij

ij



Mini-Symposium on Time Series Causality

Re(coherency) Im(coherency)

Popescu NIPS 2009 22

Phase slope index

• 1
• 0.5

0.5 1

• 1
• 0.8
• 0.6
• 0.4
• 0.2

0.2 0.4 0.6 0.8 1 re(coherency) im(coherency)

• 1
• 0.5

0.5 1

• 1
• 0.8
• 0.6
• 0.4
• 0.2

0.2 0.4 0.6 0.8 1 re(coherency) im(coherency)

Mini-Symposium on Time Series Causality

Re(coherency) Re(coherency) Im(coherency) Im(coherency)

Popescu NIPS 2009 23

PSI: benchmark data

z z z z z z sign()abs(1-)

• 1

1 Causality workbench: 1000 simulations of linear/nonlinear systems

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 24

G-Causality Results

Mini-Symposium on Time Series Causality M=1 M=2 M=3 M=4

• 10th order

AR models used

• M is the

polynomial degree of the nonlinear coupling term

Popescu NIPS 2009 25

PSI Results

Mini-Symposium on Time Series Causality M=1 M=2 M=3 M=4

Popescu NIPS 2009 26

Conclusions

 PSI does offer some advantages of Granger causality or model based method

determination.

 PSI and related extensions can give statistical estimates of causality,

dependence and non-causality and is conservative.

 Model based methods are limited by limitations in modeling technique: too

few parameters may miss interactions, too many will over-fit, covariate innovations and AR coefficients are difficult to co-estimate.

 Future developments require DAG/ acyclic causal graph inference in

multivariate time series.

 Complex non-stationarities not yet addressed.

Mini-Symposium on Time Series Causality

Popescu NIPS 2009 27

Acknowledgments

PSI: I:

• G. Nolte 1, A. Ziehe 1, N. Krämer 2, K.R.-Müller 2, Vadim V. Nikulin 3, Alois

Schlögl 4, Tom Brismar 5

1 Fraunhofer Institute FIRST, Berlin , 2 TU Berlin, 3 Charite Klinikum Berlin, 4 TU Graz, 5 Karolinska Institutet, Stockholm

PSI I RE REFERENC RENCES: S:

Nolte G, Ziehe A, Nikulin VV, Schlögl A, Krämer N, Brismar T, Müller KR. “Robustly estimating the flow direction of information in complex physical systems.” Physical Review Letters 00(23):234101 . 2008. Nolte G, Ziehe A, Krämer N, Popescu F, and Müller KR, "Comparison of Granger causality and phase slope index," Journal of Machine Learning Research Workshop and Conference Proceedings, in press., 2009. Mini-Symposium on Time Series Causality