[PPT] - Hawkes Processes with Stochastic Excitations Young Lee , Kar Wai PowerPoint Presentation

SLIDE 1

Hawkes Processes with Stochastic Excitations

Young Lee∗, Kar Wai Lim†, Cheng Soon Ong†

∗ National ICT of Australia & London School of Economics †National ICT of Australia & Australian National University Lee, Lim and Ong Stochastic Hawkes June 21, 2016 1 / 22

SLIDE 2

Outline

1 Motivation for Stochastic Hawkes 2 Simulation and Inference 3 Experimental Result 4 Summary

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 2 / 22

SLIDE 3

Motivation for Stochastic Hawkes

Background Simple point processes:

(Ti)i a sequence of non-negative random variables such that Ti < Ti+1. Also known as random times.

Counting processes:

Given simple point process (Ti)i N(t) =

i>0

1Ti ≤t is called the counting process associated with T.

Interarrival times:

The process ∆ defined by ∆i = Ti − Ti−1 is called the interarrival times associated with T.

Intensity process: The intensity process is defined as λ(t) = lim

h→0

1 hE[N(t + h) − N(t)|Ft]

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 3 / 22

SLIDE 4

Motivation for Stochastic Hawkes

Recap: Poisson → Hawkes → Stochastic Hawkes Nt as the number of arrivals or events of the process by time t. λ = const.(Poisson), does not take the history of events into account. However, if an arrival causes the intensity function to increase then the process is said to be self-exciting (Hawkes Process). Hawkes flavour: λ(t) = ˆ λ0(t) +

i:t>Ti

Y (Ti) ν(t − Ti), (1) where the function ν takes the form ν(z) = e−δz. ∃ different formulations for Y

1

Constant, Hawkes (1971), Hawkes & Oakes (1974)

2

Random excitations, Br´ emaud & Massouli´ e (2002), Dassios & Zhao (2013),

3

Stochastic differential equations.

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 4 / 22

SLIDE 5

Motivation for Stochastic Hawkes

Illustration of Stochastic Hawkes λ(t)

Note the variation of heights with Cov(Y5, Y6) = 0

Z10 = 1 Z20 = 1 Z32 = 1 T1 T2 T3 T4 T5 T6T7

Y5 Y6

Figure: A sample path of the intensity function λ(·).

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 5 / 22

SLIDE 6

Motivation for Stochastic Hawkes

Our model The intensity function λ(t) = ˆ λ0(t)

Base intensity

+

i:t>Ti

Y (Ti)

Contagion process/Levels of excitation

ν(t − Ti) where ˆ λ0 : R → R+ is a deterministic base intensity, Y is a stochastic process and ν : R+ → R+ conveys the positive influence of the past events Ti on the current value of the intensity process. Base intensity ˆ λ0 Contagion process / Levels of excitation (Yi)i=1,2,..,NT measure the impact of clustering of the event times We take ν to be the exponential kernel of the form ν(t) = e−δt.

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 6 / 22

SLIDE 7

Motivation for Stochastic Hawkes

Stochastic differential equations to describe evolution of Y Changes in the levels of excitation Y is assumed to satisfy Y· = · ˆ µ(t, Yt)dt + · ˆ σ(t, Yt)dBt where B is a standard Brownian motion and t ∈ [0, T] where T < ∞. Standing assumption: Yt > 0, ∀t ≥ 0. Geometric Brownian Motion (GBM): Exponential Langevin:

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 7 / 22

SLIDE 8

Motivation for Stochastic Hawkes

Two representations for Stochastic Hawkes Intensity based. λt = a + (λ0 − a)e−δt +

Nt

i: Ti<t

Yi e−δ(t−Ti) (2) Cluster based. Immigrants and offsprings. We say an event time Ti is an

1

immigrant if it is generated from the base intensity a + (λ0 − a)e−δt,

therwise

2

we say Ti is an offspring.

It is natural to introduce a variable that describes the specific process to which each event time Ti corresponds to. Zi0 = 1 if event i is an immigrant, Zij = 1 if event i is an offspring of j

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 8 / 22

SLIDE 9

Motivation for Stochastic Hawkes

Quick recap - Stochastic Hawkes λ(t)

Note the variation of heights with Cov(Y5, Y6) = 0

Z10 = 1 Z20 = 1 Z32 = 1 T1 T2 T3 T4 T5 T6T7

Y5 Y6

Figure: A sample path of the intensity function λ(·).

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 9 / 22

SLIDE 10

Simulation and Inference

Outline

1 Motivation for Stochastic Hawkes 2 Simulation and Inference 3 Experimental Result 4 Summary

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 10 / 22

SLIDE 11

Simulation and Inference

Simulation & Inference Simulation framework of Dassios & Zhao (2011) is adopted, Decompose the inter-arrival event times into two independent simpler random variables: S(1), S(2); Sj+1 is the inter-arrival time for the (j + 1)-th jump: Sj+1 = Tj+1 − Tj . Given the intensity function, we can derive the cumulative density function for Sj+1 as FSj+1(s) = 1 − exp

−
λT +

j − a

1 − e−δs δ − as

.

Decompose Sj+1 into S(1)

j+1 and S(2) j+1:

P(Sj+1 > s) = exp

−
λT +

j − a

1 − e−δs δ

× e−as

= P

S(1)

j+1 > s

× P
S(2)

j+1 > s

= P
min
S(1)

j+1, S(2) j+1

> s
.

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 11 / 22

SLIDE 12

Simulation and Inference

Simulation & Inference FS(1)

j+1(s) = P

S(1)

j+1 ≤ s

= 1 − exp
−
λT +

j − a

1 − e−δs δ

,

FS(2)

j+1(s) = P

S(2)

j+1 ≤ s

= 1 − e−as.

for 0 ≤ s < ∞. To simulate Sj+1, we simply need to independently simulate both S(1)

j+1 and S(2) j+1. Simulating S(2) j+1 is trivial since S(2) j+1 follows an exponential

distribution with rate parameter a. To simulate S(1)

j+1, we use the inverse CDF

approach: S∗

j+1 = −1

δ ln

1 + δ ln(v)

λT +

j − a

if exp
−

λT +

j − a

δ

≤ v < 1,

we discard S∗

j+1 otherwise, that is, v < exp

−

λT+

j −a

δ

(this corresponds to the

defective part), where v is simulated from a standard uniform distribution V ∼ U(0, 1).

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 12 / 22

SLIDE 13

Simulation and Inference

Simulation & Inference Inference - Hybrid of MH and Gibbs The employment of branching representation enables the use of Gibbs sampling to learn Z,µ and σ, Other parameters a, λ0, k and Y are learned with the vanilla MH algorithm.

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 13 / 22

SLIDE 14

Experimental Result

Outline

1 Motivation for Stochastic Hawkes 2 Simulation and Inference 3 Experimental Result 4 Summary

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 14 / 22

SLIDE 15

Experimental Result

Synthetic validation Inference algorithm is first tested on synthetic data generated from Stochastic Hawkes Event times are generated assuming Y follows iid Gamma, GBM or Exponential Langevin, Performing experiments to recalibrate the parameters and subsequently sample the posterior Y gives the following interesting results

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 15 / 22

SLIDE 16

Experimental Result

Inference learns Gamma ground truth

Ground truth Y Gamma

50 100 150 200 250 300 350 400 450 0.5 1 1.5 2 2.5 Yt Time t 50 100 150 200 250 300 350 400 450 0.5 1 1.5 2 2.5 Time t Yt

GBM Exp Langevin

50 100 150 200 250 300 350 400 450 0.5 1 1.5 2 2.5 Yt Time t 50 100 150 200 250 300 350 400 450 0.5 1 1.5 2 2.5 Yt Time t

All seems good.

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 16 / 22

SLIDE 17

Experimental Result

Inference learns G.B.M.

Ground truth Y Gamma

50 100 150 200 250 300 350 400 450 0.5 1 1.5 2 2.5 Yt Time t 50 100 150 200 250 300 350 400 450 0.5 1 1.5 2 2.5 Yt Time t

GBM Exp Langevin

50 100 150 200 250 300 350 400 450 0.5 1 1.5 2 2.5 Yt Time t 50 100 150 200 250 300 350 400 450 0.5 1 1.5 2 2.5 Yt Time t

iid Gamma fails, but a posteriori trying to capture a downward trend. GBM learns well. Exp Langevin too!!

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 17 / 22

SLIDE 18

Experimental Result

Japanese Earthquakes Data (Di Giacomo et. al 2015) Plot of Y vs time:

50 100 150 200 250 0.5 1 1.5 2 2.5 3 3.5 4 Time t Yt Japanese Earthquakes 5 10 15 20 −0.4 −0.2 0.2 0.4 0.6 0.8 1 Lag Sample Autocorrelation Sample Autocorrelation Function

Y might not be iid as earthquake occurrence tend to be correlated. Geophysical TS are frequently autocorrelated because of inertia or carryover processes in physical system. Autocorrelations should be near-zero for randomness, else will be significantly non-zero

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 18 / 22

SLIDE 19

Experimental Result

Autocorrelation functions - SDEs retrieve correlated Y

Ground truth Y Gamma

5 10 15 20 −0.4 −0.2 0.2 0.4 0.6 0.8 1 Lag Sample Autocorrelation Sample Autocorrelation Function 5 10 15 20 −0.2 0.2 0.4 0.6 0.8 Lag Sample Autocorrelation Sample Autocorrelation Function

GBM Exp Langevin

5 10 15 20 −0.2 0.2 0.4 0.6 0.8 Lag Sample Autocorrelation Sample Autocorrelation Function 5 10 15 20 −0.2 0.2 0.4 0.6 0.8 Lag Sample Autocorrelation Sample Autocorrelation Function

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 19 / 22

SLIDE 20

Experimental Result

Prediction - Stochastic Hawkes performs reasonable well

Table: Prediction of number of Earthquakes on Test Set. Result is averaged over 5 runs. Model Predicted Observed Diff Poisson Process 62.80 ± 0.00 73.00

10.20 ± 0.00

Classical Hawkes 61.13 ± 2.80 73.00

11.87 ± 2.80

Stochastic Hawkes (GBM) 64.38 ± 6.82 73.00

8.62 ± 6.82

Stochastic Hawkes (Langevin) 63.54 ± 4.09 73.00

9.46 ± 4.09

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 20 / 22

SLIDE 21

Summary

Outline

1 Motivation for Stochastic Hawkes 2 Simulation and Inference 3 Experimental Result 4 Summary

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 21 / 22

SLIDE 22

Summary

Summary Motivation for Stochastic Hawkes λt = a + (λ0 − a)e−δt +

Nt

i: Ti<t

Yi e−δ(t−Ti)

1

Constant

2

Independent and identically distributed

3

Stochastic differential equations

Simulation and Inference - with Z Experiments - Synthetic / Earthquake Poster #32, 3pm - 7pm later today

Lee, Lim and Ong Stochastic Hawkes June 21, 2016 22 / 22