Last Time Similarity search Euclidean distance Time-warping - - PowerPoint PPT Presentation

▶
last time
SMART_READER_LITE
LIVE PREVIEW

Last Time Similarity search Euclidean distance Time-warping - - PowerPoint PPT Presentation

Dec 10, 2023 45 likes •712 views

CSE 6242 / CX 4242 Time Series Nonlinear Forecasting; Visualization; Applications Duen Horng (Polo) Chau Georgia Tech Some lectures are partly based on materials by Professors Guy Lebanon, Jeffrey Heer, John Stasko, Christos Faloutsos, Le

slide-1
SLIDE 1

Time Series

Nonlinear Forecasting; Visualization; Applications

CSE 6242 / CX 4242 Duen Horng (Polo) Chau
 Georgia Tech

Some lectures are partly based on materials by 
 Professors Guy Lebanon, Jeffrey Heer, John Stasko, Christos Faloutsos, Le Song

slide-2
SLIDE 2

Last Time

Similarity search

  • Euclidean distance
  • Time-warping

Linear Forecasting

  • AR (Auto Regression) methodology
  • RLS (Recursive Least Square) 


= fast, incremental least square

2

slide-3
SLIDE 3

This Time

Linear Forecasting

  • Co-evolving time sequences

Non-linear forecasting

  • Lag-plots + k-NN

Visualization and Applications

3

slide-4
SLIDE 4

Co-Evolving Time Sequences

  • Given: A set of correlated time sequences
  • Forecast ‘Repeated(t)’

Number of packets

23 45 68 90

Time Tick

1 4 6 9 11

sent lost repeated

??

slide-5
SLIDE 5

Solution:

Q: what should we do?

slide-6
SLIDE 6

Solution:

Least Squares, with

  • Dep. Variable: Repeated(t)
  • Indep. Variables:
  • Sent(t-1) … Sent(t-w);
  • Lost(t-1) …Lost(t-w);
  • Repeated(t-1), Repeated(t-w)
  • (named: ‘MUSCLES’ [Yi+00])

Number of packets

23 45 68 90

Time Tick

1 4 6 9 11

sent lost repeated

slide-7
SLIDE 7

Forecasting - Outline

  • Auto-regression
  • Least Squares; recursive least squares
  • Co-evolving time sequences
  • Examples
  • Conclusions
slide-8
SLIDE 8

Examples - Experiments

  • Datasets

– Modem pool traffic (14 modems, 1500 time-ticks; #packets per time unit) – AT&T WorldNet internet usage (several data streams; 980 time-ticks)

  • Measures of success

– Accuracy : Root Mean Square Error (RMSE)

slide-9
SLIDE 9

Accuracy - “Modem”

MUSCLES outperforms AR & “yesterday”

RMSE 1 2 3 4 Modems 1 2 3 4 5 6 7 8 9 10 11 12 13 14 AR yesterday MUSCLES

slide-10
SLIDE 10

Accuracy - “Internet”

RMSE 0.35 0.7 1.05 1.4 Streams 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 AR yesterday MUSCLES

MUSCLES consistently outperforms AR & “yesterday”

slide-11
SLIDE 11

Linear forecasting - Outline

  • Auto-regression
  • Least Squares; recursive least squares
  • Co-evolving time sequences
  • Examples
  • Conclusions
slide-12
SLIDE 12

Conclusions - Practitioner’s guide

  • AR(IMA) methodology: prevailing method

for linear forecasting

  • Brilliant method of Recursive Least Squares

for fast, incremental estimation.

slide-13
SLIDE 13

Resources: software and urls

  • MUSCLES: Prof. Byoung-Kee Yi:

http://www.postech.ac.kr/~bkyi/ or christos@cs.cmu.edu

  • R

http://cran.r-project.org/

slide-14
SLIDE 14

Books

  • George E.P. Box and Gwilym M. Jenkins and

Gregory C. Reinsel, Time Series Analysis: Forecasting and Control, Prentice Hall, 1994 (the classic book on ARIMA, 3rd ed.)

  • Brockwell, P. J. and R. A. Davis (1987). Time Series:

Theory and Methods. New York, Springer Verlag.

slide-15
SLIDE 15

Additional Reading

  • [Papadimitriou+ vldb2003] Spiros Papadimitriou,

Anthony Brockwell and Christos Faloutsos Adaptive, Hands-Off Stream Mining VLDB 2003, Berlin, Germany, Sept. 2003

  • [Yi+00] Byoung-Kee Yi et al.: Online Data Mining

for Co-Evolving Time Sequences, ICDE 2000. (Describes MUSCLES and Recursive Least Squares)

slide-16
SLIDE 16

Outline

  • Motivation
  • ...
  • Linear Forecasting
  • Non-linear forecasting
  • Conclusions
slide-17
SLIDE 17

Chaos & non-linear forecasting

slide-18
SLIDE 18

Reference:

[ Deepay Chakrabarti and Christos Faloutsos F4: Large-Scale Automated Forecasting using Fractals CIKM 2002, Washington DC, Nov. 2002.]

slide-19
SLIDE 19

Detailed Outline

  • Non-linear forecasting

– Problem – Idea – How-to – Experiments – Conclusions

slide-20
SLIDE 20

Recall: Problem #1

Given a time series {xt}, predict its future course, that is, xt+1, xt+2, ...

Time Value

slide-21
SLIDE 21

Datasets

Logistic Parabola:
 xt = axt-1(1-xt-1) + noise 
 Models population of flies [R. May/1976]

time

x(t)

Lag-plot ARIMA: fails

slide-22
SLIDE 22

How to forecast?

  • ARIMA - but: linearity assumption

Lag-plot ARIMA: fails

slide-23
SLIDE 23

How to forecast?

  • ARIMA - but: linearity assumption
  • ANSWER: ‘Delayed Coordinate Embedding’

= Lag Plots [Sauer92] ~ nearest-neighbor search, for past incidents

slide-24
SLIDE 24

General Intuition (Lag Plot)

xt-1 xt Lag = 1,
 k = 4 NN

slide-25
SLIDE 25

General Intuition (Lag Plot)

xt-1 xt New Point Lag = 1,
 k = 4 NN

slide-26
SLIDE 26

General Intuition (Lag Plot)

xt-1 xt 4-NN New Point Lag = 1,
 k = 4 NN

slide-27
SLIDE 27

General Intuition (Lag Plot)

xt-1 xt 4-NN New Point Lag = 1,
 k = 4 NN

slide-28
SLIDE 28

General Intuition (Lag Plot)

xt-1 xt 4-NN New Point Interpolate these… Lag = 1,
 k = 4 NN

slide-29
SLIDE 29

General Intuition (Lag Plot)

xt-1 xt 4-NN New Point Interpolate these… To get the final prediction Lag = 1,
 k = 4 NN

slide-30
SLIDE 30

Questions:

  • Q1: How to choose lag L?
  • Q2: How to choose k (the # of NN)?
  • Q3: How to interpolate?
  • Q4: why should this work at all?
slide-31
SLIDE 31

Q1: Choosing lag L

  • Manually (16, in award winning system by

[Sauer94])

slide-32
SLIDE 32

Q2: Choosing number of neighbors k

  • Manually (typically ~ 1-10)
slide-33
SLIDE 33

Q3: How to interpolate?

How do we interpolate between the
 k nearest neighbors? A3.1: Average A3.2: Weighted average (weights drop with distance - how?)

slide-34
SLIDE 34

Q3: How to interpolate?

A3.3: Using SVD - seems to perform best ([Sauer94] - first place in the Santa Fe forecasting competition)

Xt-1

xt

slide-35
SLIDE 35

Q3: How to interpolate?

A3.3: Using SVD - seems to perform best ([Sauer94] - first place in the Santa Fe forecasting competition)

Xt-1

xt

slide-36
SLIDE 36

Q3: How to interpolate?

A3.3: Using SVD - seems to perform best ([Sauer94] - first place in the Santa Fe forecasting competition)

Xt-1

xt

slide-37
SLIDE 37

Q3: How to interpolate?

A3.3: Using SVD - seems to perform best ([Sauer94] - first place in the Santa Fe forecasting competition)

Xt-1

xt

slide-38
SLIDE 38

Q4: Any theory behind it?

A4: YES!

slide-39
SLIDE 39

Theoretical foundation

  • Based on the ‘Takens theorem’ [Takens81]
  • which says that long enough delay vectors can

do prediction, even if there are unobserved variables in the dynamical system (= diff. equations)

slide-40
SLIDE 40

Detailed Outline

  • Non-linear forecasting

– Problem – Idea – How-to – Experiments – Conclusions

slide-41
SLIDE 41

Datasets

Logistic Parabola:
 xt = axt-1(1-xt-1) + noise 
 Models population of flies [R. May/1976]

time

x(t)

Lag-plot

slide-42
SLIDE 42

Datasets

Logistic Parabola:
 xt = axt-1(1-xt-1) + noise 
 Models population of flies [R. May/1976]

time

x(t)

Lag-plot ARIMA: fails

slide-43
SLIDE 43

Logistic Parabola

Timesteps Value

Our Prediction from here

slide-44
SLIDE 44

Logistic Parabola

Timesteps Value Comparison of prediction to correct values

slide-45
SLIDE 45

Datasets

LORENZ: Models convection currents in the air dx / dt = a (y - x) dy / dt = x (b - z) - y dz / dt = xy - c z

Value

slide-46
SLIDE 46

LORENZ

Timesteps Value Comparison of prediction to correct values

slide-47
SLIDE 47

Datasets

Time Value

  • LASER: fluctuations in a

Laser over time (used in Santa Fe competition)

slide-48
SLIDE 48

Laser

Timesteps Value Comparison of prediction to correct values

slide-49
SLIDE 49

Conclusions

  • Lag plots for non-linear forecasting (Takens’

theorem)

  • suitable for ‘chaotic’ signals
slide-50
SLIDE 50

References

  • Deepay Chakrabarti and Christos Faloutsos F4: Large-Scale

Automated Forecasting using Fractals CIKM 2002, Washington DC, Nov. 2002.

  • Sauer, T. (1994). Time series prediction using delay

coordinate embedding. (in book by Weigend and Gershenfeld, below) Addison-Wesley.

  • Takens, F. (1981). Detecting strange attractors in fluid
  • turbulence. Dynamical Systems and Turbulence. Berlin:

Springer-Verlag.

slide-51
SLIDE 51

References

  • Weigend, A. S. and N. A. Gerschenfeld (1994). Time Series

Prediction: Forecasting the Future and Understanding the Past, Addison Wesley. (Excellent collection of papers on chaotic/non-linear forecasting, describing the algorithms behind the winners of the Santa Fe competition.)

slide-52
SLIDE 52

Overall conclusions

  • Similarity search: Euclidean/time-warping;

feature extraction and SAMs

  • Linear Forecasting: AR (Box-Jenkins)

methodology;

  • Non-linear forecasting: lag-plots (Takens)
slide-53
SLIDE 53

Must-Read Material

  • Byong-Kee Yi, Nikolaos D. Sidiropoulos,

Theodore Johnson, H.V. Jagadish, Christos Faloutsos and Alex Biliris, Online Data Mining for Co-Evolving Time Sequences, ICDE, Feb 2000.

  • Chungmin Melvin Chen and Nick Roussopoulos,

Adaptive Selectivity Estimation Using Query Feedbacks, SIGMOD 1994

slide-54
SLIDE 54

Time Series Visualization + Applications

46

slide-55
SLIDE 55

Why Time Series Visualization?

Time series is the most common data type

  • But why is time series so common?

47

slide-56
SLIDE 56

How to build time series visualization?

Easy way: use existing tools, libraries

  • Google Public Data Explorer (Gapminder)


http://goo.gl/HmrH

  • Google acquired Gapminder 


http://goo.gl/43avY


(Hans Rosling’s TED talk http://goo.gl/tKV7)

  • Google Annotated Time Line 


http://goo.gl/Upm5W

  • Timeline, from MIT’s SIMILE project


http://simile-widgets.org/timeline/

  • Timeplot, also from SIMILE


http://simile-widgets.org/timeplot/

  • Excel, of course

48

slide-57
SLIDE 57

How to build time series visualization?

The harder way:

  • R (ggplot2)
  • Matlab
  • gnuplot
  • ...

The even harder way:

  • D3, for web
  • JFreeChart (Java)
  • ...

49

slide-58
SLIDE 58

Time Series Visualization

Why is it useful? When is visualization useful? (Why not automate everything? Like using the forecasting techniques you learned last time.)

50

slide-59
SLIDE 59

Time Series User Tasks

  • When was something greatest/least?
  • Is there a pattern?
  • Are two series similar?
  • Do any of the series match a pattern?
  • Provide simpler, faster access to the series
  • Does data element exist at time t ?
  • When does a data element exist?
  • How long does a data element exist?
  • How often does a data element occur?
  • How fast are data elements changing?
  • In what order do data elements appear?
  • Do data elements exist together?

Muller & Schumann 03 citing MacEachern 95

slide-60
SLIDE 60

horizontal axis is time

slide-61
SLIDE 61

http://www.patspapers.com/blog/item/what_if_everybody_flushed_at_once_Edmonton_water_gold_medal_hockey_game/

slide-62
SLIDE 62

http://www.patspapers.com/blog/item/what_if_everybody_flushed_at_once_Edmonton_water_gold_medal_hockey_game/

slide-63
SLIDE 63

Gantt Chart

Useful for project

How to create in Excel:

http://www.youtube.com/watch?v=sA67g6zaKOE

slide-64
SLIDE 64

ThemeRiver Stacked graph Streamgraph

http://www.nytimes.com/interactive/2008/02/23/movies/20080223_REVENUE_GRAPHIC.html http://bl.ocks.org/mbostock/3943967

slide-65
SLIDE 65

TimeSearcher

support queries

http://hcil2.cs.umd.edu/video/2005/2005_timesearcher2.mpg

slide-66
SLIDE 66

GeoTime


Infovis 2004

http://www.youtube.com/watch?v=inkF86QJBdA

http://vadl.cc.gatech.edu/documents/ 55_Wright_KaplerWright_GeoTime_InfoViz_Jrnl_05_send.pdf

58