DL Approaches to Tim ime Series Data Miro Enev, DL Solution - - PowerPoint PPT Presentation

DL Approaches to Tim ime Series Data

Miro Enev, DL Solution Architect Jeff Weiss, Director West SAs

Agenda

• Define Time Series

[ Examples & Brief Summary of Considerations ]

• Semi-supervised Anomaly Detection

[ with Deep Autoencoders ]

• Ensemble event detection and classification

[ with MLPs & CNN-MLPs ]

• Prediction

[ with Dual Attentional RNNs (DA-RNNs) ]

Time Series [ [ One Defi finition ]

• A time series is a series of values recorded over equally spaced time intervals.
• The amount of time between observations is the sampling interval or sampling rate.
• The time series represent some underlying, partially-random (“stochastic”)

process which generated the data.

• We want to use the data to make guesses (“inferences”) about the process, and

want to make reliable guesses while being clear about the uncertainty involved.

• The complication is that each observation is dependent on all the other observations, and in

fact its this dependence that we want to learn and draw inferences about.

Popularity of f French Names

NVIDIA Stock

https://www.google.com/finance?chdnp=0&chdd=0&chds=1& chdv=1&chvs=Linear&chdeh=0&chfdeh=0&chdet=1494360000 000&chddm=98532&chls=IntervalBasedLine&q=NASDAQ:NVD A&ntsp=0&ei=N_kRWemBAoqxjAHRgZDwBA

Categories of f Time Series Analysis

• Prediction/Forecasting
• Weather, Sales, Inventory, Financial Markets
• Anomaly Detection
• Predictive Maintenance
• Reconstruction
• Virtual Sensors
• Classification
• Event/Activity Recognition

Time Series Best Practices

• Think hard about the input(s)
• Time and frequency domain representations
• Lags and windowing (multi-time scale)
• Study correlational structure
• Use semi/unsupervised approaches [ free ]
• Feature Building, Data Buckets, Anomaly Detection
• Gather and/or generate ground truth
• Class imbalance & sample weighting
• Try DL fabrics with increasing complexity & Ensemble
• Use a non DL baseline [ ARIMA, SVMs, random forests ]
• Use forward chained cross-validation for evaluation
• Train attentional mechanism*

DL for Anomaly Detection

Deep Autoencoder Anomaly Detection

13 Sensors, 100Hz, NASA Dataset, .5 seconds window, 650 dimensions per sample [ 256, 196, 136, 76, 14 ]

Anomaly Detection and Fault Disambiguation in Large Flight Data: A Multi-modal Deep Auto-encoder Approach, K. Reddy et al, United Technologies Research Center (PHM16)

Reconstruction & Feature Learning

Exceptionally low normalized RMS reconstruction error (0.04 – 0.09)

Anomaly Detection and Fault Disambiguation in Large Flight Data: A Multi-modal Deep Auto-encoder Approach, K. Reddy et al, United Technologies Research Center (PHM16)

Sample Reconstructions

Anomaly Detection and Fault Disambiguation in Large Flight Data: A Multi-modal Deep Auto-encoder Approach, K. Reddy et al, United Technologies Research Center (PHM16)

11-layer 14-dimensional bottleneck DAE yields 97.8% true positive detection rate with 0.0% false alarm Artifcially created anomalies [ Spall Fault ; Ballscrew Jam ]

Anomaly Analysis

Anomaly Detection and Fault Disambiguation in Large Flight Data: A Multi-modal Deep Auto-encoder Approach, K. Reddy et al, United Technologies Research Center (PHM16)

11-layer 14-dimensional bottleneck DAE yields 97.8% true positive detection rate with 0.0% false alarm Artificially created anomalies [ Spall Fault ; Ballscrew Jam ]

Anomaly Analysis/Interpretation

DL for Time Series Classification

Io IoT Case Study – Water Disagg ggregation

Frequency Domain Utilization

Automated Ground Truth Building

Open Edges Close Edges

Event/Edge Detection

Open Edge Clo Close ed edge

Edge Characterization, , Example 1

Edge Characterization, , Example 2

Clo Close ed edge Open Edge

DL for Time Series Prediction

Dual Attention RNNs

Performance @ NASDAQ 100

• In NASDAQ 100 Stock dataset, we collect the stock prices of 81 major corporations under NASDAQ 100, which are

used as the driving time series. The index value of NASDAQ 100 is used as the target series. The frequency of the data collection is one-minute. This data covers the period from July 26, 2016 to December 26, 2016, in total 104

• days. Each day contains 390 data points from the opening to closing of the market. In our experiments, we use the

first 90 days as the training set and the following seven days as the validation.

Performance @ SML 2010

SML 2010 is a public dataset used for indoor temperature forecasting. This dataset is collected from a monitor system mounted in a domestic house. We use room temperature as the target series and select 16 relevant driving series which contains approximately 40 days of monitoring data. The data was sampled every minute and was smoothed with 15 minute means. In our experiment, we use the first 3200 data points as the training set, the following 400 data points as the validation set, and the last 537 data points as the test set.

Attention Mechanism at Work

Train Set [ NASDAQ ] Test Set [ NASDAQ ]

Thanks!