Kalman Filter with Exponential Moving Average and Dynamic - - PowerPoint PPT Presentation

Robust Online Model Adaptation by Extended Kalman Filter with Exponential Moving Average and Dynamic Multi-Epoch Strategy

Abulikemu Abuduweili1,2, and Changliu Liu2

1School of EECS, Peking University, P.R. China 2Robotics Institute, Carnegie Mellon University, USA

L4DC 2020

Peking University Carnegie Mellon University June 5, 2020

Introduction – Behavior Prediction

• 𝑍

𝑢 = 𝑔(𝜄, 𝑌𝑢)

• Xt = [𝑦𝑢; 𝑦𝑢−1;···; 𝑦𝑢−𝑜] denotes the n-step past measurement. Yt = [𝑧𝑢+1; 𝑧𝑢+2;···; 𝑧𝑢+𝑛]

denotes the m-step future behavior. 𝜄 is the parameter of the model.

2

Trajectory prediction on autonomous driving Human motion prediction during human-robot collaboration

Abuduweili A, et al. “Adaptable Human Intention and Trajectory Prediction for Human-Robot Collaboration”. arXiv preprint arXiv:1909.05089 2019; Yujiao Cheng, et al. “Human motion prediction using semi-adaptable neural networks”. ACC , pages 4884–4890. IEEE, 2019.

Introduction -Why Adaptation?

• Performance of the trained model can drop significantly under a slightly different

data distribution. For tasks without annotated corpora from the test domain, adaptation techniques are required to deal with the lack of domain-specific data.

3

Distribution Difference between train set and test set. Dataset Train Set Test Set MSE (m2) 1.492 2.559 Performance comparison between train set and test set (NGSIM).

Train set distribution Test set distribution

Wenwen Si, et al. “Agen: Adaptable generative prediction networks for autonomous driving”. In 2019 IEEE Intelligent Vehicles Symposium (IV), 2019.

Introduction - Online Adaptation Framework

• Online adaptation explores local overfitting to minimize the prediction error, which

corresponds to a nonlinear least square (NLS) problem: min

෡ 𝜄𝑢

Yt − 𝑔( ෠ 𝜄𝑢, Xt)

𝑞

4

Introduction - Related Works

5

Bhasin S, et al. “Robust identification-based state derivative estimation for nonlinear systems”. IEEE Transactions on Automatic Control 2012; 58(1); Wenwen Si, et al. “Agen: Adaptable generative prediction networks for autonomous driving”. In 2019 IEEE Intelligent Vehicles Symposium (IV), 2019.

Previous Methods Problems SGD based slow convergence; sub-optimal RLS-PAA

• nly applies to linear

models

Adapting the last layer of GRU model with RLS-PAA. Previous methods for online adaptation.

Robust Online Model Adaptation

• Our adaptation algorithm is based by the recursive EKF method. Why?
• By assuming that the ground truth changes very slowly, we can pose the

parameter adaptation problem as a static state estimation problem with the following dynamics:

6

Angelo Alessandri et al. “A recursive algorithm for nonlinear least-squares problems”. Computational Optimization and Applications, 38(2), 2007.

Adaptation algorithms Convergence rate Applicable for nonlinear system SGD based Slower Yes RLS based Faster No EKF based (Ours) Faster Yes

Robust Online Model Adaptation

• Modified EKF with Exponential Moving Average (EMA) filtering

7

Angelo Alessandri et al. “A recursive algorithm for nonlinear least-squares problems”. Computational Optimization and Applications, 38(2), 2007.

Our Extensions Why Explanation (Methods) Forgetting Factor Data in the distant past is no longer relevant for modeling the current. We consider a nonlinear recursive least squares (NLS) problem with forgetting factor 𝜇: min

෡ 𝜄𝑢

σ𝑣=1

𝑢

𝜇𝑢−𝑗 𝑧𝑗 − 𝑔

1( መ

𝜄𝑗−1, X𝑗−1)

2 2, 0 < 𝜇 ≤ 1

EMA filtering EMA is typically applied to parameter update in practice, which can reduce the variance of convergence curve. (E.g.: Polyak averaging and momentum for SGD) EMA-V: EMA-V calculates the step size of the parameter by decreasing exponentially the older step size. EMA-P: P

t is an uncertainty matrix in the parameter

estimates in EKF. We can smooth the inner state of the

• ptimizer by EMA pre-filtering P

t.

Robust Online Model Adaptation

• Modified EKF with Exponential Moving Average (EMA) filtering

8

Robust Online Model Adaptation

• Dynamic multi-epoch update strategy (DME)
• DME increase the sample efficiency. It is practically useful to differentiate “easy”

samples from “hard” samples.

• Simple criterion: Two thresholds 𝜁1 and 𝜁2 are used to discriminate “easy”, “hard”,

and “anomaly” samples by error

• Easy sample: single-epoch update
• Hard sample: two-epoch update
• Anomaly sample: skip the update

9

Sampling Explanation (Methods) Previous method All data are equally considered. We run the adaptation algorithm chronologically from the first data to the last data Dynamic multi-epoch update strategy (Ours) Define a criterion 𝐷 to determine the number of epochs 𝜆𝑢 = 𝐷 X𝑢−1, 𝑧𝑢, ො 𝑧𝑢, 𝜄∗ . We reuse the input-output pair (X𝑢−1; 𝑧𝑢) 𝜆𝑢 times to adapt the parameter 𝜄∗

Yoshua Bengio, et al. “Curriculum learning”. ICML , pages 41–48. ACM, 2009.

Robust Online Model Adaptation

• Dynamic multi-epoch update strategy

10

Numerical Experiments: Multi-Task Prediction

• In the experiment, we considers a multi-task prediction problem for simultaneous

intention and trajectory prediction.

• In the online adaptation of multi-task learning, the adaptation algorithm updates

the prediction model only considering the error measured between the predicted trajectory and the ground truth trajectory.

11

Online adaptation framework for a multi-task model.

Numerical Experiments: Design

• Neural Network Architecture: RNN based encoder-decoder-classifier structure.
• Dataset: Mocap dataset (human-motion) and NGSIM dataset (vehicle).
• Online adaptation on hidden weights of encoder of the offline-trained models.

12

Neural network architecture in experiments.

Numerical Experiments: Results

• Table 1 shows the prediction performance of online adapted models using different
• ptimizers. Compared to the SGD-based algorithms, the EKF-based methods perform
• better. In addition, MEKFEMA−DME has the best performance among all.

13

Numerical Experiments: Results

• Table 2 shows the effectiveness of the proposed extensions. EMA-P slightly improves

the performance. DME improves the performance apparently.

14

Demo

• Human-motion trajectory prediction with proposed online adaptation.

15

Demo

• Human-motion trajectory and intention prediction with proposed online adaptation.

16

Conclusions

• This work studied online adaptation of neural network-based prediction models

for behavior prediction.

• In order to improve the performance and convergence rate, EMA filtering was

investigated, including EMA-V and EMA-P.

• This paper introduced a dynamic multi-epoch update strategy, which is

compatible with any optimizer.

• By combining all extensions with the EKF based algorithm, we introduced the

robust online adaptation algorithm MEKFEMA−DME

• The

source code is

• pen-sourced

in the following link: https://github.com/intelligent-control-lab/MEKF_MAME .

17

Thank you for your attention

Abulikemu Abuduweili: abduwali@pku.edu.cn , Changliu Liu: cliu6@andrew.cmu.edu. Questions?