Biased Resampling Strategies for Imbalanced Spatio-temporal - - PowerPoint PPT Presentation

▶
biased resampling strategies for imbalanced
SMART_READER_LITE
LIVE PREVIEW

Biased Resampling Strategies for Imbalanced Spatio-temporal - - PowerPoint PPT Presentation

Jul 11, 2023 27 likes •311 views

Biased Resampling Strategies for Imbalanced Spatio-temporal Forecasting M A R I AN A O L IV EI R A , N U N O M O N I Z , L U S T O R G O A N D V T O R S A N T O S C O S TA 5 - 8 O C T O B

slide-1
SLIDE 1

Biased Resampling Strategies for Imbalanced Spatio-temporal Forecasting

M A R I AN A O L IV EI R A¹ ⋅ ² , N U N O M O N I Z ¹ ⋅ ² , L U Í S T O R G O ¹ ⋅ ² ⋅ ³ A N D V Í T O R S A N T O S C O S TA ¹ ⋅ ² 5 - 8 O C T O B E R 2 0 1 9

1 2 3

slide-2
SLIDE 2

Spatio-temporal Data

Remote monitoring equipment (Source: NDSU) Air quality measurement station network (Source: Zheng et al., 2013) PM 2.5 pollution levels (time series)

2/28

slide-3
SLIDE 3

Imbalanced Numeric Forecasting

IMBALANCED DOMAIN RELEVANCE FUNCTION

3/28

slide-4
SLIDE 4

Our Contribution

  • Random resampling

approaches are often used to tackle this problem

  • However, our data is

not i.i.d. -- there are spatial and temporal dependencies

Motivation

  • Will introducing a

sampling bias that takes into account spatio-temporal dependencies improve performance?

  • Should we weight the

dimensions differently?

Research Questions

4/28

slide-5
SLIDE 5

Biased Resampling

5/28

slide-6
SLIDE 6

Proposed Resampling Strategies

Spatio-temporal Random Under-sampling (STRUS)

  • Keep all extreme cases
  • Keep only u% of normal cases, 0 < u < 100 (with sampling bias)

Spatio-temporal Random Over-sampling (STROS)

  • Keep all (normal and extreme) cases
  • Add o% replicas of extreme cases, o > 0 (with sampling bias)

6/28

slide-7
SLIDE 7

Spatio-temporal Sampling Bias

Temp mporal weight

More recent

  • bservations

Spatialweight (extreme cases)

Isolated rare cases

Spatialweight (normal cases)

Far away from rare cases

Which cases should have higher probability of being selected during resampling?

At each time-step

7/28

slide-8
SLIDE 8

Spatio-temporal Sampling Bias

What if spatial and temporal dimensions havedifferent impacts? Add weighting parameter α 8/28

slide-9
SLIDE 9

Experiments

9/28

slide-10
SLIDE 10

Datasets

Data source ID # time IDs # loc IDs % available % extreme MESA 10 280 20 100 7.3 NCDC 20 105 72 100 6.0 30 6.3 TCE 31 330 26 100 3.8 32 2.4 Rural 40 4k 70 ~49 7.5 50 3.5 Beijing Air 51 11k 36 ~40 5.5 52 8.6 53 3.8

10/28

slide-11
SLIDE 11

Calculate spatio- temporal indicators

Feature engineering

  • None
  • RUS
  • ROS
  • STRUS
  • STROS

Resampling

  • MARS
  • Random

Forest

  • RPART

Model

Learning Process

PRE-PROCESSING 11/28

slide-12
SLIDE 12

Experimental Evaluation

Evaluation metrics Performance estimation procedure

12/28

slide-13
SLIDE 13

Evaluation Metrics

  • Utility-based precision and recall

for numeric prediction:

13/28

slide-14
SLIDE 14

Performance Estimation Procedure

train test

time y x

  • Prequential temporal block evaluation

14/28

slide-15
SLIDE 15

Parametrization

Internal tuning Fixed a priori Optimal a posteriori

15/28

slide-16
SLIDE 16

Parametrization

Internal tuning Fixed a priori Optimal a posteriori

16/28

slide-17
SLIDE 17

Internal Tuning

INTERNAL ESTIMATION PROCEDURE For each training set: Temporal-block CV PARAMETER GRID SEARCH

Parameter Values u 0.2; 0.4; 0.6; 0.8; 0.95

  • 0.5; 1; 2; 3; 4

α 0; 0.25; 0.5; 0.75; 1

time y x

17/28

slide-18
SLIDE 18

Parametrization

Internal tuning Fixed a priori Optimal a posteriori

18/28

slide-19
SLIDE 19

Fixed a priori

For all training sets: Fixed parameters at middle of the grid. FIXED PARAMETERS

Parameter Values u 0.2; 0.4; 0.6; 0.8; 0.95

  • 0.5; 1; 2; 3; 4

α 0; 0.25; 0.5; 0.75; 1 19/28

slide-20
SLIDE 20

Parametrization

Internal tuning Fixed a priori Optimal a posteriori

20/28

slide-21
SLIDE 21

Optimal a posteriori

EXTERNAL ESTIMATION PROCEDURE For each data set: Choose parameters with best results on the external (prequential) procedure. PARAMETER GRID SEARCH

Parameter Values u 0.2; 0.4; 0.6; 0.8; 0.95

  • 0.5; 1; 2; 3; 4

α 0; 0.25; 0.5; 0.75; 1 21/28

slide-22
SLIDE 22

Results

22/28

slide-23
SLIDE 23

Average Rank of F1u

Parametrization None ROS STROS RUS STRUS Internal tuning 4.60 3.07 2.37 2.67 2.30 Fixed a priori 4.53 2.77 2.73 2.57 2.40 Optimal a posteriori 5.00 3.07 2.27 2.93 1.73

23/28

slide-24
SLIDE 24

Parameter Sensitivity Analysis

TWO PARAMETERS DIMENSION WEIGHTING

RUS/ROS RUS/ROS RUS/ROS

24/28

slide-25
SLIDE 25

Precision and Recall Trade-off

PRECISION RECALL

RUS/ROS RUS/ROS RUS/ROS RUS/ROS RUS/ROS RUS/ROS

25/28

slide-26
SLIDE 26

Conclusion

26/28

slide-27
SLIDE 27

Conclusion

  • Including spatio-temporal bias when resampling improves performance
  • The contributions of each dimension should be weigthed:
  • When over-sampling: favour temporal weight and prioritize more recent observations
  • When under-sampling: favour spatial weight and prioritize isolated rare cases and normal

cases that are spatially distant from extreme cases

  • Future work:
  • Study the impact of data characteristics on performance
  • Consider local instead of global definitions of extreme values

27/28

slide-28
SLIDE 28

Thank you!

Code available at https://github.com/mrfoliveira/STResampling-DSAA2019

28/28