ADVANCED MACHINE LEARNING Caveats and Techniques to Deal with - - PowerPoint PPT Presentation

1

ADVANCED MACHINE LEARNING

1

ADVANCED MACHINE LEARNING Caveats and Techniques to Deal with Imbalanced Datasets

(Adapted from H. He and E. A. Garcia, “Learning from Imbalanced Data,” IEEE Trans. Knowledge and Data Engineering, vol. 21, issue 9, pp. 1263--‐1284, 2009 )

2

ADVANCED MACHINE LEARNING

2

ML is now everywhere

❖ Increase of storage capacity  easy to build large datasets e.g. companies can store activities of clients with a variety of attributes ❖ Widely available code for ML techniques  Easy to use by laymen ➢ ML used widely in a variety of domains ❖ One hopes that ML will solve problems with which companies struggle (vast amount of data, very noisy, high-dimensional)

3

ADVANCED MACHINE LEARNING

3

ML techniques assume balanced datasets

ML algorithms assumes that data is balanced ➢ In classification: comparative number of instances of each class What would be the result of running SVM on the imbalanced dataset?

Balanced dataset Imbalanced dataset 826 dataset points: 790 positive class, 36 negative class

5

ADVANCED MACHINE LEARNING

5

Imbalanced Data: Why and When?

6

ADVANCED MACHINE LEARNING

6

Imbalanced Data: Example

Finance: Number of clients who closed account / nm clients who did not: 1% Robotics: Number of points where the two arms intersect / nm points with no intersection: 4-5% One has usually much less datapoints from the adverse class. This is unfortunate as we care a lot about avoiding misclassifying elements of this class.

7

ADVANCED MACHINE LEARNING

7

Imbalanced Data: Example

What would be the effect on SVR?

8

ADVANCED MACHINE LEARNING

8

Imbalanced Data: Example

Poor interpolation with missing data

9

ADVANCED MACHINE LEARNING

9

Types of Imbalance

Balanced between-class datasets but white dataset not representative with more data is some regions Few data locally

10

ADVANCED MACHINE LEARNING

10

11

ADVANCED MACHINE LEARNING

11

Imbalance and curse of dimensionality

12

ADVANCED MACHINE LEARNING

12

Approaches to learning with imbalanced datasets

13

ADVANCED MACHINE LEARNING

13

Learning with imbalanced datasets

Sampling Methods Cost-Sensitive Methods

Two main approaches

Act on the data Act on the cost function

14

ADVANCED MACHINE LEARNING

14

15

ADVANCED MACHINE LEARNING

15

Sampling Methods

Increase Dataset Generate new data points for the smallest class Decrease Dataset Remove redundant datapoints from the largest class

Compensate the lack of data by:

16

ADVANCED MACHINE LEARNING

16

Undersampling

Remove redundant datapoints Looses statistics – good only if enough datapoints on undersampled class and for low dimensional datasets

17

ADVANCED MACHINE LEARNING

17

Oversampling

Pick neighbour and create new datapoint Risk overfitting, especially if one does this for points that are noise

18

ADVANCED MACHINE LEARNING

18

SMOTE: Synthetic minority oversampling technique

No Neighbors of the same class  noise Several Neighbors

• f the same class

Surrounded by the

• ther class

 in danger Surrounded only on one side by the other class  safe Generate new samples inbetween existing datapoints based on their local density and their borders with the other class. Can use cleaning techniques (undersampling) to remove redundancy in the end.

19

ADVANCED MACHINE LEARNING

19

20

ADVANCED MACHINE LEARNING

20

21

ADVANCED MACHINE LEARNING

21

22

ADVANCED MACHINE LEARNING

22

How to treat Imbalanced Datasets with SVM

 SVM with asymmetric misclassification cost  Vary the penalty placed on each datapoint depending on its class  SVM class boundary adjustement  Changing b changes the boundary between the two classes

 

2 * 1

1 C minimize + 2

M i i i i

w M   







 

1

sgn ,

i

M i i

y k x x b 



       



23

ADVANCED MACHINE LEARNING

23

How to treat Imbalanced Datasets with SVM

24

ADVANCED MACHINE LEARNING

24

Taking into account imbalanced datasets in the assessment of performance

Traditional accuracy measure are sensitive to the data distribution The F-measure is better adapted as it evaluated performance on one class

25

ADVANCED MACHINE LEARNING

25

TOOLBOX https://github.com/scikit-learn-contrib/imbalanced-learn.