Improving Bug Prediction Accuracy by Regularization and - - PowerPoint PPT Presentation

▶
improving bug prediction accuracy by regularization and
SMART_READER_LITE
LIVE PREVIEW

Improving Bug Prediction Accuracy by Regularization and - - PowerPoint PPT Presentation

Nov 10, 2023 150 likes •353 views

Improving Bug Prediction Accuracy by Regularization and Hyperparameter Optimization Haidar Osman Mohammad Ghafari Oscar Nierstrasz 1 Improving Bug Prediction Accuracy by Regularization and Hyperparameter Optimization Haidar Osman Mohammad

slide-1
SLIDE 1

Improving Bug Prediction Accuracy by Regularization and Hyperparameter Optimization

Haidar Osman Mohammad Ghafari Oscar Nierstrasz

1

slide-2
SLIDE 2

Improving Bug Prediction Accuracy by Regularization and Hyperparameter Optimization

Haidar Osman Mohammad Ghafari Oscar Nierstrasz

2

slide-3
SLIDE 3

3

slide-4
SLIDE 4

Number of Bugs Class (Buggy or Clean) Confusion Matrix Prediction Error Cost Effectiveness Source Code Metrics Change Metrics Organizational Metrics Package Class Filters Wrappers

4

slide-5
SLIDE 5

Number of Bugs Class (Buggy or Clean) Confusion Matrix Prediction Error Cost Effectiveness Source Code Metrics Change Metrics Organizational Metrics Package Class Filters Wrappers KNN SVM Linear Regression Poisson Regression

5

slide-6
SLIDE 6

Number of Bugs Class (Buggy or Clean) Confusion Matrix Prediction Error Cost Effectiveness Source Code Metrics Change Metrics Organizational Metrics Package Class Filters Wrappers KNN SVM Linear Regression Poisson Regression

6

slide-7
SLIDE 7

KNN SVM

Complexity Kernel

Hyperparameters

Exponent Gamma Sigma Omega Search Algorithm Evaluation # Neighbors

7

slide-8
SLIDE 8

Number of Classes 500 1000 1500 2000

629 1'620 195 1'288 798 62 242 129 209 199

Buggy Clean

JDT Core

Mylyn

PDE UI

8

slide-9
SLIDE 9

0.8 0.81 0.65 0.63 1.05 0.94 0.41 0.42 0.53 0.53

SVM No Yes No Yes No Yes No Yes No Yes

0.5 1.0 1.5 2.0

Tuned RMSE

Prediction Error

JDT Core

Mylyn

PDE UI

Tuned?

SVM

9

slide-10
SLIDE 10

0.99 0.79 0.8 0.62 1.19 1.03 0.55 0.43 0.66 0.52

Eclipse JDT Core Eclipse PDE UI Equinox Lucene Mylyn IBK

0.5 1.0 1.5 2.0

RMSE

No Yes No Yes No Yes No Yes No Yes

Tuned

Prediction Error

JDT Core

Mylyn

PDE UI

Tuned?

KNN

10

slide-11
SLIDE 11

Number of Bugs Class (Buggy or Clean) Confusion Matrix Prediction Error Cost Effectiveness Source Code Metrics Change Metrics Organizational Metrics Package Class Filters Wrappers KNN SVM Linear Regression Poisson Regression

11

slide-12
SLIDE 12

Number of Bugs Class (Buggy or Clean) Confusion Matrix Prediction Error Cost Effectiveness Source Code Metrics Change Metrics Organizational Metrics Package Class Filters Wrappers KNN SVM Linear Regression Poisson Regression

12

slide-13
SLIDE 13

13

slide-14
SLIDE 14

Filter Train

Filter Feature Selection

14

slide-15
SLIDE 15

subset Train Train Train Train

Wrapper Feature Selection

15

slide-16
SLIDE 16

Train

Embedded Feature Selection

Lasso Ridge Elastic

16

slide-17
SLIDE 17

0.96 0.8 0.82 0.81 0.59 0.57 0.58 0.58 0.98 0.92 1.01 1.01 0.4 0.38 0.38 0.38 0.52 0.51 0.51 0.51

Eclipse JDT Core Eclipse PDE UI Equinox Lucene Mylyn Linear Regression

0.0 0.5 1.0 1.5 2.0

Prediction Error

JDT Core

Mylyn

PDE UI

Linear Regression

None Ridge Lasso Elastic None Ridge Lasso Elastic None Ridge Lasso Elastic None Ridge Lasso Elastic None Ridge Lasso Elastic 17

slide-18
SLIDE 18

1.82 0.91 0.89 0.86 0.69 0.6 0.6 0.6 1.37 1.02 0.92 0.91 0.59 0.4 0.4 0.4 0.71 0.54 0.54 0.53 None Ridge Lasso ElasticNet None Ridge Lasso ElasticNet None Ridge Lasso ElasticNet None Ridge Lasso ElasticNet None Ridge Lasso ElasticNet

0.0 0.5 1.0 1.5 2.0

Prediction Error

JDT Core

Mylyn

PDE UI

Poisson Regression

None Ridge Lasso Elastic None Ridge Lasso Elastic None Ridge Lasso Elastic None Ridge Lasso Elastic None Ridge Lasso Elastic 18

slide-19
SLIDE 19