SLIDE 1
Regularization in Directable Environments with Application to - - PowerPoint PPT Presentation
Regularization in Directable Environments with Application to - - PowerPoint PPT Presentation
Regularization in Directable Environments with Application to Tetris Jan Malte Lichtenberg zgr im ek Shrinkage Toward Equal Weights (STEW) Lichtenberg J.M. and im ek . Regularization in Directable Environments with
SLIDE 2
SLIDE 3
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
Shrinkage Toward Equal Weights (STEW)
0.0 0.1 0.2 0.3 10−4 10−2 100 102 104
λ (log scale) Weight estimates STEW (q = 2)
SLIDE 4
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
Shrinkage Toward Equal Weights (STEW)
Equal Weights
0.0 0.1 0.2 0.3 10−4 10−2 100 102 104
λ (log scale) Weight estimates STEW (q = 2)
SLIDE 5
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
Shrinkage Toward Equal Weights (STEW)
Equal Weights
0.0 0.1 0.2 0.3 10−4 10−2 100 102 104
λ (log scale) Weight estimates STEW (q = 2)
0.0 0.1 0.2 0.3 10−3 10−2 10−1 100 101 102
λ (log scale) Weight estimates Ridge regression:
SLIDE 6
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
SLIDE 7
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
“1 / N rule“ (DeMiguel et al., 2009)
SLIDE 8
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
SLIDE 9
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
Are feature directions known?
SLIDE 10
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
Are feature directions known? “Direct” all features
SLIDE 11
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
Are feature directions known? “Direct” all features
SLIDE 12
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
Are feature directions known? “Direct” all features
SLIDE 13
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
1 −2 −1 1 2 β
Density
SLIDE 14
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
1 −2 −1 1 2 β
Density
- 5
10 15 20 25 15 20 30 50 100 200
Training set size (log scale) MSE
- EW
Ridge Lasso NNLasso STEW
SLIDE 15
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
1 −2 −1 1 2 β
Density
1 −2 −1 1 2 β
Density
1 −2 −1 1 2 β
Density
Environment less directable
- 2
4 6 8 15 20 30 50 100 200
Training set size (log scale) MSE
- 2.5
5.0 7.5 10.0 15 20 30 50 100 200
Training set size (log scale) MSE
- 5
10 15 20 25 15 20 30 50 100 200
Training set size (log scale) MSE
- EW
Ridge Lasso NNLasso STEW
SLIDE 16
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
- 5
10 15 15 20 30 50 100 200
Training set size (log scale) MSE
- Ridge
Lasso NNLasso STEW
1 −2 2 β
Density
SLIDE 17
Lichtenberg J.M. and Şimşek Ö. Regularization in Directable Environments with Application to Tetris.
- 5
10 15 15 20 30 50 100 200
Training set size (log scale) MSE
- Ridge
Lasso NNLasso STEW 5 10 15 15 20 30 50 100 200
Training set size (log scale) Error component
- Sq. Bias Ridge
- Sq. Bias STEW
Variance Ridge Variance STEW
1 −2 2 β
Density
SLIDE 18