SLIDE 7 7
Many ML Algorithms aren’t Stable wrt Updates
When trained on more data (same distribution)…
- Updates (h2) increase ROC,
- But have low compatibility score,
first “stochasticity”– defined |¬ satisfied, satisfied. first “one-sided”: |¬ classifier fied. “two-sided”: classifier satisfied. fix classifier’ fix classifier first type– classifier classifier classifier Classifier Dataset ROC h1 ROC h2 CS LR Recidivism 0.68 0.72 0.74 Credit Risk 0.72 0.77 0.68 Mortality 0.68 0.77 0.54 MLP Recidivism 0.59 0.73 0.62 Credit Risk 0.70 0.80 0.69 Mortality 0.71 0.84 0.77 classifier
specifics ⇢ –1– define “kind” –2– defines
finity
C(h1, h2) = 1 − count(h1 = y, h2 6 = y) count(h2 6 = y)
20
defines –1– classification defines classification. · − · − –2– define λ · D defines classification λ classification compatibility– defines · user’
”performance” ”accuracy”
But for Teams, …
Team Performance Time
