SUPPORT VECTOR MACHINES FOR DIFFERENTIAL PREDICTION Finn Kuusisto 1 - - PowerPoint PPT Presentation
SUPPORT VECTOR MACHINES FOR DIFFERENTIAL PREDICTION Finn Kuusisto 1 , Vitor Santos Costa 2 , Houssam Nassif 3 , Elizabeth Burnside 1 , David Page 1 , and Jude Shavlik 1 1 University of Wisconsin Madison 2 University of Porto 3 Amazon 9/18/2014
Finn Kuusisto 1, Vitor Santos Costa 2, Houssam Nassif3, Elizabeth Burnside 1, David Page 1, and Jude Shavlik 1
9/18/2014 Support Vector Machines for Differential Prediction
1University of Wisconsin β Madison 2University of Porto 3Amazon
9/18/2014 Support Vector Machines for Differential Prediction
Use modeling techniques to gain insight about the differences between two subgroups of a population.
9/18/2014 Support Vector Machines for Differential Prediction
( R A D C L I F F E & S I M P S O N , 2 0 0 8 )
How do we choose which customers to target with some marketing activity?
9/18/2014 Support Vector Machines for Differential Prediction
( R A D C L I F F E & S I M P S O N , 2 0 0 8 )
Persuadables Customers who respond positively to marketing activity. Sure Things Customers who respond positively regardless. Lost Causes Customers who respond negatively regardless. Sleeping Dogs Customers who respond negatively to marketing activity.
How do we choose which customers to target with some marketing activity?
Target Control Response No Response Response No Response Persuadables, Sure Things Sleeping Dogs, Lost Causes Sleeping Dogs, Sure Things Persuadables, Lost Causes
9/18/2014 Support Vector Machines for Differential Prediction
( R A D C L I F F E & S I M P S O N , 2 0 0 8 )
True customer groups are unknown.
9/18/2014 Support Vector Machines for Differential Prediction
π©π½π½ = π©π½π΄πΌ β π©π½π΄π«
The number of true positives that a classifier achieves at a given proportion of the population labeled positive.
The difference in lift produced by a classifier between target and control subgroups.
ο‘ Non-steroidal anti-inflammatory drug (NSAID) ο‘ Significantly reduced occurrence of adverse gastrointestinal effects common to other NSAIDs (e.g. ibuprofen) ο‘ Rapid and widespread acceptance for treatment of ailments such as arthritis ο‘ Later clinical trials showed increased risk of myocardial infarction (MI), or βheart attackβ
9/18/2014 Support Vector Machines for Differential Prediction
9/18/2014 Support Vector Machines for Differential Prediction
Identify patients who demonstrate an increased risk of MI as a direct result of being treated with COX-2 inhibitors.
Patients with an increased risk of MI due to treatment with COX-2 inhibitors are directly analogous to customers with an increased chance of buying due to targeting β the persuadables.
ο‘ Compared SVMUpl against 4 alternate SVM methods ο‘ 10-fold cross-validation for evaluation ο‘ Cost parameters selected from 10 through 10β6 ο‘ Mann-Whitney test at 95% confidence for per-fold AUU comparison
9/18/2014 Support Vector Machines for Differential Prediction
Model AUU COX-2 AUL No COX-2 AUL SVMUpl π-value ππππππ 50.7 123.4 72.7
20.0 126.2 106.3 0.004 * COX-2-Only 13.8 151.5 137.7 0.002 * Standard 1.2 147.7 146.5 0.002 * Flipped 28.5 102.2 73.6 0.037 * Baseline 0.0 0.0 0.0 0.002 *
9/18/2014 Support Vector Machines for Differential Prediction
9/18/2014 Support Vector Machines for Differential Prediction
Extend previous SVM work maximizing AUC (Joachims, 2005) to maximize AUU instead.
9/18/2014 Support Vector Machines for Differential Prediction
9/18/2014 Support Vector Machines for Differential Prediction
π΅ππ = π Γ π 2 + 1 β π π΅ππ· Let the positive skew of data be: π = π π + π Then (Tuffery, 2011):
9/18/2014 Support Vector Machines for Differential Prediction
π΅ππ = π΅πππ β π΅πππ· = ππ Γ ππ 2 + 1 β ππ π΅ππ·π β ππ· Γ ππ· 2 + 1 β ππ· π΅ππ·π· πππ¦ π΅ππ β‘ πππ¦ π΅ππ·π β ππ΅ππ·π· πππ¦ π΅ππ β‘ πππ¦ ππ Γ 1 β ππ π΅ππ·π β ππ· Γ 1 β ππ· π΅ππ·π· β πππ¦ π΅ππ·π β ππ· Γ 1 β ππ· ππ Γ 1 β ππ π΅ππ·π· π
ο‘ Most common cancer in women ο‘ Two basic stages: In situ and invasive
ο‘ Younger women tend to have more aggressive in situ cancer ο‘ Older women sometimes have indolent in situ cancer
9/18/2014 Support Vector Machines for Differential Prediction
Identify older patients with in situ breast cancer that is distinct from that of younger patients.
Older patients with in situ breast cancer that is distinct from that of younger patients, who tend to have aggressive cancer, have a decreased risk of invasive progression.
9/18/2014 Support Vector Machines for Differential Prediction
Model AUU Older AUL Younger AUL SVMUpl π-value ππππππ 19.2 64.3 45.1
13.5 74.3 60.8 0.432 Older-Only 5.9 67.7 61.9 0.037 * Standard 11.0 75.4 64.3 0.049 * Flipped 4.8 53.9 49.1 0.020 * Baseline 11.0 66.0 55.0 0.004 *
9/18/2014 Support Vector Machines for Differential Prediction
9/18/2014 Support Vector Machines for Differential Prediction
9/18/2014 Support Vector Machines for Differential Prediction
ο‘ Generated synthetic customer population ο‘ Subjected customer population randomly to simulated marketing activity ο‘ Measured uplift as usual ο‘ Measured ROC with Persuadables as the positive class, others as negative
9/18/2014 Support Vector Machines for Differential Prediction
9/18/2014 Support Vector Machines for Differential Prediction
ο‘ Extended previous SVM work on AUC maximization to AUU ο‘ Results suggest SVMUpl achieves better uplift than many alternate SVM methods ο‘ May want to make performance guarantees for control group ο‘ May want to interpret learned model ο‘ Better verification that maximizing uplift is appropriate goal
9/18/2014 Support Vector Machines for Differential Prediction
Questions?
9/18/2014 Support Vector Machines for Differential Prediction
Radcliffe, N. and Simpson. R.: Identifying who can be saved and who will be driven away by retention activity. Journal of Telecommunications Management (2008). Tuffery, S.: Data Mining and Statistics for Decision Making. John Wiley & Sons, 2nd edn. (2011). Joachims, T.: A support vector method for multivariate performance
Machine Learning (2005).
9/18/2014 Support Vector Machines for Differential Prediction
9/18/2014 Support Vector Machines for Differential Prediction
9/18/2014 Support Vector Machines for Differential Prediction
( J O A C H I M S , 2 0 0 5 )
Ξπ΅ππ· π§ β², π§ = 1 2 1 β π§ππ
β² π π=1 π π=1
π±πΞ¨ π¦ , π§ β² + Ξπ΅ππ· π§ β², π§ π±πΞ¨ π² , π§ β² = π±π 1 2 π§ππ
β² π²π β π²π π π=1 π π=1
Define π§ππ
β² as a predicted label on pairs of positive (i) and negative (j) examples,
where π§ππ
β² = 1 if π§π β² > π§π β², and -1 otherwise.
Joachimsβ algorithm to maximize AUC corresponds to finding π§ππ
β that maximizes:
Where:
9/18/2014 Support Vector Machines for Differential Prediction
π΅ππ = π π 2 + 1 β π π΅ππ· Let the positive skew of data be: π = π π + π Then:
9/18/2014 Support Vector Machines for Differential Prediction
π΅ππ = π
π΅
ππ΅ 2 + 1 β ππ΅ π΅ππ·π΅ β ππΆ ππΆ 2 + 1 β ππΆ π΅ππ·πΆ πππ¦ π΅ππ β‘ πππ¦ π
π΅ 1 β ππ΅ π΅ππ·π΅ β ππΆ 1 β ππΆ π΅ππ·πΆ
β πππ¦ π΅ππ·π΅ β ππΆ 1 β ππΆ π
π΅ 1 β ππ΅
π΅ππ·πΆ Defining: π = ππΆ 1 β ππΆ π
π΅ 1 β ππ΅
Then: πππ¦ π΅ππ β‘ πππ¦ π΅ππ·π΅ β ππ΅ππ·πΆ
9/18/2014 Support Vector Machines for Differential Prediction
Ξπ΅ππ π§ β², π§ = 1 2 1 β π§ππ
β² ππ΅ π=1 ππ΅ π=1
+ π 1 2 1 β π§ππ
β² ππΆ π=1 ππΆ π=1
π±πΞ¨ π² , π§ β² = π±π 1 2 π§ππ
β² π²π β π²π ππ΅ π=1 ππ΅ π=1
+ ππ±π 1 2 π§ππ
β²
π²π β π²π
ππΆ π=1 ππΆ π=1
Now simply redefine the AUC optimization:
9/18/2014 Support Vector Machines for Differential Prediction
1. Generated random Bayesian network with 20 nodes and 30 edges 2. Selected one four-value node to define the customer group 3. Drew 10,000 samples from the network 4. Subjected customer population randomly to simulated marketing activity
ο‘ Measured uplift as usual ο‘ Measured ROC with Persuadables as the positive class, others as negative
9/18/2014 Support Vector Machines for Differential Prediction
Older Younger In Situ Invasive In Situ Invasive 132 401 110 264 COX-2 Inhibitors No COX-2 Inhibitors MI No MI MI No MI 184 1,776 184 1,776