DeepI DeepIV: A : A F Flexibl ble A Appr pproa oach for for - PowerPoint PPT Presentation

DeepI DeepIV: A : A F Flexibl ble A Appr pproa oach for for Co Counte terf rfac actu tual al Pr Predicti tion Greg Lewis * and Matt Taddy † Jason Hartford and Kevin Leyton-Brown University of British Columbia Microsoft Research & * NBER / † University of Chicago

I need a model that predicts the effect of price on ticket sales SkyHighAir Jason

Prediction with confounding effects We can raise prices and get more sales! Jason

Prediction with confounding effects " = $ ! Sales Price !

Prediction with confounding effects " = $ !, & Sales Price !

Prediction with confounding effects " = $ !, & Sales Automated pricing engine Price increases prices as the plane fills ! = '(&)

The observational distribution “response” “features / observed confounders” " = $(!, *) * Holidays Sales Price ! = '(*) “policy / treatment”

The interventional distribution “response” “features / observed confounders” -("|do !̂ , *) * Holidays Sales Price Set ! = !̂ “policy / treatment”

Identification of causal effects “response” “features / observed confounders” " = $(!, *) * Holidays Sales If *, ! & " observed, Price -("|do ! , *) is identified. See e.g. [Athey et al. ! = '(*) 2016], [ Shalit et al. 2017 ] “policy / treatment”

Identification of causal effects “response” “features / observed confounders” " = $ !, *, & * Holidays Sales & Conference Not identified without further assumptions Price “latent / unobserved confounders” ! = '(*, &) “policy / treatment”

Identification of causal effects “response” “features / observed confounders” " = $ !, * + & * Additive latent effects Holidays Sales & Conference “instrument” 3 Fuel Price “latent / unobserved confounders” Cost Variable that ! = '(*, 3, &) only affects the response indirectly via its effect on price “policy / treatment”

Simulate a world without latent effects on price Holidays Sales Conference Estimated Fuel Cost Price

Simulate a world without latent effects on price Holidays Sales Conference Estimated Fuel Cost Price

The learning problem These assumptions imply the following identity 1 , 4 " *, 3 = 4 $ !, * *, 3 = ∫ $ !, * 67(!|*, 3) So we can recover $(!, *) solve the implied inversion problem ... B A min ;∈= > " ? − ∫ $ !, * ? 67 ! *, 3 ?CD 1. This holds if 4 & *] = 0 . In general we recover $(!, *) up to a constant wrt ! – see paper for details.

� � ̇ A two-stage solution U T min M∈P > Q R − ∫ M J, K R SG J K, L RCV Stage 2: train network M N O using I J K, L using the Stage 1 : fit G H stochastic gradient descent model of your choice. with monte-carlo integration . We use mixture density 1 networks [Bishop 94] XY Z = −2 Q R − > M ] J ̇ V , K R × J V * ` J K, L ̇ ~G J V … " I J K, L G H 1 At each SGD > b N M ] J T ̇ , K R iteration |J T ̇ | ` J K, L !̇ Sample ̇ ~G J T M N O(J, K)

Causal Validation • In general, out-of-sample validation causal models is challenging / impossible … • But… both our losses depend only on observable quantities and reflect causal loss, so we can simply use standard validation sets .

Evaluation Simulation & Bing Ads Experiments

Simulation Experiments Price Sensitivity Customer features c~d{0, 1, . . , 6} i lets us smoothly vary the correlation Customer between sales and Holidays Conference type price Ticket Ticket Fuel Cost Price Sales

Simulation – low dimensional feature space

Simulation – low dimensional feature space

Simulation – low dimensional feature space

Simulation – low dimensional feature space

Simulation – low dimensional feature space

Simulation – low dimensional feature space

Simulation – low dimensional feature space [Darolles et al. 2011]

Simulation – low dimensional feature space [Darolles et al. 2011]

Implications and future directions • We recover heterogeneous treatment effects in settings with unobserved confounding effects for both discrete and continuous variables… and SGD scales naturally to very large datasets. • Can leverage the flexibility of deep nets for rich data types. E.g. raw text in our Bing ads application experiments / images in simulation. Future work: • Methods for uncertainty estimates over predictions. Code and paper available at http://bit.ly/DeepIV Poster #127