When causality matters for prediction: Investigating the practical - - PowerPoint PPT Presentation

When causality matters for prediction: Investigating the practical tradeoffs

Robert E. Tillman Peter Spirtes

Department of Philosophy Machine Learning Department College of Humanities and Social Sciences School of Computer Science

NIPS 2008 Workshop on Causality: Objectives and Assessment

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causal Discovery

Pollution HeartDis CiliaDam BreathDis LungCapac Genotype Smoker Income Parent

The Usual Setup: Unobserved data generating process i.i.d. sample

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 2 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causal Discovery

Pollution HeartDis CiliaDam BreathDis LungCapac Genotype Smoker Income Parent

The Usual Setup: Unobserved data generating process i.i.d. sample Objective: Learn structure, e.g. causal Bayesian network

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 2 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causal Discovery

Pollution HeartDis CiliaDam BreathDis LungCapac Genotype Smoker Income Parent

The Usual Setup: Unobserved data generating process i.i.d. sample Objective: Learn structure, e.g. causal Bayesian network Assessment: Compare to “ground truth”, i.e. simulations, experimental studies, expert knowledge

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 2 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causal Discovery

Pollution HeartDis CiliaDam BreathDis LungCapac Genotype Smoker Income Parent

The Usual Setup: Unobserved data generating process i.i.d. sample Objective: Learn structure, e.g. causal Bayesian network Assessment: Compare to “ground truth”, i.e. simulations, experimental studies, expert knowledge Focus: Learn network models that accurately depict the data generating mechanism

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 2 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Prediction

BB Target P1 P2 P3 P4 P5

The Standard Problem: “Target” variable associated with “predictor” variables i.i.d sample (training data)

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 3 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Prediction

BB Target P1 P2 P3 P4 P5

The Standard Problem: “Target” variable associated with “predictor” variables i.i.d sample (training data) Objective: Predict target from values of predictor variables

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 3 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Prediction

BB Target P1 P2 P3 P4 P5

The Standard Problem: “Target” variable associated with “predictor” variables i.i.d sample (training data) Objective: Predict target from values of predictor variables Assessment: Compare predictions to known target values, i.e. testing data, cross validation

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 3 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Prediction

BB Target P1 P2 P3 P4 P5

The Standard Problem: “Target” variable associated with “predictor” variables i.i.d sample (training data) Objective: Predict target from values of predictor variables Assessment: Compare predictions to known target values, i.e. testing data, cross validation Focus: Train classifier/regression model that minimizes loss function, e.g. makes accurate predictions Model need not resemble the true data generating mechanism, i.e. Naive Bayes

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 3 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causal Discovery and Prediction Previous focus: predicting the effects of possible interventions: Specify the distribution for a manipulated population Counterfactuals

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 4 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causal Discovery and Prediction Previous focus: predicting the effects of possible interventions: Specify the distribution for a manipulated population Counterfactuals Assume intervention has not been performed, e.g. no data from manipulated population

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 4 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causal Discovery and Prediction Previous focus: predicting the effects of possible interventions: Specify the distribution for a manipulated population Counterfactuals Assume intervention has not been performed, e.g. no data from manipulated population Causation and Prediction Challenge: Training data from unmanipulated population

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 4 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causal Discovery and Prediction Previous focus: predicting the effects of possible interventions: Specify the distribution for a manipulated population Counterfactuals Assume intervention has not been performed, e.g. no data from manipulated population Causation and Prediction Challenge: Training data from unmanipulated population (Structural) intervention is performed System stabilizes

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 4 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causal Discovery and Prediction Previous focus: predicting the effects of possible interventions: Specify the distribution for a manipulated population Counterfactuals Assume intervention has not been performed, e.g. no data from manipulated population Causation and Prediction Challenge: Training data from unmanipulated population (Structural) intervention is performed System stabilizes Draw i.i.d sample for predictors from manipulated population Predict target using predictor values from stabilized manipulated distribution

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 4 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causation and Prediction Challenge Results: Participants used causal methods and methods which ignore causality

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 5 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causation and Prediction Challenge Results: Participants used causal methods and methods which ignore causality Some top-ranking participants did not use causal methods, i.e. support vector machines (for feature selection and classification) Other participants using causal methods did not do as well

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 5 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causation and Prediction Challenge Results: Participants used causal methods and methods which ignore causality Some top-ranking participants did not use causal methods, i.e. support vector machines (for feature selection and classification) Other participants using causal methods did not do as well Questions: Is causality useful for standard prediction tasks?

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 5 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causation and Prediction Challenge Results: Participants used causal methods and methods which ignore causality Some top-ranking participants did not use causal methods, i.e. support vector machines (for feature selection and classification) Other participants using causal methods did not do as well Questions: Is causality useful for standard prediction tasks? Is it useful in practice?

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 5 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causation and Prediction Challenge Results: Participants used causal methods and methods which ignore causality Some top-ranking participants did not use causal methods, i.e. support vector machines (for feature selection and classification) Other participants using causal methods did not do as well Questions: Is causality useful for standard prediction tasks? Is it useful in practice? Is this a realistic scenario?

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 5 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Causation and Prediction Challenge Results: Participants used causal methods and methods which ignore causality Some top-ranking participants did not use causal methods, i.e. support vector machines (for feature selection and classification) Other participants using causal methods did not do as well Questions: Is causality useful for standard prediction tasks? Is it useful in practice? Is this a realistic scenario? Possible Explanations: Sampling error, overfitting Parametric assumptions do not hold, i.e. linearity, Gaussianity Prediction for target is invariant under the manipulation.

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 5 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Invariance of prediction under manipulations Simple example:

X Y

Bayes optimal prediction for Y is P(Y|X)

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 6 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Invariance of prediction under manipulations Simple example:

X Y

Bayes optimal prediction for Y is P(Y|X) Manipulating X does not change distribution of P(Y|X), still Bayes

• ptimal

Prediction (once system stabilizes) is invariant under manipulation

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 6 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Invariance of prediction under manipulations Simple example:

X Y

Bayes optimal prediction for Y is P(Y|X) Manipulating Y does change distribution of P(Y|X), Y depends on manipulation Incorrect predictions in stabilized manipulated population

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 7 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Terminology

CiliaDam LungCapac BreathDis Pollution HeartDis Genotype Smoker Income Parent

Predict CiliaDam

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 8 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Terminology

CiliaDam LungCapac BreathDis Smoker Pollution HeartDis Genotype Income Parent

Parents of CiliaDam

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 9 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Terminology

CiliaDam LungCapac BreathDis Pollution HeartDis Genotype Smoker Income Parent

Children of CiliaDam

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 10 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Terminology

CiliaDam LungCapac BreathDis Genotype HeartDis Pollution Smoker Income Parent

Coparents (spouses) of CiliaDam

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 11 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Terminology

CiliaDam LungCapac BreathDis Genotype HeartDis Smoker Pollution Income Parent

Definition In a causal Bayesian network B = G, P over variables V, the Markov Blanket for X ∈ V is the minimal set of variables MBG

X ⊆ V/{X} such that

X ⊥ ⊥ V/MBG

X | MBG X.

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 12 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Terminology

CiliaDam LungCapac BreathDis Genotype HeartDis Smoker Pollution Income Parent

Definition In a causal Bayesian network B = G, P over variables V, the Markov Blanket for X ∈ V is the minimal set of variables MBG

X ⊆ V/{X} such that

X ⊥ ⊥ V/MBG

X | MBG X.

Theorem (Pearl, 1988) The Markov blanket for X consists

• f the parents, children and

coparents of X in G.

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 12 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Interventions

Pollution HeartDis CiliaDam BreathDis LungCapac Genotype Smoker Income Parent Policy(Smoker)

Policy(Smoker)=0

Pollution HeartDis CiliaDam BreathDis LungCapac Genotype Smoker Income Parent Policy(Smoker)

Policy(Smoker)=1

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 13 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conditions for invariance of prediction under manipulations

CiliaDam LungCapac BreathDis Genotype HeartDis Smoker Pollution Income Parent

Theorem (Prediction invariance) In a causal Bayesian network B = G, P over variables V, let T ∈ V be a target, X ⊆ V a set of predictor variables, and Y ⊆ V the set of manipulated variables. If X ⊇ MBG

T and ∀Y ∈ Y, Y = T and

Y / ∈ Children(T), then prediction

• f T using X is invariant under the

manipulation.

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 14 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conditions for invariance of prediction under manipulations P(T | X) = P(T | MBG

T)

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 15 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conditions for invariance of prediction under manipulations P(T | X) = P(T | MBG

T)

= P(T, MBG

T)

• T P(T, MBG

T)

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 15 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conditions for invariance of prediction under manipulations P(T | X) = P(T | MBG

T)

= P(T, MBG

T)

• T P(T, MBG

T)

=

• X∈T∪Children(T)∪Parents(T)∪Coparents(T) P(X | Parents(T))
• T
• X∈T∪Children(T)∪Parents(T)∪Coparents(T) P(X | Parents(T))

in the Markov blanket subgraph

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 15 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conditions for invariance of prediction under manipulations P(T | X) = P(T | MBG

T)

= P(T, MBG

T)

• T P(T, MBG

T)

=

• X∈T∪Children(T)∪Parents(T)∪Coparents(T) P(X | Parents(T))
• T
• X∈T∪Children(T)∪Parents(T)∪Coparents(T) P(X | Parents(T))

in the Markov blanket subgraph . . . =

• X∈T∪Children(T) P(X | Parents(T))
• T
• X∈T∪Children(T) P(X | Parents(T))

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 15 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Correcting for manipulations

CiliaDam LungCapac BreathDis Genotype HeartDis Smoker Pollution Income Parent Policy(BreathDis)

Policy(BreathDis) = 0 Theorem (Causal correction) In a causal Bayesian network B = G, P over variables V, let T be a target and Y ⊆ V the set of manipulated variables. P

• T | MBG(Policy(Y))

T

• , is invariant

under the manipulation of Y if ∄Y ∈ Y, such that Y ∈ Children(T) and Y is an ancestor of some C ∈ Children(T) ∩ V/Y.

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 16 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Correcting for manipulations

CiliaDam LungCapac BreathDis Genotype HeartDis Smoker Pollution Income Parent Policy(BreathDis)

Policy(BreathDis) = 1 Theorem (Causal correction) In a causal Bayesian network B = G, P over variables V, let T be a target and Y ⊆ V the set of manipulated variables. P

• T | MBG(Policy(Y))

T

• , is invariant

under the manipulation of Y if ∄Y ∈ Y, such that Y ∈ Children(T) and Y is an ancestor of some C ∈ Children(T) ∩ V/Y.

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 17 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Correcting for manipulations

CiliaDam LungCapac BreathDis Genotype HeartDis Smoker Pollution Income Parent Policy(LungCapac)

Policy(BreathDis) = 0 Theorem (Causal correction) In a causal Bayesian network B = G, P over variables V, let T be a target and Y ⊆ V the set of manipulated variables. P

• T | MBG(Policy(Y))

T

• , is invariant

under the manipulation of Y if ∄Y ∈ Y, such that Y ∈ Children(T) and Y is an ancestor of some C ∈ Children(T) ∩ V/Y.

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 18 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Correcting for manipulations

CiliaDam LungCapac BreathDis HeartDis Smoker Pollution Genotype Income Parent Policy(LungCapac)

Policy(BreathDis) = 1 Make Correction! Theorem (Causal correction) In a causal Bayesian network B = G, P over variables V, let T be a target and Y ⊆ V the set of manipulated variables. P

• T | MBG(Policy(Y))

T

• , is invariant

under the manipulation of Y if ∄Y ∈ Y, such that Y ∈ Children(T) and Y is an ancestor of some C ∈ Children(T) ∩ V/Y.

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 19 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Model for experiments

T C0 C1 C2 C3 C4 C5 C6 C7 C8 C9 GC0 GC1 GC2 GC3 GC4 GC5 GC6 P0 S0 P1 P2 P3 P4 S1 GP0 GP1 GP2 GP3 GP4 N0 U0 When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 20 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Experiments Method: Train causal and noncausal prediction methods on unmanipulated population (linear Gaussians)

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 21 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Experiments Method: Train causal and noncausal prediction methods on unmanipulated population (linear Gaussians) Manipulate 0, 5, 10 random nonchildren of T (including Markov blanket)

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 21 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Experiments Method: Train causal and noncausal prediction methods on unmanipulated population (linear Gaussians) Manipulate 0, 5, 10 random nonchildren of T (including Markov blanket) Manipulate 0, . . . , 9 children of T in addition

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 21 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Experiments Method: Train causal and noncausal prediction methods on unmanipulated population (linear Gaussians) Manipulate 0, 5, 10 random nonchildren of T (including Markov blanket) Manipulate 0, . . . , 9 children of T in addition Predict T from manipulated distribution

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 21 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Experiments Method: Train causal and noncausal prediction methods on unmanipulated population (linear Gaussians) Manipulate 0, 5, 10 random nonchildren of T (including Markov blanket) Manipulate 0, . . . , 9 children of T in addition Predict T from manipulated distribution Hypotheses: Noncausal methods will be equivalent or better when no children are manipulated

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 21 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Experiments Method: Train causal and noncausal prediction methods on unmanipulated population (linear Gaussians) Manipulate 0, 5, 10 random nonchildren of T (including Markov blanket) Manipulate 0, . . . , 9 children of T in addition Predict T from manipulated distribution Hypotheses: Noncausal methods will be equivalent or better when no children are manipulated Causal methods will do increasingly better than noncausal methods as more children are manipulated

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 21 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Differences between distributions

1 2 3 4 5 6 7 8 9 100 200 300 400 500 600 700 Number of manipulated children of T Average squared difference between predictions 0 manipulated non−children of T 5 manipulated non−children of T 10 manipulated non−children of T

Squared difference between ground truth predictions for T using unmanipulated and manipulated model

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 22 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Prediction methods Noncausal Methods: LR-ALL linear regression using all predictors LR-MB linear regression using only the Markov blanket LASSO “least absolute shrinkage and selection operator” SVR-RBF support vector regression using radial kernel RVR-RBF relevance vector regression using radial kernel

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 23 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Prediction methods Noncausal Methods: LR-ALL linear regression using all predictors LR-MB linear regression using only the Markov blanket LASSO “least absolute shrinkage and selection operator” SVR-RBF support vector regression using radial kernel RVR-RBF relevance vector regression using radial kernel Causal Methods: LR-MB/C linear regression with Markov blanket correcting for manipulated children LR-MB/C* linear regression with Markov blanket correcting for manipulated children and active paths to unmanipulated children

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 23 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Total prediction error

1 2 3 4 5 6 7 8 9 50 100 150 200 250 300 Number of manipulated children of T Mean squared error LR−ALL LR−MB LASSO SVR−RBF RVR−RBF LR−MB/C LR−MB/C*

0 Manipulated Nonchildren of T

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 24 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Total prediction error

1 2 3 4 5 6 7 8 9 50 100 150 200 250 Number of manipulated children of T Mean squared error LR−ALL LR−MB LASSO SVR−RBF RVR−RBF LR−MB/C LR−MB/C*

5 Manipulated Nonchildren of T

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 25 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Total prediction error

1 2 3 4 5 6 7 8 9 50 100 150 200 250 Number of manipulated children of T Mean squared error LR−ALL LR−MB LASSO SVR−RBF RVR−RBF LR−MB/C LR−MB/C*

10 Manipulated Nonchildren of T

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 26 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Nonlinear data Repeated previous simulations adding nonlinear dependencies

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 27 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Nonlinear data Repeated previous simulations adding nonlinear dependencies Results so far inconclusive In general, nonparametric methods do best, though poor performance in all cases

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 27 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conclusions Is causality relevant for prediction?

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 28 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conclusions Is causality relevant for prediction? Yes, as long as a noncausal method is not invariant under the manipulation

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 28 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conclusions Is causality relevant for prediction? Yes, as long as a noncausal method is not invariant under the manipulation But causality is needed to know noncausal methods are invariant

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 28 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conclusions Is causality relevant for prediction? Yes, as long as a noncausal method is not invariant under the manipulation But causality is needed to know noncausal methods are invariant In practice? Tradeoff between errors related to causality and errors related to parametric assumptions, overfitting, etc.

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 28 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conclusions Is causality relevant for prediction? Yes, as long as a noncausal method is not invariant under the manipulation But causality is needed to know noncausal methods are invariant In practice? Tradeoff between errors related to causality and errors related to parametric assumptions, overfitting, etc. Noncausal prediction may be frequently invariant under manipulations or

• nly make small errors related to causality

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 28 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conclusions Is causality relevant for prediction? Yes, as long as a noncausal method is not invariant under the manipulation But causality is needed to know noncausal methods are invariant In practice? Tradeoff between errors related to causality and errors related to parametric assumptions, overfitting, etc. Noncausal prediction may be frequently invariant under manipulations or

• nly make small errors related to causality

Advantages of nonparametric methods and methods which deal with

• verfitting well may cancel out errors related to causality

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 28 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conclusions Is causality relevant for prediction? Yes, as long as a noncausal method is not invariant under the manipulation But causality is needed to know noncausal methods are invariant In practice? Tradeoff between errors related to causality and errors related to parametric assumptions, overfitting, etc. Noncausal prediction may be frequently invariant under manipulations or

• nly make small errors related to causality

Advantages of nonparametric methods and methods which deal with

• verfitting well may cancel out errors related to causality

Many other variables involved, analysis incomplete

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 28 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conclusions Is causality relevant for prediction? Yes, as long as a noncausal method is not invariant under the manipulation But causality is needed to know noncausal methods are invariant In practice? Tradeoff between errors related to causality and errors related to parametric assumptions, overfitting, etc. Noncausal prediction may be frequently invariant under manipulations or

• nly make small errors related to causality

Advantages of nonparametric methods and methods which deal with

• verfitting well may cancel out errors related to causality

Many other variables involved, analysis incomplete Future directions for causal discovery: Methods which deal with overfitting well

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 28 / 28

Causation and Prediction Invariance of prediction functions Experimental Results Conclusions

Conclusions Is causality relevant for prediction? Yes, as long as a noncausal method is not invariant under the manipulation But causality is needed to know noncausal methods are invariant In practice? Tradeoff between errors related to causality and errors related to parametric assumptions, overfitting, etc. Noncausal prediction may be frequently invariant under manipulations or

• nly make small errors related to causality

Advantages of nonparametric methods and methods which deal with

• verfitting well may cancel out errors related to causality

Many other variables involved, analysis incomplete Future directions for causal discovery: Methods which deal with overfitting well Less restrictive parametric assumptions

When causality matters for prediction Tillman and Spirtes NIPS 2008 Workshop on Causality 28 / 28