Causal Discovery from Observational Data Brady Neal - - PowerPoint PPT Presentation

Causal Discovery from Observational Data

Brady Neal

causalcourse.com

Brady Neal / 45

What if we don’t have the causal graph?

2

Brady Neal / 45

What if we don’t have the causal graph?

Causal discovery: data causal graph

2

Brady Neal / 45

What if we don’t have the causal graph?

Causal discovery: data causal graph Structure identification: identifying the causal graph

2

Brady Neal / 45 3

Independence-Based Causal Discovery

Assumptions Markov Equivalence and Main Theorem The PC Algorithm Can We Do Better?

Semi-Parametric Causal Discovery

No Identifiability Without Parametric Assumptions Linear Non-Gaussian Setting Nonlinear Additive Noise Setting

Brady Neal / 45 4

Independence-Based Causal Discovery

Assumptions Markov Equivalence and Main Theorem The PC Algorithm Can We Do Better?

Semi-Parametric Causal Discovery

No Identifiability Without Parametric Assumptions Linear Non-Gaussian Setting Nonlinear Additive Noise Setting

Assumptions

Brady Neal / 45

Faithfulness Assumption

5 Assumptions

Brady Neal / 45

Faithfulness Assumption

Recall the Markov assumption:

5

⊥G Y | Z = ⇒ X ⊥ ⊥P Y | Z

Assumptions

Brady Neal / 45

Faithfulness Assumption

Recall the Markov assumption:

5

⊥G Y | Z = ⇒ X ⊥ ⊥P Y | Z

Causal graph Data

Assumptions

Brady Neal / 45

Faithfulness Assumption

Recall the Markov assumption:

5

⊥G Y | Z = ⇒ X ⊥ ⊥P Y | Z

Causal graph Data Causal graph Data

Assumptions

Brady Neal / 45

Faithfulness Assumption

Recall the Markov assumption:

5

⊥G Y | Z = ⇒ X ⊥ ⊥P Y | Z

Causal graph Data Causal graph Data

⊥G Y | Z ⇐ = X ⊥ ⊥P Y | Z

Faithfulness:

Assumptions

Brady Neal / 45

Violation of Faithfulness

6 Assumptions

Brady Neal / 45

Violation of Faithfulness

6

⊥G Y | Z ⇐ = X ⊥ ⊥P Y | Z

Faithfulness:

Assumptions

Brady Neal / 45

Violation of Faithfulness

6

A B C D

⊥G Y | Z ⇐ = X ⊥ ⊥P Y | Z

Faithfulness:

Assumptions

Brady Neal / 45

Violation of Faithfulness

6

A B C D

⊥G Y | Z ⇐ = X ⊥ ⊥P Y | Z

Faithfulness:

Assumptions

Brady Neal / 45

Violation of Faithfulness

6

A B C D

⊥G Y | Z ⇐ = X ⊥ ⊥P Y | Z

Faithfulness:

⊥ D

Assumptions

Brady Neal / 45

Violation of Faithfulness

6

A B C D

⊥G Y | Z ⇐ = X ⊥ ⊥P Y | Z

Faithfulness:

⊥ D but A and D aren’t d-separated

Assumptions

Brady Neal / 45

Violation of Faithfulness

6

A B C D α γ β δ

⊥G Y | Z ⇐ = X ⊥ ⊥P Y | Z

Faithfulness:

⊥ D but A and D aren’t d-separated

Assumptions

Brady Neal / 45

Violation of Faithfulness

6

A B C D α γ β δ

C := γA D := βB + δC

⊥G Y | Z ⇐ = X ⊥ ⊥P Y | Z

Faithfulness:

⊥ D but A and D aren’t d-separated

Assumptions

Brady Neal / 45

Violation of Faithfulness

6

A B C D α γ β δ

C := γA D := βB + δC

⊥G Y | Z ⇐ = X ⊥ ⊥P Y | Z

Faithfulness:

⊥ D but A and D aren’t d-separated

Assumptions

Brady Neal / 45

Violation of Faithfulness

6

A B C D α γ β δ

C := γA D := βB + δC

⊥G Y | Z ⇐ = X ⊥ ⊥P Y | Z

Faithfulness:

⊥ D but A and D aren’t d-separated Paths cancel if

Assumptions

Brady Neal / 45

Causal Sufficiency and Acyclicity

7 Assumptions

Brady Neal / 45

Causal Sufficiency and Acyclicity

Causal Sufficiency: there are no unobserved confounders of any of the variables in the graph.

7 Assumptions

Brady Neal / 45

Causal Sufficiency and Acyclicity

Causal Sufficiency: there are no unobserved confounders of any of the variables in the graph. Acyclicity: still assuming there are no cycles in the graph.

7 Assumptions

Brady Neal / 45

Causal Sufficiency and Acyclicity

Causal Sufficiency: there are no unobserved confounders of any of the variables in the graph. Acyclicity: still assuming there are no cycles in the graph. All assumptions:

• Markov assumption
• Faithfulness
• Causal sufficiency
• Acyclicity

7 Assumptions

Brady Neal / 45

Causal Sufficiency and Acyclicity

Causal Sufficiency: there are no unobserved confounders of any of the variables in the graph. Acyclicity: still assuming there are no cycles in the graph. All assumptions:

• Markov assumption
• Faithfulness
• Causal sufficiency
• Acyclicity

7 Assumptions

Question: Why is the Markov assumption (plus causal sufficiency and acyclicity) not enough for learning causal graphs from data?

Brady Neal / 45 9

Independence-Based Causal Discovery

Assumptions Markov Equivalence and Main Theorem The PC Algorithm Can We Do Better?

Semi-Parametric Causal Discovery

No Identifiability Without Parametric Assumptions Linear Non-Gaussian Setting Nonlinear Additive Noise Setting

Markov Equivalence and Main Theorem

Brady Neal / 45

Chains and Forks Encode Same Independencies

10

X1 X2 X3

X1 X2 X3

X2 X1 X3

Markov Equivalence and Main Theorem

Brady Neal / 45

Chains and Forks Encode Same Independencies

10

X1 X2 X3

X1 X2 X3

X2 X1 X3

⊥ X3 | X2

Markov:

Markov Equivalence and Main Theorem

Brady Neal / 45

Chains and Forks Encode Same Independencies

10

X1 X2 X3

X1 X2 X3

X2 X1 X3

⊥ X3 | X2

Markov:

? ? X2

? ? X3

Minimality:

and

Markov Equivalence and Main Theorem

Brady Neal / 45

Chains and Forks Encode Same Independencies

10

X1 X2 X3

X1 X2 X3

X2 X1 X3

⊥ X3 | X2

Markov: Faithfulness:

? ? X3

? ? X2

? ? X3

Minimality:

and

Markov Equivalence and Main Theorem

Brady Neal / 45

Chains and Forks Encode Same Independencies

10

X1 X2 X3

X1 X2 X3

X2 X1 X3

⊥ X3 | X2

Markov: Faithfulness:

? ? X3

? ? X2

? ? X3

Minimality:

and

Markov equivalent (all in the same Markov equivalence class)

Markov Equivalence and Main Theorem

Brady Neal / 45

Immoralities are Special

11

X1 X2 X3

X1 X2 X3

X2 X1 X3

⊥ X3 | X2

? ? X3

Markov equivalence class where and

Markov Equivalence and Main Theorem

Brady Neal / 45

Immoralities are Special

11

X1 X2 X3

X1 X2 X3

X1 X2 X3

X2 X1 X3

⊥ X3 | X2

? ? X3

Markov equivalence class where and

Markov Equivalence and Main Theorem

Brady Neal / 45

Immoralities are Special

11

X1 X2 X3

X1 X2 X3

X1 X2 X3

X2 X1 X3

⊥ X3 | X2

? ? X3

Markov equivalence class where and

⊥ X3

Markov Equivalence and Main Theorem

Brady Neal / 45

Immoralities are Special

11

X1 X2 X3

X1 X2 X3

X1 X2 X3

X2 X1 X3

⊥ X3 | X2

? ? X3

Markov equivalence class where and and

⊥ X3

? ? X3 | X2

Markov Equivalence and Main Theorem

Brady Neal / 45

Immoralities are Special

11

X1 X2 X3

X1 X2 X3

X1 X2 X3

X2 X1 X3

⊥ X3 | X2

? ? X3

Markov equivalence class where and Markov equivalence class where and

⊥ X3

? ? X3 | X2

Markov Equivalence and Main Theorem

Brady Neal / 45

Skeletons

12

X1 X2 X3

X1 X2 X3

X2 X1 X3

⊥ X3 | X2

Markov Equivalence and Main Theorem

Brady Neal / 45

Skeletons

12

X1 X2 X3

X1 X2 X3

X2 X1 X3

X1 X2 X3

⊥ X3 | X2

Markov Equivalence and Main Theorem

Brady Neal / 45

Skeletons

12

X1 X2 X3

X1 X2 X3

X2 X1 X3

X1 X2 X3

X1 X2 X3

⊥ X3 | X2

Markov Equivalence and Main Theorem

Brady Neal / 45

Skeletons

12

X1 X2 X3

X1 X2 X3

X2 X1 X3

X1 X2 X3

X1 X2 X3

⊥ X3 | X2

? ? X3 | X2

Markov Equivalence and Main Theorem

Brady Neal / 45

Skeletons

12

X1 X2 X3

X1 X2 X3

X2 X1 X3

X1 X2 X3

X1 X2 X3

X1 X2 X3

⊥ X3 | X2

? ? X3 | X2

Markov Equivalence and Main Theorem

Brady Neal / 45

Markov Equivalence via Immoral Skeletons

13 Markov Equivalence and Main Theorem

Brady Neal / 45

Markov Equivalence via Immoral Skeletons

Two important graph qualities that we can use to distinguish graphs:

13 Markov Equivalence and Main Theorem

Brady Neal / 45

Markov Equivalence via Immoral Skeletons

Two important graph qualities that we can use to distinguish graphs:

• 1. Immoralities

13 Markov Equivalence and Main Theorem

Brady Neal / 45

Markov Equivalence via Immoral Skeletons

Two important graph qualities that we can use to distinguish graphs:

• 1. Immoralities
• 2. Skeleton

13 Markov Equivalence and Main Theorem

Brady Neal / 45

Markov Equivalence via Immoral Skeletons

Two important graph qualities that we can use to distinguish graphs:

• 1. Immoralities
• 2. Skeleton

Theorem: Two graphs are Markov equivalent if and only if they have the same skeleton and same immoralities (Verma & Pearl, 1990; Frydenburg, 1990).

13 Markov Equivalence and Main Theorem

Brady Neal / 45

Markov Equivalence via Immoral Skeletons

Two important graph qualities that we can use to distinguish graphs:

• 1. Immoralities
• 2. Skeleton

Theorem: Two graphs are Markov equivalent if and only if they have the same skeleton and same immoralities (Verma & Pearl, 1990; Frydenburg, 1990). Essential graph (aka CPDAG): skeleton + immoralities

13 Markov Equivalence and Main Theorem

Question: What graphs are Markov equivalent to the basic fork graph?

X2 X1 X3

Question: What graphs are Markov equivalent to the basic immorality?

X1 X2 X3

Question: What graphs is the following graph Markov equivalent to?

A B C D

Question: What graphs is the following graph Markov equivalent to?

A B C D

Question: What graphs is the following graph Markov equivalent to?

A B C D

A B C D

Question: What graphs is the following graph Markov equivalent to?

A B C D

A B C D

A B C D

Question: What graphs is the following graph Markov equivalent to?

A B C D

A B C D

A B C D

A B C D

C A D B

Question: What graphs is the following graph Markov equivalent to?

Question: Give a few graphs that the following graph is Markov equivalent to:

C A E D B F G H

Brady Neal / 45 19

Independence-Based Causal Discovery

Assumptions Markov Equivalence and Main Theorem The PC Algorithm Can We Do Better?

Semi-Parametric Causal Discovery

No Identifiability Without Parametric Assumptions Linear Non-Gaussian Setting Nonlinear Additive Noise Setting

The PC Algorithm

Brady Neal / 45

The PC Algorithm: Overview

20 The PC Algorithm

Brady Neal / 45

The PC Algorithm: Overview

20

C A D B E

True graph

The PC Algorithm

Brady Neal / 45

The PC Algorithm: Overview

Start with complete undirected graph

20

C A D B E

True graph

The PC Algorithm

Brady Neal / 45

The PC Algorithm: Overview

Start with complete undirected graph

20

C A D B E

C A D B E

True graph

The PC Algorithm

Brady Neal / 45

The PC Algorithm: Overview

Start with complete undirected graph Three steps:

20

C A D B E

C A D B E

True graph

The PC Algorithm

Brady Neal / 45

The PC Algorithm: Overview

Start with complete undirected graph Three steps:

• 1. Identify the skeleton

20

C A D B E

True graph

C A D B E

The PC Algorithm

Brady Neal / 45

The PC Algorithm: Overview

Start with complete undirected graph Three steps:

• 1. Identify the skeleton
• 2. Identify immoralities and orient them

20

C A D B E

True graph

C A D B E

The PC Algorithm

Brady Neal / 45

The PC Algorithm: Overview

Start with complete undirected graph Three steps:

• 1. Identify the skeleton
• 2. Identify immoralities and orient them
• 3. Orient qualifying edges that are incident
• n colliders

20

C A D B E

True graph

C A D B E

The PC Algorithm

Brady Neal / 45

Identifying the Skeleton

21 The PC Algorithm

Brady Neal / 45

Identifying the Skeleton

Start with complete undirected graph and remove edges X – Y where for some (potentially empty) conditioning set Z, starting with the empty conditioning set and increasing the size

21

⊥ Y | Z

The PC Algorithm

Brady Neal / 45

Identifying the Skeleton

Start with complete undirected graph and remove edges X – Y where for some (potentially empty) conditioning set Z, starting with the empty conditioning set and increasing the size

21

C A D B E

True graph

⊥ Y | Z

The PC Algorithm

Brady Neal / 45

Identifying the Skeleton

Start with complete undirected graph and remove edges X – Y where for some (potentially empty) conditioning set Z, starting with the empty conditioning set and increasing the size

21

C A D B E

True graph

⊥ Y | Z

C A D B E

The PC Algorithm

Brady Neal / 45

Identifying the Skeleton

Start with complete undirected graph and remove edges X – Y where for some (potentially empty) conditioning set Z, starting with the empty conditioning set and increasing the size

21

C A D B E

True graph

⊥ Y | Z

C A D B E

⊥ B

The PC Algorithm

Brady Neal / 45

Identifying the Skeleton

Start with complete undirected graph and remove edges X – Y where for some (potentially empty) conditioning set Z, starting with the empty conditioning set and increasing the size

21

C A D B E

True graph

⊥ Y | Z

C A D B E

⊥ B | {}

The PC Algorithm

Brady Neal / 45

Identifying the Skeleton

Start with complete undirected graph and remove edges X – Y where for some (potentially empty) conditioning set Z, starting with the empty conditioning set and increasing the size

21

C A D B E

True graph

⊥ Y | Z

C A D B E

⊥ B | {}

The PC Algorithm

Brady Neal / 45

Identifying the Skeleton

Start with complete undirected graph and remove edges X – Y where for some (potentially empty) conditioning set Z, starting with the empty conditioning set and increasing the size

21

C A D B E

True graph

⊥ Y | Z

C A D B E

⊥ B | {}

X ⊥ ⊥ Y | {C}

The PC Algorithm

Brady Neal / 45

Identifying the Skeleton

Start with complete undirected graph and remove edges X – Y where for some (potentially empty) conditioning set Z, starting with the empty conditioning set and increasing the size

21

C A D B E

True graph

⊥ Y | Z

C A D B E

⊥ B | {}

X ⊥ ⊥ Y | {C}

The PC Algorithm

Brady Neal / 45

C A D B E

Identifying the Immoralities

22

C A D B E

True graph

The PC Algorithm

Brady Neal / 45

Now for any paths X – Z – Y in our working graph where the following are true:

C A D B E

Identifying the Immoralities

22

C A D B E

True graph

The PC Algorithm

Brady Neal / 45

Now for any paths X – Z – Y in our working graph where the following are true:

• 1. We discovered that there is no edge between

X and Y in our previous step.

C A D B E

Identifying the Immoralities

22

C A D B E

True graph

The PC Algorithm

Brady Neal / 45

Now for any paths X – Z – Y in our working graph where the following are true:

• 1. We discovered that there is no edge between

X and Y in our previous step.

• 2. Z was not in the conditioning set that makes

X and Y conditionally independent.

C A D B E

Identifying the Immoralities

22

C A D B E

True graph

The PC Algorithm

Brady Neal / 45

Now for any paths X – Z – Y in our working graph where the following are true:

• 1. We discovered that there is no edge between

X and Y in our previous step.

• 2. Z was not in the conditioning set that makes

X and Y conditionally independent. Then, we know X – Z – Y forms an immortality.

C A D B E

Identifying the Immoralities

22

C A D B E

True graph

The PC Algorithm

Brady Neal / 45

Now for any paths X – Z – Y in our working graph where the following are true:

• 1. We discovered that there is no edge between

X and Y in our previous step.

• 2. Z was not in the conditioning set that makes

X and Y conditionally independent. Then, we know X – Z – Y forms an immortality.

C A D B E

Identifying the Immoralities

22

C A D B E

True graph

⊥ B | {}

The PC Algorithm

Brady Neal / 45

Now for any paths X – Z – Y in our working graph where the following are true:

• 1. We discovered that there is no edge between

X and Y in our previous step.

• 2. Z was not in the conditioning set that makes

X and Y conditionally independent. Then, we know X – Z – Y forms an immortality.

Identifying the Immoralities

22

C A D B E

True graph

⊥ B | {}

C A D B E

The PC Algorithm

Brady Neal / 45

C A D B E

Orienting Qualifying Edges Incident on Colliders

23

C A D B E

True graph

C A D B E

The PC Algorithm

Brady Neal / 45

Idea: use fact that we discovered all immoralities to orient more edges

C A D B E

Orienting Qualifying Edges Incident on Colliders

23

C A D B E

True graph

C A D B E

The PC Algorithm

Brady Neal / 45

Idea: use fact that we discovered all immoralities to orient more edges Any edge part of a partially directed path of the form , where there is no edge connecting X and Y can be oriented as

C A D B E

Orienting Qualifying Edges Incident on Colliders

23

C A D B E

True graph

C A D B E

The PC Algorithm

Brady Neal / 45

Idea: use fact that we discovered all immoralities to orient more edges Any edge part of a partially directed path of the form , where there is no edge connecting X and Y can be oriented as

C A D B E

Orienting Qualifying Edges Incident on Colliders

23

C A D B E

True graph

C A D B E

The PC Algorithm

Brady Neal / 45

Removing Assumptions

24 The PC Algorithm

Brady Neal / 45

Removing Assumptions

No assumed causal sufficiency: FCI algorithm (Spirtes et al., 2001)

24 The PC Algorithm

Brady Neal / 45

Removing Assumptions

No assumed causal sufficiency: FCI algorithm (Spirtes et al., 2001) No assumed acyclicity: CCD algorithm (Richardson, 1996)

24 The PC Algorithm

Brady Neal / 45

Removing Assumptions

No assumed causal sufficiency: FCI algorithm (Spirtes et al., 2001) No assumed acyclicity: CCD algorithm (Richardson, 1996) Neither causal sufficiency nor acyclicity: SAT-based causal discovery (Hyttinen et al., 2013; 2014)

24 The PC Algorithm

Brady Neal / 45

Hardness of Conditional Independence Testing

25 The PC Algorithm

Brady Neal / 45

Hardness of Conditional Independence Testing

Independence-based causal discovery algorithms rely on accurate conditional independence testing.

25 The PC Algorithm

Brady Neal / 45

Hardness of Conditional Independence Testing

Independence-based causal discovery algorithms rely on accurate conditional independence testing. Conditional independence testing is simple if we have infinite data.

25 The PC Algorithm

Brady Neal / 45

Hardness of Conditional Independence Testing

Independence-based causal discovery algorithms rely on accurate conditional independence testing. Conditional independence testing is simple if we have infinite data. However, it is a quite hard problem with finite data, and it can sometimes require a lot of data to get accurate test results (Shah & Peters, 2020).

25 The PC Algorithm

X2 X1 X3

X1 X2 X3

A B C D

C A D B

Questions:

• 1. What are the essential graphs of the following graphs?

X1 X2 X3

A B C D

C A D B

Questions:

• 1. What are the essential graphs of the following graphs?

X2 X1 X3

X1 X2 X3

A B C D

C A D B

Questions:

• 1. What are the essential graphs of the following graphs?

X2 X1 X3

X1 X2 X3

C A D B

Questions:

• 1. What are the essential graphs of the following graphs?

A B C D

X2 X1 X3

X1 X2 X3

C A D B

Questions:

• 1. What are the essential graphs of the following graphs?

A B C D

X2 X1 X3

X2 X1 X3

X1 X2 X3

A B C D

C A D B

Questions:

• 1. What are the essential graphs of the following graphs?
• 2. Walk through the steps of PC to get them.

Question: What is the essential graph for this graph?

C A E D B F G H

Brady Neal / 45 28

Independence-Based Causal Discovery

Assumptions Markov Equivalence and Main Theorem The PC Algorithm Can We Do Better?

Semi-Parametric Causal Discovery

No Identifiability Without Parametric Assumptions Linear Non-Gaussian Setting Nonlinear Additive Noise Setting

Can We Do Better?

Brady Neal / 45

Can We Do Better?

With faithfulness, we saw we can identify the essential graph (Markov equivalence class).

29 Can We Do Better?

Brady Neal / 45

Can We Do Better?

With faithfulness, we saw we can identify the essential graph (Markov equivalence class). If we have multinomial distributions (Meek, 1995) or linear Gaussian structural equations (Geiger & Pearl, 1988), we can only identify a graph up to its Markov equivalence class.

29 Can We Do Better?

Brady Neal / 45

Can We Do Better?

With faithfulness, we saw we can identify the essential graph (Markov equivalence class). If we have multinomial distributions (Meek, 1995) or linear Gaussian structural equations (Geiger & Pearl, 1988), we can only identify a graph up to its Markov equivalence class. What about non-Gaussian structural equations? Or nonlinear structural equations?

29 Can We Do Better?

Brady Neal / 45 30

Independence-Based Causal Discovery

Assumptions Markov Equivalence and Main Theorem The PC Algorithm Can We Do Better?

Semi-Parametric Causal Discovery

No Identifiability Without Parametric Assumptions Linear Non-Gaussian Setting Nonlinear Additive Noise Setting

Semi-Parametric Causal Discovery

Brady Neal / 45

Issues with Independence-Based Causal Discovery

31 Semi-Parametric Causal Discovery

Brady Neal / 45

Issues with Independence-Based Causal Discovery

• Requires faithfulness assumption

31 Semi-Parametric Causal Discovery

Brady Neal / 45

Issues with Independence-Based Causal Discovery

• Requires faithfulness assumption
• Large samples can be necessary for conditional independence tests

31 Semi-Parametric Causal Discovery

Brady Neal / 45

Issues with Independence-Based Causal Discovery

• Requires faithfulness assumption
• Large samples can be necessary for conditional independence tests
• Only identifies the Markov equivalence class

31 Semi-Parametric Causal Discovery

Brady Neal / 45 32

Independence-Based Causal Discovery

Assumptions Markov Equivalence and Main Theorem The PC Algorithm Can We Do Better?

Semi-Parametric Causal Discovery

No Identifiability Without Parametric Assumptions Linear Non-Gaussian Setting Nonlinear Additive Noise Setting

No Identifiability Without Parametric Assumptions

Brady Neal / 45

Two Variable Case: Markov Equivalence

33

Infinite data:

No Identifiability Without Parametric Assumptions

Brady Neal / 45

Two Variable Case: Markov Equivalence

33

X Y

X Y

vs.

Infinite data:

No Identifiability Without Parametric Assumptions

Brady Neal / 45

Two Variable Case: Markov Equivalence

33

X Y

X Y

X Y

vs.

Infinite data: Essential graph:

No Identifiability Without Parametric Assumptions

Brady Neal / 45

Two Variable Case: SCMs Perspective

34 No Identifiability Without Parametric Assumptions

Brady Neal / 45

Two Variable Case: SCMs Perspective

Proposition: For every joint distribution on two real-valued random variables, there is an SCM in either direction that generates data consistent with .

34

No Identifiability Without Parametric Assumptions

Brady Neal / 45

Two Variable Case: SCMs Perspective

Proposition: For every joint distribution on two real-valued random variables, there is an SCM in either direction that generates data consistent with . Mathematically, there exists a function such that and there exists a function such that where and are real-valued random variables.

34

X ⊥ ⊥ UY

Y ⊥ ⊥ UX

No Identifiability Without Parametric Assumptions

Brady Neal / 45

Two Variable Case: SCMs Perspective

Proposition: For every joint distribution on two real-valued random variables, there is an SCM in either direction that generates data consistent with . Mathematically, there exists a function such that and there exists a function such that where and are real-valued random variables.

34

X ⊥ ⊥ UY

Y ⊥ ⊥ UX

X Y

No Identifiability Without Parametric Assumptions

Brady Neal / 45

Two Variable Case: SCMs Perspective

Proposition: For every joint distribution on two real-valued random variables, there is an SCM in either direction that generates data consistent with . Mathematically, there exists a function such that and there exists a function such that where and are real-valued random variables.

34

X ⊥ ⊥ UY

Y ⊥ ⊥ UX

X Y

X Y

No Identifiability Without Parametric Assumptions

We must make assumptions about the parametric form.

Brady Neal / 45 36

Independence-Based Causal Discovery

Assumptions Markov Equivalence and Main Theorem The PC Algorithm Can We Do Better?

Semi-Parametric Causal Discovery

No Identifiability Without Parametric Assumptions Linear Non-Gaussian Setting Nonlinear Additive Noise Setting

Linear Non-Gaussian Setting

Brady Neal / 45

Linear Non-Gaussian Assumption

37 Linear Non-Gaussian Setting

Brady Neal / 45

Linear Non-Gaussian Assumption

Recall: We cannot hope to identify the graph more precisely than the Markov equivalence class in the linear Gaussian noise setting (Geiger & Pearl, 1988).

37 Linear Non-Gaussian Setting

Brady Neal / 45

Linear Non-Gaussian Assumption

Recall: We cannot hope to identify the graph more precisely than the Markov equivalence class in the linear Gaussian noise setting (Geiger & Pearl, 1988). What if the noise is non-Gaussian?

37 Linear Non-Gaussian Setting

Brady Neal / 45

Linear Non-Gaussian Assumption

Recall: We cannot hope to identify the graph more precisely than the Markov equivalence class in the linear Gaussian noise setting (Geiger & Pearl, 1988). What if the noise is non-Gaussian? Linear Non-Gaussian Assumption: All structural equations (causal mechanisms that generate the data) are of the following form: where f is a linear function, , and is distributed as some non-Gaussian.

37

⊥ U

Linear Non-Gaussian Setting

Brady Neal / 45

Theorem (Shimizu et al., 2006): In the linear non-Gaussian setting, if the true SCM is then there does not exist an SCM in the reverse direction, that can generate data consistent with .

X ⊥ ⊥ U ,

X := g(Y ) + ˜ U , Y ⊥ ⊥ ˜ U ,

Identifiability in Linear Non-Gaussian Setting

38

Linear Non-Gaussian Setting

Brady Neal / 45

Theorem (Shimizu et al., 2006): In the linear non-Gaussian setting, if the true SCM is then there does not exist an SCM in the reverse direction, that can generate data consistent with . See proof in the course book

X ⊥ ⊥ U ,

X := g(Y ) + ˜ U , Y ⊥ ⊥ ˜ U ,

Identifiability in Linear Non-Gaussian Setting

38

Linear Non-Gaussian Setting

Brady Neal / 45 39

Identifiability in Linear Non-Gaussian Setting: Linear Fit

Linear Non-Gaussian Setting

Brady Neal / 45 39

Identifiability in Linear Non-Gaussian Setting: Linear Fit

Linear Non-Gaussian Setting

Brady Neal / 45 39

Identifiability in Linear Non-Gaussian Setting: Linear Fit

Linear Non-Gaussian Setting

Brady Neal / 45 39

X := g(Y ) + ˜ U ,

Identifiability in Linear Non-Gaussian Setting: Linear Fit

Linear Non-Gaussian Setting

Brady Neal / 45 39

X := g(Y ) + ˜ U ,

Identifiability in Linear Non-Gaussian Setting: Linear Fit

Linear Non-Gaussian Setting

Brady Neal / 45 40

X ⊥ ⊥ U

Identifiability in Linear Non-Gaussian Setting: Residuals

R e s i d u a l s

Linear Non-Gaussian Setting

Brady Neal / 45 40

X ⊥ ⊥ U

Identifiability in Linear Non-Gaussian Setting: Residuals

X := g(Y ) + ˜ U , Y 6 ? ? ˜ U ,

R e s i d u a l s

Linear Non-Gaussian Setting

Brady Neal / 45

Linear Non-Gaussian Identifiability Extensions

41 Linear Non-Gaussian Setting

Brady Neal / 45

Linear Non-Gaussian Identifiability Extensions

• Multivariate: Shimizu et al., (2006)

41 Linear Non-Gaussian Setting

Brady Neal / 45

Linear Non-Gaussian Identifiability Extensions

• Multivariate: Shimizu et al., (2006)
• Drop causal sufficiency assumption: Hoyer et al. (2008)

41 Linear Non-Gaussian Setting

Brady Neal / 45

Linear Non-Gaussian Identifiability Extensions

• Multivariate: Shimizu et al., (2006)
• Drop causal sufficiency assumption: Hoyer et al. (2008)
• Drop acyclicity assumption: Lacerda et al. (2008)

41 Linear Non-Gaussian Setting

Brady Neal / 45 42

Independence-Based Causal Discovery

Assumptions Markov Equivalence and Main Theorem The PC Algorithm Can We Do Better?

Semi-Parametric Causal Discovery

No Identifiability Without Parametric Assumptions Linear Non-Gaussian Setting Nonlinear Additive Noise Setting

Nonlinear Additive Noise Setting

Brady Neal / 45

Identifiability in Nonlinear Additive Noise Setting

43 Nonlinear Additive Noise Setting

Brady Neal / 45

Identifiability in Nonlinear Additive Noise Setting

Recall: We cannot hope to identify the graph more precisely than the Markov equivalence class in the linear Gaussian noise setting (Geiger & Pearl, 1988).

43 Nonlinear Additive Noise Setting

Brady Neal / 45

Identifiability in Nonlinear Additive Noise Setting

Recall: We cannot hope to identify the graph more precisely than the Markov equivalence class in the linear Gaussian noise setting (Geiger & Pearl, 1988). What if the structural equations are nonlinear?

43 Nonlinear Additive Noise Setting

Brady Neal / 45

Identifiability in Nonlinear Additive Noise Setting

Recall: We cannot hope to identify the graph more precisely than the Markov equivalence class in the linear Gaussian noise setting (Geiger & Pearl, 1988). What if the structural equations are nonlinear? Nonlinear additive noise assumption:

43

Xi := fi(pai) + Ui

where is nonlinear

Nonlinear Additive Noise Setting

Brady Neal / 45

Identifiability in Nonlinear Additive Noise Setting

Recall: We cannot hope to identify the graph more precisely than the Markov equivalence class in the linear Gaussian noise setting (Geiger & Pearl, 1988). What if the structural equations are nonlinear? Nonlinear additive noise assumption: Theorem (Hoyer et al. 2008): Under the Markov assumption, causal sufficiency, acyclicity, the nonlinear additive noise assumption, and a technical condition from Hoyer et al. (2008), we can identify the causal graph.

43

Xi := fi(pai) + Ui

where is nonlinear

Nonlinear Additive Noise Setting

Brady Neal / 45

Post-Nonlinear Setting

Nonlinear additive noise setting:

44

X ⊥ ⊥ U

Nonlinear Additive Noise Setting

Brady Neal / 45

Post-Nonlinear Setting

Nonlinear additive noise setting:

44

X ⊥ ⊥ U

X ⊥ ⊥ U

Nonlinear Additive Noise Setting

Brady Neal / 45

Post-Nonlinear Setting

Nonlinear additive noise setting:

44

X ⊥ ⊥ U

X ⊥ ⊥ U

Nonlinear Additive Noise Setting

Brady Neal / 45

Post-Nonlinear Setting

Nonlinear additive noise setting: Post-nonlinear (Zhang & Hyvärinen, 2009):

44

X ⊥ ⊥ U

X ⊥ ⊥ U

Nonlinear Additive Noise Setting

Brady Neal / 45

Review Articles and Book

• Introduction to the Foundations of Causal Discovery (Eberhardt, 2017)
• Review of Causal Discovery Methods Based on Graphical Models

(Glymour, Zhang, & Spirtes, 2019)

• Elements of Causal Inference (Peters, Janzing, & Schölkopf, 2017)

45