Robustly Disentangled Causal Mechanisms: Validating Deep - - PowerPoint PPT Presentation

Robustly Disentangled Causal Mechanisms:

Validating Deep Representations for Interventional Robustness

Raphael Suter 1, Ðor¯ de Miladinovi´ c 1, Bernhard Schölkopf2, Stefan Bauer 2

1ETH Zurich, 2MPI for Intelligent Systems

ICML 2019 1

Contributions

• Causal Model for Representation Learning
• Interventional Robustness Score
• Visualising Robustness

2

Disentangled Representations

Observation: X ∈ Rn Feature encoding: Z = E(X) ∈ RK, n ≫ K Disentanglement ⇐

⇒ components Zi represent different

sources of variation in X

3

Definition: Disentangled Causal Process

G1 G2

· · ·

GK−1 GK C X

Disentangled Causal Mechanisms:

∀g△

j

p(gi|do(Gj ← g△

j )) = p(gi)

• = p(gi|g△

j )

• 4

Unified Causal Model

Generative Factors

G1 G2

· · ·

GK−1 GK X

· · ·

Z2 Z1 ZK′−1 ZK′

Feature Representation

5

Robust Representation

relevant factors: G1, G2 nuisance factor: GK selected features: Z1, Z2

6

Interventional Robustness

Post Interventional Disagreement d

• E[Zsel|grel)], E[Zsel|grel, do(Gnuis ← g△

nuis)]

• Interventional Robustness Score

normalised score ∈ [0, 1]

7

Theoretical Results

• Properties of a disentangled causal process
• IRS estimation from observational data

D = {(g(i), x(i))}N

i=1

• Handles confounding Gi ← C → Gj
• Efficient O(N) algorithm

8

Conclusion

• disentanglement_lib by Locatello et al. (2019):

github.com/google-research/disentanglement_lib

• Poster: Thurs 06:30 – 09:00 PM at Pacific Ballroom #29

9