What is a probability monad? Paolo Perrone Massachusetts Institute - - PowerPoint PPT Presentation

What is a probability monad?

Paolo Perrone

Massachusetts Institute

• f Technology (MIT)

Categorical Probability 2020 Tutorial video

Monads as extensions

Definition:

Let C be a category. A monad on C consists of:

• A functor T : C → C;
• A natural transformation η : idC ⇒ T called unit;
• A natural transformation µ : TT ⇒ T called composition;

such that the following diagrams commute: T TT T

Tη id µ

T TT T

ηT id µ

TTT TT TT T

Tµ µT µ µ

2 of 27

Monads as extensions

Idea:

A monad is like a consistent way of extending spaces to include generalized elements and generalized functions of a specific kind.

3 of 27

Monads as extensions

Idea:

A monad is like a consistent way of extending spaces to include generalized elements and generalized functions of a specific kind. A functor T : C → C consists of:

3 of 27

Monads as extensions

Idea:

A monad is like a consistent way of extending spaces to include generalized elements and generalized functions of a specific kind. A functor T : C → C consists of:

• 1. To each space X, an “extended” space TX.

3 of 27

Monads as extensions

Idea:

A monad is like a consistent way of extending spaces to include generalized elements and generalized functions of a specific kind. A functor T : C → C consists of:

• 1. To each space X, an “extended” space TX.
• 2. Given f : X → Y , an “extension” Tf : TX → TY .

3 of 27

Monads as extensions

Idea:

A monad is like a consistent way of extending spaces to include generalized elements and generalized functions of a specific kind. A functor T : C → C consists of:

• 1. To each space X, an “extended” space TX.
• 2. Given f : X → Y , an “extension” Tf : TX → TY .

x1 x2 x3 x1 x2 x3 x1 x2 x3 x2 x3 x1 x1 x2 x3 X TX

3 of 27

Monads as extensions

Idea:

A monad is like a consistent way of extending spaces to include generalized elements and generalized functions of a specific kind. A functor T : C → C consists of:

• 1. To each space X, an “extended” space TX.
• 2. Given f : X → Y , an “extension” Tf : TX → TY .

X Y

3 of 27

Monads as extensions

Idea:

A monad is like a consistent way of extending spaces to include generalized elements and generalized functions of a specific kind. A functor T : C → C consists of:

• 1. To each space X, an “extended” space TX.
• 2. Given f : X → Y , an “extension” Tf : TX → TY .

X Y

3 of 27

Monads as extensions

Idea:

A monad is like a consistent way of extending spaces to include generalized elements and generalized functions of a specific kind. A functor T : C → C consists of:

• 1. To each space X, an “extended” space TX.
• 2. Given f : X → Y , an “extension” Tf : TX → TY .

X Y

3 of 27

Monads as extensions

x1 x2 x3 x1 x2 x3 x1 x2 x3 x2 x3 x1 x1 x2 x3 X TX

4 of 27

Monads as extensions

x1 x2 x3 x1 x2 x3 x1 x2 x3 x2 x3 x1 x1 x2 x3 X TX

4 of 27

Monads as extensions

x1 x2 x3 x1 x2 x3 x1 x2 x3 x2 x3 x1 x1 x2 x3 X TX A natural transformation η : idC ⇒ T consists of:

• 1. To each X a map ηX : X → TX, usually monic.

4 of 27

Monads as extensions

x1 x2 x3 x1 x2 x3 x1 x2 x3 x2 x3 x1 x1 x2 x3 X TX A natural transformation η : idC ⇒ T consists of:

• 1. To each X a map ηX : X → TX, usually monic.
• 2. This diagram must commute:

X Y TX TY

ηX f ηY Tf

4 of 27

Monads as extensions

A natural transformation µ : TT ⇒ T, is:

• 1. For each X a map µX : TTX → TX;
• 2. Again a naturality diagram as before.

5 of 27

Monads as extensions

A natural transformation µ : TT ⇒ T, is:

• 1. For each X a map µX : TTX → TX;
• 2. Again a naturality diagram as before.

5 of 27

Monads as extensions

A natural transformation µ : TT ⇒ T, is:

• 1. For each X a map µX : TTX → TX;
• 2. Again a naturality diagram as before.

5 of 27

Monads as extensions

Definition:

Let T be a monad on C. A Kleisli morphism from X to Y is a morphism X → TY .

6 of 27

Monads as extensions

Definition:

Let T be a monad on C. A Kleisli morphism from X to Y is a morphism X → TY . X Y

6 of 27

Monads as extensions

Definition:

Given Kleisli morphisms k : X → TY and h : Y → TZ, their Kleisli composition is the morphism h ◦kl k given by: X TY TTZ TZ

k Th µ

7 of 27

Monads as extensions

Definition:

Given Kleisli morphisms k : X → TY and h : Y → TZ, their Kleisli composition is the morphism h ◦kl k given by: X TY TTZ TZ

k Th µ

X Y Z k

7 of 27

Monads as extensions

Definition:

Given Kleisli morphisms k : X → TY and h : Y → TZ, their Kleisli composition is the morphism h ◦kl k given by: X TY TTZ TZ

k Th µ

X Y Z k h

7 of 27

Monads as extensions

Definition:

Given Kleisli morphisms k : X → TY and h : Y → TZ, their Kleisli composition is the morphism h ◦kl k given by: X TY TTZ TZ

k Th µ

X Y Z k Th

7 of 27

Monads as extensions

Definition:

Given Kleisli morphisms k : X → TY and h : Y → TZ, their Kleisli composition is the morphism h ◦kl k given by: X TY TTZ TZ

k Th µ

X Y Z k µ ◦ Th

7 of 27

Monads as extensions

Definition:

Given Kleisli morphisms k : X → TY and h : Y → TZ, their Kleisli composition is the morphism h ◦kl k given by: X TY TTZ TZ

k Th µ

X Y Z

7 of 27

Monads as extensions

Exercise:

Prove that Kleisli morphisms form a category thanks to the commutativity of these diagrams: TX TTX TX

Tη µ

TX TTX TX

ηT µ

TTTX TTX TTX TX

Tµ µT µ µ

where the identity morphisms of the Kleisli category are given by the units η : X → TX.

8 of 27

Probability monads

Idea [Giry, 1982]:

Spaces of “random elements” generalizing usual elements.

9 of 27

Probability monads

Idea [Giry, 1982]:

Spaces of “random elements” generalizing usual elements.

• Base category C

9 of 27

Probability monads

Idea [Giry, 1982]:

Spaces of “random elements” generalizing usual elements.

X

• Base category C

9 of 27

Probability monads

Idea [Giry, 1982]:

Spaces of “random elements” generalizing usual elements.

X PX

• Base category C
• Functor X → PX

9 of 27

Probability monads

Idea [Giry, 1982]:

Spaces of “random elements” generalizing usual elements.

X PX

• Base category C
• Functor X → PX
• Unit δ : X → PX

9 of 27

Probability monads

Idea [Giry, 1982]:

Spaces of “random elements” generalizing usual elements.

1/2 1/2

• Base category C
• Functor X → PX
• Unit δ : X → PX

9 of 27

Probability monads

Idea [Giry, 1982]:

Spaces of “random elements” generalizing usual elements.

1/2 1/2 1/2 1/2

• Base category C
• Functor X → PX
• Unit δ : X → PX

9 of 27

Probability monads

Idea [Giry, 1982]:

Spaces of “random elements” generalizing usual elements.

?

1/2 1/2 1/2 1/2 1/2 1/2

• Base category C
• Functor X → PX
• Unit δ : X → PX

9 of 27

Probability monads

Idea [Giry, 1982]:

Spaces of “random elements” generalizing usual elements.

? ?

1/2 1/2 1/2 1/2 1/2 1/2 1/4 3/4

• Base category C
• Functor X → PX
• Unit δ : X → PX

9 of 27

Probability monads

Idea [Giry, 1982]:

Spaces of “random elements” generalizing usual elements.

PPX PX

? ?

1/2 1/2 1/2 1/2 1/2 1/2 1/4 3/4

• Base category C
• Functor X → PX
• Unit δ : X → PX
• Composition

E : PPX → PX

9 of 27

Probability monads

A Kleisli morphism from X to Y is a morphism X → PY . We can interpret this as a “random function” or “random transition”. X Y

10 of 27

Probability monads

Given Kleisli morphisms k : X → PY and h : Y → PZ, their Kleisli composition is the morphism h ◦kl k given by: X PY PPZ PZ

k Ph E

X Y Z

11 of 27

Probability monads

Given Kleisli morphisms k : X → PY and h : Y → PZ, their Kleisli composition is the morphism h ◦kl k given by: X PY PPZ PZ

k Ph E

X Y Z

11 of 27

Probability monads

Given Kleisli morphisms k : X → PY and h : Y → PZ, their Kleisli composition is the morphism h ◦kl k given by: X PY PPZ PZ

k Ph E

X Y Z

11 of 27

Probability monads

Given Kleisli morphisms k : X → PY and h : Y → PZ, their Kleisli composition is the morphism h ◦kl k given by: X PY PPZ PZ

k Ph E

X Y Z

11 of 27

Probability monads

Given Kleisli morphisms k : X → PY and h : Y → PZ, their Kleisli composition is the morphism h ◦kl k given by: X PY PPZ PZ

k Ph E

X Y Z

11 of 27

Probability monads

Given Kleisli morphisms k : X → PY and h : Y → PZ, their Kleisli composition is the morphism h ◦kl k given by: X PY PPZ PZ

k Ph E

X Y Z

11 of 27

The distribution monad on Set

Definition:

Let X be a set. A f.s. distribution on X is a function p : X → [0, 1] such that

• It is nonzero for finitely many x ∈ X;

x∈X p(x) = 1.

We denote by DX the set of f.s. distributions on X.

12 of 27

The distribution monad on Set

Definition:

Let X be a set. A f.s. distribution on X is a function p : X → [0, 1] such that

• It is nonzero for finitely many x ∈ X;

x∈X p(x) = 1.

We denote by DX the set of f.s. distributions on X. X

12 of 27

The distribution monad on Set

Definition:

Let X be a set. A f.s. distribution on X is a function p : X → [0, 1] such that

• It is nonzero for finitely many x ∈ X;

x∈X p(x) = 1.

We denote by DX the set of f.s. distributions on X. X

12 of 27

The distribution monad on Set

Definition:

Let f : X → Y be a function and p ∈ DX. The pushforward of p along f is the distribution f∗p ∈ DY given by f∗p(y) :=

• x∈f −1(y)

p(x). We denote the map f∗ : DX → DY by Df , this makes D a functor.

13 of 27

The distribution monad on Set

Definition:

Let f : X → Y be a function and p ∈ DX. The pushforward of p along f is the distribution f∗p ∈ DY given by f∗p(y) :=

• x∈f −1(y)

p(x). We denote the map f∗ : DX → DY by Df , this makes D a functor. X Y

13 of 27

The distribution monad on Set

Definition:

Let f : X → Y be a function and p ∈ DX. The pushforward of p along f is the distribution f∗p ∈ DY given by f∗p(y) :=

• x∈f −1(y)

p(x). We denote the map f∗ : DX → DY by Df , this makes D a functor. X Y

13 of 27

The distribution monad on Set

Definition:

Let f : X → Y be a function and p ∈ DX. The pushforward of p along f is the distribution f∗p ∈ DY given by f∗p(y) :=

• x∈f −1(y)

p(x). We denote the map f∗ : DX → DY by Df , this makes D a functor. X Y

13 of 27

The distribution monad on Set

Definition:

Let X be a set. The map δ : X → DX maps x ∈ X to the distribution δx ∈ DX given by δx(y) =

• 1

y = x; y = x. This gives a natural map δ : X → DX, a component of the unit of the monad.

14 of 27

The distribution monad on Set

Definition:

Let X be a set. The map δ : X → DX maps x ∈ X to the distribution δx ∈ DX given by δx(y) =

• 1

y = x; y = x. This gives a natural map δ : X → DX, a component of the unit of the monad. X

14 of 27

The distribution monad on Set

Definition:

Let X be a set. Given ξ ∈ DDX, define Eξ ∈ DX to be distribution given by Eξ(x) :=

• p∈DX

p(x) ξ(p). This gives a natural map E : DDX → DX, a component of the multiplication of the monad.

15 of 27

The distribution monad on Set

Definition:

Let X be a set. Given ξ ∈ DDX, define Eξ ∈ DX to be distribution given by Eξ(x) :=

• p∈DX

p(x) ξ(p). This gives a natural map E : DDX → DX, a component of the multiplication of the monad. X

15 of 27

The distribution monad on Set

Definition:

Let X be a set. Given ξ ∈ DDX, define Eξ ∈ DX to be distribution given by Eξ(x) :=

• p∈DX

p(x) ξ(p). This gives a natural map E : DDX → DX, a component of the multiplication of the monad. X

15 of 27

The distribution monad on Set

Definition:

Let X be a set. Given ξ ∈ DDX, define Eξ ∈ DX to be distribution given by Eξ(x) :=

• p∈DX

p(x) ξ(p). This gives a natural map E : DDX → DX, a component of the multiplication of the monad. X

15 of 27

The distribution monad on Set

Definition:

Let X be a set. Given ξ ∈ DDX, define Eξ ∈ DX to be distribution given by Eξ(x) :=

• p∈DX

p(x) ξ(p). This gives a natural map E : DDX → DX, a component of the multiplication of the monad. X

15 of 27

The distribution monad on Set

Definition:

Let X be a set. Given ξ ∈ DDX, define Eξ ∈ DX to be distribution given by Eξ(x) :=

• p∈DX

p(x) ξ(p). This gives a natural map E : DDX → DX, a component of the multiplication of the monad. X

15 of 27

The distribution monad on Set

Kleisli morphisms:

A Kleisli morphism for D is a function k : X → DY . In other words, it is function ¯ k : X × Y → [0, 1] such that

• For each x ∈ X, ¯

k(x, −) : Y → [0, 1] is nonzero in finitely many entries;

• For each x ∈ X,

y∈Y ¯

k(x, y) = 1.

16 of 27

The distribution monad on Set

Kleisli morphisms:

A Kleisli morphism for D is a function k : X → DY . In other words, it is function ¯ k : X × Y → [0, 1] such that

• For each x ∈ X, ¯

k(x, −) : Y → [0, 1] is nonzero in finitely many entries;

• For each x ∈ X,

y∈Y ¯

k(x, y) = 1.

Kleisli composition:

The Kleisli composition of k : X → DY and h : Y → DZ is given by the Chapman-Kolmogorov equation: (h ◦kl k)(x, z) =

• y∈Y

k(x, y) h(y, z).

16 of 27

The Giry monad on Meas

Let X be a measurable space. Define PX to be

• The set of probability measures on X

17 of 27

The Giry monad on Meas

Let X be a measurable space. Define PX to be

• The set of probability measures on X
• Equipped with the σ-algebra generated by the evaluation functions

εA : PX → R given by p − → p(A) for all A ⊆ X measurable.

17 of 27

The Giry monad on Meas

Let X be a measurable space. Define PX to be

• The set of probability measures on X
• Equipped with the σ-algebra generated by the evaluation functions

εA : PX → R given by p − → p(A) for all A ⊆ X measurable.

• Equivalently, the σ-algebra is generated by the “integration”

functions εf : PX → R given by p − →

• f dp,

for all f : X → [0, 1] measurable.

17 of 27

The Giry monad on Meas

Functoriality:

Let f : X → Y be a measurable function. Given a measure p ∈ PX, recall that the pushforward measure f∗p ∈ PY is given by f∗p(B) := p(f −1(B)). We get a measurable map Pf : PX → PY which makes P a functor.

18 of 27

The Giry monad on Meas

Functoriality:

Let f : X → Y be a measurable function. Given a measure p ∈ PX, recall that the pushforward measure f∗p ∈ PY is given by f∗p(B) := p(f −1(B)). We get a measurable map Pf : PX → PY which makes P a functor.

Unit:

Given a measurable space X, to each x ∈ X we can give the Dirac delta measure δx ∈ PX. This gives a measurable map δ : X → PX, which is natural, and forms a component of the unit of the monad.

18 of 27

The Giry monad on Meas

Multiplication:

Given a measurable space X and a measure π ∈ PPX, we define the measure Eπ ∈ PX by Eπ(A) :=

• PX

p(A)dπ(p), This gives a measurable map E : PPX → PX which is natural in X and forms a component of the monad multiplication.

19 of 27

The Giry monad on Meas

Kleisli morphisms:

A Kleisli morphism is a measurable map k : X → PY , in other words, a Markov kernel between X and Y . Denote k(x) ∈ PY by kx.

20 of 27

The Giry monad on Meas

Kleisli morphisms:

A Kleisli morphism is a measurable map k : X → PY , in other words, a Markov kernel between X and Y . Denote k(x) ∈ PY by kx.

Kleisli composition:

The composition of Kleisli morphisms reproduces the Chapman-Kolmogorov equation for general measures. Given k : X → PY and h : Y → PZ, we get that (h ◦kl k)(x)(C) =

• Y

hy(C)dkx(y) for each x ∈ X and for each C ⊆ Z measurable.

20 of 27

Other probability monads

Category Monad (P) Points of PX Set Distribution monad f.s. distributions Meas Giry monad probability measures Pol Giry monad Borel probability measures QBS

• Prob. monad
• Eq. classes of R.V.s

DCPO

• Prob. powerdomain
• cont. valuations

Top

• Ext. prob. PD
• cont. valuations

Top

• Prob. monad

τ-smooth Borel prob. measures CHaus Radon monad Radon prob. measures Met Kantorovich monad Radon prob. measures of FFM More on the nLab, “probability monad” [nLab article].

21 of 27

Joints and marginals

Idea:

Probability theory is mostly about interactions of random variables.

• Composite states

X × Y

22 of 27

Joints and marginals

Idea:

Probability theory is mostly about interactions of random variables.

X Y

• Composite states

X × Y

22 of 27

Joints and marginals

Idea:

Probability theory is mostly about interactions of random variables.

X Y Y X

• Composite states

X × Y

22 of 27

Joints and marginals

Idea:

Probability theory is mostly about interactions of random variables.

X Y Y X

• Composite states

X × Y

• Given marginals

22 of 27

Joints and marginals

Idea:

Probability theory is mostly about interactions of random variables.

X Y Y X

• Composite states

X × Y

• Given marginals
• Many possible joints

22 of 27

Joints and marginals

Idea:

Probability theory is mostly about interactions of random variables.

X Y Y X

• Composite states

X × Y

• Given marginals
• Many possible joints

22 of 27

Joints and marginals

Idea:

Probability theory is mostly about interactions of random variables.

X Y Y X

• Composite states

X × Y

• Given marginals
• Many possible joints

22 of 27

Joints and marginals

Idea:

Probability theory is mostly about interactions of random variables.

X Y Y X

• Composite states

X × Y

• Given marginals
• Many possible joints
• One canonical

choice of “independence”

22 of 27

Joints and marginals

Idea:

Given objects X and Y , a probability distribution on X × Y is not just pair of distributions on X and Y separately. However, given p ∈ PX and q ∈ PY , we get a measure p ⊗ q ∈ P(X × Y ). PX × PY P(X × Y )

∇

23 of 27

Joints and marginals

Idea:

Given objects X and Y , a probability distribution on X × Y is not just pair of distributions on X and Y separately. However, given p ∈ PX and q ∈ PY , we get a measure p ⊗ q ∈ P(X × Y ). PX × PY P(X × Y )

∇

This gives a monoidal structure to the probability monad. (Technically, we need ∇ together with a map 1 → P1, but for probability monads 1 and P1 are uniquely isomorphic.)

23 of 27

Joints and marginals

PX × PY × PZ P(X × Y ) × PZ PX × P(Y × Z) P(X × Y × Z)

∇×id id×∇ ∇ ∇

24 of 27

Joints and marginals

X × Y X Y

π1 π2

25 of 27

Joints and marginals

P(X × Y ) PX PY

Pπ1 Pπ2

25 of 27

Joints and marginals

P(X × Y ) PX PX × PY PY

Pπ1 Pπ2 π1 π2

25 of 27

Joints and marginals

P(X × Y ) PX PX × PY PY

Pπ1 Pπ2 ∆ π1 π2

P(X × Y × Z) PX × P(Y × Z) P(X × Y ) × PZ PX × PY × PZ

∆×id id×∆ ∆ ∆

25 of 27

Joints and marginals

Operations on distributions:

Let f : X × Y → Z be a binary function. Then we can form the map PX × PY P(X × Y ) PZ

∇ Pf

26 of 27

Joints and marginals

Operations on distributions:

Let f : X × Y → Z be a binary function. Then we can form the map PX × PY P(X × Y ) PZ

∇ Pf

For example, the addition as a map R × R → R gives the convolution

• f real-valued random variables.

26 of 27

Joints and marginals

Operations on distributions:

Let f : X × Y → Z be a binary function. Then we can form the map PX × PY P(X × Y ) PZ

∇ Pf

For example, the addition as a map R × R → R gives the convolution

• f real-valued random variables.

1 2 3

26 of 27

Joints and marginals

Operations on distributions:

Let f : X × Y → Z be a binary function. Then we can form the map PX × PY P(X × Y ) PZ

∇ Pf

For example, the addition as a map R × R → R gives the convolution

• f real-valued random variables.

1 2 3

+

1 2 3

26 of 27

Joints and marginals

Operations on distributions:

Let f : X × Y → Z be a binary function. Then we can form the map PX × PY P(X × Y ) PZ

∇ Pf

For example, the addition as a map R × R → R gives the convolution

• f real-valued random variables.

1 2 3

+

1 2 3

=

1 2 3

26 of 27

Some references

van Breugel, F. (2005). The Metric Monad for Probabilistic Nondeterminism. www.cse.yorku.ca/~franck/research/drafts/ monad.pdf. Fritz, T. and Perrone, P. (2018). Bimonoidal structure of probability monads. Proceedings of MFPS 34. Fritz, T. and Perrone, P. (2020). Monads, partial evaluations, and rewriting. Proceedings of MFPS 36. Giry, M. (1982). A Categorical Approach to Probability Theory. In Categorical aspects of topology and analysis, volume 915 of Lecture Notes in Mathematics. Heunen, C., Kammar, O., Staton, S., and Yang, H. (2017). A convenient category for higher-order probability theory. Proceedings of LICS’17, (77):1–12. Jacobs, B. (2018). From probability monads to commutative effectus. Journal of Logical and Algebraic Methods in Programming, 94:200–237. Keimel, K. (2008). The monad of probability measures over compact

• rdered spaces and its Eilenberg-Moore algebras.

Topology and its Applications, 156(2):227–239. nLab article. Monads of probability, measures and valuations. ncatlab.org/nlab/show/probability+monad. Perrone, P. (2018). Categorical Probability and Stochastic Dominance in Metric Spaces. PhD thesis, University of Leipzig. www.paoloperrone.org/phdthesis.pdf. Perrone, P. (2019). Notes on category theory with examples from basic mathematics. arXiv:1912.10642. 27 of 27