Autonomization of monoidal categories Antonin Delpeuch March 28, - - PowerPoint PPT Presentation

Autonomization of monoidal categories

Antonin Delpeuch March 28, 2019 SYCO 3

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 1 / 19

Outline

1 Pregroup grammars and compositional semantics 2 Free yourselves from the strings of tensors! 3 Examples of applications

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 2 / 19

Pregroup grammars and compositional semantics

Outline

1 Pregroup grammars and compositional semantics 2 Free yourselves from the strings of tensors! 3 Examples of applications

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 3 / 19

Pregroup grammars and compositional semantics

Context

Pregroup grammars (Lambek, 1993, Lambek, 1999) the np · nl film n that nr · n · npll · sl Emily np directed npr · s · npl

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 4 / 19

Pregroup grammars and compositional semantics

Context

Pregroup grammars (Lambek, 1993, Lambek, 1999) the np · nl film n that nr · n · npll · sl Emily np directed npr · s · npl

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 4 / 19

Pregroup grammars and compositional semantics

Context

Pregroup grammars (Lambek, 1993, Lambek, 1999) the np · nl film n that nr · n · npll · sl Emily np directed npr · s · npl

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 4 / 19

Pregroup grammars and compositional semantics

Context

Pregroup grammars (Lambek, 1993, Lambek, 1999) the np · nl film n that nr · n · npll · sl Emily np directed npr · s · npl

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 4 / 19

Pregroup grammars and compositional semantics

Autonomous (or rigid) categories

Objects (= types)

are closed under _ ⊗ _ (product of types), _l and _r (adjoints). contain basic types, and I, neutral for ⊗.

Arrows (= type reductions) between two objects

can be composed with ◦ (sequential composition) and ⊗ (parallel composition) ; contain 1A : A → A (identity of A) and ǫl : Al ⊗ A → I ǫr : A ⊗ Ar → I ηl : I → A ⊗ Al ηr : I → Ar ⊗ A

and such that some equations hold.

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 5 / 19

Pregroup grammars and compositional semantics

Representation

f A B f I B f A I f ◦ g A C = B g A f C f ⊗ g A ⊗ C B ⊗ D = f g A C B D ⊗ ⊗

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 6 / 19

Pregroup grammars and compositional semantics

ǫ and η

A ⊗ Ar ǫr = I Al ⊗ A ǫl = I Ar ⊗ A ηr = I A ⊗ Al ηl = I A A IA =

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 7 / 19

Pregroup grammars and compositional semantics

Some equalities

A A Ar A I I A = A A A A Al A I I A = A A

f g = g f = f g

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 8 / 19

Pregroup grammars and compositional semantics

Pregroup reductions as arrows

Clouzot directed an Italian movie n nr s nl d dr d dr n

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 9 / 19

Pregroup grammars and compositional semantics

Pregroup reductions as arrows

Clouzot directed an Italian movie n nr s nl d dr d dr n

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 9 / 19

Pregroup grammars and compositional semantics

Pregroup reductions as arrows

Clouzot directed an Italian movie n nr s nl d dr d dr n s

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 9 / 19

Pregroup grammars and compositional semantics

Compositional semantics

ns Clouzot nr

s s nl

directed d an dr d Italian drns movie Word semantics Type reduction s I I I I I Motto: Type reduction ◦ Word meanings = Sentence meaning

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 10 / 19

Pregroup grammars and compositional semantics

Distributional Compositional Categorial model

DisCoCat (Coecke, Sadrzadeh, and Clark, 2011) : use (Vect, ⊗, I), finite dimensional vector spaces over R and linear maps between them.

I n

=

    

0.73 −2.3 0.1 1.4

    

I n nr =

    

−0.3 3.9 −2.1 0.4 −2.3 2.2 1.5 −1.6 0.1 0.3 −3.8 1.2 1.4 3.4 0.1 3.2

    

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 11 / 19

Pregroup grammars and compositional semantics

Distributional Compositional Categorial model

DisCoCat (Coecke, Sadrzadeh, and Clark, 2011) : use (Vect, ⊗, I), finite dimensional vector spaces over R and linear maps between them.

I n

=

    

0.73 −2.3 0.1 1.4

    

I n nr =

    

−0.3 3.9 −2.1 0.4 −2.3 2.2 1.5 −1.6 0.1 0.3 −3.8 1.2 1.4 3.4 0.1 3.2

    

The dimension of a word representation is exponential in the length

• f the grammatical type.
• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 11 / 19

Pregroup grammars and compositional semantics

Why should we use the tensor product?

The direct sum ⊕ is cartesian, so it cannot have cups and caps: = = =

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 12 / 19

Pregroup grammars and compositional semantics

Why should we use the tensor product?

The direct sum ⊕ is cartesian, so it cannot have cups and caps: = = = General belief in the community: “sticking with the categorical framework [...] forces us to stay within the world of linear maps” (Wijnholds and Sadrzadeh, 2018).

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 12 / 19

Free yourselves from the strings of tensors!

Outline

1 Pregroup grammars and compositional semantics 2 Free yourselves from the strings of tensors! 3 Examples of applications

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 13 / 19

Free yourselves from the strings of tensors!

Just cheat and be free!

Our semantic category does not need to have caps and cups: we can freely add them. n Pat nr s nl grows n nl delicious n kiwis s

f1 +

=

s

f1 +

Trick: caps and cups can be eliminated in any sentence representation.

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 14 / 19

Free yourselves from the strings of tensors!

Constructing free autonomous categories

Preller and Lambek (2007) construct the free autonomous category generated by a category. We need to start from a monoidal category instead. We factorize their construction: Mon Nom L R ⊥ Cat L′ R′ ⊥

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 15 / 19

Examples of applications

Outline

1 Pregroup grammars and compositional semantics 2 Free yourselves from the strings of tensors! 3 Examples of applications

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 16 / 19

Examples of applications

Additive models

Observation by Mikolov et al. (2013): + + + + − − → man − − → king − − − → queen − − − − → woman So, it tempting to define royal(x) = x + − − − → queen − − − − − − → woman.

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 17 / 19

Examples of applications

Additive models

Observation by Mikolov et al. (2013): + + + + − − → man − − → king − − − → queen − − − − → woman So, it tempting to define royal(x) = x + − − − → queen − − − − − − → woman. That is forbidden in (Vect, ⊗, I)!

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 17 / 19

Examples of applications

Convolutional neural networks

Socher et al. (2013) combine vectors following a Chomskyian tree: Lewis (2019) translates this approach to the categorical model, in (Vect, ⊗, I).

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 18 / 19

Examples of applications

d a drns man nr n sl n who nr s nl ate d a drns cake a b c d a b c d = b c a d

• A. Delpeuch

Autonomization March 28, 2019 SYCO 3 19 / 19