Polymorphism and Type Inference Liam OConnor CSE, UNSW (and data61) - - PowerPoint PPT Presentation

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Polymorphism and Type Inference Liam O’Connor

CSE, UNSW (and data61) Term 3 2019

1

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Where we’re at

Syntax Foundations Concrete/Abstract Syntax, Ambiguity, HOAS, Binding, Variables, Substitution Semantics Foundations Static Semantics, Dynamic Semantics (Small-Step/Big-Step), Abstract Machines, Environments (Assignment 1) Features

Algebraic Data Types Polymorphism Polymorphic Type Inference (Assignment 2) Overloading Subtyping Modules Concurrency

2

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

A Swap Function

Consider the humble swap function in Haskell: swap :: (t1, t2) → (t2, t1) swap (a, b) = (b, a) In our MinHS with algebraic data types from last lecture, we can’t define this function.

3

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Monomorphic

In MinHS, we’re stuck copy-pasting our function over and over for every different type we want to use it with: recfun swap1 :: ((Int × Bool) → (Bool × Int)) p = (snd p, fst p) recfun swap2 :: ((Bool × Int) → (Int × Bool)) p = (snd p, fst p) recfun swap3 :: ((Bool × Bool) → (Bool × Bool)) p = (snd p, fst p) · · · This is an acceptable state of affairs for some domain-specific languages, but not for general purpose programming.

4

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Solutions

We want some way to specify that we don’t care what the types of the tuple elements are.

swap :: (∀a b. (a × b) → (b × a))

This is called parametric polymorphism (or just polymorphism in functional programming circles). In Java and some other languages, this is called generics and polymorphism refers to something else. Don’t be confused.

5

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

How it works

There are two main components to parametric polymorphism:

1

Type abstraction is the ability to define functions regardless of specific types (like the swap example before).In MinHS, we will write using type expressions like so: (the literature uses Λ) swap = type a. type b. recfun swap :: (a × b) → (b × a) p = (snd p, fst p)

2

Type application is the ability to instantiate polymorphic functions to specific types. In MinHS, we use @ signs. swap@Int@Bool (3, True)

6

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Analogies

The reason they’re called type abstraction and application is that they behave analogously to λ-calculus. We have a β-reduction principle, but for types: (type a. e)@τ →β (e[a := τ]) Example (Identity Function) (type a. recfun f :: (a → a) x = x)@Int 3 → (recfun f :: (Int → Int) x = x) 3 → 3 This means that type expressions can be thought of as functions from types to values.

7

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Type Variables

What is the type of this? (type a. recfun f :: (a → a) x = x)

∀a. a → a

Types can mention type variables now1. If id : ∀a.a → a, what is the type of id@Int? (a → a)[a := Int] = (Int → Int)

1Technically, they already could with recursive types. 8

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Typing Rules Sketch

We would like rules that look something like this: Γ ⊢ e : τ Γ ⊢ type a. e : ∀a. τ Γ ⊢ e : ∀a. τ Γ ⊢ e@ρ : τ[a := ρ] But these rules don’t account for what type variables are available

• r in scope.

9

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Type Wellformedness

With variables in the picture, we need to check our types to make sure that they only refer to well-scoped variables. t bound ∈ ∆ ∆ ⊢ t ok ∆ ⊢ Int ok ∆ ⊢ Bool ok ∆ ⊢ τ1 ok ∆ ⊢ τ2 ok ∆ ⊢ τ1 → τ2 ok ∆ ⊢ τ1 ok ∆ ⊢ τ2 ok ∆ ⊢ τ1 × τ2 ok (etc.) ∆, a bound ⊢ τ ok ∆ ⊢ ∀a. τ ok

10

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Typing Rules, Properly

We add a second context of type variables that are bound. a bound, ∆; Γ ⊢ e : τ ∆; Γ ⊢ type a. e : ∀a. τ ∆; Γ ⊢ e : ∀a. τ ∆ ⊢ ρ ok ∆; Γ ⊢ e@ρ : τ[a := ρ] (the other typing rules just pass ∆ through)

11

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Dynamic Semantics

First we evaluate the LHS of a type application as much as possible: e →M e′ e@τ →M e′@τ Then we apply our β-reduction principle: (type a. e)@τ →M e[a := τ]

12

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Curry-Howard

Previously we noted the correspondence between types and logic:

× ∧ + ∨ → ⇒ 1 ⊤ 0 ⊥ ∀ ∀

13

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Curry-Howard

The type quantifier ∀ corresponds to a universal quantifier ∀, but it is not the same as the ∀ from first-order logic. What’s the difference? First-order logic quantifiers range over a set of individuals or values, for example the natural numbers: ∀x. x + 1 > x These quantifiers range over propositions (types) themselves. It is analogous to second-order logic, not first-order: ∀A. ∀B. A ∧ B ⇒ B ∧ A ∀A. ∀B. A × B → B × A The first-order quantifier has a type-theoretic analogue too (type indices), but this is not nearly as common as polymorphism.

14

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Generality

If we need a function of type Int → Int, a polymorphic function

• f type ∀a. a → a will do just fine, we can just instantiate the type

variable to Int. But the reverse is not true. This gives rise to an

• rdering.

Generality A type τ is more general than a type ρ, often written ρ ⊑ τ, if type variables in τ can be instantiated to give the type ρ. Example (Functions) Int → Int ⊑ ∀z. z → z ⊑ ∀x y. x → y ⊑ ∀a. a

15

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Implementation Strategies

Our simple dynamic semantics belies a complex implementation headache. While we can easily define functions that operate uniformly on multiple types, when this is compiled to machine code the results may differ depending on the size of the type in question. There are two main approaches to solve this problem.

16

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Template Instantiation

Key Idea Automatically generate a monomorphic copy of each polymorphic functions based on the types applied to it. For example, if we defined our polymorphic swap function: swap = type a. type b. recfun swap :: (a × b) → (b × a) p = (snd p, fst p) Then a type application like swap@Int@Bool would be replaced statically by the compiler with the monomorphic version: swapIB = recfun swap :: (Int × Bool) → (Bool × Int) p = (snd p, fst p) A new copy is made for each unique type application.

17

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Evaluating Template Instatiation

This approach has a number of advantages:

1

Little to no run-time cost

2

Simple mental model

3

Allows for custom specialisations (e.g. list of booleans into bit-vectors)

4

Easy to implement However the downsides are just as numerous:

1

Large binary size if many instantiations are used

2

This can lead to long compilation times

3

Restricts the type system to statically instantiated type variables. Languages that use Template Instantiation: Rust, C++, Cogent, some ML dialects

18

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Polymorphic Recursion

Consider the following Haskell data type: data Dims a = Step a (Dims [a]) | Epsilon This describes a list of matrices of increasing dimensionality, e.g: Step 1 (Step [1, 2] (Step [[1, 2], [3, 4]] Epsilon)) :: Dims Int We can write a sum function like this: sumDims :: ∀a. (a → Int) → Dims a → Int sumDims f Epsilon = 0 sumDims f (Step a t) = (f a) + sumDims (sum f ) t How many different instantiations of the type variable a are there? We’d have to run the program to find out.

19

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

HM Types

Automatically generating a copy for each instantiation is great but can’t handle all polymorphic programs. In practice a statically determined subset can be carved out by restricting what sort of programs can be written:

1

Only allow ∀ quantifiers on the outermost part of a type declaration (not inside functions or type constructors).

2

Recursive functions cannot call themselves with different type parameters. This restriction is sometimes called Hindley-Milner polymorphism. This is also the subset for which type inference is both complete and tractable.

20

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Boxing

An alternative to our copy-paste-heavy template instantiation approach is to make all types represented the same way. Thus, a polymorphic function only requires one function in the generated code. Typically this is done by boxing each type. That is, all data types are represented as a pointer to a data structure on the heap. If everything is a pointer, then all values use exactly 32 (or 64) bits

• f stack space.

The extra indirection has a run-time penalty, and it can make garbage collection more necessary, but it results in smaller binaries and unrestricted polymorphism. Languages that use boxing: Haskell, Java, C♯, OCaml

21

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Constraining Implementations

How many possible implementations are there of a function of the following type? Int → Int How about this type? ∀a. a → a Polymorphic type signatures constrain implementations.

22

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Parametricity

Definition The principle of parametricity states that the result of polymorphic functions cannot depend on values of an abstracted type. More formally, suppose I have a polymorphic function g that takes a type parameter. If run any arbitrary function f : τ → τ on some values of type τ, then run the function g@τ on the result, that will give the same results as running g@τ first, then f . Example foo :: ∀a. [a] → [a] We know that every element of the output occurs in the input. The parametricity theorem we get is, for all f : foo ◦ (map f ) = (map f ) ◦ foo

23

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

More Examples

head :: ∀a. [a] → a What’s the parametricity theorems? Example (Answer) For any f : f (head ℓ) = head (map f ℓ)

24

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

More Examples

(+ +) :: ∀a. [a] → [a] → [a] What’s the parametricity theorem? Example (Answer) map f (a ++ b) = map f a ++ map f b

25

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

More Examples

concat :: ∀a. [[a]] → [a] What’s the parametricity theorem? Example (Answer) map f (concat ls) = concat (map (map f ) ls)

26

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Higher Order Functions

filter :: ∀a. (a → Bool) → [a] → [a] What’s the parametricity theorem? Example (Answer) filter p (map f ls) = map f (filter (p ◦ f ) ls)

27

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Parametricity Theorems

Follow a similar structure. In fact it can be mechanically derived, using the relational parametricity framework invented by John C. Reynolds, and popularised by Wadler in the famous paper, “Theorems for Free!”2. Upshot: We can ask lambdabot on the Haskell IRC channel for these theorems.

2https://people.mpi-sws.org/~dreyer/tor/papers/wadler.pdf 28

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Implicitly Typed MinHS

Explicitly typed languages are awkward to use3. Ideally, we’d like the compiler to determine the types for us. Example What is the type of this function? recfun f x = fst x + 1 We want the compiler to infer the most general type.

3See Java 29

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Implicitly Typed MinHS

Start with our polymorphic MinHS, then: Remove type signatures from recfun, let, etc. Remove explicit type abstractions, and type applications (the @ operator). Keep ∀-quantified types. Remove recursive types, as we can’t infer types for them.

See whiteboard for why. 30

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Typing Rules

x : τ ∈ Γ Γ ⊢ x : τ Var Γ ⊢ e1 : τ1 → τ2 Γ ⊢ e2 : τ1 Γ ⊢ e1 e2 : τ2 App Γ ⊢ e1 : τ1 Γ ⊢ e2 : τ2 Γ ⊢ (Pair e1 e2) : τ1 × τ2 ConjI Γ ⊢ e1 : Bool Γ ⊢ e2 : τ Γ ⊢ e3 : τ Γ ⊢ (If e1 e2 e3) : τ If

31

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Primitive Operators

For convenience, we treat prim ops as functions, and place their types in the environment. (+) : Int → Int → Int, Γ ⊢ (App (App (+) (Num 2)) (Num 1)) : Int

32

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Functions

x : τ1, f : τ1 → τ2, Γ ⊢ e : τ2 Γ ⊢ (Recfun (f .x. e)) : τ1 → τ2 Func

33

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Sum Types

Γ ⊢ e : τ1 Γ ⊢ InL e : τ1 + τ2 DisjI1 Γ ⊢ e : τ2 Γ ⊢ InR e : τ1 + τ2 DisjI2 Note that we allow the other side of the sum to be any type.

34

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Polymorphism

If we have a polymorphic type, we can instantiate it to any type: Γ ⊢ e : ∀a.τ Γ ⊢ e : τ[a := ρ]AllE We can quantify over any variable that has not already been used. Γ ⊢ e : τ a / ∈ TV (Γ) Γ ⊢ e : ∀a. τ AllI (Where TV (Γ) here is all type variables occurring free in the types

• f variables in Γ)

35

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

The Goal

We want an algorithm for type inference: With a clear input and output. Which terminates. Which is fully deterministic.

36

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Typing Rules

Γ ⊢ e1 : τ1 Γ ⊢ e2 : τ2 Γ ⊢ (Pair e1 e2) : τ1 × τ2 Can we use the existing typing rules as our algorithm? infer :: Context → Expr → Type This approach can work for monomorphic types, but not polymorphic ones. Why not?

37

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

First Problem

Γ ⊢ e : ∀a.τ Γ ⊢ e : τ[a := ρ]AllE The rule to add a ∀-quantifier can always be applied: . . . Γ ⊢ (Num 5) : ∀a. ∀b. Int Γ ⊢ (Num 5) : ∀a. Int AllE Γ ⊢ (Num 5) : Int AllE This makes the rules give rise to a non-deterministic algorithm – there are many possible rules for a given input. Furthermore, as it can always be applied, a depth-first search strategy may end up attempting infinite derivations.

38

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Another Problem

Γ ⊢ e : ∀a.τ Γ ⊢ e : τ[a := ρ]AllE The above rule can be applied at any time to a polymorphic type, even if it would break later typing derivations: Γ ⊢ fst : ∀a. ∀b. (a × b) → a Γ ⊢ fst : (Bool × Bool) → Bool · · · Γ ⊢ (Pair 1 True) : (Int × Bool) Γ ⊢ (Apply fst (Pair 1 True)) : ???

39

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Yet Another Problem

The rule for recfun mentions τ2 in both input and output positions. x : τ1, f : τ1 → τ2, Γ ⊢ e : τ2 Γ ⊢ (Recfun (f .x. e)) : τ1 → τ2 Func In order to infer τ2 we must provide a context that includes τ2 — this is circular. Any guess we make for τ2 could be wrong.

40

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Solution

We allow types to include unknowns, also known as unification variables or schematic variables. These are placeholders for types that we haven’t worked out yet. We shall use α, β etc. for these variables. Example (Int × α) → β is the type of a function from tuples where the left side of the tuple is Int, but no other details of the type have been determined yet. As we encounter situations where two types should be equal, we unify the two types to determine what the unknown variables should be.

41

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Example

Γ ⊢ fst : ∀a. ∀b. (a × b) → a Γ ⊢ fst : (α × β) → α · · · Γ ⊢ (Pair 1 True) : (Int × Bool) Γ ⊢ (Apply fst (Pair 1 True)) : γ (α × β) → α ∼ (Int × Bool) → γ [α := Int, β := Bool, γ := Int]

42

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Unification

We call this substitution a unifier. Definition A substitution S to unification variables is a unifier of two types τ and ρ iff Sτ = Sρ. Furthermore, it is the most general unifier, or mgu, of τ and ρ if there is no other unifier S′ where Sτ ⊑ S′τ. We write τ U ∼ ρ if U is the mgu of τ and ρ. Example (Whiteboard) α × (α × α) ∼ β × γ (α × α) × β ∼ β × γ Int + α ∼ α + Bool (α × α) × α ∼ α × (α × α)

43

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Back to Type Inference

We will decompose the typing judgement to allow for an additional

• utput — a substitution that contains all the unifiers we have

found about unknowns so far. Inputs Expression, Context Outputs Type, Substitution We will write this as SΓ ⊢ e : τ, to make clear how the original typing judgement may be reconstructed.

44

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Application, Elimination

S1Γ ⊢ e1 : τ1 S2S1Γ ⊢ e2 : τ2 S2τ1

U

∼ (τ2 → α) US2S1Γ ⊢ (Apply e1 e2) : Uα (α fresh) (x : ∀a1. ∀a2. . . . ∀an. τ) ∈ Γ Γ ⊢ x : τ[a1 := α1, a2 := α2, . . . , an = αn] (α1 . . . αn fresh) Example (Whiteboard) (fst : ∀a b. (a × b) → a) ⊢ (Apply fst (Pair 1 2))

45

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Functions

S(Γ, x : α1, f : α2) ⊢ e : τ Sα2

U

∼ (Sα1 → τ) USΓ ⊢ (Recfun (f .x. e)) : U(Sα1 → τ) (α1, α2 fresh) Example (Whiteboard) (Recfun (f .x. (Pair x x))) (Recfun (f .x. (Apply f x)))

46

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Generalisation

In our typing rules, we could generalise a type to a polymorphic type by introducing a ∀ at any point in the typing derivation. We want to be able to restrict this to only occur in a syntax-directed way. Consider this example: let f = (recfun f x = (x, x)) in (fst (f 4), fst (f True)) Where should generalisation happen?

47

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Let-generalisation

To make type inference tractable, we restrict generalisation to only

• ccur when a binding is added to the context via a let expression.

This means that let expressions are now not just sugar for a function application, they actually play a vital role in the language’s syntax, as a place for generalisation to occur. We define Gen(Γ, τ) = ∀(TV (τ) \ TV (Γ)). τ Then we have: S1Γ ⊢ e1 : τ S2(S1Γ, x : Gen(S1Γ, τ)) ⊢ e2 : τ ′ S2S1Γ ⊢ (Let e1(x. e2)) : τ ′

48

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Summary

The rest of the rules are straightforward from their typing rule. We’ve specified Robin Milner’s algorithm W for type

• inference. Many other algorithms exist, for other kinds of type

systems, including explicit constraint-based systems. This algorithm is restricted to the Hindley-Milner subset of decidable polymorphic instantiations, and requires that polymorphism is top-level — polymorphic functions are not first class. We still need an algorithm to compute the unifiers.

49

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Unification

unify :: Type → Type → Maybe Unifier (where the Type arguments do not include any ∀ quantifiers and the Unifier returned is the mgu) We shall discuss cases for unify τ1 τ2

50

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Cases

Both type variables: τ1 = v1 and τ2 = v2: v1 = v2 ⇒ empty unifier v1 = v2 ⇒ [v1 := v2]

51

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Cases

Both primitive type constructors: τ1 = C1 and τ2 = C2: C1 = C2 ⇒ empty unifier C1 = C2 ⇒ no unifier

52

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Cases

Both are product types τ1 = τ11 × τ12 and τ2 = τ21 × τ22.

1

Compute the mgu S of τ11 and τ21.

2

Compute the mgu S′ of Sτ12 and Sτ22.

3

Return S ∪ S′ (same for sum, function types)

53

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Cases

One is a type variable v, the other is just any term t. v occurs in t ⇒ no unifier

• therwise ⇒ [v := t]

54

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Unification

Done

Implementing this algorithm will be the focus of Assignment 2. Assignment 2 will be released tonight. You should allow plenty of time to tackle it. You will be given a generous deadline but it requires time to complete. Haskell-wise, this code will use a monad to track errors and the state needed to generate fresh unification variables.

55