[PPT] - The Computational Essence of Sorting Algorithms Ralf Hinze PowerPoint Presentation

SLIDE 1

The Computational Essence of Sorting Algorithms — Prologue WG 2.8

The Computational Essence of Sorting Algorithms

Ralf Hinze

Department of Computer Science, University of Oxford Wolfson Building, Parks Road, Oxford, OX1 3QD, England ralf.hinze@cs.ox.ac.uk http://www.cs.ox.ac.uk/ralf.hinze/

November 2012

Joint Work with Daniel W. H. James, Thomas Harper, Nicolas Wu and Jos´ e Pedro Magalh˜ aes. University of Oxford — Ralf Hinze 1-86

SLIDE 2

The Computational Essence of Sorting Algorithms — Prologue WG 2.8

Section 1 Prologue

University of Oxford — Ralf Hinze 2-86

SLIDE 3

The Computational Essence of Sorting Algorithms — Prologue WG 2.8

algorithmics


programming languages

University of Oxford — Ralf Hinze 3-86

SLIDE 4

The Computational Essence of Sorting Algorithms — Prologue WG 2.8

Any customer can have a car painted any colour that he wants so long as it is black. My Life and Work (1922) by Henry Ford The limits of my language mean the limits of my world. Tractatus Logico-Philosophicus (1922) by Ludwig Wittgenstein translated by C. K. Ogden and D. F. Pears

University of Oxford — Ralf Hinze 4-86

SLIDE 5

The Computational Essence of Sorting Algorithms — Prologue WG 2.8

design of sorting algorithms


semantics of programming languages

University of Oxford — Ralf Hinze 5-86

SLIDE 6

The Computational Essence of Sorting Algorithms — Prologue WG 2.8

categorical algorithmics

University of Oxford — Ralf Hinze 6-86

SLIDE 7

The Computational Essence of Sorting Algorithms — Prologue WG 2.8 University of Oxford — Ralf Hinze 7-86

SLIDE 8

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

Section 2 Insertion and selection sort I

University of Oxford — Ralf Hinze 8-86

SLIDE 9

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

. . . the area of sorting and searching provides an ideal framework for discussing a wide variety of important general issues:

How are good algorithms discovered?
. . .

Indeed, I believe that virtually every important aspect of programming arises somewhere in the context of sorting or searching! The Art of Computer Programming Volume 3: Sorting and Searching by Donald E. Knuth

University of Oxford — Ralf Hinze 9-86

SLIDE 10

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8 University of Oxford — Ralf Hinze 10-86

SLIDE 11

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.1 Insertion sort

also known as “bridge player sort”
idea: maintain an ordered sequence (invariant)
consume the input sequence one by one
insert each element into the ordered sequence
the focus is on the input

University of Oxford — Ralf Hinze 11-86

SLIDE 12

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.1 Knuth’s straight insertion sort

University of Oxford — Ralf Hinze 12-86

SLIDE 13

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.1 A functional insertion sort

insertSort :: (Ord a) ⇒ [a] → [a] insertSort [ ] = [ ] insertSort (x : xs) = x ‘insert‘ insertSort xs insert :: (Ord a) ⇒ a → [a] → [a] insert x [ ] = [x] insert x (y : xs) | x y = x : y : xs | otherwise = y : insert x xs

University of Oxford — Ralf Hinze 13-86

SLIDE 14

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.1 insertSort is an instance of foldr

x1 ‘insert‘ (x2 ‘insert‘ · · · ‘insert‘ (xn−1 ‘insert‘ (xn ‘insert‘ [ ]))) x1 : (x2 : · · · : (xn−1 : (xn : [ ]))) ❄ foldr insert [ ]

University of Oxford — Ralf Hinze 14-86

SLIDE 15

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.1 foldr captures a recursion pattern

x1 ⊗ (x2 ⊗ · · · ⊗ (xn−1 ⊗ (xn ⊗ e))) x1 : (x2 : · · · : (xn−1 : (xn : [ ]))) ❄ foldr (⊗) e Slogan: replacing constructors by functions.

University of Oxford — Ralf Hinze 15-86

SLIDE 16

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.1 Insertion sort, revisited

insertSort :: (Ord a) ⇒ [a] → [a] insertSort = foldr insert [ ] insert :: (Ord a) ⇒ a → [a] → [a] insert x [ ] = [x] insert x (y : xs) | x y = x : y : xs | otherwise = y : insert x xs But what about insert?

University of Oxford — Ralf Hinze 16-86

SLIDE 17

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.2 Selection sort

idea: produce an ordered sequence by repeatedly

selecting the minimum element

initial seed or state: input sequence
the focus is on the output
selection sort is in some sense dual to insertion sort

University of Oxford — Ralf Hinze 17-86

SLIDE 18

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.2 Knuth’s straight selection sort

University of Oxford — Ralf Hinze 18-86

SLIDE 19

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.2 A functional selection sort

selectSort :: (Ord a) ⇒ [a] → [a] selectSort = unfoldr select select :: (Ord a) ⇒ [a] → Maybe (a, [a]) select [ ] = Nothing select (x : xs) = case select xs of Nothing → Just (x, [ ]) Just (y, ys) | x y → Just (x, xs) | otherwise → Just (y, x : ys) But what about select?

University of Oxford — Ralf Hinze 19-86

SLIDE 20

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.2 Intermediate summary

unfoldr is the categorical dual of foldr
insertion is in some sense dual to selection
the details of this relationship are somewhat shrouded

by our language

to illuminate the connection, we use a type-driven

approach to algorithm design

University of Oxford — Ralf Hinze 20-86

SLIDE 21

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8 University of Oxford — Ralf Hinze 21-86

SLIDE 22

The Computational Essence of Sorting Algorithms — Background WG 2.8

Section 3 Background

University of Oxford — Ralf Hinze 22-86

SLIDE 23

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.0 Background

the list datatype is recursively defined

data [a] = [ ] | a : [a]

semantics: fixed point of an associated higher-order

type constructor data List list a = Nil | Cons a (list a)

for simplicity, let’s fix the type of elements,

data List list = Nil | Cons K list where K is some key type that admits ordering

categorical concept: initial algebra of a functor
NB. List is the non-recursive base functor of the list

datatype; µList is then the recursive list type

University of Oxford — Ralf Hinze 23-86

SLIDE 24

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.1 Functor in Haskell

in Haskell, a functor is given by a datatype definition

data List list = Nil | Cons K list

and an associated Functor declaration

instance Functor List where map f Nil = Nil map f (Cons k ks) = Cons k (f ks)

the mapping function changes the elements of a

container, but keeps its structure intact

the type parameter list marks the ‘recursive’ component

University of Oxford — Ralf Hinze 24-86

SLIDE 25

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.2 Algebra and algebra homomorphism

an F-algebra a is an arrow of type F A → A

F A a

A

F A a

F h

F B b

A

h B F B b

B
an F-algebra homomorphism h preserves the structure
F-algebras and homomorphisms form a category

University of Oxford — Ralf Hinze 25-86

SLIDE 26

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.2 Initial algebra

the initial object in this category, the initial F-algebra, is

the ‘least’ fixed point of F F (µF) in

F (fold a) F A

a

µF

fold a A

initiality entails that there is a unique homomorphism

from the initial algebra to any algebra a, called fold a

initial List-algebra: finite lists (in Set)

University of Oxford — Ralf Hinze 26-86

SLIDE 27

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.2 Initial algebra in Haskell

in Haskell, µf can be defined

newtype µf = In {in◦ :: f (µf )}

as an aside, In a will be written as ⌈a⌉
since in is an isomorphism, we can turn the commuting

diagram into a generic definition of fold fold :: (Functor f ) ⇒ (f a → a) → (µf → a) fold f = f · map (fold f ) · in◦

University of Oxford — Ralf Hinze 27-86

SLIDE 28

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.3 Duality

we obtain the categorical dual by reversing the arrows
algebras dualise to coalgebras
initial algebras dualise to final coalgebras

University of Oxford — Ralf Hinze 28-86

SLIDE 29

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.3 Coalgebra

an F-coalgebra c is an arrow of type C → F C

C c

F C

C c

h

D d

F C

F h F D D d

F D
an F-coalgebra homomorphism h preserves the structure
F-coalgebras and homomorphisms form a category

University of Oxford — Ralf Hinze 29-86

SLIDE 30

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.3 Final coalgebra

the final object in this category, the final F-coalgebra, is

the ‘greatest’ fixed point of F C c

unfold c

νF

ut
F C

F (unfold c) F (νF)

finality entails that there is a unique homomorphism to

the final coalgebra from any coalgebra c, called unfold c

final List-coalgebra: finite and infinite lists (in Set)

University of Oxford — Ralf Hinze 30-86

SLIDE 31

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.3 Final coalgebra in Haskell

in Haskell, νf can be defined

newtype νf = Out◦ {out :: f (νf )}

as an aside, Out◦ a will be written as ⌊a⌋
since out is an isomorphism, we can turn the

commuting diagram into a generic definition of unfold unfold :: (Functor f ) ⇒ (a → f a) → (a → νf ) unfold f = out◦ · map (unfold f ) · f

Haskell: initial algebras and final coalgebras coincide!

University of Oxford — Ralf Hinze 31-86

SLIDE 32

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf

University of Oxford — Ralf Hinze 32-86

SLIDE 33

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold ... : µF → νF ...

University of Oxford — Ralf Hinze 32-86

SLIDE 34

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold ... : µF → νF ... : F (νF) → νF

University of Oxford — Ralf Hinze 32-86

SLIDE 35

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold (unfold c) : µF → νF unfold c : F (νF) → νF

University of Oxford — Ralf Hinze 32-86

SLIDE 36

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold (unfold c) : µF → νF unfold c : F (νF) → νF c : F (νF) → F (F (νF))

University of Oxford — Ralf Hinze 32-86

SLIDE 37

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold (unfold c) : µF → νF unfold c : F (νF) → νF c : F (νF) → F (F (νF)) . . . or as an unfold: unfold (fold a) : µF → νF

University of Oxford — Ralf Hinze 32-86

SLIDE 38

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold (unfold c) : µF → νF unfold c : F (νF) → νF c : F (νF) → F (F (νF)) . . . or as an unfold: unfold (fold a) : µF → νF fold a : µF → F (µF)

University of Oxford — Ralf Hinze 32-86

SLIDE 39

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold (unfold c) : µF → νF unfold c : F (νF) → νF c : F (νF) → F (F (νF)) . . . or as an unfold: unfold (fold a) : µF → νF fold a : µF → F (µF) a : F (F (µF)) → F (µF)

University of Oxford — Ralf Hinze 32-86

SLIDE 40

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold (unfold c) : µF → νF unfold c : F (νF) → νF c : F (νF) → F (F (νF)) . . . or as an unfold: unfold (fold a) : µF → νF fold a : µF → F (µF) a : F (F (µF)) → F (µF) Obvious candidates: c = map out and a = map in.

University of Oxford — Ralf Hinze 32-86

SLIDE 41

The Computational Essence of Sorting Algorithms — Background WG 2.8

The coalgebra fold (map in) is the inverse of in; the algebra unfold (map out) is the inverse of out. Moreover, F (µF) in

F (νF)
ut◦ = unfold (map out)
µF

fold (map in) = in◦

fold out◦

unfold in◦ νF

ut
F (µF)

F (νF) (The triples µF, in, in◦ and νF, out◦, out are examples of bialgebras, more later.)

University of Oxford — Ralf Hinze 33-86

SLIDE 42

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Intermediate summary

initial algebra: syntax (finite trees)
folds: replacing constructors by functions
(denotational semantics: compositional valuation

function that maps syntax to semantics—folding over syntax trees)

final coalgebra: behaviour (finite and infinite trees)
unfolds: tracing a state space
(operational semantics: unfolding to transition trees)
we have seen a glimpse of type-driven program

development

running time (assuming a strict setting):
fold: proportional to the size of the input
unfold: proportional to the size of the output

(output-sensitive algorithm)

University of Oxford — Ralf Hinze 34-86

SLIDE 43

The Computational Essence of Sorting Algorithms — Background WG 2.8 University of Oxford — Ralf Hinze 35-86

SLIDE 44

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

Section 4 Exchange sort

University of Oxford — Ralf Hinze 36-86

SLIDE 45

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

4.0 Back to sorting

A sorting function takes a list to an ordered list, sort :: µList → νList where νList is the datatype of ordered lists: data List list = Nil | Cons K list instance Functor List where map f Nil = Nil map f (Cons k ks) = Cons k (f ks) (No guarantees, we use List for emphasis.)

University of Oxford — Ralf Hinze 37-86

SLIDE 46

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

To define a sorting function let us follow a type-directed approach: f :: µList → νList f = unfold c

University of Oxford — Ralf Hinze 38-86

SLIDE 47

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

To define a sorting function let us follow a type-directed approach: f :: µList → νList f = unfold c c :: µList → List (µList) c = fold a

University of Oxford — Ralf Hinze 38-86

SLIDE 48

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

To define a sorting function let us follow a type-directed approach: f :: µList → νList f = unfold c c :: µList → List (µList) c = fold a a :: List (List (µList)) → List (µList) a Nil = Nil a (Cons x Nil) = Cons x ⌈Nil⌉ a (Cons x (Cons y xs)) | x y = Cons x ⌈Cons y xs⌉ | otherwise = Cons y ⌈Cons x xs⌉

University of Oxford — Ralf Hinze 38-86

SLIDE 49

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

4.1 Bubble sort

We have re-invented bubble sort! bubbleSort :: µList → νList bubbleSort = unfold bubble bubble :: µList → List (µList) bubble = fold bub bub :: List (List (µList)) → List (µList) bub Nil = Nil bub (Cons x Nil) = Cons x ⌈Nil⌉ bub (Cons x (Cons y xs)) | x y = Cons x ⌈Cons y xs⌉ | otherwise = Cons y ⌈Cons x xs⌉

University of Oxford — Ralf Hinze 39-86

SLIDE 50

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

Dually, we can start with a fold: f :: µList → νList f = fold a

University of Oxford — Ralf Hinze 40-86

SLIDE 51

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

Dually, we can start with a fold: f :: µList → νList f = fold a a :: List (νList) → νList a = unfold c

University of Oxford — Ralf Hinze 40-86

SLIDE 52

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

Dually, we can start with a fold: f :: µList → νList f = fold a a :: List (νList) → νList a = unfold c c :: List (νList) → List (List (νList)) c Nil = Nil c (Cons x ⌊Nil⌋) = Cons x Nil c (Cons x ⌊Cons y xs⌋) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs)

University of Oxford — Ralf Hinze 40-86

SLIDE 53

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

4.2 Na ¨ ıve insertion sort

We obtain a na¨ ıve variant of insertion sort! naiveInsertionSort :: µList → νList naiveInsertionSort = fold naiveInsert naiveInsert :: List (νList) → νList naiveInsert = unfold naiveIns naiveIns :: List (νList) → List (List (νList)) naiveIns Nil = Nil naiveIns (Cons x ⌊Nil⌋) = Cons x Nil naiveIns (Cons x ⌊Cons y xs⌋) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs) Why na¨ ıve?

University of Oxford — Ralf Hinze 41-86

SLIDE 54

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

The algebra and the coalgebra are almost identical:

a :: List (List (µList)) → List (µList) a Nil = Nil a (Cons x Nil) = Cons x ⌈Nil⌉ a (Cons x (Cons y xs)) | x y = Cons x ⌈Cons y xs⌉ | otherwise = Cons y ⌈Cons x xs⌉ University of Oxford — Ralf Hinze 42-86

SLIDE 55

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

The algebra and the coalgebra are almost identical:

a :: List (List (µList)) → List (µList) a Nil = Nil a (Cons x Nil) = Cons x ⌈Nil⌉ a (Cons x (Cons y xs)) | x y = Cons x ⌈Cons y xs⌉ | otherwise = Cons y ⌈Cons x xs⌉ c :: List (νList) → List (List (νList)) c Nil = Nil c (Cons x ⌊Nil⌋) = Cons a Nil c (Cons x ⌊Cons y xs⌋) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs) University of Oxford — Ralf Hinze 42-86

SLIDE 56

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

The algebra and the coalgebra are almost identical:

a :: List (List (µList)) → List (µList) a Nil = Nil a (Cons x Nil) = Cons x ⌈Nil⌉ a (Cons x (Cons y xs)) | x y = Cons x ⌈Cons y xs⌉ | otherwise = Cons y ⌈Cons x xs⌉ c :: List (νList) → List (List (νList)) c Nil = Nil c (Cons x ⌊Nil⌋) = Cons a Nil c (Cons x ⌊Cons y xs⌋) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs)

We can unify them in a single natural transformation: swap :: List (List a) → List (List a) swap Nil = Nil swap (Cons x Nil) = Cons x Nil swap (Cons x (Cons y xs)) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs)

University of Oxford — Ralf Hinze 42-86

SLIDE 57

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

swap :: List (List x) → List (List x) swap Nil = Nil swap (Cons x Nil) = Cons x Nil swap (Cons x (Cons y l)) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs)

University of Oxford — Ralf Hinze 43-86

SLIDE 58

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

swap :: List (List x) → List (List x) swap Nil = Nil swap (Cons x Nil) = Cons x Nil swap (Cons x (Cons y l)) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs) We can re-define bubble and na¨ ıve insertion sort using swap: bubbleSort :: µList → νList bubbleSort = unfold (fold (map in · swap)) naiveInsertionSort :: µList → νList naiveInsertionSort = fold (unfold (swap · map out)) In a sense, swap extracts the computational ‘essence’ of bubble and na¨ ıve insertion sorting.

University of Oxford — Ralf Hinze 43-86

SLIDE 59

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

bubble sort initial input 2 4 1 3 2 ↔ 1 4 ↔ 1 1 ↔ 3 1 2 4 3 2 ↔ 3 4 ↔ 3 1 2 3 4 3 ↔ 4 1 2 3 4 1 2 3 4

utput

na¨ ıve insertion sort input 2 4 1 3 1 ↔ 3 2 4 1 3 4 ↔ 1 4 ↔ 3 2 4 1 3 2 ↔ 1 2 ↔ 3 3 ↔ 4 2 1 3 4 1 2 3 4 final output

University of Oxford — Ralf Hinze 44-86

SLIDE 60

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

4.2 Intermediate summary

swap exchanges adjacent elements
swap is the computational essence of bubble sort and

na¨ ıve insertion sort

running time Θ(n2)
how can we write true insertion sort?
first: proof that bubbleSort and naiveInsertionSort are

equal (in a strong sense)

University of Oxford — Ralf Hinze 45-86

SLIDE 61

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8 University of Oxford — Ralf Hinze 46-86

SLIDE 62

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

Section 5 Bialgebras and distributive laws

University of Oxford — Ralf Hinze 47-86

SLIDE 63

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

Recall that bubble is a List-algebra homomorphism. List (µList) in

map bubble List (List (µList))

swap

List (List (µList))

map in

µList

bubble List (µList)

University of Oxford — Ralf Hinze 48-86

SLIDE 64

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

Let us rearrange the diagram. List (µList) in

map bubble
List (List (µList))

swap

µList

bubble

List (List (µList))

map in

List (µList)

The algebra in and the coalgebra bubble form a swap-bialgebra: µList, in, bubble.

University of Oxford — Ralf Hinze 49-86

SLIDE 65

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

Recall that naiveInsert is a List-coalgebra homomorphism. List (νList) naiveInsert

map out
List (List (νList))

swap

νList
ut
List (List (νList))

map naiveInsert

List (νList)

The algebra naiveInsert and the coalgebra out also form a swap-bialgebra: νList, naiveInsert, out.

University of Oxford — Ralf Hinze 50-86

SLIDE 66

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

5.1 Bialgebra

For an algebra a and coalgebra c to be a swap-bialgebra, we must have that List X a

List c
List (List X)

swap

X

c

List (List X)

List a

List X

University of Oxford — Ralf Hinze 51-86

SLIDE 67

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

5.2 Bialgebra homomorphism

A swap-bialgebra homomorphism h is simultaneously an List-algebra and a List-coalgebra homomorphism. List X a

List h

List Y b

X

c

h

Y d

List X

List h List Y swap-bialgebras and homomorphisms form a category.

University of Oxford — Ralf Hinze 52-86

SLIDE 68

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

5.2 Initial and final bialgebra

The initial object in this category is µList, in, bubble; the final object is νList, naiveInsert, out. List (µList) in

List (νList)

naiveInsert

µList

bubble

fold naiveInsert

unfold bubble νList

ut
List (µList)

List (νList) By uniqueness, naiveInsertionSort and bubbleSort are equal.

University of Oxford — Ralf Hinze 53-86

SLIDE 69

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

5.2 Intermediate summary

swap is a distributive law
µList, in, bubble is the initial swap-bialgebra
νList, naiveInsert, out is the final swap-bialgebra
bubble sort and na¨

ıve insertion sort are two (strongly related) variations of the same idea: repeatedly exchanging adjacent elements

University of Oxford — Ralf Hinze 54-86

SLIDE 70

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8 University of Oxford — Ralf Hinze 55-86

SLIDE 71

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

Section 6 Insertion and selection sort II

University of Oxford — Ralf Hinze 56-86

SLIDE 72

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

sorting algorithms as folds of unfolds or unfolds of

folds necessarily have a running time of Θ(n2)

to define insertion and selection sort, we need variants
f folds and unfolds, so-called para- and apomorphisms

University of Oxford — Ralf Hinze 57-86

SLIDE 73

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

6.1 Paramorphism

we start by defining products

data a × b = As {outl :: a, outr :: b} (△) :: (c → a) → (c → b) → (c → a × b) (f △ g) x = As (f x) (g x)

we write As a b as a

b (we use it like Haskell’s a@b).

University of Oxford — Ralf Hinze 58-86

SLIDE 74

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

6.1 Paramorphism

we start by defining products

data a × b = As {outl :: a, outr :: b} (△) :: (c → a) → (c → b) → (c → a × b) (f △ g) x = As (f x) (g x)

we write As a b as a

b (we use it like Haskell’s a@b).

we are now ready to define paramorphisms:

para :: (Functor f ) ⇒ (f (µf × a) → a) → (µf → a) para f = f · map (id △ para f ) · in◦ a paramorphism also provides the intermediate input: the ‘algebra’ has type f (µf × a) → a instead of f a → a

slogan: eats its argument and keeps it too

University of Oxford — Ralf Hinze 58-86

SLIDE 75

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

6.2 Apomorphism

products dualise to sums

data a + b = Stop a | Play b (▽) :: (a → c) → (b → c) → (a + b → c) (f ▽ g) (Stop a) = f a (f ▽ g) (Play b) = g b

we write Stop a as a ◾, and Play b as ▸ b

University of Oxford — Ralf Hinze 59-86

SLIDE 76

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

6.2 Apomorphism

products dualise to sums

data a + b = Stop a | Play b (▽) :: (a → c) → (b → c) → (a + b → c) (f ▽ g) (Stop a) = f a (f ▽ g) (Play b) = g b

we write Stop a as a ◾, and Play b as ▸ b
paramorphisms dualise to apomorphisms:

apo :: (Functor f ) ⇒ (a → f (νf + a)) → (a → νf ) apo f = out◦ · map (id ▽ apo f ) · f the corecursion is split into two branches, with no recursive call on the left

apomorphisms improve the running time

University of Oxford — Ralf Hinze 59-86

SLIDE 77

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

With apomorphisms, we can write the insertion function as

ne that stops scanning after inserting an element:

insertSort :: µList → νList insertSort = fold insert

University of Oxford — Ralf Hinze 60-86

SLIDE 78

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

With apomorphisms, we can write the insertion function as

ne that stops scanning after inserting an element:

insertSort :: µList → νList insertSort = fold insert insert :: List (νList) → νList insert = apo ins

University of Oxford — Ralf Hinze 60-86

SLIDE 79

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

With apomorphisms, we can write the insertion function as

ne that stops scanning after inserting an element:

insertSort :: µList → νList insertSort = fold insert insert :: List (νList) → νList insert = apo ins ins :: List (νList) → List (νList + List (νList)) ins Nil = Nil ins (Cons x ⌊Nil⌋) = Cons x (⌊Nil⌋ ◾) ins (Cons x ⌊Cons y xs⌋) | x y = Cons x (⌊Cons y xs⌋ ◾) | otherwise = Cons y (▸ (Cons x xs))

University of Oxford — Ralf Hinze 60-86

SLIDE 80

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

From ins we can extract a natural transformation, which we call swop for swap‘n’stop: swop :: List (a × List a) → List (a + List a) swop Nil = Nil swop (Cons x (xs Nil)) = Cons x (xs ◾) swop (Cons x (xs (Cons y ys))) | x y = Cons x (xs ◾) | otherwise = Cons y (▸ (Cons x ys))

University of Oxford — Ralf Hinze 61-86

SLIDE 81

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

From swop we get both insertion and selection sort: insertSort :: µList → νList insertSort = fold (apo (swop · map (id △ out))) selectSort :: µList → νList selectSort = unfold (para (map (id ▽ in) · swop)) In general, a natural transformation such as swop gives rise to two algorithms. Algorithms for free!

University of Oxford — Ralf Hinze 62-86

SLIDE 82

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

6.3 Intermediate summary

apomorphisms improve the running time
running time of insertion sort: worst case still Θ(n2),

but best case Θ(n)

(paramorphisms don’t improve the running time)
the computational essence of insertion and selection

sort is the natural transformation swop

in general, we shall seek natural transformation of type

F (A × G A) → G (A + F A)

(proof of equality involves (co-) pointed functors)

University of Oxford — Ralf Hinze 63-86

SLIDE 83

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8 University of Oxford — Ralf Hinze 64-86

SLIDE 84

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

Section 7 Quicksort and treesort

University of Oxford — Ralf Hinze 65-86

SLIDE 85

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

so far: one-phase sorting algorithms

µList → νList

to improve performance we need to exchange

non-adjacent elements

next: two-phase sorting algorithms that make use of an

intermediate data structure µList → νTree → µTree → νList

the intermediate data structure can sometimes be

deforested (turning a data into a control structure)

we can play our game for each phase

University of Oxford — Ralf Hinze 66-86

SLIDE 86

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

7.0 Search trees

an obvious intermediate data structure is a binary tree

data Tree tree = Empty | Node tree K tree instance Functor Tree where map f Empty = Empty map f (Node l k r) = Node (f l) k (f r)

we assume a ‘horizontal’ ordering

type SearchTree = Tree

University of Oxford — Ralf Hinze 67-86

SLIDE 87

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

7.1 Phase one: growing a search tree

the essence of growing a search tree

sprout :: List (a × SearchTree a) → SearchTree (a + List a) sprout Nil = Empty sprout (Cons x (t Empty)) = Node (t ◾) x (t ◾) sprout (Cons x (t (Node l y r))) | x y = Node (▸ (Cons x l)) y (r ◾) | otherwise = Node (l ◾) y (▸ (Cons x r))

this is the only sensible definition: no choices
we compare elements across some distance

University of Oxford — Ralf Hinze 68-86

SLIDE 88

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

we can either recursively partition a list, building

subtrees from the resulting sublists, or start with an empty tree and repeatedly insert the elements into it grow :: µList → νSearchTree grow = unfold (para (map (id ▽ in) · sprout)) grow′ :: µList → νSearchTree grow′ = fold (apo (sprout · map (id △ out)))

the algebra is a useful function on its own: insertion

into a search tree

efficient insertion into a tree is necessarily an

apomorphism

University of Oxford — Ralf Hinze 69-86

SLIDE 89

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

7.2 Phase two: withering a search tree

the essence of withering a search tree

wither :: SearchTree (a × List a) → List (a + SearchTree a) wither Empty = Nil wither (Node (l Nil) x (r )) = Cons x (r ◾) wither (Node (l (Cons x l′)) y (r )) = Cons x (▸ (Node l′ y r))

again, this is the only sensible definition

University of Oxford — Ralf Hinze 70-86

SLIDE 90

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

this should surprise no one: the second phase would

surely be an in-order traversal flatten :: µSearchTree → νList flatten = fold (apo (wither · map (id △ out))) flatten′ :: µSearchTree → νList flatten′ = unfold (para (map (id ▽ in) · wither))

the algebra is essentially a ternary version of append
the coalgebra deletes the leftmost element from a

search tree

University of Oxford — Ralf Hinze 71-86

SLIDE 91

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

7.2 Putting things together

We obtain the famous quicksort and the less prominent treesort algorithms, quickSort :: µList → νList quickSort = flatten · downcast · grow treeSort :: µList → νList treeSort = flatten · downcast · grow′ where downcast :: (Functor f ) ⇒ νf → µf projects the final coalgebra onto the initial algebra.

University of Oxford — Ralf Hinze 72-86

SLIDE 92

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

7.2 Intermediate summary

once the intermediate data structure has been fixed,

everything falls into place: no choices

observation: only the first phase performs comparisons
quicksort and treesort are are two (strongly related)

variations of the same idea

running time: worst case still Θ(n2), but average case

Θ(n log n)

University of Oxford — Ralf Hinze 73-86

SLIDE 93

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8 University of Oxford — Ralf Hinze 74-86

SLIDE 94

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

Section 8 Heapsort and minglesort

University of Oxford — Ralf Hinze 75-86

SLIDE 95

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

8.0 Heaps

a search tree imposes a horizontal ordering
we can also assume a ‘vertical’ ordering

type Heap = Tree

University of Oxford — Ralf Hinze 76-86

SLIDE 96

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

8.1 Phase one: piling up a heap

the essence of piling up a heap

pile :: List (a × Heap a) → Heap (a + List a) pile Nil = Empty pile (Cons x (t Empty)) = Node (t ◾) x (t ◾) pile (Cons x (t (Node l y r))) | x y = Node (▸ (Cons y r)) x (l ◾) | otherwise = Node (▸ (Cons x r)) y (l ◾)

now we have a choice (3rd equation)! Braun’s trick!

University of Oxford — Ralf Hinze 77-86

SLIDE 97

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

8.1 Phase one: piling up a heap

the essence of piling up a heap

pile :: List (a × Heap a) → Heap (a + List a) pile Nil = Empty pile (Cons x (t Empty)) = Node (t ◾) x (t ◾) pile (Cons x (t (Node l y r))) | x y = Node (▸ (Cons y r)) x (l ◾) | otherwise = Node (▸ (Cons x r)) y (l ◾)

now we have a choice (3rd equation)! Braun’s trick!
let a = x ‘min‘ y and b = x ‘max‘ y,

= Node (▸ (Cons b l)) a (r ◾) = Node (r ◾) a (▸ (Cons b l)) = Node (l ◾) a (▸ (Cons b r)) = Node (▸ (Cons b r)) a (l ◾)

University of Oxford — Ralf Hinze 77-86

SLIDE 98

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

as usual we obtain two algorithms

heapify :: µList → νHeap heapify = unfold (para (map (id ▽ in) · pile)) heapify′ :: µList → νHeap heapify′ = fold (apo (pile · map (id △ out)))

the algebra is a useful function on its own: insertion

into a heap

University of Oxford — Ralf Hinze 78-86

SLIDE 99

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

8.2 Phase two: sifting through a heap

the essence of sifting through a heap

sift :: Heap (a × List a) → List (a + Heap a) sift Empty = Nil sift (Node (l Nil) x (r )) = Cons x (r ◾) sift (Node (l ) x (r Nil)) = Cons x (l ◾) sift (Node (l (Cons y l′)) x (r (Cons z r′))) | y z = Cons x (▸ (Node l′ y r)) | otherwise = Cons x (▸ (Node l z r′))

when constructing the heap node to continue with, we

have the option to swap left with right, but this buys us nothing

University of Oxford — Ralf Hinze 79-86

SLIDE 100

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

again, we obtain two algorithms

unheapify :: µHeap → νList unheapify = fold (apo (sift · map (id △ out))) unheapify′ :: µHeap → νList unheapify′ = unfold (para (map (id ▽ in) · sift))

the coalgebra deletes the mimimum element from a

heap

University of Oxford — Ralf Hinze 80-86

SLIDE 101

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

8.2 Putting things together

we obtain heapsort and a variant of heapsort that

behaves suspiciously like mergesort heapSort :: µList → νList heapSort = unheapify · downcast · heapify mingleSort :: µList → νList mingleSort = unheapify′ · downcast · heapify′

trimerous mergesort, called minglesort, builds a heap by

repeatedly dividing the input into three parts: two lists

f balanced length along with the minimum element
the merging phase is a similarly trimerous operation

University of Oxford — Ralf Hinze 81-86

SLIDE 102

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

8.2 Intermediate summary

the intermediate data structure provides us with a

choice (heaps are more flexible than search trees)

observation: both phases use comparisons
running time: worst case Θ(n log n)—insensitive to the

input

minglesort is a recent variant of heapsort (closely

related to Heap-Mergesort, see Computer and Mathematics with Applications 39, 2000)

an analogous approach using binary leaf trees works for

true mergesort

University of Oxford — Ralf Hinze 82-86

SLIDE 103

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8 University of Oxford — Ralf Hinze 83-86

SLIDE 104

The Computational Essence of Sorting Algorithms — Epilogue WG 2.8

Section 9 Epilogue

University of Oxford — Ralf Hinze 84-86

SLIDE 105

The Computational Essence of Sorting Algorithms — Epilogue WG 2.8

9.1 Summary

categorical algorithmics
type-driven algorithm design building on a few

principled recursion operators

few design decisions: intermediate data structures
no rabbits!
categorical duality gives us algorithms for free
see WGP ’12 paper: Sorting with Bialgebras and

Distributive Laws by Ralf Hinze, Daniel W. H. James, Thomas Harper, Nicolas Wu and Jos´ e Pedro Magalh˜ aes

University of Oxford — Ralf Hinze 85-86

SLIDE 106

The Computational Essence of Sorting Algorithms — Epilogue WG 2.8

9.2 Summary and future work

computational essence
na¨

ıve insertion and bubble sort: swap

insertion and selection sort: swop
growing a search tree: sprout
withering a search tree: wither
piling up a heap: pile
sifting through a heap: sift
top-down algorithms: regular datatypes
bottom-up algorithms: nested datatypes

University of Oxford — Ralf Hinze 86-86