The Computational Essence of Sorting Algorithms Ralf Hinze - - PowerPoint PPT Presentation

the computational essence of sorting algorithms
SMART_READER_LITE
LIVE PREVIEW

The Computational Essence of Sorting Algorithms Ralf Hinze - - PowerPoint PPT Presentation

The Computational Essence of Sorting Algorithms Prologue WG 2.8 The Computational Essence of Sorting Algorithms Ralf Hinze Department of Computer Science, University of Oxford Wolfson Building, Parks Road, Oxford, OX1 3QD, England


slide-1
SLIDE 1

The Computational Essence of Sorting Algorithms — Prologue WG 2.8

The Computational Essence of Sorting Algorithms

Ralf Hinze

Department of Computer Science, University of Oxford Wolfson Building, Parks Road, Oxford, OX1 3QD, England ralf.hinze@cs.ox.ac.uk http://www.cs.ox.ac.uk/ralf.hinze/

November 2012

Joint Work with Daniel W. H. James, Thomas Harper, Nicolas Wu and Jos´ e Pedro Magalh˜ aes. University of Oxford — Ralf Hinze 1-86

slide-2
SLIDE 2

The Computational Essence of Sorting Algorithms — Prologue WG 2.8

Section 1 Prologue

University of Oxford — Ralf Hinze 2-86

slide-3
SLIDE 3

The Computational Essence of Sorting Algorithms — Prologue WG 2.8

algorithmics

  • programming languages

University of Oxford — Ralf Hinze 3-86

slide-4
SLIDE 4

The Computational Essence of Sorting Algorithms — Prologue WG 2.8

Any customer can have a car painted any colour that he wants so long as it is black. My Life and Work (1922) by Henry Ford The limits of my language mean the limits of my world. Tractatus Logico-Philosophicus (1922) by Ludwig Wittgenstein translated by C. K. Ogden and D. F. Pears

University of Oxford — Ralf Hinze 4-86

slide-5
SLIDE 5

The Computational Essence of Sorting Algorithms — Prologue WG 2.8

design of sorting algorithms

  • semantics of programming languages

University of Oxford — Ralf Hinze 5-86

slide-6
SLIDE 6

The Computational Essence of Sorting Algorithms — Prologue WG 2.8

categorical algorithmics

University of Oxford — Ralf Hinze 6-86

slide-7
SLIDE 7

The Computational Essence of Sorting Algorithms — Prologue WG 2.8 University of Oxford — Ralf Hinze 7-86

slide-8
SLIDE 8

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

Section 2 Insertion and selection sort I

University of Oxford — Ralf Hinze 8-86

slide-9
SLIDE 9

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

. . . the area of sorting and searching provides an ideal framework for discussing a wide variety of important general issues:

  • How are good algorithms discovered?
  • . . .

Indeed, I believe that virtually every important aspect of programming arises somewhere in the context of sorting or searching! The Art of Computer Programming Volume 3: Sorting and Searching by Donald E. Knuth

University of Oxford — Ralf Hinze 9-86

slide-10
SLIDE 10

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8 University of Oxford — Ralf Hinze 10-86

slide-11
SLIDE 11

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.1 Insertion sort

  • also known as “bridge player sort”
  • idea: maintain an ordered sequence (invariant)
  • consume the input sequence one by one
  • insert each element into the ordered sequence
  • the focus is on the input

University of Oxford — Ralf Hinze 11-86

slide-12
SLIDE 12

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.1 Knuth’s straight insertion sort

University of Oxford — Ralf Hinze 12-86

slide-13
SLIDE 13

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.1 A functional insertion sort

insertSort :: (Ord a) ⇒ [a] → [a] insertSort [ ] = [ ] insertSort (x : xs) = x ‘insert‘ insertSort xs insert :: (Ord a) ⇒ a → [a] → [a] insert x [ ] = [x] insert x (y : xs) | x y = x : y : xs | otherwise = y : insert x xs

University of Oxford — Ralf Hinze 13-86

slide-14
SLIDE 14

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.1 insertSort is an instance of foldr

x1 ‘insert‘ (x2 ‘insert‘ · · · ‘insert‘ (xn−1 ‘insert‘ (xn ‘insert‘ [ ]))) x1 : (x2 : · · · : (xn−1 : (xn : [ ]))) ❄ foldr insert [ ]

University of Oxford — Ralf Hinze 14-86

slide-15
SLIDE 15

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.1 foldr captures a recursion pattern

x1 ⊗ (x2 ⊗ · · · ⊗ (xn−1 ⊗ (xn ⊗ e))) x1 : (x2 : · · · : (xn−1 : (xn : [ ]))) ❄ foldr (⊗) e Slogan: replacing constructors by functions.

University of Oxford — Ralf Hinze 15-86

slide-16
SLIDE 16

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.1 Insertion sort, revisited

insertSort :: (Ord a) ⇒ [a] → [a] insertSort = foldr insert [ ] insert :: (Ord a) ⇒ a → [a] → [a] insert x [ ] = [x] insert x (y : xs) | x y = x : y : xs | otherwise = y : insert x xs But what about insert?

University of Oxford — Ralf Hinze 16-86

slide-17
SLIDE 17

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.2 Selection sort

  • idea: produce an ordered sequence by repeatedly

selecting the minimum element

  • initial seed or state: input sequence
  • the focus is on the output
  • selection sort is in some sense dual to insertion sort

University of Oxford — Ralf Hinze 17-86

slide-18
SLIDE 18

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.2 Knuth’s straight selection sort

University of Oxford — Ralf Hinze 18-86

slide-19
SLIDE 19

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.2 A functional selection sort

selectSort :: (Ord a) ⇒ [a] → [a] selectSort = unfoldr select select :: (Ord a) ⇒ [a] → Maybe (a, [a]) select [ ] = Nothing select (x : xs) = case select xs of Nothing → Just (x, [ ]) Just (y, ys) | x y → Just (x, xs) | otherwise → Just (y, x : ys) But what about select?

University of Oxford — Ralf Hinze 19-86

slide-20
SLIDE 20

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8

2.2 Intermediate summary

  • unfoldr is the categorical dual of foldr
  • insertion is in some sense dual to selection
  • the details of this relationship are somewhat shrouded

by our language

  • to illuminate the connection, we use a type-driven

approach to algorithm design

University of Oxford — Ralf Hinze 20-86

slide-21
SLIDE 21

The Computational Essence of Sorting Algorithms — Insertion and selection sort I WG 2.8 University of Oxford — Ralf Hinze 21-86

slide-22
SLIDE 22

The Computational Essence of Sorting Algorithms — Background WG 2.8

Section 3 Background

University of Oxford — Ralf Hinze 22-86

slide-23
SLIDE 23

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.0 Background

  • the list datatype is recursively defined

data [a] = [ ] | a : [a]

  • semantics: fixed point of an associated higher-order

type constructor data List list a = Nil | Cons a (list a)

  • for simplicity, let’s fix the type of elements,

data List list = Nil | Cons K list where K is some key type that admits ordering

  • categorical concept: initial algebra of a functor
  • NB. List is the non-recursive base functor of the list

datatype; µList is then the recursive list type

University of Oxford — Ralf Hinze 23-86

slide-24
SLIDE 24

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.1 Functor in Haskell

  • in Haskell, a functor is given by a datatype definition

data List list = Nil | Cons K list

  • and an associated Functor declaration

instance Functor List where map f Nil = Nil map f (Cons k ks) = Cons k (f ks)

  • the mapping function changes the elements of a

container, but keeps its structure intact

  • the type parameter list marks the ‘recursive’ component

University of Oxford — Ralf Hinze 24-86

slide-25
SLIDE 25

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.2 Algebra and algebra homomorphism

  • an F-algebra a is an arrow of type F A → A

F A a

  • A

F A a

  • F h

F B b

  • A

h B F B b

  • B
  • an F-algebra homomorphism h preserves the structure
  • F-algebras and homomorphisms form a category

University of Oxford — Ralf Hinze 25-86

slide-26
SLIDE 26

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.2 Initial algebra

  • the initial object in this category, the initial F-algebra, is

the ‘least’ fixed point of F F (µF) in

  • F (fold a) F A

a

  • µF

fold a A

  • initiality entails that there is a unique homomorphism

from the initial algebra to any algebra a, called fold a

  • initial List-algebra: finite lists (in Set)

University of Oxford — Ralf Hinze 26-86

slide-27
SLIDE 27

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.2 Initial algebra in Haskell

  • in Haskell, µf can be defined

newtype µf = In {in◦ :: f (µf )}

  • as an aside, In a will be written as ⌈a⌉
  • since in is an isomorphism, we can turn the commuting

diagram into a generic definition of fold fold :: (Functor f ) ⇒ (f a → a) → (µf → a) fold f = f · map (fold f ) · in◦

University of Oxford — Ralf Hinze 27-86

slide-28
SLIDE 28

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.3 Duality

  • we obtain the categorical dual by reversing the arrows
  • algebras dualise to coalgebras
  • initial algebras dualise to final coalgebras

University of Oxford — Ralf Hinze 28-86

slide-29
SLIDE 29

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.3 Coalgebra

  • an F-coalgebra c is an arrow of type C → F C

C c

  • F C

C c

  • h

D d

  • F C

F h F D D d

  • F D
  • an F-coalgebra homomorphism h preserves the structure
  • F-coalgebras and homomorphisms form a category

University of Oxford — Ralf Hinze 29-86

slide-30
SLIDE 30

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.3 Final coalgebra

  • the final object in this category, the final F-coalgebra, is

the ‘greatest’ fixed point of F C c

  • unfold c

νF

  • ut
  • F C

F (unfold c) F (νF)

  • finality entails that there is a unique homomorphism to

the final coalgebra from any coalgebra c, called unfold c

  • final List-coalgebra: finite and infinite lists (in Set)

University of Oxford — Ralf Hinze 30-86

slide-31
SLIDE 31

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.3 Final coalgebra in Haskell

  • in Haskell, νf can be defined

newtype νf = Out◦ {out :: f (νf )}

  • as an aside, Out◦ a will be written as ⌊a⌋
  • since out is an isomorphism, we can turn the

commuting diagram into a generic definition of unfold unfold :: (Functor f ) ⇒ (a → f a) → (a → νf ) unfold f = out◦ · map (unfold f ) · f

  • Haskell: initial algebras and final coalgebras coincide!

University of Oxford — Ralf Hinze 31-86

slide-32
SLIDE 32

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf

University of Oxford — Ralf Hinze 32-86

slide-33
SLIDE 33

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold ... : µF → νF ...

University of Oxford — Ralf Hinze 32-86

slide-34
SLIDE 34

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold ... : µF → νF ... : F (νF) → νF

University of Oxford — Ralf Hinze 32-86

slide-35
SLIDE 35

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold (unfold c) : µF → νF unfold c : F (νF) → νF

University of Oxford — Ralf Hinze 32-86

slide-36
SLIDE 36

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold (unfold c) : µF → νF unfold c : F (νF) → νF c : F (νF) → F (F (νF))

University of Oxford — Ralf Hinze 32-86

slide-37
SLIDE 37

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold (unfold c) : µF → νF unfold c : F (νF) → νF c : F (νF) → F (F (νF)) . . . or as an unfold: unfold (fold a) : µF → νF

University of Oxford — Ralf Hinze 32-86

slide-38
SLIDE 38

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold (unfold c) : µF → νF unfold c : F (νF) → νF c : F (νF) → F (F (νF)) . . . or as an unfold: unfold (fold a) : µF → νF fold a : µF → F (µF)

University of Oxford — Ralf Hinze 32-86

slide-39
SLIDE 39

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold (unfold c) : µF → νF unfold c : F (νF) → νF c : F (νF) → F (F (νF)) . . . or as an unfold: unfold (fold a) : µF → νF fold a : µF → F (µF) a : F (F (µF)) → F (µF)

University of Oxford — Ralf Hinze 32-86

slide-40
SLIDE 40

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Embedding initial into final

Least fixed points can be embedded into greatest fixed points. upcast :: (Functor f ) ⇒ µf → νf How to define upcast? We can write it as a fold . . . fold (unfold c) : µF → νF unfold c : F (νF) → νF c : F (νF) → F (F (νF)) . . . or as an unfold: unfold (fold a) : µF → νF fold a : µF → F (µF) a : F (F (µF)) → F (µF) Obvious candidates: c = map out and a = map in.

University of Oxford — Ralf Hinze 32-86

slide-41
SLIDE 41

The Computational Essence of Sorting Algorithms — Background WG 2.8

The coalgebra fold (map in) is the inverse of in; the algebra unfold (map out) is the inverse of out. Moreover, F (µF) in

  • F (νF)
  • ut◦ = unfold (map out)
  • µF

fold (map in) = in◦

  • fold out◦

unfold in◦ νF

  • ut
  • F (µF)

F (νF) (The triples µF, in, in◦ and νF, out◦, out are examples of bialgebras, more later.)

University of Oxford — Ralf Hinze 33-86

slide-42
SLIDE 42

The Computational Essence of Sorting Algorithms — Background WG 2.8

3.4 Intermediate summary

  • initial algebra: syntax (finite trees)
  • folds: replacing constructors by functions
  • (denotational semantics: compositional valuation

function that maps syntax to semantics—folding over syntax trees)

  • final coalgebra: behaviour (finite and infinite trees)
  • unfolds: tracing a state space
  • (operational semantics: unfolding to transition trees)
  • we have seen a glimpse of type-driven program

development

  • running time (assuming a strict setting):
  • fold: proportional to the size of the input
  • unfold: proportional to the size of the output

(output-sensitive algorithm)

University of Oxford — Ralf Hinze 34-86

slide-43
SLIDE 43

The Computational Essence of Sorting Algorithms — Background WG 2.8 University of Oxford — Ralf Hinze 35-86

slide-44
SLIDE 44

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

Section 4 Exchange sort

University of Oxford — Ralf Hinze 36-86

slide-45
SLIDE 45

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

4.0 Back to sorting

A sorting function takes a list to an ordered list, sort :: µList → νList where νList is the datatype of ordered lists: data List list = Nil | Cons K list instance Functor List where map f Nil = Nil map f (Cons k ks) = Cons k (f ks) (No guarantees, we use List for emphasis.)

University of Oxford — Ralf Hinze 37-86

slide-46
SLIDE 46

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

To define a sorting function let us follow a type-directed approach: f :: µList → νList f = unfold c

University of Oxford — Ralf Hinze 38-86

slide-47
SLIDE 47

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

To define a sorting function let us follow a type-directed approach: f :: µList → νList f = unfold c c :: µList → List (µList) c = fold a

University of Oxford — Ralf Hinze 38-86

slide-48
SLIDE 48

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

To define a sorting function let us follow a type-directed approach: f :: µList → νList f = unfold c c :: µList → List (µList) c = fold a a :: List (List (µList)) → List (µList) a Nil = Nil a (Cons x Nil) = Cons x ⌈Nil⌉ a (Cons x (Cons y xs)) | x y = Cons x ⌈Cons y xs⌉ | otherwise = Cons y ⌈Cons x xs⌉

University of Oxford — Ralf Hinze 38-86

slide-49
SLIDE 49

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

4.1 Bubble sort

We have re-invented bubble sort! bubbleSort :: µList → νList bubbleSort = unfold bubble bubble :: µList → List (µList) bubble = fold bub bub :: List (List (µList)) → List (µList) bub Nil = Nil bub (Cons x Nil) = Cons x ⌈Nil⌉ bub (Cons x (Cons y xs)) | x y = Cons x ⌈Cons y xs⌉ | otherwise = Cons y ⌈Cons x xs⌉

University of Oxford — Ralf Hinze 39-86

slide-50
SLIDE 50

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

Dually, we can start with a fold: f :: µList → νList f = fold a

University of Oxford — Ralf Hinze 40-86

slide-51
SLIDE 51

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

Dually, we can start with a fold: f :: µList → νList f = fold a a :: List (νList) → νList a = unfold c

University of Oxford — Ralf Hinze 40-86

slide-52
SLIDE 52

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

Dually, we can start with a fold: f :: µList → νList f = fold a a :: List (νList) → νList a = unfold c c :: List (νList) → List (List (νList)) c Nil = Nil c (Cons x ⌊Nil⌋) = Cons x Nil c (Cons x ⌊Cons y xs⌋) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs)

University of Oxford — Ralf Hinze 40-86

slide-53
SLIDE 53

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

4.2 Na ¨ ıve insertion sort

We obtain a na¨ ıve variant of insertion sort! naiveInsertionSort :: µList → νList naiveInsertionSort = fold naiveInsert naiveInsert :: List (νList) → νList naiveInsert = unfold naiveIns naiveIns :: List (νList) → List (List (νList)) naiveIns Nil = Nil naiveIns (Cons x ⌊Nil⌋) = Cons x Nil naiveIns (Cons x ⌊Cons y xs⌋) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs) Why na¨ ıve?

University of Oxford — Ralf Hinze 41-86

slide-54
SLIDE 54

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

The algebra and the coalgebra are almost identical:

a :: List (List (µList)) → List (µList) a Nil = Nil a (Cons x Nil) = Cons x ⌈Nil⌉ a (Cons x (Cons y xs)) | x y = Cons x ⌈Cons y xs⌉ | otherwise = Cons y ⌈Cons x xs⌉ University of Oxford — Ralf Hinze 42-86

slide-55
SLIDE 55

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

The algebra and the coalgebra are almost identical:

a :: List (List (µList)) → List (µList) a Nil = Nil a (Cons x Nil) = Cons x ⌈Nil⌉ a (Cons x (Cons y xs)) | x y = Cons x ⌈Cons y xs⌉ | otherwise = Cons y ⌈Cons x xs⌉ c :: List (νList) → List (List (νList)) c Nil = Nil c (Cons x ⌊Nil⌋) = Cons a Nil c (Cons x ⌊Cons y xs⌋) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs) University of Oxford — Ralf Hinze 42-86

slide-56
SLIDE 56

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

The algebra and the coalgebra are almost identical:

a :: List (List (µList)) → List (µList) a Nil = Nil a (Cons x Nil) = Cons x ⌈Nil⌉ a (Cons x (Cons y xs)) | x y = Cons x ⌈Cons y xs⌉ | otherwise = Cons y ⌈Cons x xs⌉ c :: List (νList) → List (List (νList)) c Nil = Nil c (Cons x ⌊Nil⌋) = Cons a Nil c (Cons x ⌊Cons y xs⌋) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs)

We can unify them in a single natural transformation: swap :: List (List a) → List (List a) swap Nil = Nil swap (Cons x Nil) = Cons x Nil swap (Cons x (Cons y xs)) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs)

University of Oxford — Ralf Hinze 42-86

slide-57
SLIDE 57

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

swap :: List (List x) → List (List x) swap Nil = Nil swap (Cons x Nil) = Cons x Nil swap (Cons x (Cons y l)) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs)

University of Oxford — Ralf Hinze 43-86

slide-58
SLIDE 58

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

swap :: List (List x) → List (List x) swap Nil = Nil swap (Cons x Nil) = Cons x Nil swap (Cons x (Cons y l)) | x y = Cons x (Cons y xs) | otherwise = Cons y (Cons x xs) We can re-define bubble and na¨ ıve insertion sort using swap: bubbleSort :: µList → νList bubbleSort = unfold (fold (map in · swap)) naiveInsertionSort :: µList → νList naiveInsertionSort = fold (unfold (swap · map out)) In a sense, swap extracts the computational ‘essence’ of bubble and na¨ ıve insertion sorting.

University of Oxford — Ralf Hinze 43-86

slide-59
SLIDE 59

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

bubble sort initial input 2 4 1 3 2 ↔ 1 4 ↔ 1 1 ↔ 3 1 2 4 3 2 ↔ 3 4 ↔ 3 1 2 3 4 3 ↔ 4 1 2 3 4 1 2 3 4

  • utput

na¨ ıve insertion sort input 2 4 1 3 1 ↔ 3 2 4 1 3 4 ↔ 1 4 ↔ 3 2 4 1 3 2 ↔ 1 2 ↔ 3 3 ↔ 4 2 1 3 4 1 2 3 4 final output

University of Oxford — Ralf Hinze 44-86

slide-60
SLIDE 60

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8

4.2 Intermediate summary

  • swap exchanges adjacent elements
  • swap is the computational essence of bubble sort and

na¨ ıve insertion sort

  • running time Θ(n2)
  • how can we write true insertion sort?
  • first: proof that bubbleSort and naiveInsertionSort are

equal (in a strong sense)

University of Oxford — Ralf Hinze 45-86

slide-61
SLIDE 61

The Computational Essence of Sorting Algorithms — Exchange sort WG 2.8 University of Oxford — Ralf Hinze 46-86

slide-62
SLIDE 62

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

Section 5 Bialgebras and distributive laws

University of Oxford — Ralf Hinze 47-86

slide-63
SLIDE 63

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

Recall that bubble is a List-algebra homomorphism. List (µList) in

  • map bubble List (List (µList))

swap

  • List (List (µList))

map in

  • µList

bubble List (µList)

University of Oxford — Ralf Hinze 48-86

slide-64
SLIDE 64

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

Let us rearrange the diagram. List (µList) in

  • map bubble
  • List (List (µList))

swap

  • µList

bubble

  • List (List (µList))

map in

  • List (µList)

The algebra in and the coalgebra bubble form a swap-bialgebra: µList, in, bubble.

University of Oxford — Ralf Hinze 49-86

slide-65
SLIDE 65

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

Recall that naiveInsert is a List-coalgebra homomorphism. List (νList) naiveInsert

  • map out
  • List (List (νList))

swap

  • νList
  • ut
  • List (List (νList))

map naiveInsert

  • List (νList)

The algebra naiveInsert and the coalgebra out also form a swap-bialgebra: νList, naiveInsert, out.

University of Oxford — Ralf Hinze 50-86

slide-66
SLIDE 66

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

5.1 Bialgebra

For an algebra a and coalgebra c to be a swap-bialgebra, we must have that List X a

  • List c
  • List (List X)

swap

  • X

c

  • List (List X)

List a

  • List X

University of Oxford — Ralf Hinze 51-86

slide-67
SLIDE 67

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

5.2 Bialgebra homomorphism

A swap-bialgebra homomorphism h is simultaneously an List-algebra and a List-coalgebra homomorphism. List X a

  • List h

List Y b

  • X

c

  • h

Y d

  • List X

List h List Y swap-bialgebras and homomorphisms form a category.

University of Oxford — Ralf Hinze 52-86

slide-68
SLIDE 68

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

5.2 Initial and final bialgebra

The initial object in this category is µList, in, bubble; the final object is νList, naiveInsert, out. List (µList) in

  • List (νList)

naiveInsert

  • µList

bubble

  • fold naiveInsert

unfold bubble νList

  • ut
  • List (µList)

List (νList) By uniqueness, naiveInsertionSort and bubbleSort are equal.

University of Oxford — Ralf Hinze 53-86

slide-69
SLIDE 69

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8

5.2 Intermediate summary

  • swap is a distributive law
  • µList, in, bubble is the initial swap-bialgebra
  • νList, naiveInsert, out is the final swap-bialgebra
  • bubble sort and na¨

ıve insertion sort are two (strongly related) variations of the same idea: repeatedly exchanging adjacent elements

University of Oxford — Ralf Hinze 54-86

slide-70
SLIDE 70

The Computational Essence of Sorting Algorithms — Bialgebras and distributive laws WG 2.8 University of Oxford — Ralf Hinze 55-86

slide-71
SLIDE 71

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

Section 6 Insertion and selection sort II

University of Oxford — Ralf Hinze 56-86

slide-72
SLIDE 72

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

  • sorting algorithms as folds of unfolds or unfolds of

folds necessarily have a running time of Θ(n2)

  • to define insertion and selection sort, we need variants
  • f folds and unfolds, so-called para- and apomorphisms

University of Oxford — Ralf Hinze 57-86

slide-73
SLIDE 73

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

6.1 Paramorphism

  • we start by defining products

data a × b = As {outl :: a, outr :: b} (△) :: (c → a) → (c → b) → (c → a × b) (f △ g) x = As (f x) (g x)

  • we write As a b as a

b (we use it like Haskell’s a@b).

University of Oxford — Ralf Hinze 58-86

slide-74
SLIDE 74

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

6.1 Paramorphism

  • we start by defining products

data a × b = As {outl :: a, outr :: b} (△) :: (c → a) → (c → b) → (c → a × b) (f △ g) x = As (f x) (g x)

  • we write As a b as a

b (we use it like Haskell’s a@b).

  • we are now ready to define paramorphisms:

para :: (Functor f ) ⇒ (f (µf × a) → a) → (µf → a) para f = f · map (id △ para f ) · in◦ a paramorphism also provides the intermediate input: the ‘algebra’ has type f (µf × a) → a instead of f a → a

  • slogan: eats its argument and keeps it too

University of Oxford — Ralf Hinze 58-86

slide-75
SLIDE 75

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

6.2 Apomorphism

  • products dualise to sums

data a + b = Stop a | Play b (▽) :: (a → c) → (b → c) → (a + b → c) (f ▽ g) (Stop a) = f a (f ▽ g) (Play b) = g b

  • we write Stop a as a ◾, and Play b as ▸ b

University of Oxford — Ralf Hinze 59-86

slide-76
SLIDE 76

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

6.2 Apomorphism

  • products dualise to sums

data a + b = Stop a | Play b (▽) :: (a → c) → (b → c) → (a + b → c) (f ▽ g) (Stop a) = f a (f ▽ g) (Play b) = g b

  • we write Stop a as a ◾, and Play b as ▸ b
  • paramorphisms dualise to apomorphisms:

apo :: (Functor f ) ⇒ (a → f (νf + a)) → (a → νf ) apo f = out◦ · map (id ▽ apo f ) · f the corecursion is split into two branches, with no recursive call on the left

  • apomorphisms improve the running time

University of Oxford — Ralf Hinze 59-86

slide-77
SLIDE 77

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

With apomorphisms, we can write the insertion function as

  • ne that stops scanning after inserting an element:

insertSort :: µList → νList insertSort = fold insert

University of Oxford — Ralf Hinze 60-86

slide-78
SLIDE 78

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

With apomorphisms, we can write the insertion function as

  • ne that stops scanning after inserting an element:

insertSort :: µList → νList insertSort = fold insert insert :: List (νList) → νList insert = apo ins

University of Oxford — Ralf Hinze 60-86

slide-79
SLIDE 79

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

With apomorphisms, we can write the insertion function as

  • ne that stops scanning after inserting an element:

insertSort :: µList → νList insertSort = fold insert insert :: List (νList) → νList insert = apo ins ins :: List (νList) → List (νList + List (νList)) ins Nil = Nil ins (Cons x ⌊Nil⌋) = Cons x (⌊Nil⌋ ◾) ins (Cons x ⌊Cons y xs⌋) | x y = Cons x (⌊Cons y xs⌋ ◾) | otherwise = Cons y (▸ (Cons x xs))

University of Oxford — Ralf Hinze 60-86

slide-80
SLIDE 80

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

From ins we can extract a natural transformation, which we call swop for swap‘n’stop: swop :: List (a × List a) → List (a + List a) swop Nil = Nil swop (Cons x (xs Nil)) = Cons x (xs ◾) swop (Cons x (xs (Cons y ys))) | x y = Cons x (xs ◾) | otherwise = Cons y (▸ (Cons x ys))

University of Oxford — Ralf Hinze 61-86

slide-81
SLIDE 81

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

From swop we get both insertion and selection sort: insertSort :: µList → νList insertSort = fold (apo (swop · map (id △ out))) selectSort :: µList → νList selectSort = unfold (para (map (id ▽ in) · swop)) In general, a natural transformation such as swop gives rise to two algorithms. Algorithms for free!

University of Oxford — Ralf Hinze 62-86

slide-82
SLIDE 82

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8

6.3 Intermediate summary

  • apomorphisms improve the running time
  • running time of insertion sort: worst case still Θ(n2),

but best case Θ(n)

  • (paramorphisms don’t improve the running time)
  • the computational essence of insertion and selection

sort is the natural transformation swop

  • in general, we shall seek natural transformation of type

F (A × G A) → G (A + F A)

  • (proof of equality involves (co-) pointed functors)

University of Oxford — Ralf Hinze 63-86

slide-83
SLIDE 83

The Computational Essence of Sorting Algorithms — Insertion and selection sort II WG 2.8 University of Oxford — Ralf Hinze 64-86

slide-84
SLIDE 84

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

Section 7 Quicksort and treesort

University of Oxford — Ralf Hinze 65-86

slide-85
SLIDE 85

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

  • so far: one-phase sorting algorithms

µList → νList

  • to improve performance we need to exchange

non-adjacent elements

  • next: two-phase sorting algorithms that make use of an

intermediate data structure µList → νTree → µTree → νList

  • the intermediate data structure can sometimes be

deforested (turning a data into a control structure)

  • we can play our game for each phase

University of Oxford — Ralf Hinze 66-86

slide-86
SLIDE 86

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

7.0 Search trees

  • an obvious intermediate data structure is a binary tree

data Tree tree = Empty | Node tree K tree instance Functor Tree where map f Empty = Empty map f (Node l k r) = Node (f l) k (f r)

  • we assume a ‘horizontal’ ordering

type SearchTree = Tree

University of Oxford — Ralf Hinze 67-86

slide-87
SLIDE 87

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

7.1 Phase one: growing a search tree

  • the essence of growing a search tree

sprout :: List (a × SearchTree a) → SearchTree (a + List a) sprout Nil = Empty sprout (Cons x (t Empty)) = Node (t ◾) x (t ◾) sprout (Cons x (t (Node l y r))) | x y = Node (▸ (Cons x l)) y (r ◾) | otherwise = Node (l ◾) y (▸ (Cons x r))

  • this is the only sensible definition: no choices
  • we compare elements across some distance

University of Oxford — Ralf Hinze 68-86

slide-88
SLIDE 88

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

  • we can either recursively partition a list, building

subtrees from the resulting sublists, or start with an empty tree and repeatedly insert the elements into it grow :: µList → νSearchTree grow = unfold (para (map (id ▽ in) · sprout)) grow′ :: µList → νSearchTree grow′ = fold (apo (sprout · map (id △ out)))

  • the algebra is a useful function on its own: insertion

into a search tree

  • efficient insertion into a tree is necessarily an

apomorphism

University of Oxford — Ralf Hinze 69-86

slide-89
SLIDE 89

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

7.2 Phase two: withering a search tree

  • the essence of withering a search tree

wither :: SearchTree (a × List a) → List (a + SearchTree a) wither Empty = Nil wither (Node (l Nil) x (r )) = Cons x (r ◾) wither (Node (l (Cons x l′)) y (r )) = Cons x (▸ (Node l′ y r))

  • again, this is the only sensible definition

University of Oxford — Ralf Hinze 70-86

slide-90
SLIDE 90

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

  • this should surprise no one: the second phase would

surely be an in-order traversal flatten :: µSearchTree → νList flatten = fold (apo (wither · map (id △ out))) flatten′ :: µSearchTree → νList flatten′ = unfold (para (map (id ▽ in) · wither))

  • the algebra is essentially a ternary version of append
  • the coalgebra deletes the leftmost element from a

search tree

University of Oxford — Ralf Hinze 71-86

slide-91
SLIDE 91

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

7.2 Putting things together

We obtain the famous quicksort and the less prominent treesort algorithms, quickSort :: µList → νList quickSort = flatten · downcast · grow treeSort :: µList → νList treeSort = flatten · downcast · grow′ where downcast :: (Functor f ) ⇒ νf → µf projects the final coalgebra onto the initial algebra.

University of Oxford — Ralf Hinze 72-86

slide-92
SLIDE 92

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8

7.2 Intermediate summary

  • once the intermediate data structure has been fixed,

everything falls into place: no choices

  • observation: only the first phase performs comparisons
  • quicksort and treesort are are two (strongly related)

variations of the same idea

  • running time: worst case still Θ(n2), but average case

Θ(n log n)

University of Oxford — Ralf Hinze 73-86

slide-93
SLIDE 93

The Computational Essence of Sorting Algorithms — Quicksort and treesort WG 2.8 University of Oxford — Ralf Hinze 74-86

slide-94
SLIDE 94

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

Section 8 Heapsort and minglesort

University of Oxford — Ralf Hinze 75-86

slide-95
SLIDE 95

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

8.0 Heaps

  • a search tree imposes a horizontal ordering
  • we can also assume a ‘vertical’ ordering

type Heap = Tree

University of Oxford — Ralf Hinze 76-86

slide-96
SLIDE 96

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

8.1 Phase one: piling up a heap

  • the essence of piling up a heap

pile :: List (a × Heap a) → Heap (a + List a) pile Nil = Empty pile (Cons x (t Empty)) = Node (t ◾) x (t ◾) pile (Cons x (t (Node l y r))) | x y = Node (▸ (Cons y r)) x (l ◾) | otherwise = Node (▸ (Cons x r)) y (l ◾)

  • now we have a choice (3rd equation)! Braun’s trick!

University of Oxford — Ralf Hinze 77-86

slide-97
SLIDE 97

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

8.1 Phase one: piling up a heap

  • the essence of piling up a heap

pile :: List (a × Heap a) → Heap (a + List a) pile Nil = Empty pile (Cons x (t Empty)) = Node (t ◾) x (t ◾) pile (Cons x (t (Node l y r))) | x y = Node (▸ (Cons y r)) x (l ◾) | otherwise = Node (▸ (Cons x r)) y (l ◾)

  • now we have a choice (3rd equation)! Braun’s trick!
  • let a = x ‘min‘ y and b = x ‘max‘ y,

= Node (▸ (Cons b l)) a (r ◾) = Node (r ◾) a (▸ (Cons b l)) = Node (l ◾) a (▸ (Cons b r)) = Node (▸ (Cons b r)) a (l ◾)

University of Oxford — Ralf Hinze 77-86

slide-98
SLIDE 98

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

  • as usual we obtain two algorithms

heapify :: µList → νHeap heapify = unfold (para (map (id ▽ in) · pile)) heapify′ :: µList → νHeap heapify′ = fold (apo (pile · map (id △ out)))

  • the algebra is a useful function on its own: insertion

into a heap

University of Oxford — Ralf Hinze 78-86

slide-99
SLIDE 99

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

8.2 Phase two: sifting through a heap

  • the essence of sifting through a heap

sift :: Heap (a × List a) → List (a + Heap a) sift Empty = Nil sift (Node (l Nil) x (r )) = Cons x (r ◾) sift (Node (l ) x (r Nil)) = Cons x (l ◾) sift (Node (l (Cons y l′)) x (r (Cons z r′))) | y z = Cons x (▸ (Node l′ y r)) | otherwise = Cons x (▸ (Node l z r′))

  • when constructing the heap node to continue with, we

have the option to swap left with right, but this buys us nothing

University of Oxford — Ralf Hinze 79-86

slide-100
SLIDE 100

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

  • again, we obtain two algorithms

unheapify :: µHeap → νList unheapify = fold (apo (sift · map (id △ out))) unheapify′ :: µHeap → νList unheapify′ = unfold (para (map (id ▽ in) · sift))

  • the coalgebra deletes the mimimum element from a

heap

University of Oxford — Ralf Hinze 80-86

slide-101
SLIDE 101

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

8.2 Putting things together

  • we obtain heapsort and a variant of heapsort that

behaves suspiciously like mergesort heapSort :: µList → νList heapSort = unheapify · downcast · heapify mingleSort :: µList → νList mingleSort = unheapify′ · downcast · heapify′

  • trimerous mergesort, called minglesort, builds a heap by

repeatedly dividing the input into three parts: two lists

  • f balanced length along with the minimum element
  • the merging phase is a similarly trimerous operation

University of Oxford — Ralf Hinze 81-86

slide-102
SLIDE 102

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8

8.2 Intermediate summary

  • the intermediate data structure provides us with a

choice (heaps are more flexible than search trees)

  • observation: both phases use comparisons
  • running time: worst case Θ(n log n)—insensitive to the

input

  • minglesort is a recent variant of heapsort (closely

related to Heap-Mergesort, see Computer and Mathematics with Applications 39, 2000)

  • an analogous approach using binary leaf trees works for

true mergesort

University of Oxford — Ralf Hinze 82-86

slide-103
SLIDE 103

The Computational Essence of Sorting Algorithms — Heapsort and minglesort WG 2.8 University of Oxford — Ralf Hinze 83-86

slide-104
SLIDE 104

The Computational Essence of Sorting Algorithms — Epilogue WG 2.8

Section 9 Epilogue

University of Oxford — Ralf Hinze 84-86

slide-105
SLIDE 105

The Computational Essence of Sorting Algorithms — Epilogue WG 2.8

9.1 Summary

  • categorical algorithmics
  • type-driven algorithm design building on a few

principled recursion operators

  • few design decisions: intermediate data structures
  • no rabbits!
  • categorical duality gives us algorithms for free
  • see WGP ’12 paper: Sorting with Bialgebras and

Distributive Laws by Ralf Hinze, Daniel W. H. James, Thomas Harper, Nicolas Wu and Jos´ e Pedro Magalh˜ aes

University of Oxford — Ralf Hinze 85-86

slide-106
SLIDE 106

The Computational Essence of Sorting Algorithms — Epilogue WG 2.8

9.2 Summary and future work

  • computational essence
  • na¨

ıve insertion and bubble sort: swap

  • insertion and selection sort: swop
  • growing a search tree: sprout
  • withering a search tree: wither
  • piling up a heap: pile
  • sifting through a heap: sift
  • top-down algorithms: regular datatypes
  • bottom-up algorithms: nested datatypes

University of Oxford — Ralf Hinze 86-86