A Graph-Based Definition of Distillation Geoff Hamilton Gavin - - PowerPoint PPT Presentation

A Graph-Based Definition of Distillation

Geoff Hamilton Gavin Mendel-Gleason {hamilton,ggleason}@computing.dcu.ie

Lero@DCU School of Computing Dublin City University Dublin, Ireland

META 2010

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Background

This work is inspired by the supercompilation transformation algorithm developed by Turchin. Although originally developed in the 1960s/70s, supercompilation did not become more widely known

• utside Russia until much later:

Published only in less accessible journals. Defined on the unconventional language Refal.

Supercompilation became more widely known through the positive supercompilation algorithm (Sørensen, Glück and Jones):

Simplified algorithm. Defined on a more familiar functional language.

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Motivation

Only linear improvements in performance are possible using the positive supercompilation algorithm. Transformations such as the following are therefore not possible:

nrev xs where nrev = λxs.case xs of [] ⇒ [] | x′ : xs′ ⇒ app (nrev xs′) (x′ : []) app = λxs.λys.case xs of [] ⇒ ys | x′ : xs′ ⇒ x′ : (app xs′ ys) ⇓ arev xs where arev = λxs.arev′ xs [] arev′ = λxs.λys.case xs of [] ⇒ ys | x′ : xs′ ⇒ arev′ xs′ (x′ : ys)

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Language Syntax

We use the following higher-order functional language: prog ::= e0 where f1 = e1 . . . fk = ek Program e ::= v Variable | c e1 . . . ek Constructor | f Function Call | λv.e λ-Abstraction | e0 e1 Application | case e0 of p1 ⇒ e1 | · · · | pk ⇒ ek Case Expression p ::= c v1 . . . vk Pattern The operational semantics of this language is normal-order reduction.

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation

Central concept is that of driving, which constructs a potentially infinite process tree, representing all possible computations of the program by normal order reduction. Generalization is required to ensure the termination of driving. Folding can be performed on encountering an instance of a previously encountered term, thus producing a finite partial process tree. A (hopefully) more efficient recursive program can be extracted from the resulting partial process tree.

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Process Trees

e → t1, . . . , tn is the process tree with root labelled e and n children which are the subtrees t1, . . . , tn respectively. root(t) denotes the root node of process tree t. t(α) denotes the label of node α within process tree t. anc(t, α) denotes the set of ancestors of node α within t. t{α := t′} denotes the tree obtained by replacing the subtree with root α in t by the tree t′.

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Expression Instance

An expression e′ is an instance of expression e, denoted by e ⋖e e′, if there is a substitution θ such that e θ ≡ e′. When an instance is encountered, a repeat node is constructed. e α denotes a repeat node where the expression e is an instance of the label of node α.

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Expression Embedding

The homeomorphic embedding relation e is a well-quasi order which provides a so-called whistle to stop driving due to potential divergence, and to indicate that generalization should be performed.

e1 ⊳e e2 e1 e e2 e1 ⊲ ⊳e e2 e1 e e2 e e (e′[v/v′]) λv.e ⊲ ⊳e λv′.e′ ∃i ∈ {1 . . . n}.e e ei e ⊳e φ(e1, . . . , en) ∀i ∈ {1 . . . n}.ei e e′

i

φ(e1, . . . , en) ⊲ ⊳e φ(e′

1, . . . , e′ n)

e0 e e′ ∀i ∈ {1 . . . n}.∃θi.pi ≡ (p′

i θi) ∧ ei e (e′ i θi)

(case e0 of p1 : e1| . . . |pn : en) ⊲ ⊳e (case e′

0 of p′ 1 : e′ 1| . . . |p′ n : e′ n)

e1 e e2 iff ∃θ.e1 θ ⊲ ⊳e e2

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Expression Generalization

A generalization of expressions e and e′ is a triple (eg, θ, θ′) where eg θ ≡ e and eg θ′ ≡ e′. e ⊓e e′ =            (φ(eg

1, . . . , eg n), n i=1 θi, n i=1 θ′ i),

if e e e′ where e = φ(e1, . . . , en) e′ = φ(e′

1, . . . , e′ n)

(eg

i , θi, θ′ i) = ei ⊓e e′ i

(v, [e/v], [e′/v]),

• therwise

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Expression Generalization

When we encounter an expression e′ which is a coupling

• f a previously encountered expression e, we perform

generalization. To represent the result of generalization, we introduce a let construct of the form let v1 = e1, . . . , vn = en in e0 into

• ur language.

The expression e is replaced by its generalized form, which is constructed using the abstracte operation: abstracte(e, e′) = let v1 = e1, . . . , vn = en in eg where e ⊓e e′ = (eg, {v1 := e1, . . . , vn := en}, θ)

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Algorithm

If the partially constructed process tree t contains a node β in which the redex is a function, the node is processed as follows: if ∃α ∈ anc(t, β).t(α) ⋖e t(β) then t{β := t(β) α} else if ∃α ∈ anc(t, β).t(α) e t(β) then t{α := S[ [abstracte(t(α), t(β))] ]} else t{β := t(β) → S[ [unfold(t(β))] ]}

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ case (nrev xs′) of . . . Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ case (nrev xs′) of . . . case (case xs′ of . . .) of . . . Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ case (nrev xs′) of . . . case (case xs′ of . . .) of . . . x′ : [] xs′ = [] Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ case (nrev xs′) of . . . case (case xs′ of . . .) of . . . x′ : [] xs′ = [] case (app (nrev xs′′) (x′′ : [])) of . . . xs′ = x′′ : xs′′ Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ case (nrev xs′) of . . . case (case xs′ of . . .) of . . . x′ : [] xs′ = [] case (app (nrev xs′′) (x′′ : [])) of . . . xs′ = x′′ : xs′′ case (case (nrev xs′′) of . . .) of . . . Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ let vs = nrev xs′ in case vs of . . . Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ let vs = nrev xs′ in case vs of . . . nrev xs′ Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ let vs = nrev xs′ in case vs of . . . nrev xs′ case vs of . . . Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ let vs = nrev xs′ in case vs of . . . nrev xs′ case vs of . . . x′ : [] vs = [] Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ let vs = nrev xs′ in case vs of . . . nrev xs′ case vs of . . . x′ : [] vs = [] v′ : (app vs′ (x′ : [])) vs = v′ : vs′ Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ let vs = nrev xs′ in case vs of . . . nrev xs′ case vs of . . . x′ : [] vs = [] v′ : (app vs′ (x′ : [])) vs = v′ : vs′ v′ Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ let vs = nrev xs′ in case vs of . . . nrev xs′ case vs of . . . x′ : [] vs = [] v′ : (app vs′ (x′ : [])) vs = v′ : vs′ v′ app vs′ (x′ : []) Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

nrev xs case xs of . . . [] xs = [] app (nrev xs′) (x′ : []) xs = x′ : xs′ let vs = nrev xs′ in case vs of . . . nrev xs′ case vs of . . . x′ : [] vs = [] v′ : (app vs′ (x′ : [])) vs = v′ : vs′ v′ app vs′ (x′ : []) Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Positive Supercompilation: Example

The following program is constructed from this process graph: f xs where f = λxs.case xs of [] ⇒ [] | x′ : xs′ ⇒ case (f xs′) of [] ⇒ [x′] | x′′ : xs′′ ⇒ x′′ : (f ′ xs′′ x′) f ′ = λxs.λy.case xs of [] ⇒ [y] | x′ : xs′ ⇒ x′ : (f ′ xs′ y)

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Distillation

Central concept is also driving to construct a potentially infinite process tree representing all possible computations

• f the program by normal order reduction.

The terms in the nodes of this process tree are transformed by positive supercompilation to obtain their corresponding transition system representation. Generalization and folding are performed with respect to these transition systems. As a result, it is possible to obtain a super-linear improvement in performance as computationally expensive terms can be extracted from within loops in these transition systems.

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Transition System Instance

Two transition systems are equivalent if the following relation is satisfied: cone → t1, . . . , tn ≡ con′e′ → t′

1, . . . , t′ n,

iff e e e′ ∧ ∀i ∈ {1 . . . n}.ti ≡ t′

i

e t ≡ e′ t′, iff t ≡ t′ A transition system t′ is an instance of another transition system t (denoted by t ⋖t t′) iff there is a substitution θ s.t. t ≡ t′ θ.

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Transition System Embedding

The embedding relation t on transition systems is defined as follows:

t1 ⊳t t2 t1 t t2 t1 ⊲ ⊳t t2 t1 t t2 t t (t′[v/v′]) λv.e → t ⊲ ⊳t λv′.e′ → t′ e ⊲ ⊳e e′ ∀i ∈ {1 . . . n}.ti t t′

i

cone → t1, . . . , tn ⊲ ⊳t con′e′ → t′

1, . . . , t′ n

∃i ∈ {1 . . . n}.t t ti t ⊳t e → t1, . . . , tn t ⊲ ⊳t t′ e t ⊲ ⊳t e′ t′ t0 t t′ ∀i ∈ {1 . . . n}.∃θi.pi ≡ (p′

i θi) ∧ ti t (t′ i θi)

(case e0 of p1 : e1| . . . |pn : en) → t0, . . . , tn ⊲ ⊳t (case e′

0 of p′ 1 : e′ 1| . . . |p′ n : e′ n) → t′ 0, . . . , t′ n

t1 t t2 iff ∃θ.t1 θ ⊲ ⊳t t2

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Transition System Generalization

A generalization of transition systems t and t′ is a triple (tg, θ, θ′) where tg θ ≡ t and tg θ′ ≡ t′. t ⊓t t′ =                    (e → tg

1 , . . . , tg n , n i=1 θi, n i=1 θ′ i),

if t t t′ where t = e → t1, . . . , tn t′ = e′ → t′

1, . . . , t′ n

(tg

i , θi, θ′ i) = ti ⊓t t′ i

(v v1 . . . vk, {v := λv1 . . . vk.t}, {v := λv1 . . . vk.t′}),

• therwise

where {v1 . . . vk} = bv(t) ∪ bv(t′)

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Transition System Generalization

When we encounter a transition system t′ which is a coupling of a previously encountered transition system t, we perform generalization. To represent the result of generalization, we introduce a generalized node of the form let v1 = t1, . . . , vn = tn in t0 into our transition systems. The transition system t is replaced by its generalized form, which is constructed using the abstractt operation: abstractt(t, t′) = let v1 = t1, . . . , vn = tn in tg where t ⊓t t′ = (tg, {v1 := t1, . . . , vn := tn}, θ)

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Distillation: Algorithm

If the partially constructed process tree t contains a node β in which the redex is a function, the node is processed as follows: if ∃α ∈ anc(t, β).S[ [t(α)] ] ⋖t S[ [t(β)] ] then t{β := t(β) α} else if ∃α ∈ anc(t, β).S[ [t(α)] ] t S[ [t(β)] ] then t{α := D[ [abstractt(S[ [t(α)] ], S[ [t(β)] ])] ]} else t{β := t(β) → D[ [unfold(t(β))] ]}

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Distillation Example: nrev xs

† case (nrev xs′) of . . . case (case xs′ of . . .) of . . . x′ : [] xs′ = [] case (app (nrev xs′′) (x′′ : [])) of . . . xs′ = x′′ : xs′′ ‡ case (case (nrev xs′′) of . . .) of . . . case (case (case xs′′ of . . .) of . . .) of . . . x′′ : x′ : [] xs′′ = [] case (case (app (nrev xs′′′) (x′′′ : [])) of . . .) of . . . xs′′ = x′′′ : xs′′′ ∗ case (case (case (nrev xs′′) of . . .) of . . .) of . . . Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Distillation Example: nrev xs

The following transition system is constructed from the term †:

let vs = nrev xs′ in case vs of . . . nrev xs′ case vs of . . . x′ : [] vs = [] v′ : app vs′ (x′ : []) vs = v′ : vs′ v′ app vs′ (x′ : []) case vs′ of . . . x′ : [] vs′ = [] v′′ : app vs′′ (x′ : []) vs′ = v′′ : vs′′ v′′ app vs′′ (x′ : []) Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Distillation Example: nrev xs

The following transition system is constructed from the term ‡:

let vs = nrev xs′′ in case (case vs of . . .) of . . . nrev xs′′ case (case vs of . . .) of . . . x′′ : x′ : [] vs = [] v′ : case (app vs′ (x′′ : [])) of . . . vs = v′ : vs′ v′ case (app vs′ (x′′ : [])) of . . . case (case vs′ of . . .) of . . . x′′ : x′ : [] vs′ = [] v′′ : case (app vs′′ (x′′ : [])) of . . . vs′ = v′′ : vs′′ v′′ case (app vs′′ (x′′ : [])) of . . . Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Distillation Example: nrev xs

The following transition system is constructed from the term ∗:

let vs = nrev xs′′′ in case (case (case vs of . . .) of . . .) of . . . nrev xs′′′ case (case (case vs of . . .) of . . .) of . . . x′′′ : x′′ : x′ : [] vs = [] v′ : case (case (app vs′ (x′′′ : [])) of . . .) of . . . vs = v′ : vs′ v′ case (case (app vs′ (x′′′ : [])) of . . .) of . . . case (case vs′ of . . .) of . . . x′′′ : x′′ : x′ : [] vs′ = [] v′′ : case (case (app vs′′ (x′′′ : [])) of . . .) of . . . vs′ = v′′ : vs′′ v′′ case (case (app vs′′ (x′′′ : [])) of . . .) of . . . Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Distillation Example: nrev xs

Transition system † is generalized with respect to transition system ‡ to give:

let vs = nrev xs′ in case vs of . . . nrev xs′ case vs of . . . x′ : v vs = [] v′ : app vs′ (x′ : []) vs = v′ : vs′ v′ app vs′ (x′ : []) case vs′ of . . . x′ : v vs′ = [] v′′ : app vs′′ (x′ : []) vs′ = v′′ : vs′′ v′′ app vs′′ (x′ : []) Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Distillation Example: nrev xs

Transition system ‡ is generalized with respect to transition system ∗ to give:

let vs = nrev xs′′ in case (case vs of . . .) of . . . nrev xs′′ case (case vs of . . .) of . . . x′′ : x′ : v vs = [] v′ : case (app vs′ (x′′ : [])) of . . . vs = v′ : vs′ v′ case (app vs′ (x′′ : [])) of . . . case (case vs′ of . . .) of . . . x′′ : x′ : v vs′ = [] v′′ : case (app vs′′ (x′′ : [])) of . . . vs′ = v′′ : vs′′ v′′ case (app vs′′ (x′′ : [])) of . . . Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Distillation Example: nrev xs

The second of these transition systems is an instance of the first, so folding is performed to obtain the following program: case xs of [] ⇒ [] | x′ : xs′ ⇒ f x′ xs′ [] where f = λx′.λxs′.λv.case xs′ of [] ⇒ x′ : v | x′′ : xs′′ ⇒ f x′′ xs′′ (x′ : v) This program has a run-time which is linear with respect to the length of the input list, while the original program is quadratic.

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Distillation Example: app (arev′ xs ys) zs

The following program is obtained after applying distillation: f xs (g ys zs) where f = λxs.λv.case xs of [] ⇒ v | x′ : xs′ ⇒ f xs′ (x′ : v) g = λys.λzs.case ys of [] ⇒ zs | y′ : ys ⇒ y′ : (g ys′ zs) The intermediate list (arev′ xs ys) within the initial program has therefore been eliminated. This intermediate list is not removed using positive supercompilation.

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Distillation Example: fib n

The fib function: fib n where fib = λn.case n of Zero ⇒ Succ Zero | Succ n′ ⇒ case n′ of Zero ⇒ Succ Zero | Succ n′′ ⇒ add (fib n′) (fib n′′) add = λx.λy.case x of Zero ⇒ y | Succ x′ ⇒ Succ (add x′ y) is transformed to something similar to the following by distillation: f n (λn.Succ n) (λn.Succ n) where f = λn.λg.λh.case n of Zero ⇒ g Zero | Succ n′ ⇒ f n′ h (λn.h (g n))

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Conclusions

Distillation and positive supercompilation are very similar algorithms. Distillation extends the range of transformations which can be performed beyond those which can be performed by positive supercompilation. Distillation can produce a superlinear speedup in programs, which is not possible using positive supercompilation. The extra power of distillation is obtained by extracting computationally expensive terms from within loops in transition systems. The extra power of distillation comes at a price; there is an exponential increase in the number of steps required in the worst case.

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation

Further Work

Use of the distillation algorithm to facilitate the proof of temporal formulae in the µ-calculus. Proofs of termination and correctness of the distillation algorithm. More detailed comparison between distillation and higher-level supercompilation. Incorporation of the distillation algorithm into the Haskell programming language. Use of the distillation algorithm to facilitate the parallelization of code.

Geoff Hamilton, Gavin Mendel-Gleason A Graph-Based Definition of Distillation