Program Extraction in Churchs Simple Theory of Types with - - PowerPoint PPT Presentation

Program Extraction in Church’s Simple Theory of Types with Applications to Computable Analysis

Ulrich Berger

Swansea University

Computation and Correctness in Analysis (CCA) Nancy, 9 July 2013

1 / 36

A logical approach to computable analysis

In this talk I present a logical approach to computable analysis based on program extraction from proofs. I’ll explain the basic principle, give some examples and highlight aspects that might make this approach interesting to people working in computable analysis. The main attraction of the approach is, in my opinion, the fact that one can work directly with abstract mathematical objects without having to construct particular representations. The work reported in this talk is joint with Monika Seisenberger and Tie Hou from Swansea and Helmut Schichtenberg and Kenji Miyamoto in Munich. Parts of it is implemented in the the proof system Minlog.

2 / 36

The COMPUTAL project http://computal.uni-trier.de/

Participating Universities: Cambridge, Cape Town, Darmstadt, Hagen, Kanazawa, Ljubljana, Munich, Novosibirsk, Pretoria, Siegen, Swansea, Trier.

2nd COMPUTAL workshop, Gregynog, 26-30 June 2013

Topics: computable analysis, domain theory, topology, exact real number computation, program extraction.

3 / 36

Overview

◮ Introduction: from constructive ideas to program extraction ◮ What is exact real number computation? ◮ Formalizing real numbers and continuous functions ◮ Extracting exact real number arithmetic ◮ Efficient continuity ◮ Church’s simple theory of types ◮ Realizability interpretation of CST ◮ Conclusion

4 / 36

From constructive ideas to program extraction

Origins

◮ Kronecker, Brouwer, Heyting, Kolmogorov, Bishop:

Mathematics as a theory of mental constructions (Intuitionism, Constructivism)

◮ G¨

• del, Kleene, Kreisel: Functional- and Realizability

interpretations

◮ Curry-Howard correspondence: Formulas-as-types,

Proofs-as-programs Trends

◮ Proof Mining based on Functional Interpretation ◮ Constructive Type Theory ◮ Computational interpretations of classical proofs ◮ Program specification based on realizability ◮ Program extraction based on realizability

5 / 36

What is program extraction?

(1) Instead of defining what it means for a formula A to be true

• ne defines what it means for a program to realize A.

Intuitively, a realizer is a solution of the computational problem expressed by A. (2) The Soundness Theorem states that from a constructive proof of A one can extract (automatically) a realizer of A (and a proof that it is a realizer). (3) The Adequacy Theorem states that programs that denote concrete data (e.g. natural numbers) evaluate to canonical representations of the data (e.g. 3 + 4 evaluates to 7). I.o.w. programms are not just formal expressions, but they compute as expected. Domain Theory is essential for the proof! For a category-theoretic formulation of (1), which leads to a general theory of representations and specification of programs, see e.g. Andrej Bauer’s PhD thesis “The Realizability Approach to Computable Analysis and Topology” and “Implementing real numbers with RZ” (CCA 2007).

6 / 36

Program Extraction from a Computer Science Perspective

Traditional approach to verified software: Problem ⇒ Program ⇒ Proof Program extraction: Problem ⇒ Proof ⇒ Program Advantages:

◮ the step “Proof

⇒ Program” is automatic,

◮ correctness of extracted program is proved automatically, ◮ data structures are generated automatically, ◮ new data strures and algorithms may be discovered, ◮ problem and proof can be writen in usual mathematics

(no “constructivisation” necessary),

◮ target programming language can be low level (and fast) since

not used for manual programming.

◮ Moreover, all sub-programs are specified and proven correct,

which supports safe modification of programs.

7 / 36

State of the art

Program extraction (PE) is implemented in Nuprl PX Minlog Coq Isabel Agda . . . Minlog (Schwichtenberg, Munich):

◮ active research in PE ◮ PE from concrete and and abstract mathematics ◮ PE from constructive and classical proofs ◮ PE based on realizability or functional (Dialectica)

interpretation

◮ case studies in constructive analysis, infinitary combinatorics,

lambda calculus, parsing, sat solving, . . . .

8 / 36

What is exact real number computation?

A Double precision computation: f(x) = 1+x-(x^2)*(x+1)*((1/x)-(1/(x+1))) *Main> f (10^9)

• 149.21128177642822

The problem is not that the result is wrong (it should be 1), but that floating point arithmetic doesn’t warn us. Exact real number computation

◮ provides exact error bounds, ◮ can make errors arbitrarily small, ◮ comes with a rigorous proof of these facts, ◮ should be fast (iRRAM, M¨

uller, Trier)

9 / 36

Formalizing real numbers

We assume that the structure R of real numbers with 0, 1, +, −, ∗, /, =, <, sin, sg, . . . is given axiomatically (no implementation or computational model provided). Any true disjunction-free first-order formulas are allowed as axioms. Since in classical logic disjunction can be expressed by other logical connectives, all classically true statements can be axioms. In addition true higher-order formulas satisfying certain syntactic criteria (details later) are allowed. For example, completeness: X nonempty and bounded → X has l.u.b

10 / 36

Discontinuous and partial fuctions

Discontinuous functions are allowed. E.g. the sign function with the axioms x < 0 → sg(x) = −1 x = 0 → sg(x) = 0 x > 0 → sg(x) = 1 The partial function 1/x can be thought of as being totalized, however, without stating anything about 1/0: x = 0 → x ∗ 1/x = 1

11 / 36

Natural numbers, integers, rational numbers

. . . are defined as subsets of R: x ∈ N

µ

≡ x = 0 ∨ x − 1 ∈ N x ∈ Z ≡ x ∈ N ∨ −x ∈ N x ∈ Q ≡ ∃n ∈ Z, m ∈ N \ {0} . x = n/m where “µ” means that N is inductively defined, i.e. it is the least set satisfying the equation. The realizers of “t ∈ N” will be unary numerals. To obtain binary one defines x ∈ N2

µ

≡ x ∈ {0, 1} ∨ ∃y > 0 (y ∈ N2 ∧ ∃d ∈ {0, 1} (x = 2y + d)) Here is our first theorem we extract a program from: Theorem ∀x (N(x) ⇔ N2(x)). The extracted program translates between unary and binary notation.

12 / 36

Realizability by example: natural numbers

In order to obtain the definition of a r N(x) (“a realizes the fact that x is a natural number”) we first compute the type of potential realizers as follows:

◮ replace “t ∈ N” by a name for the data type, say Nat. ◮ replace other atomic formulas by the unit or ‘void’ type 1, ◮ delete all quantifiers and object terms, ◮ replace ∨ by + (disjoint sum) and ∧ by × (cartesian product), ◮ carry out obvious simplifications (e.g. replace α × 1 by α).

Hence, the definition x ∈ N

µ

≡ x = 0 ∨ x − 1 ∈ N yields the free algebra of unary numbers: Nat

µ

≡ 1 + Nat. The definition of a r N(x), where a ∈ Nat, is a copy of the definition of N (we call the constructors of Nat Z and S): a r N(x)

µ

≡ (a = Z ∧ x = 0) ∨ (a = S(b) ∧ b r N(x − 1)) Hence a r N(x) iff n is a unary representation of x.

13 / 36

Realizability of implication and “for all”

A realizer of an implication A → B is a function mapping realizers

• f A to realizers of B (as in the theory of representations).

a r ∀x A(x) means ∀x (a r A(x)) (where a does not depend on x). Putting things together, a realizer of the formula ∀x . N(x) → N2(x) is a function f such that whenenver a realizes N(x) (i.e. a is a unary representation of x), then f (a) realizes N2(x) (i.e. f (a) is a binary representation of x). The function f is extracted from the proof of the formula.

14 / 36

Approximating real numbers

Two (equivalent) ways of saying that a real number x ∈ I := [−1, 1] can be approximated: A(x) ⇔ |x| < 1 ∧ ∀n ∈ N ∃q ∈ Q x ∈ Bn(q) C0(x)

ν

⇔ |x| < 1 ∧ ∃d ∈ {−1, 0, 1} C0(2 ∗ x − d) where x ∈ Bn(q) ≡ |x − q| < 2−n. A realizer of A(x) is a fast rational Cauchy sequence converging to x. A realizer of C0(x) is an infinite stream of signed digits, d0 : d1 : . . . representing x, i.e. x =

∞

• di ∗ 2i+1

Theorem A(x) ↔ C0(x). The extracted program translates between Cauchy and signed digit representation.

15 / 36

Extracting exact real number arithmetic

Theorem If x, y ∈ C0 then x+y

2

∈ C0. Theorem If x, y ∈ C0 then xy ∈ C0. From these theorems one extracts implementations of addition and multiplication w.r.t. the signed digit representation. Similar implementations were studied by Edalat, Potts, Heckmann, Escardo, Ciaffaglione, Gianantonio, e.t.c. The difference is that we extract the programs

16 / 36

Approximating continuous functions

A function f : I → I is continuous iff ∀l ∈ N ∃k ∈ N ∀p ∈ Q ∃q ∈ Q f [Bk(p)] ⊆ Bl(q) Using this as a definition, a realizer of the statement “f is continuous” consists of α: N → N l → k (modulus) g : N × Q → Q (l, p) → q (approximating function) such that ∀l ∈ N ∀p ∈ Q f [Bα(l)(p)] ⊆ Bl(g(l, p)) This corresponds to the usual notion of representations of continuous functions.

17 / 36

Approximating continuous functions coinductively

Alternatively, one can define continuous functions by a nested inductive/coinductive definition. In order to carry this out conveniently, we introduce explicit

• perators for least and greatest fixed points of monotone operators

Φ : P(X) → P(X): µ Φ := least fixed point of Φ ν Φ := largest fixed point of Φ Hence, for example, N = µ (λX . {x | x = 0∨x−1 ∈ X}) =: µ X . {x | x = 0∨x−1 ∈ X} We define C1 ⊆ II by a nested inductive/coinductive definition as follows (F, G range over subsets of II): C1 = ν F . µ G . {f : II | (∃e ∈ SD vae ◦ f ∈ F) ∨ (∀d ∈ SD f ◦ avd ∈ G)}

18 / 36

Memo tries for continuous functions

Theorem h is continuous iff h ∈ C1. From the proof of this theorem one extracts programs translating between realisers of “f is continuous” (where continuity has to be defined in a constructively meaningful way) and realisers of “f ∈ C1”. What is a realiser of “f ∈ C1”? It is a finitely branching non-wellfounded tree describing when f emits and absorbs digits. I.p. it is a data structure, not a function. Similar trees have been studied by P. Hancock, D. Pattinson, N. Ghani.

• P. Hancock, D. Pattinson, N. Ghani. Representations of Stream

Processors Using Nested Fixed Points, LMCS 5, 2009.

19 / 36

Tree of the logistic map, fa(x) = a(1 − x2) − 1, with a = 2/3

N N P Z N Z Z Z P N Z Z Z P N Z Z Z P Z Z N Z P Z Z N Z Z Z N N P

20 / 36

Extracting memoized exact real arithmetic

The definition of C1 ⊆ II can be generalised to Cn ⊆ I(In). Theorem The average function lies in C2. Theorem Multiplication lies in C2. Theorem If f ∈ Cn and g1, . . . gn ∈ Cm, then f ◦ (g1, . . . , gn) ∈ Cm. From these Theorems one extracts implementations of addition and multiplication as memo-tries, and of composition as a function

• n memo-tries.

Experiments show considerable speed-up when sampling “hard” functions (e.g. high iterations of the logistic map) on a very fine grid.

21 / 36

Integration

Let

• f denote the definite integral

1

−1 f (x)dx.

We assume the following “axioms” about

• f :

(a)

• f = 1

2

• (vad ◦ f ) + d

where vad(x) := 2x − d. (b)

• f = 1

2(

• (f ◦ av−1) +
• (f ◦ av1)).

Theorem If f ∈ C1, then

• f ∈ A, i.e.

∀n ∈ N ∃q ∈ Q |

• f − q| ≤ 2−n.

The proof is very short and uses only the equations (a), (b) above. The extracted program is reasonably efficient.

22 / 36

Towards “efficient” continuity

Both notions of continuity considered so far have disadvantages regarding realizability: The first one is “backward” in the sense that it computes a modulus of continuity (which can be expensive and is often not needed). The coinductive notion C1(f ) memoizes which is bad in most cases. We introduce a weaker notion of continuity with a nonconstructive notion of modulus.

23 / 36

m-continuity

A relation m ⊆ N × N is called a modulus if ∀l ∈ N ¬¬∃k ∈ N m[

∨

k] ⊆

∨

l where

∨

k:= {k′ ∈ N | k′ ≥ k} and m[K] := {l | ∃k ∈ K m(k, l)}. A function f : X → X is m-continuous if ∀k ∈ N ∀p ∈ Q ∃l ∈ N (m(k, l) ∧ ∃q ∈ Q f [Bk(p)] ⊆ Bl(q)) A realizer of “f is m-continuous” is a function g : N × Q → N × Q such that ∀k ∈ N ∀p ∈ Q m(k, l) ∧ f [Bk(p)] ⊆ Bl(q) where (l, q) := g(k, p) Hence the realizer is entirely “forward”.

24 / 36

Composition

Theorem If f is m-continuous and f ′ is m′-continuous, then f ◦ f ′ is m′ ◦ m-continuous. Realizer extracted from the proof: Comp(g, g′) = g ◦ g′. Note that no moduli occur in the realizer. It is not difficult to see that a function is continuous iff it is m-continuous for some modulus m.

25 / 36

Application

Theorem (EMP) If f is m-continuous and x is approximable, then f (x) is approximable. Proof needs a slight extension of Markov’s principle (EMP). Realizer extracted from the proof of the theorem: App(g, ϕ) = λl.q where (l′, q) = g(k, ϕ(k)) with k minimal such that l′ ≥ l The modulus guarantees that l′ ≥ l, eventually.

26 / 36

Extended Markov’s principle (EMP)

Markov’s Principle (MP) ∀x (N(x) → D(x) ∨ ¬D(x)) ¬¬∃x (N(x) ∧ D(x)) ∃x (N(x) ∧ D(x)) Extended Markov’s Principle (EMP) ∀x, y (A(x, y) → D(y) ∨ ¬D(y)) ∀x (N(x) → ∃y A(x, y)) ¬¬∃x (N(x) ∧ ∀y (A(x, y) → D(y))) ∃x, y (N(x) ∧ A(x, y) ∧ D(y))

27 / 36

Realizability of EMP

EMP can be realized using the least number operator. EMP follows from MP and countable choice. Is countable choice necessary?

28 / 36

Church’s theory of simple types (CST)

Alonzo Church: A Formulation of the Simple Theory of Types. The Journal of Symbolic Logic, Vol. 5, No. 2. 1940). CST is a formal system for higher order-logic presented as a simply typed lambda calculus. Types a set I of base types for individuals; the base type o (type of propositions, or truth values); ρ × σ, ρ → σ. Constants →, ∧, ∨ :

• → o → o

∀ρ, ∃ρ : (ρ → o) → o (ρ arbitrary) Terms TER ∋ M, N ::= x ∈ VAR | c ∈ C| | λx : ρ.M | MN | M, N | π0(M) | π1(M)

29 / 36

Semantics of CST

CST admits a straightforward classical semantics:

• = {0, 1}

× = cartesian product → = full function space The propositional connective are interpreted as usual: ∀ρ(p) = min{p(x) | x ∈ ρ} ∃ρ(p) = max{p(x) | x ∈ ρ}

30 / 36

Proofs in CST

A typing context is a sequence Γ = x1 : ρ1, . . . , xn : ρn. A Γ-formula is a term that has type o in the typing context Γ. One can define an intuitionistic proof calculus for sequents of the form ∆ ⊢Γ A where ∆ is a finite set of Γ-formulas and A is a Γ-formula.

31 / 36

Negation, equality, least and greatest fixed points

Truth, falsity and negation can be defined as ⊤ := ∃x : o .x ⊥ := ∀x : o .x ¬A := A → ⊥ Equality can be defined as x =ρ y := ∀p : ρ → o . p x → p y One can also define least and greatest fixed point operators µρ, νρ : (ρ → ρ) → ρ for “predicate types” ρ (e.g. ρ = ι → o). The expected rules can be derived.

32 / 36

Realizability for CST (brief sketch)

Let δ be a new type of “programs” or “potential realizers” (the semantics of δ is a suitable Scott domain). For formulas A : o realizability is clear: the realizability interptetation of A must define a set of realizers, i.e. a term Rr(A)

• f type δ → o.

But what is the realizability interpretation of a term of higher type, say Φ : (ι → o) → ι → o? The crucial insight is that the type of the realizability interpretation of Φ is obtained by simply replacing in the type of Φ each occurrence of o by ι → o. Hence for the example above, Rr(Φ) : (ι → δ → o) → ι → δ → o The rest of the definition of realizability is rather straight forward. What is not straight forward is program extraction and the proof of the Soundness Theorem.

33 / 36

Remarks

◮ The realizability interpretation of CST is a vast generalization

• f existing interpretations, e.g. by M Tatsuta and

implementations in Minlog and Coq. I.p. only strictly positive induction and coinduction had been considered so far.

◮ The soundness proof for monotone induction and coinduction

seems to need induction and coinduction for non-mononotone

• perators.

◮ A prototype implementation is under development (joint work

with T Hou).

34 / 36

Conclusion

Program extraction via realizability . . .

◮ defines representations implicitely by predicates (e.g. A ⊆ R

instead of ρ ⊆: NN → R);

◮ analyses the computational content of a theorem, but not of

its statement (as in Weihrauch degrees), but of its proof;

◮ considers computations rather than computability; ◮ aims at correctness rather than speed. ◮ provides a useful guideline for designing data, representations

and algorithms, even in the absence of a proof assistant.

35 / 36

Some References

◮ B. and Monika Seisenberger, Proofs, programs, processes.

Theory of Computing Systems 51(3), 2012.

◮ B., Tie Hou, Typed vs. Untyped realizability, Electronic Notes

in Computer Science 286, 2012.

◮ Andrew Lawrence, B., Monika Seisenberger, Extracting a

DPLL Algorithm. Electronic Notes in Computer Science 286, 2012.

◮ B., From coinductive proofs to exact real arithmetic: theory

and applications. LMCS 7(1), 2011.

◮ B., Kenji Miyamoto, Helmut Schwichtenberg, and Monika

Seisenberger, Minlog - A Tool for Program Extraction for Supporting Algebra and Coalgebra. Lecture Notes in Computer Science, 2011.

◮ B., Realisability for Induction and Coinduction with

Applications to Constructive Analysis. Journal of Universal Computer Science, 2010.

36 / 36