Real Vector Spaces, the Cauchy-Schwarz Inequality, & Convex Functions in ACL2(r) Carl Kwan Mark R. Greenstreet University of British Columbia 15th International Workshop on the ACL2 Theorem Prover and Its Applications Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 1 / 26
Introduction Outline: ◮ Framework for reasoning about real vector spaces and convex functions ◮ The Cauchy-Schwarz inequality ◮ Proof “engineering” ◮ Design proofs such that theorem statements are clear and concise ◮ Avoid fundamental logical limitations Motivation: ◮ Reasoning about convex optimisation algorithms ◮ Cauchy-Schwarz is useful and elegant ◮ Top 100 Theorems / Formalising 100 Theorems 1 1 cs.ru.nl/ ∼ freek/100 Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 2 / 26
Vector Spaces ( R n , R , · , +) such that ◮ + : R n × R n → R is associative and commutative ◮ Identity elements: 0 + v = v and 1 v = v ◮ Inverse elements: v + ( − v ) = 0 ◮ Compatibility: a ( bv ) = ( ab ) v ◮ Distributivity (two ways): a ( u + v ) = au + uv and ( a + b ) v = av + bv au u + v v u u Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 3 / 26
Inner Product Spaces Inner Product Space = Vector Space + Inner Product �− , −� : R n × R n → R ◮ Positive-definiteness: � u , u � ≥ 0 and � u , u � = 0 ⇐ ⇒ u = 0 ◮ Symmetry 2 : � u , v � = � v , u � ◮ Linearity of the first coordinate: � au + v , w � = a � u , w � + � v , w � For R n and u = ( u i ) n i =1 , v = ( v i ) n i =1 , use the dot product: n � � u , v � = u i v i i =1 2 when over R Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 4 / 26
Cauchy-Schwarz Theorem 1 (The Cauchy-Schwarz Inequality) Let u , v ∈ R n . Then |� u , v �| 2 ≤ � u , u �� v , v � (CS1) or, equivalently, |� u , v �| ≤ � u � · � v � (CS2) � with equality iff u , v are linearly dependent. Here � u � := � u , u � . How to prove it? Clever set-up + basic algebraic manipulations Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 5 / 26
Proof of |� u , v �| 2 ≤ � u , u �� v , v � How to prove it? Clever set-up + basic algebraic manipulations: Proof (sketch). From positive-definiteness: 0 ≤ � u − av , u − av � = � u , u � − 2 a � u , v � + a 2 � v , v � . Set a = � u , v � � v , v � and rearrange (a bunch) to get � u , u � − � u , v � 2 � � − 2 � u , v � � v , v � + � u , v � 0 ≤ · · · = � u , u � + � u , v � = � v , v � . � v , v � � � � � � � � � � How to formalise it? Follow (mostly) from the classical proofs. Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 6 / 26
Structure of Cauchy-Schwarz Approach: easy, √ · CS2 easy, ( · ) 2 classic Axioms CS1 CS1 easy, “=” = ⇒ “ ≤ ” ? EQ CS1: |� u , v �| 2 ≤ � u , u �� v , v � CS2: |� u , v �| ≤ � u �� v � EQ: |� u , v �| 2 = � u , u �� v , v � ⇐ ⇒ ∃ a ∈ R , u = av Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 7 / 26
Cauchy-Schwarz: Conditions for Equality EQ CS1 |� u , v �| 2 = � u , u �� v , v � ∃ a ∈ R , u = av In ACL2, just reverse and use positive-definiteness � u , u � − � u , v � 2 0 ≤ � u − av , u − av � = · · · = � v , v � . � � � � � � � � � How to express “ ∃ a ”? 1. Explicitly compute a from |� u , v �| 2 = � u , u �� v , v � - hard & annoying 2. Use Skolem functions - much easier Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 8 / 26
Using Skolem Functions for Cauchy-Schwarz Skolem functions have bodies with outermost quantifiers 3 : (defun-sk linear-dependence (u v) (exists a (equal u (scalar-* a v)))) Requires a witness: 0 = � u − av , u − av � ⇐ ⇒ u − av = 0 ⇐ ⇒ u = av where a = � u , v � � v , v � from before. 3 scalar-* is scalar-vector multiplication Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 9 / 26
Real Vector Spaces & Cauchy-Schwarz - Summary Results: ◮ Reason about real vector & inner product spaces ◮ Formalised Cauchy-Schwarz inequality Proof design issues: ◮ Exhibiting linear dependence in Cauchy-Schwarz ◮ Use Skolem functions ◮ Explicitly computing coefficients is hard - why compute when you don’t need to? Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 10 / 26
Metric Spaces � u − v , u − v � = � u − v � 2 = d 2 ( u , v ) inner products → norms − → metrics ( M , d ) where d : M × M → R such that 1. Indiscernibility: d ( x , y ) = 0 ⇐ ⇒ x = y 2. Symmetry: d ( x , y ) = d ( y , x ) 3. Triangle inequality: d ( x , y ) ≤ d ( x , z ) + d ( z , y ) Let M = R n and d ( x , y ) = � x − y � : 1. & 2. Immediate 3. Use Cauchy-Schwarz: let x = x ′ − z , y = z − y ′ � x + y � 2 = � x � 2 + 2 � x , y � + � y � 2 ≤ � x � 2 + 2 � x �� y � + � y � 2 = ( � x � + � y � ) 2 Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 11 / 26
Univariate/Multivariate Non-standard Analysis 4 A number x is standard if it satisfies our usual definition of real. A number x > 0 is i-small if it is less than any positive standard. Continuity: A function f is continuous at a standard x if for any y d ( x , y ) i-small = ⇒ d ( f ( x ) , f ( y )) i-small Univariate Multivariate f : R n → R , d = � · � f : R → R , d = | · | Differentiability: The derivative of f is a function f ′ satisfying the conditions below for “i-small” h Univariate Multivariate f ′ ( x ) = f ( x + h ) − f ( x ) � f ( x + h ) − f ( x ) −� f ′ ( x ) , h �� = 0 h � h � What does “i-small” mean for a vector in R n ? 4 informal Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 12 / 26
Recognizing “i-small” Vectors Want: (defun i-small -vecp (vec) (if (null vec) t (and (i-small (car vec)) (i-small -vecp (cdr vec ))))) NO! Non-classical 5 recursive functions are prohibited! Instead, � n � � � z 2 � x � = i ≥ max | x i | ≥ | x i | � i i =1 so � x � i-small = ⇒ | x i | i-small ∀ i ∈ [1 , n ] 5 functions defined only in ACL2(r) Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 13 / 26
Recognizing “i-small” Vectors � x � i-small = ⇒ | x i | i-small ∀ i ∈ [1 , n ] Avoid recursion by reasoning over i : (defthm eu -norm -i-small -implies -elements -i-small (implies (and (real -listp x) (i-small (eu -norm x)) (natp i) (< i (len x))) (i-small (nth i x)))) eu-norm is the Euclidean norm Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 14 / 26
Real Vector & Metric Spaces - Summary Results: ◮ Reason about real vector spaces ◮ Reason about real metric spaces ◮ Multivariate continuity & differentiability Proof design issues: ◮ Exhibiting linear dependence in Cauchy-Schwarz ◮ Defining continuity ◮ Non-classical recursive functions are prohibited ◮ Show the largest entry in the vector is i-small ◮ Reason about the index of arbitrary entries in the vector to avoid recursion Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 15 / 26
Convex Functions A function f : R n → R is convex if for all α ∈ [0 , 1] ⊂ R , x , y ∈ R n f ( α x + (1 − α ) y ) ≤ α f ( x ) + (1 − α ) f ( y ) . Theorem 2 Let f , g : R n → R , h : R → R be convex. Then 1. a · f is convex for all a ∈ R ≥ 0 , 2. f + g is convex, 3. h ◦ f is convex. But how do we reason about functions? Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 16 / 26
Encapsulating Convex Functions Encapsulate and suppress function definitions after proving hypotheses: (encapsulate ... (local (defun cvfn -1 (x) ... 1337)) ... (defthm cvfn -1- convex (implies ... ;; hypotheses (<= (cvfn -1 (vec -+ (scalar -* a x) (scalar -* (- 1 a) y))) (+ (* a (cvfn -1 x)) (* (- 1 a) (cvfn -1 y))))) ...) (local (in -theory (disable cvfn -1))) ... ;; prove theorems about cvfn -1 ) How do we reason about the convexity of a function? Carl Kwan & Mark R. Greenstreet (UBC) Re. Vec. Spaces, C.S. Inequality, & Conv. Func. (ACL2 2018) 2018-11-06 17 / 26
Recommend
More recommend
