Daisy a framework for sound accuracy analysis of numerical programs - PowerPoint PPT Presentation

Daisy a framework for sound accuracy analysis of numerical programs Eva Darulova Anastasiia Izycheva eva@mpi-sws.org izycheva@in.tum.de

Daisy a framework for sound accuracy analysis of numerical programs } and optimization Eva Darulova Anastasiia Izycheva eva@mpi-sws.org izycheva@in.tum.de

Observation:

Observation: many applications use more resources than they really need

Observation: many applications use more resources than they really need ➡ particularly important for embedded systems

Observation: many applications use more resources than they really need ➡ particularly important for embedded systems ➡ in numerical programs:

Observation: many applications use more resources than they really need ➡ particularly important for embedded systems ➡ in numerical programs: ‣ too much precision (among other issues)

Observation: many applications use more resources than they really need ➡ particularly important for embedded systems ➡ in numerical programs: ‣ too much precision (among other issues) Why?

Observation: many applications use more resources than they really need ➡ particularly important for embedded systems ➡ in numerical programs: ‣ too much precision (among other issues) Why? challenging to optimise manually

Observation: many applications use more resources than they really need ➡ particularly important for embedded systems ➡ in numerical programs: ‣ too much precision (among other issues) Why? challenging to optimise manually ‣ verification of finite-precision

Observation: many applications use more resources than they really need ➡ particularly important for embedded systems ➡ in numerical programs: ‣ too much precision (among other issues) Why? challenging to optimise manually ‣ verification of finite-precision ‣ too many options for optimisation

Daisy’s Goal real-valued program with accuracy spec Daisy optimised finite-precision program

Daisy’s Goal general real-valued program with accuracy spec Daisy optimised finite-precision program

Daisy’s Goal general real-valued program with accuracy spec Daisy automated optimised finite-precision program

real-valued program with accuracy spec finite-precision program

real-valued program with accuracy spec 1: def sine(x: Real ): Real = { 2: require (-1.5 <= x && x <= 1.5 && x +/- 1e-11) 3: 4: x - (x^3)/6.0 + (x^5)/120.0 - (x^7)/5040.0 5: 6: } ensuring (res => res +/- 1.001e-11) finite-precision program

real-valued program with accuracy spec 1: def sine(x: Real ): Real = { 2: require (-1.5 <= x && x <= 1.5 && x +/- 1e-11) 3: 4: x - (x^3)/6.0 + (x^5)/120.0 - (x^7)/5040.0 5: 6: } ensuring (res => res +/- 1.001e-11) finite-precision program floating-point arithmetic def sine(x: Double ): Double = { require (-1.5 <= x && x <= 1.5) x - (x^3)/6.0 + (x^5)/120.0 - (x^7)/5040.0 }

real-valued program with accuracy spec 1: def sine(x: Real ): Real = { 2: require (-1.5 <= x && x <= 1.5 && x +/- 1e-11) 3: 4: x - (x^3)/6.0 + (x^5)/120.0 - (x^7)/5040.0 5: 6: } ensuring (res => res +/- 1.001e-11) finite-precision program floating-point arithmetic fixed-point arithmetic def sine(x: Long): Long = { def sine(x: Double ): Double = { require (-1.5 <= x && x <= 1.5) require (-1.5 <= x && x <= 1.5) val _t1 = ((x * x) >> 31) val _t2 = ((_t1 * x) >> 30) x - (x^3)/6.0 + (x^5)/120.0 - (x^7)/5040.0 val _t3 = ((_t2 << 30) / 1610612736l) val _t4 = ((x << 1) - _t3) } . . .

real-valued program with accuracy spec 1: def sine(x: Real ): Real = { 2: require (-1.5 <= x && x <= 1.5 && x +/- 1e-11) 3: 4: x - (x^3)/6.0 + (x^5)/120.0 - (x^7)/5040.0 5: 6: } ensuring (res => res +/- 1.001e-11) 1. verification 2. optimisation finite-precision program floating-point arithmetic fixed-point arithmetic def sine(x: Long): Long = { def sine(x: Double ): Double = { require (-1.5 <= x && x <= 1.5) require (-1.5 <= x && x <= 1.5) val _t1 = ((x * x) >> 31) val _t2 = ((_t1 * x) >> 30) x - (x^3)/6.0 + (x^5)/120.0 - (x^7)/5040.0 val _t3 = ((_t2 << 30) / 1610612736l) val _t4 = ((x << 1) - _t3) } . . .

Accuracy Verification def sine(x: Real ): Real = { require (-1.5 <= x && x <= 1.5 && x +/- 1e-11) x - (x^3)/6.0 + (x^5)/120.0 - (x^7)/5040.0 } ensuring (res => res +/- 1.001e-11) How to measure accuracy? x ∈ I | f ( x ) − ˜ max f (˜ x ) | absolute errors

Accuracy Verification def sine(x: Real ): Real = { require (-1.5 <= x && x <= 1.5 && x +/- 1e-11) x - (x^3)/6.0 + (x^5)/120.0 - (x^7)/5040.0 } ensuring (res => res +/- 1.001e-11) How to measure accuracy? � � f ( x ) − ˜ f (˜ x ) � � x ∈ I | f ( x ) − ˜ max f (˜ x ) | max � � f ( x ) � � x ∈ I � � absolute errors relative errors

Accuracy Verification def sine(x: Real ): Real = { require (-1.5 <= x && x <= 1.5 && x +/- 1e-11) x - (x^3)/6.0 + (x^5)/120.0 - (x^7)/5040.0 } ensuring (res => res +/- 1.001e-11) How to measure accuracy? � � f ( x ) − ˜ f (˜ x ) � � x ∈ I | f ( x ) − ˜ max f (˜ x ) | max � � f ( x ) � � x ∈ I � � absolute errors relative errors Challenge: automatically and accurately bound worst-case errors

Bounding Absolute Errors x ∈ I | f ( x ) − ˜ max f (˜ x ) | Static dataflow analysis ‣ fully automated ‣ sound upper bound

Bounding Absolute Errors x ∈ I | f ( x ) − ˜ max f (˜ x ) | Static dataflow analysis ‣ fully automated ‣ sound upper bound for each arithmetic operation 1. compute real-valued range for intermediate value 2. propagate existing errors 3. compute new roundoff error

Bounding Absolute Errors [1] x ∈ I | f ( x ) − ˜ max f (˜ x ) | Static dataflow analysis ‣ fully automated ‣ sound upper bound Challenge: tight bounds for nonlinear arithmetic

Bounding Absolute Errors [1] x ∈ I | f ( x ) − ˜ max f (˜ x ) | Static dataflow analysis ‣ fully automated ‣ sound upper bound Challenge: tight bounds for nonlinear arithmetic Challenge: conditionals (control-flow may diverge)

Bounding Absolute Errors [1] x ∈ I | f ( x ) − ˜ max f (˜ x ) | Static dataflow analysis ‣ fully automated ‣ sound upper bound Challenge: tight bounds for nonlinear arithmetic Challenge: conditionals (control-flow may diverge) Challenge: loops (errors may grow unboundedly)

Bounding Absolute Errors x ∈ I | f ( x ) − ˜ max f (˜ x ) | Challenge: tight bounds for nonlinear arithmetic abstract domains:

Bounding Absolute Errors x ∈ I | f ( x ) − ˜ max f (˜ x ) | Challenge: tight bounds for nonlinear arithmetic abstract domains: ‣ interval arithmetic

Bounding Absolute Errors x ∈ I | f ( x ) − ˜ max f (˜ x ) | Challenge: tight bounds for nonlinear arithmetic abstract domains: ‣ interval arithmetic ‣ affine arithmetic

Bounding Absolute Errors x ∈ I | f ( x ) − ˜ max f (˜ x ) | Challenge: tight bounds for nonlinear arithmetic abstract domains: ‣ interval arithmetic ‣ affine arithmetic additional techniques:

Bounding Absolute Errors x ∈ I | f ( x ) − ˜ max f (˜ x ) | Challenge: tight bounds for nonlinear arithmetic abstract domains: ‣ interval arithmetic ‣ affine arithmetic additional techniques: ‣ SMT (nonlinear decision procedure)

Bounding Absolute Errors x ∈ I | f ( x ) − ˜ max f (˜ x ) | Challenge: tight bounds for nonlinear arithmetic abstract domains: ‣ interval arithmetic ‣ affine arithmetic additional techniques: ‣ SMT (nonlinear decision procedure) ‣ interval subdivision

Bounding Absolute Errors x ∈ I | f ( x ) − ˜ max f (˜ x ) | Challenge: tight bounds for nonlinear arithmetic abstract domains: ‣ interval arithmetic ‣ affine arithmetic additional techniques: ‣ SMT (nonlinear decision procedure) ‣ interval subdivision error bounds often within one order of magnitude of true errors

Bounding Relative Errors state-of-the-art definition � � x ) − ˜ f ( ~ x ) − ˜ max x ∈ I | f ( ⃗ f ( ⃗ x ) | f ( ~ x ) ˜ ˜ � � max � � 6 = f ( ~ x ) min x ∈ I | f ( ⃗ x ) | � � x ∈ I � � Challenge: tight bounds

Bounding Relative Errors state-of-the-art definition � � x ) − ˜ f ( ~ x ) − ˜ max x ∈ I | f ( ⃗ f ( ⃗ x ) | f ( ~ x ) ˜ ˜ � � max � � 6 = f ( ~ x ) min x ∈ I | f ( ⃗ x ) | � � x ∈ I � � Challenge: tight bounds Challenge: division by zero

Tight Relative Error Bounds Assume NO division by zero � � x ) − ˜ f ( ~ f ( ~ x ) ˜ � � err rel = max � � f ( ~ x ) � � x ∈ I � � Our solution : ‣ evaluate the relative error expression directly ‣ interval subdivision

Relative Errors Directly Replace ˜ f ( ~ e, ~ x ) with its abstraction ˜ f ( ~ d ) x, ~ according to IEEE754 f ( x ) = x + 0 . 3 Naive approach:

Relative Errors Directly Replace ˜ f ( ~ e, ~ x ) with its abstraction ˜ f ( ~ d ) x, ~ according to IEEE754 f ( x ) = x + 0 . 3 Naive approach: e, ⃗ f ( ⃗ d ) = ( x (1 + e x ) + d x + 0 . 3(1 + e 0 . 3 ) + d 0 . 3 )(1 + e + ) + d + x, ⃗