Hack the Derivative! Erik Taubeneck GameChanger Media - Software - - PowerPoint PPT Presentation

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Hack the Derivative!

Erik Taubeneck

GameChanger Media - Software Engineer and Data Maven We’re hiring! https://gc.com/about/careers

July 25th, 2015

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Outline

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Try It Yourself

The functions used through out this presentations can be installed with pip!

$ mkdir somewhere_new $ cd somewhere_new $ virtualenv venv $ source venv/bin/activate $ pip install hackthederivative $ python >>> from hackthederivative import *

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Taking the Derivative

Let f (x) be a function from R → X ⊆ R. e.g.

• f (x) = x2
• g(x) = sin(x)
• h(x) = 5x3 − 6x + 1

x 1

1Note that h(x) is not defined at x = 0.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Taking the Derivative

The derivative of f (x), f ′(x), is a new function which tells us the slope of the tangent line that intersects f at any given x.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Taking the Derivative

Formally, the derivative of f (x) at x is f ′(x) = lim

h→0

f (x + h) − f (x) h

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Taking the Derivative

Ex: Let f (x) = x2.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Taking the Derivative

Ex: Let f (x) = x2. Then f ′(x) = lim

h→0

(x + h)2 − x2 h

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Taking the Derivative

Ex: Let f (x) = x2. Then f ′(x) = lim

h→0

(x + h)2 − x2 h = lim

h→0

x2 + 2xh + h2 − x2 h

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Taking the Derivative

Ex: Let f (x) = x2. Then f ′(x) = lim

h→0

(x + h)2 − x2 h = lim

h→0

x2 + 2xh + h2 − x2 h = lim

h→0

2xh + h2 h

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Taking the Derivative

Ex: Let f (x) = x2. Then f ′(x) = lim

h→0

(x + h)2 − x2 h = lim

h→0

x2 + 2xh + h2 − x2 h = lim

h→0

2xh + h2 h = lim

h→0 2x + h

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Taking the Derivative

Ex: Let f (x) = x2. Then f ′(x) = lim

h→0

(x + h)2 − x2 h = lim

h→0

x2 + 2xh + h2 − x2 h = lim

h→0

2xh + h2 h = lim

h→0 2x + h

= 2x

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Finite Difference

Suppose we wish to find the derivative for some function f (x) a x0.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Finite Difference

Suppose we wish to find the derivative for some function f (x) a x0. One approximation is the finite difference method.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Finite Difference

Suppose we wish to find the derivative for some function f (x) a x0. One approximation is the finite difference method. Recall f ′(x) = lim

h→0

f (x + h) − f (x) h

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Finite Difference

Suppose we wish to find the derivative for some function f (x) a x0. One approximation is the finite difference method. Recall f ′(x) = lim

h→0

f (x + h) − f (x) h We can estimate f ′(xo) ≈ f (x0 + h) − f (x0) h for some small h.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Finite Difference

Implemented in Python

In [1]: def finite_difference(f, x, h): return (f(x+h) - f(x))/h

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Finite Difference

Implemented in Python

In [1]: def finite_difference(f, x, h): return (f(x+h) - f(x))/h In [2]: f = lambda x: x**2 In [2]: finite_difference(f, 1.0, 0.00001) Out [3]: 2.00001000001393

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Finite Difference

But how do we choose h?

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Finite Difference

But how do we choose h? The limit as h → 0 approaches the derivative, so the smaller the better!

In [4]: import sys In [5]: h = sys.float_info.min

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Finite Difference

But how do we choose h? The limit as h → 0 approaches the derivative, so the smaller the better!

In [4]: import sys In [5]: h = sys.float_info.min In [6]: print finite_difference(f, 1.0, h) Out [6]: 0.0

Uh oh.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Finite Difference

Hmm, 0.00001 seems pretty small, and worked well earlier. Maybe that will just work.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Finite Difference

Hmm, 0.00001 seems pretty small, and worked well earlier. Maybe that will just work.

In [7]: finite_difference(f, 1.0e20, 0.00001) Out [7]: 0.0

Nope, that can break too. To figure out why, let’s look closer at Floating Point numbers.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Floating Point Numbers

A Float64, or a double-precision floating-point number, consists of and represents the real number x ∈ R with sign, e, and bi such that (−1)sign(1.b51b50...b0)2 2e−1023

• r (possibly more readable)

(−1)sign

• 1 +

52

• i=1

b52−i2−i

• × 2e−1023

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Floating Point Numbers

This means that the floating point numbers have gaps!

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Floating Point Gaps

In [1]: import sys In [2]: plus_epsilon_identity(x, eps): return x + eps == x In [3]: eps = sys.float_info.min

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Floating Point Gaps

In [1]: import sys In [2]: plus_epsilon_identity(x, eps): return x + eps == x In [3]: eps = sys.float_info.min In [4]: plus_epsilon_identity(0.0, eps) Out [4]: False In [5]: plus_epsilon_identity(1.0, eps) Out [5]: True

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Floating Point Gaps

In [1]: import sys In [2]: plus_epsilon_identity(x, eps): return x + eps == x In [3]: eps = sys.float_info.min In [4]: plus_epsilon_identity(0.0, eps) Out [4]: False In [5]: plus_epsilon_identity(1.0, eps) Out [5]: True In [6]: plus_epsilon_identity(1.0e20, 1.0) Out [6]: True

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Problems with the Derivative

If we choose h that is so small such that float(x) + float(h) = float(x)

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Problems with the Derivative

If we choose h that is so small such that float(x) + float(h) = float(x) then f (x + h) − f (x) h = f (x) − f (x) h = 0 h = 0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Choosing Epsilon

In [1]: def eps(x): e = float(max(sys.float_info.min, abs(x))) while not plus_epsilon_identity(x, e): last = e e = e / 2. return last

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Choosing Epsilon

In [1]: def eps(x): e = float(max(sys.float_info.min, abs(x))) while not plus_epsilon_identity(x, e): last = e e = e / 2. return last In [2]: eps(1.0) Out[2]: 2.220446049250313e-16 In [3]: eps(1.0e10) Out[3]: 1.1102230246251565e-06

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Choosing Epsilon

In [1]: def eps(x): e = float(max(sys.float_info.min, abs(x))) while not plus_epsilon_identity(x, e): last = e e = e / 2. return last In [2]: eps(1.0) Out[2]: 2.220446049250313e-16 In [3]: eps(1.0e10) Out[3]: 1.1102230246251565e-06 In [4]: eps(1.0e20) Out[4]: 11102.230246251565

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Choosing Epsilon

From lecture notes posted by Karen Kopecky 2, we see we should use h = √u ∗ max(|x|, 1) where u = eps(1).

2http://www.karenkopecky.net/Teaching/eco613614/Notes_

NumericalDifferentiation.pdf

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Choosing Epsilon

From lecture notes posted by Karen Kopecky 2, we see we should use h = √u ∗ max(|x|, 1) where u = eps(1).

In [1]: def finite_difference(f, x, h=None): if not h: h = sqrt(eps(1.0)) * max(abs(x), 1.0) return (f(x+h) - f(x))/h

2http://www.karenkopecky.net/Teaching/eco613614/Notes_

NumericalDifferentiation.pdf

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [2]: def error(f, df, x): return abs(finite_difference(f, x) - df(x)) In [3]: def error_rate(f, df, x): return error(f, df, x) / df(x)

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [2]: def error(f, df, x): return abs(finite_difference(f, x) - df(x)) In [3]: def error_rate(f, df, x): return error(f, df, x) / df(x) In [4]: f, df = lambda x:x**2, lambda x:2*x

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [2]: def error(f, df, x): return abs(finite_difference(f, x) - df(x)) In [3]: def error_rate(f, df, x): return error(f, df, x) / df(x) In [4]: f, df = lambda x:x**2, lambda x:2*x In [5]: error_rate(f, df, 1.0) Out[5]: 7.450580596923828e-09

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [2]: def error(f, df, x): return abs(finite_difference(f, x) - df(x)) In [3]: def error_rate(f, df, x): return error(f, df, x) / df(x) In [4]: f, df = lambda x:x**2, lambda x:2*x In [5]: error_rate(f, df, 1.0) Out[5]: 7.450580596923828e-09 In [6]: error_rate(f, df, 1.0e5) Out[6]: 9.045761108398438e-09

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [2]: def error(f, df, x): return abs(finite_difference(f, x) - df(x)) In [3]: def error_rate(f, df, x): return error(f, df, x) / df(x) In [4]: f, df = lambda x:x**2, lambda x:2*x In [5]: error_rate(f, df, 1.0) Out[5]: 7.450580596923828e-09 In [6]: error_rate(f, df, 1.0e5) Out[6]: 9.045761108398438e-09 In [7]: error_rate(f, df, 1.0e20) Out[7]: 4.5369065472e-09

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [8]: import math In [9]: f, df = math.sin, math.cos

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [8]: import math In [9]: f, df = math.sin, math.cos In [10]: error_rate(f, df, 1.0) Out[10]: 1.2780011808656197e-08

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [8]: import math In [9]: f, df = math.sin, math.cos In [10]: error_rate(f, df, 1.0) Out[10]: 1.2780011808656197e-08 In [11]: error_rate(f, df, 1.0e10) Out[11]: 1.002014830004253

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [8]: import math In [9]: f, df = math.sin, math.cos In [10]: error_rate(f, df, 1.0) Out[10]: 1.2780011808656197e-08 In [11]: error_rate(f, df, 1.0e10) Out[11]: 1.002014830004253 In [12]: error_rate(f, df, 1.0e20) Out[12]: 0.9999999999998509

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

We Can Do Better!

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

We Can Do Better!

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

A Crash Course in Complex Analysis

• Def: i = √−1.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

A Crash Course in Complex Analysis

• Def: i = √−1.
• Why do we need i???

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

A Crash Course in Complex Analysis

• Def: i = √−1.
• Why do we need i???
• Solve x2 + 1 = 0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

A Crash Course in Complex Analysis

• Def: i = √−1.
• Why do we need i???
• Solve x2 + 1 = 0
• Def: C = {x + iy ∀ x, y ∈ R2}

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

A Crash Course in Complex Analysis

• Def: i = √−1.
• Why do we need i???
• Solve x2 + 1 = 0
• Def: C = {x + iy ∀ x, y ∈ R2}
• Fun fact! All polynomials of degree n have n zeros in C.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

A Crash Course in Complex Analysis

Ex: f (z) = z2

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

A Crash Course in Complex Analysis

Ex: f (z) = z2 Let z = x + iy. Then f (z) = f (x + iy) = (x + iy)2 = x2 + 2ixy + i2y2 = x2 + 2ixy − y2

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

A Crash Course in Complex Analysis

Ex: f (z) = z2 Let z = x + iy. Then f (z) = f (x + iy) = (x + iy)2 = x2 + 2ixy + i2y2 = x2 + 2ixy − y2 We can always write f (x + iy) = u(x, y) + iv(x, y). Note that u(x, y) = R(f (x + iy)) and v(x, y) = I(f (x + iy))

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

A Crash Course in Complex Analysis

Ex: f (z) = z2 Let z = x + iy. Then f (z) = f (x + iy) = (x + iy)2 = x2 + 2ixy + i2y2 = x2 + 2ixy − y2 We can always write f (x + iy) = u(x, y) + iv(x, y). Note that u(x, y) = R(f (x + iy)) and v(x, y) = I(f (x + iy)) In our example u(x, y) = x2 − y2 and v(x, y) = 2xy

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

A Crash Course in Complex Analysis

u(x, y) = x2 − y2 and v(x, y) = 2xy We can now take 4 different derivatives. ∂u ∂x = 2x ∂v ∂x = 2y ∂u ∂y = −2y ∂v ∂y = 2x

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

A Crash Course in Complex Analysis

u(x, y) = x2 − y2 and v(x, y) = 2xy We can now take 4 different derivatives. ∂u ∂x = 2x ∂v ∂x = 2y ∂u ∂y = −2y ∂v ∂y = 2x Note that: ∂u ∂x = ∂v ∂y ∂u ∂y = −∂v ∂x

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Cauchy-Riemann Equations

These are the Cauchy-Riemann equations and hold for all analytic functions on C! 3 ∂u ∂x = ∂v ∂y ∂u ∂y = −∂v ∂x

3See your favorite Complex Analysis textbook for a proof.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Time for the Hack!

We have f : R → X ⊆ R. Rewrite f (z) = f (x + iy) = u(x, y) + iv(x, y)

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Time for the Hack!

We have f : R → X ⊆ R. Rewrite f (z) = f (x + iy) = u(x, y) + iv(x, y) Now, for all ¯ z ∈ R, we have y = 0, and so f (¯ z) = f (x, 0) = u(x, 0) + iv(x, 0)

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Time for the Hack!

We have f : R → X ⊆ R. Rewrite f (z) = f (x + iy) = u(x, y) + iv(x, y) Now, for all ¯ z ∈ R, we have y = 0, and so f (¯ z) = f (x, 0) = u(x, 0) + iv(x, 0) Since f : R → X ⊆ R, f (¯ z) ∈ R for all ¯ z ∈ R. Therefore v(x, 0) = 0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Time for the Hack!

We have f : R → X ⊆ R. Rewrite f (z) = f (x + iy) = u(x, y) + iv(x, y) Now, for all ¯ z ∈ R, we have y = 0, and so f (¯ z) = f (x, 0) = u(x, 0) + iv(x, 0) Since f : R → X ⊆ R, f (¯ z) ∈ R for all ¯ z ∈ R. Therefore v(x, 0) = 0 and so f (x, 0) = u(x, 0)

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Time for the Hack!

We have f : R → X ⊆ R. Rewrite f (z) = f (x + iy) = u(x, y) + iv(x, y) Now, for all ¯ z ∈ R, we have y = 0, and so f (¯ z) = f (x, 0) = u(x, 0) + iv(x, 0) Since f : R → X ⊆ R, f (¯ z) ∈ R for all ¯ z ∈ R. Therefore v(x, 0) = 0 and so f (x, 0) = u(x, 0) We want to estimate df

∂x . Since for all z ∈ R, f = u,

df ∂x = ∂u ∂x = ∂v ∂y

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Time for the Hack!

Again, using the definition of the derivative ∂v ∂y = lim

h→0

v(x, y + h) − v(x, y) h

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Time for the Hack!

Again, using the definition of the derivative ∂v ∂y = lim

h→0

v(x, y + h) − v(x, y) h Since y = 0 for all ¯ z ∈ R, ∂v ∂y = lim

h→0

v(x, h) − v(x, 0) h

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Time for the Hack!

Again, using the definition of the derivative ∂v ∂y = lim

h→0

v(x, y + h) − v(x, y) h Since y = 0 for all ¯ z ∈ R, ∂v ∂y = lim

h→0

v(x, h) − v(x, 0) h Recall that v(x, 0) = 0, so ∂v ∂y = lim

h→0

v(x, h) h

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Time for the Hack!

Again, using the definition of the derivative ∂v ∂y = lim

h→0

v(x, y + h) − v(x, y) h Since y = 0 for all ¯ z ∈ R, ∂v ∂y = lim

h→0

v(x, h) − v(x, 0) h Recall that v(x, 0) = 0, so ∂v ∂y = lim

h→0

v(x, h) h Now, to estimate df

∂x = ∂v ∂y , for a very small h

df ∂x ≈ v(x, h) h = Im(f (x + ih)) h

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Complex Step Finite Difference

Implemented in Python

In [1]: import sys In [2]: def complex_step_finite_diff(f, x): h = sys.float_info.min return (f(x+h*1.0j)).imag / h

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Complex Step Finite Difference

Implemented in Python

In [3]: f, df = lambda x:x**2, lambda x:2*x

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Complex Step Finite Difference

Implemented in Python

In [3]: f, df = lambda x:x**2, lambda x:2*x In [4]: complex_step_finite_diff(f, 1.0) Out[4]: 2.0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Complex Step Finite Difference

Implemented in Python

In [3]: f, df = lambda x:x**2, lambda x:2*x In [4]: complex_step_finite_diff(f, 1.0) Out[4]: 2.0 In [5]: complex_step_finite_diff(f, 1.0e10) Out[5]: 20000000000.0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Complex Step Finite Difference

Implemented in Python

In [3]: f, df = lambda x:x**2, lambda x:2*x In [4]: complex_step_finite_diff(f, 1.0) Out[4]: 2.0 In [5]: complex_step_finite_diff(f, 1.0e10) Out[5]: 20000000000.0 In [6]: complex_step_finite_diff(f, 1.0e20) Out[6]: 2e+20

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [7]: def cerror(f, df, x): return abs(complex_step_finite_diff(f, x) - df(x)) In [8]: def cerror_rate(f, df, x): return cerror(f, df, x) / x

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [7]: def cerror(f, df, x): return abs(complex_step_finite_diff(f, x) - df(x)) In [8]: def cerror_rate(f, df, x): return cerror(f, df, x) / x In [9]: cerror_rate(f, df, 1.0) Out[9]: 0.0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [7]: def cerror(f, df, x): return abs(complex_step_finite_diff(f, x) - df(x)) In [8]: def cerror_rate(f, df, x): return cerror(f, df, x) / x In [9]: cerror_rate(f, df, 1.0) Out[9]: 0.0 In [10]: cerror_rate(f, df, 1.0e10) Out[10]: 0.0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [7]: def cerror(f, df, x): return abs(complex_step_finite_diff(f, x) - df(x)) In [8]: def cerror_rate(f, df, x): return cerror(f, df, x) / x In [9]: cerror_rate(f, df, 1.0) Out[9]: 0.0 In [10]: cerror_rate(f, df, 1.0e10) Out[10]: 0.0 In [11]: cerror_rate(f, df, 1.0e20) Out[11]: 0.0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [7]: def cerror(f, df, x): return abs(complex_step_finite_diff(f, x) - df(x)) In [8]: def cerror_rate(f, df, x): return cerror(f, df, x) / x In [9]: cerror_rate(f, df, 1.0) Out[9]: 0.0 In [10]: cerror_rate(f, df, 1.0e10) Out[10]: 0.0 In [11]: cerror_rate(f, df, 1.0e20) Out[11]: 0.0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [11]: import cmath In [11]: f, df = cmath.sin, cmath.cos

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [11]: import cmath In [11]: f, df = cmath.sin, cmath.cos In [12]: cerror_rate(f, df, 1.0) Out[12]: 0.0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [11]: import cmath In [11]: f, df = cmath.sin, cmath.cos In [12]: cerror_rate(f, df, 1.0) Out[12]: 0.0 In [13]: cerror_rate(f, df, 1.0e10) Out[13]: 0.0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Testing It Out

In [11]: import cmath In [11]: f, df = cmath.sin, cmath.cos In [12]: cerror_rate(f, df, 1.0) Out[12]: 0.0 In [13]: cerror_rate(f, df, 1.0e10) Out[13]: 0.0 In [14]: cerror_rate(f, df, 1.0e20) Out[14]: 0.0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

It’s Not Perfect, But It’s Pretty Good

In [15]: cerror_rate(f, df, 1.0e5) Out[15]: .110933124815182e-16

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

It’s Not Perfect, But It’s Pretty Good

In [15]: cerror_rate(f, df, 1.0e5) Out[15]: .110933124815182e-16 In [16]: f = lambda x: cmath.exp(x) / cmath.sqrt(x) In [17]: df = lambda x: (cmath.exp(x)* (2*x - 1))/(2*x**(1.5))

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

It’s Not Perfect, But It’s Pretty Good

In [15]: cerror_rate(f, df, 1.0e5) Out[15]: .110933124815182e-16 In [16]: f = lambda x: cmath.exp(x) / cmath.sqrt(x) In [17]: df = lambda x: (cmath.exp(x)* (2*x - 1))/(2*x**(1.5)) In [18]: cerror_rate(f, df, 1.0) Out[18]: 0.0

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

It’s Not Perfect, But It’s Pretty Good

In [15]: cerror_rate(f, df, 1.0e5) Out[15]: .110933124815182e-16 In [16]: f = lambda x: cmath.exp(x) / cmath.sqrt(x) In [17]: df = lambda x: (cmath.exp(x)* (2*x - 1))/(2*x**(1.5)) In [18]: cerror_rate(f, df, 1.0) Out[18]: 0.0 In [19]: cerror_rate(f, df, 1.0e2) Out[19]: 1.1570932160552342e-16

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

It’s Not Perfect, But It’s Pretty Good

In [15]: cerror_rate(f, df, 1.0e5) Out[15]: .110933124815182e-16 In [16]: f = lambda x: cmath.exp(x) / cmath.sqrt(x) In [17]: df = lambda x: (cmath.exp(x)* (2*x - 1))/(2*x**(1.5)) In [18]: cerror_rate(f, df, 1.0) Out[18]: 0.0 In [19]: cerror_rate(f, df, 1.0e2) Out[19]: 1.1570932160552342e-16 In [20]: cerror_rate(f, df, 5.67) Out[20]: 1.2795419601231268e-16

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

It’s Not Perfect, But It’s Pretty Good

In [15]: cerror_rate(f, df, 1.0e5) Out[15]: .110933124815182e-16 In [16]: f = lambda x: cmath.exp(x) / cmath.sqrt(x) In [17]: df = lambda x: (cmath.exp(x)* (2*x - 1))/(2*x**(1.5)) In [18]: cerror_rate(f, df, 1.0) Out[18]: 0.0 In [19]: cerror_rate(f, df, 1.0e2) Out[19]: 1.1570932160552342e-16 In [20]: cerror_rate(f, df, 5.67) Out[20]: 1.2795419601231268e-16 In [21]: cerror_rate(f, df, 1.0e3) Out [21]: OverflowError: math range error

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Limits

• Must satisfy Cauchy Riemann equations, so does not work for

abs or a function with < or > in it.

• Only works for functions from R → X ⊆ R.

Calculus Crash Course Finite Difference Floating Point Arithmetic Complex Analysis Crash Course Hack the Derivative

Thanks!

• Follow me on twitter: @taubeneck
• Fork me on Github: eriktaubeneck
• Blog: http://skien.cc/
• Slides: http:

//slides.skien.cc/hack-the-derivative-pydata.pdf

• Other Presentations: http://slides.skienc.cc