Truncation errors: using Taylor series to approximation functions - - PowerPoint PPT Presentation

Truncation errors: using Taylor series to approximation functions

Let’s say we want to approximate a function 𝑔(𝑦) with a polynomial For simplicity, assume we know the function value and its derivatives at 𝑦! = 0 (we will later generalize this for any point). Hence,

Approximating functions using polynomials:

𝑔 𝑦 = 𝑏! + 𝑏" 𝑦 + 𝑏# 𝑦# + 𝑏$ 𝑦$ + 𝑏% 𝑦% + ⋯

𝑔 0 = 𝑏!

𝑔" 𝑦 = 𝑏# + 2 𝑏$ 𝑦 + 3 𝑏% 𝑦$ + 4 𝑏& 𝑦% + ⋯

𝑔′ 0 = 𝑏"

𝑔""(𝑦) = 2 𝑏$ + 3×2 𝑏% 𝑦 + (4×3)𝑏& 𝑦$ + ⋯

𝑔′′ 0 = 2 𝑏#

𝑔′′′ 0 = (3×2) 𝑏! 𝑔"# 0 = (4×3×2) 𝑏$

𝑔"""(𝑦) = 3×2 𝑏% + (4×3×2)𝑏& 𝑦 + ⋯ 𝑔"'(𝑦) = (4×3×2)𝑏& + ⋯ 𝑔()) 0 = 𝑗! 𝑏)

Taylor Series

𝑔 𝑦 = 𝑔 0 + 𝑔" 0 𝑦 + 𝑔"" 0 2! 𝑦$ + 𝑔""" 0 3! 𝑦% + ⋯

Taylor Series approximation about point 𝑦! = 0

𝑔 𝑦 = 0

)+,

• 𝑔 ) (0)

𝑗! 𝑦)

Demo “Polynomial Approximation with Derivatives” – Part 1

𝑔 𝑦 = 𝑏! + 𝑏" 𝑦 + 𝑏# 𝑦# + 𝑏$ 𝑦$ + 𝑏% 𝑦% + ⋯

Taylor Series

𝑔 𝑦 = 𝑔 𝑦! + 𝑔$ 𝑦! (𝑦 − 𝑦!) + 𝑔$$ 𝑦! 2! (𝑦 − 𝑦!)#+ 𝑔$$$ 𝑦! 3! (𝑦 − 𝑦!)%+ ⋯

In a more general form, the Taylor Series approximation about point 𝑦! is given by:

𝑔 𝑦 = 0

)+,

• 𝑔 ) (𝑦!)

𝑗! (𝑦 − 𝑦!))

Iclicker question

Assume a finite Taylor series approximation that converges everywhere for a given function 𝑔(𝑦) and you are given the following information: 𝑔 1 = 2; 𝑔&(1) = −3; 𝑔&&(1) = 4; 𝑔 ' 1 = 0 ∀ 𝑜 ≥ 3 Evaluate 𝑔 4 A) 29 B) 11 C) -25 D) -7 E) None of the above

Taylor Series

We cannot sum infinite number of terms, and therefore we have to truncate. How big is the error caused by truncation? Let’s write ℎ = 𝑦 − 𝑦!

𝑔 𝑦% + ℎ − 0

&'( ) 𝑔 & 𝑦%

𝑗! ℎ & = 0

&')*+ ,

𝑔 & (𝑦%) 𝑗! (ℎ)& 𝑔 𝑦% + ℎ − 0

&'( ) 𝑔 & 𝑦%

𝑗! ℎ & ≤ C 5 ℎ)*+

And as ℎ → 0 we write:

Error due to Taylor approximation of degree n

𝑔 𝑦% + ℎ − 0

&'( ) 𝑔 & 𝑦%

𝑗! ℎ & = 𝑃(ℎ)*+)

Demo “Polynomial Approximation with Derivatives” – Part 2

Taylor series with remainder

𝑔 𝑦% + ℎ − 0

&'( ) 𝑔 & 𝑦%

𝑗! ℎ & = 0

&')*+ ,

𝑔 & (𝑦%) 𝑗! (ℎ)&

Let 𝑔 be (𝑜 + 1)-times differentiable on the interval (𝑦!, 𝑦) with 𝑔(') continuous on [𝑦!, 𝑦], and ℎ = 𝑦 − 𝑦! Then there exists a 𝜊 ∈ (𝑦!, 𝑦) so that

𝑔 𝑦% + ℎ − 0

&'( ) 𝑔 & 𝑦%

𝑗! ℎ & = 𝑔 )*+ (𝜊) (𝑜 + 1)! (𝜊 − 𝑦%))*+

And since 𝜊 − 𝑦! ≤ ℎ

𝑔 𝑦% + ℎ − 0

&'( ) 𝑔 & 𝑦%

𝑗! ℎ & ≤ 𝑔 )*+ (𝜊) (𝑜 + 1)! (ℎ))*+

Taylor remainder 𝑔 𝑦 − 𝑈 𝑦 = 𝑆(𝑦)

Demo: Polynomial Approximation with Derivatives

Demo: Polynomial Approximation with Derivatives

Iclicker question

A) B) C) D) E)

Demo “Taylor of exp(x) about 2”

Making error predictions

Demo “Polynomial Approximation with Derivatives” – Part 3

Using Taylor approximations to obtain derivatives

Let’s say a function has the following Taylor series expansion about 𝑦 = 2.

𝑔 𝑦 = 5 2 − 5 2 𝑦 − 2 ! + 15 8 𝑦 − 2 " − 5 4 𝑦 − 2 # + 25 32 𝑦 − 2 $ + O((𝑦 − 2)%)

Therefore the Taylor polynomial of order 4 is given by

𝑢 𝑦 = 5 2 − 5 2 𝑦 − 2 ! + 15 8 𝑦 − 2 "

where the first derivative is

𝑢"(𝑦) = −5 𝑦 − 2 + 15 2 𝑦 − 2 !

1 2 3 4 2 4 6 8

𝑔 𝑦 𝑢 𝑦

Using Taylor approximations to obtain derivatives

We can get the approximation for the derivative of the function 𝑔 𝑦 using the derivative of the Taylor approximation:

𝑢"(𝑦) = −5 𝑦 − 2 + 15 2 𝑦 − 2 ! For example, the approximation for 𝑔′ 2.3 is 𝑔" 2.3 ≈ 𝑢" 2.3 = −1.2975 (note that the exact value is 𝑔" 2.3 = −1.31444

1 2 3 4 2 4 6 8

𝑔 𝑦 𝑢 𝑦 What happens if we want to use the same method to approximate 𝑔′ 3 ?

The function 𝑔 𝑦 = cos 𝑦 𝑦# +

)*+(#,) ,-#,& '

is approximated by the following Taylor polynomial of degree 𝑜 = 2 about 𝑦 = 2π

𝑢- 𝑦 = 39.4784 + 12.5664 𝑦 − 2π − 18.73922 𝑦 − 2𝜌 -

Determine an approximation for the first derivative of 𝑔 𝑦 at 𝑦 = 6.1 A) 18.7741 B) 12.6856 C) 19.4319 D) 15.6840

Iclicker question

Computing integrals using Taylor Series

A function 𝑔 𝑦 is approximated by a Taylor polynomial of order 𝑜 around 𝑦 = 0.

𝑢) = 0

&'( ) 𝑔 & 0

𝑗! 𝑦 &

We can find an approximation for the integral ∫

. / 𝑔 𝑦 𝑒𝑦 by integrating the

polynomial: Where we can use ∫

. / 𝑦0 𝑒𝑦 = /()* 0-" − .()* 0-"

Demo “Computing PI with Taylor”

A function 𝑔 𝑦 is approximated by the following Taylor polynomial: 𝑢. 𝑦 = 10 + 𝑦 − 5 𝑦- − 𝑦! 2 + 5𝑦$ 12 + 𝑦. 24 − 𝑦/ 72

Determine an approximated value for ∫

1% " 𝑔 𝑦 𝑒𝑦

A) -10.27 B)

• 11.77

C) 11.77 D) 10.27

Iclicker question

Finite difference approximation

For a given smooth function 𝑔 𝑦 , we want to calculate the derivative

𝑔′ 𝑦 at 𝑦 = 1. Suppose we don’t know how to compute the analytical expression for 𝑔′ 𝑦 , but we have available a code that evaluates the function value:

We know that:

𝑔′ 𝑦 = lim

2→4

𝑔 𝑦 + ℎ − 𝑔(𝑦) ℎ

Can we just use 𝑔′ 𝑦 ≈ > ?@A B> ?

A

? How do we choose ℎ? Can we get estimate the error of our approximation?

For a differentiable function 𝑔: ℛ → ℛ, the derivative is defined as: 𝑔′ 𝑦 = lim

+→-

𝑔 𝑦 + ℎ − 𝑔(𝑦) ℎ Let’s consider the finite difference approximation to the first derivative as 𝑔′ 𝑦 ≈ 𝑔 𝑦 + ℎ − 𝑔 𝑦 ℎ Where ℎ is often called a “perturbation”, i.e. a “small” change to the variable 𝑦. By the Taylor’s theorem we can write: 𝑔 𝑦 + ℎ = 𝑔 𝑦 + 𝑔. 𝑦 ℎ + 𝑔′′(𝜊) ℎ! 2 For some 𝜊 ∈ [𝑦, 𝑦 + ℎ]. Rearranging the above we get: 𝑔. 𝑦 = 𝑔 𝑦 + ℎ − 𝑔(𝑦) ℎ − 𝑔′′(𝜊) ℎ 2 Therefore, the truncation error of the finite difference approximation is bounded by M

+ !, where M is

a bound on 𝑔.. 𝜊 for 𝜊 near 𝑦.

Demo: Finite Difference

𝑔 𝑦 = 𝑓0 − 2 𝑒𝑔𝑓𝑦𝑏𝑑𝑢 = 𝑓0 𝑒𝑔𝑏𝑞𝑞𝑠𝑝𝑦 = 𝑓0*1 − 2 − (𝑓0−2) ℎ 𝑓𝑠𝑠𝑝𝑠(ℎ) = 𝑏𝑐𝑡(𝑒𝑔𝑓𝑦𝑏𝑑𝑢 − 𝑒𝑔𝑏𝑞𝑞𝑠𝑝𝑦) We want to obtain an approximation for 𝑔′ 1

ℎ 𝑓𝑠𝑠𝑝𝑠

𝑓𝑠𝑠𝑝𝑠 < 𝑔"" 𝜊 ℎ 2

truncation error

Demo: Finite Difference

Should we just keep decreasing the perturbation ℎ, in order to approach the limit ℎ → 0 and obtain a better approximation for the derivative?

𝑔′ 𝑦 = lim

+→-

𝑔 𝑦 + ℎ − 𝑔(𝑦) ℎ

Uh-Oh!

What happened here? 𝑔 𝑦 = 𝑓0 − 2 𝑔′ 𝑦 = 𝑓0 → 𝑔′ 1 ≈ 2.7

𝑔′ 1 = lim

+→-

𝑔 1 + ℎ − 𝑔(1) ℎ

Rounding error!

1) for a “very small” ℎ (ℎ < 𝜗) → 𝑔 1 + ℎ = 𝑔(1) → 𝑔′ 1 = 0 2) for other still “small” ℎ (ℎ > 𝜗) → 𝑔 1 + ℎ − 𝑔 1 gives results with fewer significant digits

(We will later define the meaning of the quantity 𝜗)

𝑓𝑠𝑠𝑝𝑠~𝑁 ℎ 2

Truncation error: Rounding error:

𝑓𝑠𝑠𝑝𝑠~ 2𝜗 ℎ

Minimize the error 2𝜗 ℎ + 𝑁 ℎ 2 Gives ℎ = 2 𝜗/𝑁