Finite Difference Method
Motivation For a given smooth function π π¦ , we want to calculate the derivative πβ² π¦ at a given value of π¦. Suppose we donβt know how to compute the analytical expression for πβ² π¦ , or it is computationally very expensive. However you do know how to evaluate the function value: We know that: π π¦ + β β π(π¦) πβ² π¦ = lim β !β# ! "#$ %! " Can we just use πβ² π¦ β as an approximation? How do we choose β ? $ Can we get estimate the error of our approximation?
Finite difference method For a differentiable function π: β β β , the derivative is defined as: π π¦ + β β π(π¦) πβ² π¦ = lim β !β# Taylor Series centered at π¦ , where Μ π¦ = π¦ + β ! ! ! " π π¦ + β = π π¦ + π $ π¦ β + πβ²β² π¦ % +πβ²β²β² π¦ & + β― π π¦ + β = π π¦ + π $ π¦ β + π(β % ) We define the Forward Finite Difference as: Therefore, the truncation error of the forward finite difference approximation is bounded by:
In a similar way, we can write: π π¦ β β = π π¦ β π ! π¦ β + π(β " ) β π ! π¦ = π π¦ β π π¦ β β + π(β) β And define the Backward Finite Difference as: ππ π¦ = π π¦ β π π¦ β β β π ! π¦ = ππ π¦ + π(β) β And subtracting the two Taylor approximations # ! # " π π¦ + β = π π¦ + π ! π¦ β + πβ²β² π¦ " +πβ²β²β² π¦ $ + β― # ! # " π π¦ β β = π π¦ β π ! π¦ β + πβ²β² π¦ " βπβ²β²β² π¦ $ + β― π π¦ + β β π π¦ β β = 2π ! π¦ β + πβ²β²β² π¦ β % 6 + π(β & ) π ! π¦ = π π¦ + β β π π¦ β β + π(β " ) 2β And define the Central Finite Difference as: ππ π¦ = π¦ + β β π π¦ β β β π ! π¦ = ππ π¦ + π(β " ) 2β
How accurate is the finite difference approximation? How many function evaluations (in additional to π π¦ )? Forward Finite Difference: ' ()# *' ( β π ! π¦ = ππ π¦ + π(β) Truncation error: π(β) ππ π¦ = # Cost: 1 function evaluation Backward Finite Difference: β π ! π¦ = ππ π¦ + π(β) ' ( *' (*# Truncation error: π(β) ππ π¦ = # Cost: 1 function evaluation Central Finite Difference: β π ! π¦ = ππ π¦ + π(β " ) Truncation error: π(β " ) ππ π¦ = ' ()# *' (*# "# Cost: 2 function evaluation2 Our typical trade-off issue! We can get better accuracy with Central Finite Difference with the (possible) increased computational cost. How small should the value of π ?
Example ππ π ππ β π π¦ = π $ β 2 πβ² π¦ = π $ We want to obtain an approximation for πβ² 1 ππππππ ππ¦ = (π $%! β2) β (π $ β2) β Truncation error ππ π ππ (β) = πππ‘(πβ² π¦ β ππππππ ππ¦)
Example Should we just keep decreasing the perturbation β , in order to approach the limit β β 0 and obtain a better approximation for the derivative?
Uh-Oh ! What happened here? π π¦ = π $ β 2, πβ² π¦ = π $ β πβ² 1 β 2.7 Forward Finite Difference ππ 1 = π 1 + β β π(1) β
When computing the finite difference approximation, we have two competing source of errors: Truncation errors and Rounding errors ππ(π¦) = π π¦ + β β π(π¦) β€ π + |π π¦ | β β
Loss of accuracy due to rounding Minimize the total error ππ π ππ ~ π + |π π¦ | ππ π ππ ~π β Optimal βhβ Truncation error: + πβ β Gives ππ π ππ ~ π & |π π¦ | β = π + |π π¦ |/π Rounding error: β
Recommend
More recommend
