CE 341/441 - Lecture 11 - Fall 2004

p. 11.1
LECTURE 11
NUMERICAL DIFFERENTIATION
To nd discrete approximations to differentiation (since computers can only deal with
functional values at discrete points)
Uses of numerical differentiation
To represent the terms in o.d.e.s and p.d.e.s in a discrete manner
Many error estimates include derivatives of a function. This function is typically not
available, but values of the function at discrete points are.
Notation
Nodes are data points at which functional values are available or at which you wish
to compute functional values
At the nodes f x
i
( ) f
i

f
i-2
f
i-1
f
i
f
i+1
f
i+2

x
i-2
x
i-1
x
i
x
i+1
x
i+2

f(x)
x
node i-2 i-1 i i+1 i+2
CE 341/441 - Lecture 11 - Fall 2004
p. 11.2
Node index i indicates which node or point in space-time we are considering
(here only one spatial or temporal direction)
For equi-spaced nodal points,
Taylor Series Expansion for f(x) About a Typical Node i _______________________________________________
i 0 1 2 i 20 21 N=22
i=1, N
i=0,N
h x
i 1 +
x
i
=
f x ( ) f x
i
( ) x x
i
( ) f
1 ( )
x
i
( )
x x
i
( )
2
2!
-------------------- f
2 ( )
x
i
( )
x x
i
( )
3
3!
-------------------- f
3 ( )
x
i
( ) + + + =
x x
i
( )
4
4!
-------------------- + f
4 ( )
x
i
( )
x x
i
( )
5
5!
-------------------- f
5 ( )
x
i
( )
x x
i
( )
6
6!
-------------------- f
6 ( )
x
i
( ) + + + +
CE 341/441 - Lecture 11 - Fall 2004
p. 11.3
For the present analysis we will consider only the rst four terms of the T.S. expansion
(may have to consider more)
where

If the Taylor series is convergent, each subsequent term in the error series should be
becoming smaller.
f x ( ) f x
i
( ) x x
i
( ) f
1 ( )
x
i
( )
x x
i
( )
2
2!
-------------------- f
2 ( )
x
i
( )
x x
i
( )
3
3!
-------------------- f
3 ( )
x
i
( ) E + + + + =
E
x x
i
( )
4
4!
-------------------- f
4 ( )
x
i
( )
x x
i
( )
5
5!
-------------------- f
5 ( )
x
i
( )
x x
i
( )
6
6!
-------------------- f
6 ( )
x
i
( ) + + + =
E
x x
i
( )
4
4!
-------------------- f
4 ( )
( ) = x
i
x
E
x x
i
( )
4
4!
-------------------- f
4 ( )
x
i
( )
E O x x
i
( )
4

CE 341/441 - Lecture 11 - Fall 2004

p. 11.4
The terms in the error series may be expressed
Exactly as
We note that the value of is not known
This single term exactly represents all the truncated terms in the Taylor series
Approximately as
This is the leading order truncated term in the series
This approximation for the error can also be thought of as being derived fromthe
exact single term representation of the error with the approximation
In terms of an order of magnitude only as
This termis often carried simply to ensure that all terms of the correct order have
been carried in the derivations.
This error term is indicative of how the error relatively depends on the size of the
interval!
E
x x
i
( )
4
4!
-------------------- f
4 ( )
( ) =

E
x x
i
( )
4
4!
-------------------- f
4 ( )
x
i
( )
f
4 ( )
( ) f
4 ( )
x
i
( )
E O x x
i
( )
4

CE 341/441 - Lecture 11 - Fall 2004

p. 11.5
Evaluate

Evaluate

f x
i 1 +
( )
f x
i 1 +
( ) f x
i
( ) x
i 1 +
x
i
( ) f
1 ( )
x
i
( )
x
i 1 +
x
i
( )
2
2!
----------------------------- f
2 ( )
x
i
( ) + + =
x
i 1 +
x
i
( )
3
3!
----------------------------- f
3 ( )
x
i
( ) O x
i 1 +
x
i
( )
4
+ +
f
i 1 +
f
i
h f
i
1 ( ) h
2
2
----- f
i
2 ( ) h
3
6
----- f
i
3 ( )
O h ( )
4
+ + + + =
f x
i 2 +
( )
f x
i 2 +
( ) f x
i
( ) x
i 2 +
x
i
( ) f
1 ( )
x
i
( )
x
i 2 +
x
i
( )
2
2!
----------------------------- f
2 ( )
x
i
( ) + + =
x
i 2 +
x
i
( )
3
3!
----------------------------- f
3 ( )
x
i
( ) O x
i 2 +
x
i
( )
4
+ +
f
i 2 +
f
i
2h f
i
1 ( )
2h
2
f
i
2 ( ) 4
3
---h
3
f
i
3 ( )
O h ( )
4
+ + + + =
CE 341/441 - Lecture 11 - Fall 2004
p. 11.6
Evaluate

Similarly we can evaluate

f x
i 1
( )
f x
i 1
( ) f x
i
( ) x
i 1
x
i
( ) f
1 ( )
x
i
( )
x
i 1
x
i
( )
2
2!
---------------------------- f
2 ( )
x
i
( ) + + =
x
i 1
x
i
( )
3
3!
---------------------------- + f
3
x
i
( ) O x
i 1
x
i
( )
4
+
f
i 1
f
i
h f
i
1 ( )

h
2
2
----- f
i
2 ( ) h
3
6
----- f
i
3 ( )
O h ( )
4
+ + =
f x
i 2
( )
f
i 2
f
i
2h f
i
1 ( )
2h
2
f
i
2 ( ) 4
3
---h
3
f
i
3 ( )
O h ( )
4
+ + =
CE 341/441 - Lecture 11 - Fall 2004
p. 11.7
Approximating Derivatives by Linearly Combining Functional Values at Nodes
Forward rst order accurate approximation to the rst derivative
Consider 2 nodes, and
Combine the difference of the functional values at these two nodes

i i 1 +
i i+1
f
i
f
i+1
f
i 1 +
f
i
f
i
h f
i
1 ( ) h
2
2
----- f
i
2 ( ) h
3
6
----- f
i
3 ( )
O h ( )
4
f
i
+ + + + =
h f
i
1 ( )
f
i 1 +
f
i

h
2
2
----- f
i
2 ( )

h
3
6
----- f
i
3 ( )
O h ( )
4
+ =
f
i
1 ( )
f
i 1 +
f
i

h
----------------------
h
2
--- f
i
2 ( ) h
2
6
----- f
i
3 ( )
O h ( )
3
+ =
CE 341/441 - Lecture 11 - Fall 2004
p. 11.8
First derivative of at node is approximated as
where
This is the rst forward difference and the error is called rst order in (i.e. )
Notes:
There is a clear dependence of the error on
The rst forward difference approximation is exact for 1
st
degree polynomials
f i
f
i
1 ( )
f
i 1 +
f
i

h
----------------------- E + = E
h
2
--- f
i
2 ( )

h E O h ( )
x
i
x
i+1
= x
i
+h
f
i
f
i+1
approximate slope f
i+1
- f
i
h
actual slope f
i
(1)
h
f(x)
h
CE 341/441 - Lecture 11 - Fall 2004
p. 11.9
Backward rst order accurate approximation to the rst derivative
Consider nodes and and dene

First backward difference of is then dened as:

Error is again rst order in
i 1 i f
i
f
i 1

f
i
f
i 1
f
i
f
i
h f
i
1 ( )

h
2
2
----- f
i
2 ( ) h
3
6
----- f
i
3 ( )
O h ( )
4
+ +


=
f
i
f
i 1
h f
i
1 ( ) h
2
2
----- f
i
2 ( )

h
3
6
----- f
i
3 ( )
O h ( )
4
+ + =
f
f
i
1 ( )
f
i
f
i 1

h
----------------------- E + =
h
E
1
2
---h f
i
2 ( )
O h ( ) =
CE 341/441 - Lecture 11 - Fall 2004
p. 11.10
Central second order accurate approximation to the rst derivative
Consider nodes , and and examine

Central difference approximation to the rst derivative is

Formula has an error which is second order in
i i 1 i 1 + f
i 1 +
f
i 1

f
i 1 +
f
i 1
f
i
h f
i
1 ( ) h
2
2
----- f
i
2 ( ) h
3
6
----- f
i
3 ( )
O h ( )
4
+ + + +


= f
i
h f
i
1 ( )

h
2
2
----- f
i
2 ( ) h
3
6
----- f
i
3 ( )
O h ( )
4
+ +

f
i 1 +
f
i 1
2h f
i
1 ( ) h
3
3
----- f
i
3 ( )
O h ( )
4
+ + =
f
i
1 ( )
f
i 1 +
f
i 1

2h
------------------------------ E + =
h
E
h
2
6
----- f
i
3 ( )
O h ( )
2
=
CE 341/441 - Lecture 11 - Fall 2004
p. 11.11
The smaller , the smaller the error
Error is obviously generally better for the central formula than the forward or
backward formulae!
Expression is exact for 2nd degree polynomials due to the third derivative in the expres-
sion for E
x
i-1
f
i-1
f
i+1
approximate slope f
i+1
- f
i-1
2h
actual slope f
i
(1)
f
i i+1
h
O h ( )
2
O h ( )
CE 341/441 - Lecture 11 - Fall 2004
p. 11.12
Strictly the order of the error is indicative of the rate of convergence as opposed to the
absolute error
1
1
1
2
log h
log(E)=log(f
i
(1)
- F.D. approx)
1
s
t

o
r
d
e
r
2
n
d

o
r
d
e
r
CE 341/441 - Lecture 11 - Fall 2004
p. 11.13
Forward rst order accurate approximation to the second derivative
Now consider nodes , and and the linear combination of functional values

Forward difference approximation to second derivative
Error rst order in
i i 1 + i 2 +
f
i 2 +
2 f
i 1 +
f
i
+
f
i 2 +
2 f
i 1 +
f
i
+ f
i
2h f
i
1 ( )
2h
2
f
i
2 ( ) 8
6
---h
3
f
i
3 ( )
O h ( )
4
+ + + +


=
2 f
i
h f
i
1 ( ) h
2
2
----- f
i
2 ( ) 1
6
-- -h
3
f
i
3 ( )
O h ( )
4
+ + + +


f
i
( ) +
f
i 2 +
2 f
i 1 +
f
i
+ h
2
f
i
2 ( )
h
3
f
i
3 ( )
= O h ( )
4
+
f
i
2 ( )
f
i 2 +
2 f
i 1 +
f
i
+
h
2
--------------------------------------------- E + =
h
E h f
i
3 ( )
O h ( ) = =
CE 341/441 - Lecture 11 - Fall 2004
p. 11.14
TABLE OF DIFFERENCE APPROXIMATIONS
First Derivative Approximations
Forward difference approximations:
,
,
,
Backward difference approximations:
,
,
,
f
i
1 ( )
f
i 1 +
f
i

h
---------------------- E + = E
1
2
---h f
i
2 ( )

f
i
1 ( )
f
i 2 +
4 f
i 1 +
3 f
i
+
2h
---------------------------------------------------- E + = E
1
3
---h
2
f
i
3 ( )

f
i
1 ( )
2 f
i 3 +
9 f
i 2 +
18 f
i 1 +
11 f
i
+
6h
---------------------------------------------------------------------------- E + = E
1
4
--- h
3
f
i
4 ( )

f
i
1 ( )
f
i
f
i 1

h
---------------------- E + = E
1
2
---h f
i
2 ( )

f
i
1 ( )
3 f
i
4 f
i 1
f
i 2
+
2h
----------------------------------------------- E + = E
1
3
---h
2
f
i
3 ( )

f
i
1 ( )
11 f
i
18 f
i 1
9 f
i 2
2 f
i 3
+
6h
----------------------------------------------------------------------------- E + = E
1
4
---h
3
f
i
4 ( )

CE 341/441 - Lecture 11 - Fall 2004

p. 11.15
Central difference approximations:
,
,
Second Derivative Approximations
Forward difference approximations
,
,
f
i
1 ( )
f
i 1 +
f
i 1

2h
----------------------------- E + = E
1
6
---h
2
f
i
3 ( )

f
i
1 ( )
f
i 2 +
8 f
i 1 +
8 f
i 1
f
i 2
+ +
12h
----------------------------------------------------------------------------- E + = E
1
30
------h
4
f
i
5 ( )

f
i
2 ( )
f
i 2 +
2 f
i 1 +
f
i
+
h
2
-------------------------------------------- E + = E h f
i
3 ( )

f
i
2 ( )
f
i 3 +
4 f
i 2 +
5 f
i 1 +
2 f
i
+ +
h
2
-------------------------------------------------------------------------- E + = E
11
12
------h
2
f
i
4 ( )

CE 341/441 - Lecture 11 - Fall 2004

p. 11.16
Backward difference approximations:
,
,
Central difference approximations:
,
,
All the derivative approximations we have examined are linear combinations of
functional values at nodes!!
What is a general technique for nding the associated coefcients?
f
i
2 ( )
f
i
2 f
i 1
f
i 2
+
h
2
------------------------------------------- E + = E h f
i
3 ( )

f
i
2 ( )
2 f
i
5 f
i 1
4 f
i 2
f
i 3
+
h
2
-------------------------------------------------------------------- E + = E
11
12
------h
2
f
i
4 ( )

f
i
2 ( )
f
i 1 +
2 f
i
f
i 1
+
h
2
-------------------------------------------- E + = E
1
12
------h
2
f
i
4 ( )

f
i
2 ( )
f
i 2 +
16 f
i 1 +
30 f
i
16 f
i 1
f
i 2
+ +
12h
2
---------------------------------------------------------------------------------------------------- E + = E
1
90
------h
4
f
i
6 ( )