file:nd&i.ppt p.

1
NUMERICAL DIFFERENTIATION
AND INTEGRATION
ENGR 351
Numerical Methods for Engineers
Southern Illinois University Carbondale
College of Engineering
Dr. L.R. Chevalier
Dr. B.A. DeVantier

file:nd&i.ppt p. 2
Numerical Differentiation and
Integration
Calculus is the mathematics of change.
Engineers must continuously deal with
systems and processes that change, making
calculus an essential tool of our profession.
At the heart of calculus are the related
mathematical concepts of differentiation and
integration.
file:nd&i.ppt p. 3
Differentiation
Dictionary definition of differentiate - to
mark off by differences, distinguish; ..to
perceive the difference in or between
Mathematical definition of derivative -
rate of change of a dependent variable
with respect to an independent variable
( ) ( )
x
x f x x f
x
y
i i
A
A +
=
A
A
file:nd&i.ppt p. 4
( ) ( ) A
A
A
A
y
x
f x x f x
x
i i
=
+
( ) f x
i
( ) f x x
i
+ A
Ay
Ax
f(x)
x
file:nd&i.ppt p. 5
Integration
The inverse process of differentiation
Dictionary definition of integrate - to bring
together, as parts, into a whole; to unite; to
indicate the total amount
Mathematically, it is the total value or
summation of f(x)dx over a range of x. In fact
the integration symbol is actually a stylized
capital S intended to signify the connection
between integration and summation.
file:nd&i.ppt p. 6
f(x)
x
( ) I f x dx
a
b
=
}
file:nd&i.ppt p. 7
Mathematical Background
d
dx
x
d
dx
e
d
dx
x
d
dx
a
d
dx
x
d
dx
x
d
dx
x
if u and v are functions of x
d
dx
u
d
dx
uv
x
x
n
n
sin ? ?
cos ? ?
tan ? ?
ln ?
? ( ) ?
= =
= =
= =
=
= =
file:nd&i.ppt p. 8
udv
u du
a dx
dx
x
e dx
n
bx
ax
=
=
=
=
=
}
}
}
}
}
?
?
?
?
?
Mathematical Background
file:nd&i.ppt p. 9
Overview
Newton-Cotes Integration Formulas
Trapezoidal rule
Simpsons Rules
Unequal Segments
Open Integration
Integration of Equations
Romberg Integration
Gauss Quadrature
Improper Integrals
file:nd&i.ppt p. 10
Overview
Numerical Differentiation
High accuracy formulas
Richardsons extrapolation
Unequal spaced data
Uncertain data
Applied problems
file:nd&i.ppt p. 11
Specific Study Objectives
Understand the derivation of the
Newton-Cotes formulas
Recognize that the trapezoidal and
Simpsons 1/3 and 3/8 rules represent
the areas of 1st, 2nd, and 3rd order
polynomials
Be able to choose the best among
these formulas for any particular
problem

file:nd&i.ppt p. 12
Specific Study Objectives
Recognize the difference between open
and closed integration formulas
Understand the theoretical basis of
Richardson extrapolation and how it is
applied in the Romberg integration
algorithm and for numerical
differentiation


file:nd&i.ppt p. 13
Specific Study Objectives
Recognize why both Romberg
integration and Gauss quadrature have
utility when integrating equations (as
opposed to tabular or discrete data).
Understand the application of high-
accuracy numerical-differentiation.
Recognize data error on the processes
of integration and differentiation.
file:nd&i.ppt p. 14
Newton-Cotes Integration
Common numerical integration scheme
Based on the strategy of replacing a complicated
function or tabulated data with some approximating
function that is easy to integrate
file:nd&i.ppt p. 15
Newton-Cotes Integration
Common numerical integration scheme
Based on the strategy of replacing a complicated
function or tabulated data with some approximating
function that is easy to integrate
( ) ( )
( )
I f x dx f x dx
f x a a x a x
a
b
n
a
b
n n
n
= ~
= + + +
} }
0 1
....
file:nd&i.ppt p. 16
Newton-Cotes Integration
Common numerical integration scheme
Based on the strategy of replacing a complicated
function or tabulated data with some approximating
function that is easy to integrate
( ) ( )
( )
I f x dx f x dx
f x a a x a x
a
b
n
a
b
n n
n
= ~
= + + +
} }
0 1
.... f
n
(x) is an nth order
polynomial
file:nd&i.ppt p. 17
The approximation of an integral by the area under
- a first order polynomial
- a second order polynomial
We can also approximated the integral by using a
series of polynomials applied piece wise.
0
1
2
3
4
5
0 5 10
x
f
(
x
)
0
1
2
3
4
5
0 5 10
x
f
(
x
)
file:nd&i.ppt p. 18
0
1
2
3
4
5
0 1 2 3 4 5 6 7 8 9 10
x
f
(
x
)
An approximation of an integral by the area under straight
line segments.
file:nd&i.ppt p. 19
Newton-Cotes Formulas
Closed form - data is at the beginning and
end of the limits of integration
Open form - integration limits extend
beyond the range of data.
0
1
2
3
4
5
0 1 2 3 4 5 6 7 8 9 10
x
f
(
x
)
0
1
2
3
4
5
0 1 2 3 4 5 6 7 8 9 10
x
f
(
x
)
file:nd&i.ppt p. 20
Trapezoidal Rule
First of the Newton-Cotes closed
integration formulas
Corresponds to the case where the
polynomial is a first order
( ) ( )
( )
I f x dx f x dx
f x a a x
a
b
a
b
n
= ~
= +
} }
1
0 1
file:nd&i.ppt p. 21
( ) ( )
( )
I f x dx f x dx
f x a a x
a
b
a
b
n
= ~
= +
} }
1
0 1
A straight line can be represented as:
( ) ( )
( ) ( )
( ) f x f a
f b f a
b a
x a
1
= +

file:nd&i.ppt p. 22
( ) ( )
( )
( ) ( )
( )
I f x dx f x dx
f a
f b f a
b a
x a dx
a
b
a
b
a
b
= ~
= +

} }
}
1
Integrate this equation. Results in the trapezoidal rule.
( )
( ) ( )
I b a
f a f b
~
+
2
file:nd&i.ppt p. 23
( )
( ) ( )
I b a
f a f b
~
+
2
The concept is the same but the trapezoid is on its
side.
h
e
i
g
h
t

base
base
width
h
e
i
g
h
t

h
e
i
g
h
t

file:nd&i.ppt p. 24
Error of the Trapezoidal Rule
( )( ) E f b a
where a b
t
=
< <
1
12
3
' '

This indicates that is the function being integrated is

linear, the trapezoidal rule will be exact.

Otherwise, for section with second and higher order
derivatives (that is with curvature) error can occur.

A reasonable estimate of x is the average value of
b and a
file:nd&i.ppt p. 25
Multiple Application of the
Trapezoidal Rule
Improve the accuracy by dividing the
integration interval into a number of
smaller segments
Apply the method to each segment
Resulting equations are called multiple-
application or composite integration
formulas
file:nd&i.ppt p. 26
( ) ( ) ( ) ( ) ( ) ( )
I f x dx f x dx f x dx
I h
f x f x
h
f x f x
h
f x f x
x
x
x
x
x
x
n n
n
n
= + + +
~
+
+
+
+
+
} } }

( ) ( ) ( )
0
1
1
2
1
0 1 1 2 1
2 2 2

Multiple Application of the

Trapezoidal Rule
where there are n+1 equally spaced base points.
file:nd&i.ppt p. 27
( ) ( ) ( ) ( ) ( ) ( )
( )
( ) ( ) ( )
I f x dx f x dx f x dx
I h
f x f x
h
f x f x
h
f x f x
I b a
f x f x f x
n
x
x
x
x
x
x
n n
i n
i
n
n
n
= + + +
~
+
+
+
+
+
~
+ +
} } }

( ) ( ) ( )
0
1
1
2
1
0 1 1 2 1
0
1
1
2 2 2
2
2

We can group terms to express a general form

}

}

width average height
file:nd&i.ppt p. 28
( )
( ) ( ) ( )
I b a
f x f x f x
n
i n
i
n
~
+ +
=

0
1
1
2
2 }

}

width average height
The average height represents a weighted average
of the function values

Note that the interior points are given twice the weight
of the two end points
( )
E
b a
n
f
a
=

3
2
12
' '
file:nd&i.ppt p. 29
EXAMPLE
Evaluate the following integral using the
trapezoidal rule and h = 0.1
( )
( ) ( ) ( )
I b a
f x f x f x
n
i n
i
n
~
+ +
=

0
1
1
2
2
I e dx
x
=
}
2
1
1 6 .
Example Problem
0
10
20
30
40
50
60
0 0.5 1 1.5 2
x
f
(
x
)
h
b a
n
=

file:nd&i.ppt p. 30
Simpsons 1/3 Rule
Corresponds to the case where the
function is a second order polynomial
( ) ( )
( )
I f x dx f x dx
f x a a x a x
a
b
a
b
n
= ~
= + +
} }
2
0 1 2
2
file:nd&i.ppt p. 31
Simpsons 1/3 Rule
Designate a and b as x
0
and x
2
, and
estimate f
2
(x) as a second order
Lagrange polynomial
( ) ( )
( )( )
( )( )
( )
I f x dx f x dx
x x x x
x x x x
f x dx
a
b
a
b
x
x
= ~
=


+

(
} }
}
2
1 2
0 1 0 2
0
0
2
.......
file:nd&i.ppt p. 32
Simpsons 1/3 Rule
After integration and algebraic
manipulation, we get the following
equations
( ) ( ) ( )
| |
( )
( ) ( ) ( )
I
h
f x f x f x
b a
f x f x f x
~ + +
~
+ +
3
4
4
6
0 1 2
0 1 2
}

}

width average height
file:nd&i.ppt p. 33
Error
( )( ) E f b a
where a b
t
=
< <
1
12
3
' '

Single application of Trapezoidal Rule.

Single application of Simpsons 1/3 Rule
( )( ) E f b a
t
=
1
2880
4
5
( )

file:nd&i.ppt p. 34
Multiple Application of
Simpsons 1/3 Rule
( )
( ) ( )
( )
( )
( )
( )
I f x dx f x dx f x dx
I b a
f x f x f x f x
n
E
b a
n
f
x
x
x
x
x
x
i j n
j
n
i
n
a
n
n
= + + +
~
+ + +
=

} } }


=

( ) ( ) ( )
, , .. , , ..
0
1
1
2
1
0
2 4 6
2
1 3 5
1
5
4
4
4 2
3
180

file:nd&i.ppt p. 35
( )
( ) ( )
( )
( )
I b a
f x f x f x f x
n
i j n
j
n
i
n
~
+ + +
=

0
2 4 6
2
1 3 5
1
4 2
3
, , .. , , ..
The odd points represent the middle term for each
application. Hence carry the weight 4
The even points are common to adjacent
applications and are counted twice.
file:nd&i.ppt p. 36
Simpsons 3/8 Rule
Corresponds to the case where the
function is a third order polynomial
( ) ( )
( )
( ) ( ) ( ) ( )
| |
I f x dx f x dx
f x a a x a x a x
I
h
f x f x f x f x
a
b
a
b
n
= ~
= + + +
~ + + +
} }
3
0 1 2
2
3
3
0 1 2 3
3
8
3 3
file:nd&i.ppt p. 37
Integration of Unequal
Segments
Experimental and field study data is often
unevenly spaced
In previous equations we grouped the term
(i.e. h
i
) which represented segment width.
( )
( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )
I b a
f x f x f x
n
I h
f x f x
h
f x f x
h
f x f x
i n
i
n
n n
~
+ +
~
+
+
+
+
+
=

0
1
1
0 1 1 2 1
2
2
2 2 2

file:nd&i.ppt p. 38
Integration of Unequal
Segments
We should also consider alternately using
higher order equations if we can find data in
consecutively even segments
trapezoidal
rule
1/3
rule
3/8
rule
trapezoidal
rule
file:nd&i.ppt p. 39
EXAMPLE
Integrate the following using the trapezoidal rule,
Simpsons 1/3 Rule and a multiple application of
the trapezoidal rule with n=2. Compare results with
the analytical solution.
xe dx
x 2
0
4
}
0
5000
10000
15000
20000
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
f
(
x
)
file:nd&i.ppt p. 40
0
5000
10000
15000
20000
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
f
(
x
)
Simpsons 1/3 Rule

f(2) = 109.196
( ) ( ) ( )
| |
( )
( ) ( ) ( )
( )
( ) ( )
I
h
f x f x f x
b a
f x f x f x
t
~ + +
~
+ +
~
+ +
=
=

=
3
4
4
6
4 0
0 4 109196 1192383
6
8240 41
5216 93 8240 41
5216 93
100 57 96%
0 1 2
0 1 2
. .
.
. .
.
. c
file:nd&i.ppt p. 41
( )
( ) ( ) ( )
( )
( )
( )
I b a
f x f x f x
n
i n
i
n
t
~
+ +
~
+ +
=
=

=
=

0
1
1
2
2
4 0
0 2 109196 1192383
2 2
12142 22
5216 93 12142 22
5216 93
100 133%
. .
.
. .
.
c
Multiple Application of
the Trapezoidal Rule
We are obviously not doing
very well on our estimates.
Lets consider a scheme
where we weight the
estimates
....end of example
0
5000
10000
15000
20000
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
f
(
x
)
file:nd&i.ppt p. 42
Integration of Equations
Integration of analytical as opposed to
tabular functions
Romberg Integration
Richardsons Extrapolation
Romberg Integration Algorithm
Gauss Quadrature
Improper Integrals
file:nd&i.ppt p. 43
Richardsons Extrapolation
Use two estimates of an integral to compute a third
more accurate approximation
The estimate and error associated with a multiple
application trapezoidal rule can be represented
generally as:
I = I(h) + E(h)
where I is the exact value of the integral
I(h) is the approximation from an n-segment
application
E(h) is the truncation error
h is the step size (b-a)/n
file:nd&i.ppt p. 44
Make two separate estimates using step sizes
of h
1
and h
2
.

I(h
1
) + E(h
1
) = I(h
2
) + E(h
2
)

Recall the error of the multiple-application of the trapezoidal
rule
E
b a
h f =

12
2
' '
Assume that is constant regardless of the step size
f ' '
( )
( )
E h
E h
h
h
1
2
1
2
2
2
~
file:nd&i.ppt p. 45
( )
( )
( ) ( )
E h
E h
h
h
E h E h
h
h
1
2
1
2
2
2
1 2
1
2
2
~
~
|
\

|
.
|
Substitute into previous equation:

I(h
1
) + E(h
1
) = I(h
2
) + E(h
2
)
( )
( ) ( )
E h
I h I h
h
h
2
1 2
1
2
2
1
=

|
\

|
.
|
file:nd&i.ppt p. 46
Thus we have developed an estimate of the truncation error in
terms of the integral estimates and their step sizes. This
estimate can then be substituted into:

I = I(h
2
) + E(h
2
)

to yield an improved estimate of the integral:
( ) ( ) ( )
| |
I I h
h
h
I h I h ~ +
|
\

|
.
|

(
(
(

2
1
2
2 2 1
1
1
( )
( ) ( )
E h
I h I h
h
h
2
1 2
1
2
2
1
=

|
\

|
.
|
file:nd&i.ppt p. 47
( ) ( ) ( )
| |
I I h
h
h
I h I h ~ +
|
\

|
.
|

(
(
(

2
1
2
2 2 1
1
1
What is the equation for the special case
where the interval is halved?

i.e. h
2
= h
1
/ 2
file:nd&i.ppt p. 48
( ) ( ) ( )
| |
( ) ( )
h
h
h
h
I I h I h I h
collecting terms
I I h I h
1
2
2
2
2
2
2 1
2 1
2
2
1
2 1
4
3
1
3
|
\

|
.
|
=
|
\

|
.
|
=
~ +

(

~
file:nd&i.ppt p. 49
EXAMPLE
Use Richardsons extrapolation to evaluate:
xe dx
x 2
0
4
}
0
5000
10000
15000
20000
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
f
(
x
)
file:nd&i.ppt p. 50
( ) ( )
( ) ( )
( ) ( )
I I h I h
I I h I h
I I h I h
I
I I
m l
m l
j k
k
j k j k
k
~
~
~
~

4
3
1
3
16
15
1
15
64
63
1
63
4
4 1
2 1
1
1 1 1
1
,
, ,
We can continue to improve the estimate by successive
halving of the step size to yield a general formula:
k = 2; j = 1
ROMBERG INTEGRATION
Note:
the subscripts
m and l refer to
more and less
accurate estimates
file:nd&i.ppt p. 51
Error orders for j values
i = 1 i = 2 i = 3 i = 4
j O(h
2
) O(h
4
) O(h
6
) O(h
8
)

1 h I
1,1
I
1,2
I
1,3
I
1,4


2 h/2 I
2,1
I
2,2
I
2,3


3 h/4 I
3,1
I
3,2


4 h/8 I
4,1


Trapezoidal Simpsons 1/3 Simpsons 3/8 Booles

Rule Rule Rule Rule

( ) ( )

h I
63
1
h I
63
64
I
m
~
Following a similar pattern to Newton divided
differences, Rombergs Table can be produced
file:nd&i.ppt p. 52
Gauss Quadrature
f(x)
x
f(x)
x
Extend the area
under the straight
line
file:nd&i.ppt p. 53
Method of Undetermined
Coefficients
Recall the trapezoidal rule
( )
( ) ( )
I b a
f a f b
~
+
2
This can also be expressed as
( ) ( ) I c f a c f b ~ +
0 1
where the cs are constant
Before analyzing
this method,
answer this
question.
What are two
functions that
should be evaluated
exactly
by the trapezoidal
rule?
file:nd&i.ppt p. 54
The two cases that should be evaluated exactly
by the trapezoidal rule: 1) y = constant
2) a straight line
f(x)
x
y = 1
(b-a)/2
-(b-a)/2
f(x)
x
y = x
(b-a)/2
-(b-a)/2
file:nd&i.ppt p. 55
Thus, the following equalities should hold.
( ) ( )
( )
( )
( )
( )
I c f a c f b
c c dx
c
b a
c
b a
xdx
b a
b a
b a
b a
~ +
+ ~

|
\

|
.
|
+

|
\

|
.
|
~

}
}
0 1
0 1
2
2
0 1
2
2
1
2 2
FOR y=1
since f(a) = f(b) =1
FOR y =x
since f(a) = x =-(b-a)/2
and
f(b) = x =(b-a)/2
file:nd&i.ppt p. 56
Evaluating both integrals
c c b a
c
b a
c
b
0 1
0 1
2
1
2
0
+ =

+

=
For y = 1


For y = x
Now we have two equations and two unknowns, c
0
and c
1
.

Solving simultaneously, we get :

c
0
= c
1
= (b-a)/2

Substitute this back into:

( ) ( ) I c f a c f b ~ +
0 1
file:nd&i.ppt p. 57
( )
( ) ( )
I b a
f a f b
~
+
2
We get the equivalent of the trapezoidal rule.
DERIVATION OF THE TWO-POINT
GAUSS-LEGENDRE FORMULA


( ) ( ) I c f x c f x ~ +
0 0 1 1
Lets raise the level of sophistication by:
- considering two points between -1 and 1
- i.e. open integration
file:nd&i.ppt p. 58
f(x)
x
-1 x
0
x
1
1
Previously ,we assumed that the equation fit the
integrals of a constant and linear function.

Extend the reasoning by assuming that it also fits
the integral of a parabolic and a cubic function.
file:nd&i.ppt p. 59
( ) ( ) ( ) ( )
( ) ( )
( ) ( )
( ) ( )
c f x c f x dx c c
c f x c f x xdx c x c x
c f x c f x x dx c x c x
c f x c f x x dx c x c x
0 0 1 1
1
1
0 1
0 0 1 1
1
1
0 0 1 1
0 0 1 1
2
1
1
0 0
2
1 1
2
0 0 1 1
3
1
1
0 0
3
1 1
3
1 2 1 1 2
0 0
2 3 2 3
0 0
+ = = + =
+ = = + =
+ = = + =
+ = = + =

}
}
}
}
`
/ /
Solve these equations simultaneously
f(x
i
) is either 1, x
i
, x
i
2
or x
i
3
file:nd&i.ppt p. 60
c c
x
x
I f f
0 1
0
1
1
1
3
1
3
1
3
1
3
= =
=

=
~

|
\

|
.
|
+
|
\

|
.
|
This results in the following
The interesting result is that the simple addition
of the function values at
1
3
1
3
and
file:nd&i.ppt p. 61
However, we have set the limit of integration at -1 and 1.

This was done to simplify the mathematics. A simple
change in variables can be use to translate other limits.

Assume that the new variable x
d
is related to the
original variable x in a linear fashion.

x = a
0
+ a
1
x
d


Let the lower limit x = a correspond to x
d
= -1 and the upper
limit x=b correspond to x
d
=1

a = a
0
+ a
1
(-1) b = a
0
+ a
1
(1)

file:nd&i.ppt p. 62
a = a
0
+ a
1
(-1) b = a
0
+ a
1
(1)
SOLVE THESE EQUATIONS
SIMULTANEOUSLY
a
b a
a
b a
0 1
2 2
=
+
=

( ) ( )
x a a x
b a b a x
d
d
= + =
+ +
0 1
2
substitute
file:nd&i.ppt p. 63
( ) ( )
x a a x
b a b a x
dx
b a
dx
d
d
d
= + =
+ +
=

0 1
2
2
These equations are substituted for x and dx respectively.

Lets do an example to appreciate the theory
behind this numerical method.
file:nd&i.ppt p. 64
EXAMPLE
Estimate the following using two-point Gauss Legendre:
xe dx
x 2
0
4
}
0
5000
10000
15000
20000
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
f
(
x
)
file:nd&i.ppt p. 65
Higher-Point Formulas
( ) ( ) ( )
I c f x c f x c f x
n n
~ + + +
0 0 1 1 1 1

For two point, we determined that c

0
=c
1
= 1

For three point:

c
0
= 0.556 x
0
=-0.775
c
1
= 0.889 x
1
=0.0
c
2
= 0.556 x
2
=0.775


file:nd&i.ppt p. 66
Higher-Point Formulas
Your text goes on to provide additional weighting
factors (c
i
s) and function arguments (x
i
s)
in Table 22.1 p. 623.
file:nd&i.ppt p. 67
Improper Integrals
( )
f x dx
b

}
How do we deal with
integrals that do not
have finite limits
and bounded
integrands?

Our answer will focus
on improper integrals
where one limit (upper or
lower) is infinity.
file:nd&i.ppt p. 68
( ) ( ) ( ) f x dx f x dx f x dx
b A
A
b

} } }
= +
Choose -A so that is is sufficiently large to
approach zero asymptotically.

Once chosen, evaluate the second part using a
Newton-Cotes formula.

Evaluate the first part with the following identity.
( ) f x dx
t
f
t
dt for ab
a
b
b
a
} }
=
|
\

|
.
|
>
1 1
0
2
1
1
file:nd&i.ppt p. 69
Numerical Differentiation
Forward finite divided difference
Backward finite divided difference
Center finite divided difference
All based on the Taylor Series
( ) ( ) ( )
( )
.........
! 2
' '
'
2
1
+ + + =
+
h
x f
h x f x f x f
i
i i i
file:nd&i.ppt p. 70
Forward Finite Difference
( ) ( ) ( )
( )
( )
( ) ( )
( )
( )
( ) ( ) ( )
( )
2
1
1
2
1
2
' '
'
'
.........
! 2
' '
'
h O h
x f
h
x f x f
x f
h O
h
x f x f
x f
h
x f
h x f x f x f
i i i
i
i i
i
i
i i i
+

=
+

=
+ + + =
+
+
+
file:nd&i.ppt p. 71
Forward Divided Difference
( )
( ) ( )
( ) ( ) f x
f x f x
x x
O x x
f
h
O h
i
i i
i i
i i
i
' =

+ = +
+
+
+
1
1
1
A
f(x)
x
(x
i
, y
i
)
(x
i+1
,y
i+1
)
file:nd&i.ppt p. 72
( ) ( ) f x
f
h
O h
i
i
' = +
A
first forward divided difference
Error is proportional to
the step size
O(h
2
) error is proportional to the square of the step size

O(h
3
) error is proportional to the cube of the step size
file:nd&i.ppt p. 73
f(x)
x
(x
i
,y
i
)
(x
i-1
,y
i-1
)
file:nd&i.ppt p. 74
( ) ( ) ( )
( )
( ) ( ) ( )
( )
( )
( ) ( )
f x f x f x h
f x
h
f x f x f x h
f x
h
f x
f x f x
h
f
h
i i i
i
i i i
i
i
i i i
+

= + + +
= + +
=

=
V
1
2
1
2
1
2
2
'
' '
!
......
'
' '
!
.....
'
Backward Difference Approximation of the
First Derivative

Expand the Taylor series backwards
The error is still O(h)
file:nd&i.ppt p. 75
Centered Difference Approximation of the
First Derivative

Subtract backward difference approximation
from forward Taylor series expansion
( ) ( ) ( )
( )
( )
( ) ( )
( ) ( ) ( )
( )
( )
( ) ( )
( )
2 1 1
2
1 1
1
1
2
1
2
'
6
' ' '
' 2
'
....
! 2
' '
'
h O
h
x f x f
x f
h
x f
h x f x f x f
x x
x f x f
x f
h
x f
h x f x f x f
i i
i
i
i i i
i i
i i
i
i
i i i

=
+ + + =

=
+ + + =
+

+
+
+

file:nd&i.ppt p. 76
f(x)
x
(x
i
,y
i
)
(x
i-1
,y
i-1
)
(x
i+1
,y
i+1
)
( )
( ) ( )
( )
2
1 1
2
' h O
h
x f x f
x f
i i
i

=
+
file:nd&i.ppt p. 77
f(x)
x
f(x)
x
f(x)
x
f(x)
x
true derivative
forward
finite divided
difference approx.
backward
finite divided
difference approx.
centered
finite divided
difference approx.
file:nd&i.ppt p. 78
Numerical Differentiation
Forward finite divided differences Fig.
23.1
Backward finite divided differences Fig.
23.2
Centered finite divided differences Fig.
23.3
First - Fourth derivative
file:nd&i.ppt p. 79
Richardson Extrapolation
Two ways to improve derivative estimates
decrease step size
use a higher order formula that employs more
points
Third approach, based on Richardson
extrapolation, uses two derivatives estimates
to compute a third, more accurate
approximation
file:nd&i.ppt p. 80
Richardson Extrapolation
( ) ( ) ( )
| |
( ) ( )
( ) ( )
I I h
h
h
I h I h
Special case where h
h
I I h I h
In a similar fashion
D D h D h
~ +
|
\

|
.
|


=
~
~
2
1
2
2 2 1
2
1
2 1
2 1
1
1
2
4
3
1
3
4
3
1
3
For a centered difference
approximation with
O(h
2
) the application of
this formula will yield
a new derivative estimate
of O(h
4
)
file:nd&i.ppt p. 81
EXAMPLE
Given the following function, use Richardsons
extrapolation to determine the derivative at 0.5.

f(x) = -0.1x
4
- 0.15x
3
- 0.5x
2
- 0.25x +1.2

Note:

f(0) = 1.2
f(0.25) =1.1035
f(0.75) = 0.636
f(1) = 0.2

file:nd&i.ppt p. 82
Derivatives of Unequally
Spaced Data
Common in data from experiments or field studies
Fit a second order Lagrange interpolating polynomial
to each set of three adjacent points, since this
polynomial does not require that the points be
equispaced
Differentiate analytically
( ) ( )
( )( )
( )
( )( )
( )
( )( )
f x f x
x x x
x x x x
f x
x x x
x x x x
f x
x x x
x x x x
i
i i
i i i i
i
i i
i i i i
i
i i
i i i i
' =


+


+

+
+
+
+
+

+ +
1
1
1 1 1
1 1
1 1
1
1
1 1 1
2 2
2
file:nd&i.ppt p. 83
Derivative and Integral
Estimates for Data with Errors
In addition to unequal spacing, the other problem
related to differentiating empirical data is
measurement error
Differentiation amplifies error
Integration tends to be more forgiving
Primary approach for determining derivatives of
imprecise data is to use least squares regression to
fit a smooth, differentiable function to the data
In absence of other information, a lower order
polynomial regression is a good first choice
file:nd&i.ppt p. 84
0
100
200
0 10 20
t
y
0
10
20
30
0 5 10 15
t
d
y
/
d
t
0
50
100
150
200
0 5 10 15
t
y
0
10
20
30
40
0 5 10 15
t
d
y
/
d
t