5

Overview of Discretization
Methods
5.1 Finite-Dierence Method
5.1.1 Basic Ideas: Collocation and Truncation
The basic idea underlying the nite-dierence method is two-fold:
1. At each discrete location at which the governing equation is to be solved
numerically, replace the spatial derivatives by discrete derivatives. Taking
the Burgers equation as an example, we can write the semi-discrete equation
at the location x
0
as
du
dt

x0
+u
x0
u
x

x0
= 0. (5.1)
Note that we require the semi-discrete equation to be satised exactly.
2. Discrete derivatives are approximated by divided dierences of the function Although the innitesimal
calculus has been a splendid
success, yet there remain
problems in which it is
cumbrous or unworkable.
When such diculties are
encountered it may be well
to return to the manner in
which they did things before
the calculus was invented,
postponing the passage to
the limit until after the
problem had been solved for
a moderate number of
moderately small
dierences.
L.F. Richardson, 1927
evaluated at discrete locations. This idea is simply based on the denition
of the x-derivative of a function (x) at x
0
, namely,
d
dx

x0
= lim
x0
(x
0
+ x) (x
0
)
x
. (5.2)
The nite-dierence method approximates the derivative by foregoing the
limiting process and evaluating the right-hand side of equation (5.2) at a
nite value of x,

x0
=
(x
0
+ x) (x
0
)
x
. (5.3)
The lack of the limiting process causes the so-called truncation error, which
is discussed in more detail below.
It is clear from the above that the precise manner in which the discrete deriva-
tives are computed is the key to the accuracy of the nite-dierence method.
Before continuing to show how various nite-dierence approximations can be
derived, we provide a preliminary discussion of the accuracy of nite-dierence
approximations. More detailed discussions are presented in section 6.3.
We can expect that the derivative will be approximated accurately for a
small but nite value of x provided that (x) is suciently smooth in the
neighborhood of x
0
. In fact, the mean-value theorem of calculus tells us that the
approximation will be exact at some point within the interval [x
0
, x
0
+ x].
29
30 Overview of Discretization Methods
x
s s s s E
' E
i i + 1 x i i 1
s
Figure 5.1: Finite-dierence grid in one dimension.
The vague statements about the accuracy of the approximation for small but
nite x can be made somewhat more precise by using Taylor series. For example,
the Taylor-series expansion for (x
0
+ x) about x
0
is
(x
0
+ x) = (x
0
) + x
d
dx

x0
+
(x)
2
2
d
2

dx
2

x0
+
(x)
3
3!
d
3

dx
3

x0
+. . . . (5.4)
Hence we can write
d
dx

x0
=
(x
0
+ x) (x
0
)
x

_
x
2
d
2

dx
2

x0
+
(x)
2
3!
d
3

dx
3

x0
+. . .
_
. (5.5)
At this stage, it is convenient to identify locations such as x
0
and (x
0
+x) with
vertices on a grid such as that depicted in gure 5.1. For simplicity, we assume
that the grid has constant spacing x. Hence each vertex can be addressed by
an index i and the location x
0
can be expressed as (ix). Equation (5.5) can
now be written
d
dx

i
=

i+1

i
x

_
x
2
d
2

dx
2

i
+
(x)
2
3!
d
3

dx
3

i
+. . .
_
. (5.6)
We recognize the rst term on the right-hand side as the nite-dierence approx-
imation given by equation (5.3). Written in terms of the grid point locations, this
so-called forward-dierence approximation can be expressed as

i
=

i+1

i
x
. (5.7)
Equation (5.7) can be interpreted geometrically as shown in gure 5.2(a). With
equation (5.7), equation (5.6) can be written as
d
dx

i
=

x

_
x
2
d
2

dx
2

i
+
(x)
2
3!
d
3

dx
3

i
+. . .
_
. (5.8)
Hence we see that the derivative and its nite-dierence approximation dier by
an innite series of terms. This innite series of terms is the truncation error
(te) alluded to above. The designation truncation error is motivated by the
fact that, when the derivative is simply replaced by the discrete derivative, we
are tacitly truncating after the rst term on the right-hand side and incurring an
error in doing so.
Each term in the te consists of products of fractions, powers of the grid
spacing, and higher-order derivatives of the solution evaluated at the location
where the derivative is approximated. In the limit of x 0, the leading term of
the te dominates subsequent terms if the function being approximated is smooth
and we can write equation (5.6) as
d
dx

i
=

i+1

i
x
+O(x). (5.9)
Id
5.1. Finite-Difference Method 31
i 1 i i + 1
(x)

i1

i+1
ii1
x
i+1i
x
(a) Forward- and backward-dierence approximations.
i 1 i i + 1
(x)

i1

i+1
i+1i1
2x
(b) Central-dierence approximation.
Figure 5.2: Geometric interpretation of nite-dierence approxima-
tions to rst derivative.
The power of the grid spacing in the leading term of the te is called the order of
accuracy of the approximation or simply the order of approximation. Accordingly,
the forward-dierence approximation is a rst-order accurate approximation of
the rst derivative. It is important to note that the notation O(x) does not
quantify the exact value of the te, but only its functional dependence as x 0.
Note that the order of the derivative being approximated, the order of accuracy
of the approximation, and the order of the derivative of the leading term in the
truncation error are related by dimensional arguments.
The order of accuracy of a nite-dierence approximation as dened above
is also referred to as the nominal order of accuracy. This term is used to
distinguish between the theoretical order of accuracy as specied when deriving
a given nite-dierence approximation and the order of accuracy achieved in
practice, the so-called actual order of accuracy. The determination of the actual
order of accuracy is described in sections 6.3.1 and 6.3.3.
Id
32 Overview of Discretization Methods
5.1.2 Explicit Finite-Dierence Approximations
Equation (5.7) is termed an explicit nite-dierence approximation because the
derivative can be computed explicitly in terms of the function values. In sec-
tion 5.1.3, we will discuss implicit nite-dierence approximations in which the
derivative at a given point depends on derivatives at other points.
The approximation given by equation (5.7) is not the only possibility for com-
puting the rst derivative. By considering the Taylor series
(x
0
x) = (x
0
) x
d
dx

x0
+
(x)
2
2
d
2

dx
2

x0
. . . (5.10)
we can derive the rst-order accurate backward-dierence approximation

i
=

i

i1
x
. (5.11)
Similarly, we can combine equations (5.4) and (5.10) to eliminate (x
0
, y
0
) and
obtain the second-order accurate central-dierence approximation,

i
=

i+1

i1
2x
. (5.12)
Equations (5.11) and (5.12) can be interpreted geometrically as shown in g-
ure 5.2(a) and gure 5.2(b). By including more points, i.e., Taylor-series ex-
pansions for (x
0
2x), for example, higher-order approximations to the rst
derivative can be derived.
Approximations for higher-order derivatives can also be derived. For example,
adding equations (5.4) and (5.10) leads to the following second-order accurate
approximation

x
2

i
=

i+1
2
i
+
i1
(x)
2
. (5.13)
Higher-order approximations for second derivatives can be derived similarly.
5.1.2.1 Derivation of Finite-Dierence Approximations
In the above, Taylor series were employed to derive nite-dierence approxima-
tions. Several other methods can be used to derive nite-dierence approxima-
tions as detailed below. For simplicity, we assume uniform grid spacing, but these
methods are applicable to arbitrary variations of the grid spacing. We begin by
describing the Taylor-series method in a more formal setting.
Taylor Series. The derivation of a nite-dierence approximation to a given
derivative

i
=
d
dx

i
+ TE (5.14)
starts by specifying the functional form of the approximation, i.e.,

i
=
r

j=l

i+j
. (5.15)
Note that we specify which grid points are included in the stencil, i.e., which
values contribute to the approximation. The goal is to compute the weights
j
such that the approximation attains a specied order of accuracy. Dimensional We shall see later, e.g., in
section 9.1.5, that attaining
a specic order of accuracy
is not always the most
appropriate goal and that
accuracy can be measured
in more ways than just
through the order of
accuracy.
considerations indicate that the weights must have dimensions of inverse length,
i.e.,
j
(x)
1
.
Id
5.1. Finite-Difference Method 33
To make these ideas more specic, we rst consider a specic example and then
proceed to the general case. Let us nd a second-order accurate approximation for
(d/dx)
i
using
i
,
i+1
, and
i+2
, i.e., we put l = 0 and r = 2 in equation (5.15)
to get

i
=
2

j=0

i+j
=
0

i
+
1

i+1
+
2

i+2
. (5.16)
Next, we use Taylor-series expansions to express each term on the right-hand
side of the approximation given by equation (5.15) at the same grid point at which
the left-hand side is evaluated. Expanding
i+j
about x
i
gives

i+j
=
i
+

k=1
(jx)
k
k!
d
k

dx
k

i
, (5.17)
where we made use of the assumption of uniform grid spacing so that x
i+j
x
i
=
jx. Therefore, we can rewrite equation (5.16) as

i
=
0

i
+
1
_

i
+ x
d
dx

i
+
1
2
(x)
2
d
2

dx
2

i
+
1
3!
(x)
3
d
3

dx
3

i
+. . .
_
+
2
_

i
+ 2x
d
dx

i
+
1
2
(2x)
2
d
2

dx
2

i
+
1
3!
(2x)
3
d
3

dx
3

i
+. . .
_
.
Collecting terms gives

i
= (
0
+
1
+
2
)
i
+ (
1
x + 2
2
x)
d
dx

i
+
1
2
_

1
(x)
2
+
2
(2x)
2
_
d
2

dx
2

i
+
1
3!
_

1
(x)
3
+
2
(2x)
3
_
d
3

dx
3

i
+. . . .
(5.18)
We need to compute three weights and therefore have to extract as many
conditions from equation (5.18). These conditions can be derived by assigning
specic values to the terms in parentheses that multiply the function value and
the rst and second derivatives:
1. We require that (/x)
i
does not depend on
i
because adding a constant
to a function does not change the derivative. Therefore the rst condition
is

0
+
1
+
2
= 0. (5.19)
2. Our goal is for (/x)
i
to be an approximation to (d/dx)
i
and therefore
the second condition is

1
x + 2
2
x = 1. (5.20)
Equations (5.19) and (5.20) are also referred to as conditions for rst-order
accuracy. This is because if the weights satisfy only these two conditions
(note this would leave one free parameter), the term in parentheses multi-
plying the second derivative in equation (5.18) would become the TE and be
O(x). Note that equation (5.19) by itself is sometimes called a condition
for zeroth-order accuracy.
Id
34 Overview of Discretization Methods
3. To obtain second-order accuracy, the weights also have to satisfy the third
condition

1
(x)
2
+
2
(2x)
2
= 0. (5.21)
The three conditions give a linear system that can be solved to obtain the
weights,

0
=
3
2x
,
1
=
2
x
,
2
=
1
2x
. (5.22)
Substituting back into equation (5.15) gives the desired approximation as

i
=
3
i
+ 4
i+1

i+2
2x
. (5.23)
This simple approach to deriving nite-dierence approximations can in principle
be used for any order derivative and any order of accuracy.

i

0
i

00
i

000
i
(=x)
i
0 1 0 0
!
0

i
!
0
0 0 0
!
1

i+1
!
1
!
1
x
1
2
!
1
(x)
2 1
6
!
1
(x)
3
!
2

i+2
!
2
2!
2
x 2!
2
(x)
2 4
3
!
2
(x)
3
Thisrowcontainsthefunction
valueandderivativesinthe
Taylor-seriesexpansion.
Thisentrycontainsthe
discreteoperatorforwhich
anexpressionissought.
Theentriesinthiscolumn
containthetermsthatappear
intheexpressionforthe
discreteoperator.
Step1:Constructionofbasictable(firstrowandcolumn)
Thislineservestoseparate
thediscreteoperatorfrom
thetermsintheexpressionfor
thediscreteoperator(i.e.,the
left-andright-handsidesof
Eq.(x.yz)).

i

0
i

00
i

000
i
(=x)
i
0 1 0 0
!
0

i
!
0
0 0 0
!
1

i+1
!
1
!
1
x
1
2
!
1
(x)
2 1
6
!
1
(x)
3
!
2

i+2
!
2
2!
2
x 2!
2
(x)
2 4
3
!
2
(x)
3
Thisrowcontainsentries
thatspecifywhichcontinuous
operatoristobeapproximated
bythediscreteoperator.Here,
the`1'belowthefirstderivative
indicatesthatthediscrete
operatorisanapproximationof
thefirstderivative.
Step2:Specificationofapproximation

i

0
i

00
i

000
i
(=x)
i
0 1 0 0
!
0

i
!
0
0 0 0
!
1

i+1
!
1
!
1
x
1
2
!
1
(x)
2 1
6
!
1
(x)
3
!
2

i+2
!
2
2!
2
x 2!
2
(x)
2 4
3
!
2
(x)
3
Thisrowcontainsthe
coefficientsintheTaylor-series
expansionof
multipliedbytheweight.The
rowsaboveandbeloware
treatedinthesamemanner.
about

i+1
xi

Step3:Taylor-seriesexpansionofapproximation

i

0
i

00
i

000
i
(=x)
i
0 1 0 0
!
0

i
!
0
0 0 0
!
1

i+1
!
1
!
1
x
1
2
!
1
(x)
2 1
6
!
1
(x)
3
!
2

i+2
!
2
2!
2
x 2!
2
(x)
2 4
3
!
2
(x)
3
!
0
+ !
1
+ !
2
= 0,
!
1
x + 2!
2
x = 1,
1
2
!
1
(x)
2
+ 2!
2
(x)
2
= 0.
Todeterminetheweights,the
entriesabovetheseparationline
ineachcolumnareequatedto
thesumoftheentriesbelowthe
separationlineinthatcolumn.
Thisisdoneforasmanycolumns
asthereareweightstobe
determined.
Withthreeweightstobe
determined,weobtainfrom
thethreehighlightedcolumns
thelinearsystem
Step4:Determinationofweights
Figure 5.3: Construction and use of Taylor table for the derivation
of the second-order accurate forward-dierence approximation to the
rst derivative.
The derivation of nite-dierence approximations with Taylor series can be
made more systematic through the so-called Taylor table. The construction and
use of a Taylor table is explained in gure 5.3, taking the second-order accurate
forward-dierence approximation for the rst derivative as an example. As in-
dicated in the gure, the construction and use of a Taylor table can be divided
into four steps. The main advantage of constructing Taylor tables is that the
Id
5.1. Finite-Difference Method 35
conditions to be satised by the weights are easily extracted. For the specic
example considered here, the extracted conditions are equal to equations (5.20)
and (5.21), of course, and we again obtain equation (5.23). To determine the te,
we inspect the rst column of the Taylor table that was not used to determine
the weights; in this case, the column for

i
. Rather than equating the entry in
the second row to the sum of those in the third, fourth, and fth rows, we simply
sum the entries of the latter, i.e.,
TE =
_
1
6

1
+
4
3

2
_
(x)
3
d
3

dx
3

i
=
1
3
(x)
2
d
3

dx
3

i
. (5.24)
Taylor tables are especially useful for the derivation of implicit nite-dierence
approximations, which we will meet in section 5.1.3.
Before presenting another method of deriving nite-dierence approximations,
we reconsider the Taylor-series approach in a general manner, make some obser-
vations, and introduce some terminology that will be used in future sections and
chapters. Assume we wish to approximate the nth derivative on a uniform grid
with spacing h, i.e.,

x
n

i
=
1
h
n
r

j=l

i+j
. (5.25)
Using equation (5.17) and grouping terms gives

x
n

i
=

k=0
h
k
k!
d
k

dx
k

i
r

j=l

j
j
k
. (5.26)
We observe immediately that the sum
r

j=l

j
j
k
plays a central role in the analysis of approximations. This sum is called the kth
moment of the weights or simply the kth weight moment. Now assume that we
want the approximation to have order of accuracy p. This suggests breaking the
sum over k in the above equation as follows,

x
n

i
=
n1

k=0
h
kn
k!
d
k

dx
k

i
r

j=l

j
j
k
+
1
n!
d
n

dx
n

i
r

j=l

j
j
n
+
p+n1

k=n+1
h
kn
k!
d
k

dx
k

i
r

j=l

j
j
k
+

k=p+n
h
kn
k!
d
k

dx
k

i
r

j=l

j
j
k
.
(5.27)
Therefore, we extract the following conditions for the approximation of the nth
derivative to be pth-order accurate:
r

j=l

j
j
k
=
_

_
0 for 0 k n 1,
n! for k = n,
0 for n + 1 k p +n 1.
(5.28)
Id
36 Overview of Discretization Methods
The rst set of conditions ensures that we do not obtain an approximation of a
derivative lower than n. The second condition leads to the discrete nth derivative
being equal to the exact nth derivative for polynomials of degree n. The third
set of conditions requires that the rst non-zero term in the truncation error is
O(h
p
). Overall, the weight moments thus must satisfy n + 1 + (p 1) = p + n
conditions. To satisfy these conditions, we therefore require an equal number of
degrees of freedom. These are provided by the rl +1 stencil weights. Therefore,
we have derived the following important condition:
r l + 1 = p +n or p = (r l + 1) n. (5.29)
Lagrangian Polynomial Interpolation. An alternative way of determining
nite-dierence approximations is through the use of Lagrangian polynomial in-
terpolation. Given a function (x) at r l + 1 points x
j
, the Lagrangian inter-
polation polynomial is
p(x) =
r

j=l
L
j
(x) (x
j
) =
r

j=l
L
j
(x)
j
, (5.30)
where the Lagrangian basis functions are dened as
L
j
(x) =
r

k=l
k=j
x x
k
x
j
x
k
. (5.31)
The basis functions are polynomials of degree r l and have the properties
L
j
(x
i
) =
ij
and
r

j=l
L
j
(x) = 1,
where
ij
is the Kronecker delta.
Discrete derivatives are formed from derivatives of the interpolation polyno-
mial. Thus an approximation to the rst derivative is determined from

i
= p

(x
i
) =
r

j=l
L

j
(x
i
)
j
, (5.32)
where, from equation (5.31),
L

j
(x) =
r

m=l
m=j
r

k=l
k=j,m
(x x
k
)
r

k=l
k=j
(x
j
x
k
)
. (5.33)
By way of example, consider the case with l = 0 and r = 2, for which we have
p(x) =
(x x
1
)(x x
2
)
(x
0
x
1
)(x
0
x
2
)

0
+
(x x
0
)(x x
2
)
(x
1
x
0
)(x
1
x
2
)

1
+
(x x
0
)(x x
1
)
(x
2
x
0
)(x
2
x
1
)

2
.
If x
1
= x
0
+h and x
2
= x
1
+h, p

(x) can be written as

p

(x) =
2(x x
0
) 3h
2h
2

0

2(x x
0
) 2h
h
2

1
+
2(x x
0
) h
2h
2

2
. (5.34)
Id
5.1. Finite-Difference Method 37
Therefore, we have that

0
= p

(x
0
) =
3
0
+ 4
1

2
2h
,
consistent with equation (5.23).
Analogously, an approximation to the second derivative is determined from

x
2

i
= p

(x
i
) =
r

j=l
L

j
(x
i
)
j
, (5.35)
where, from equation (5.31) or equation (5.33),
L

j
(x) =
r

m=l
m=j
r

n=l
n=j,m
r

k=l
k=j,m,n
(x x
k
)
r

k=l
k=j
(x
j
x
k
)
. (5.36)
Approximations to higher derivatives can be derived in a similar manner. Note
that the discrete derivatives are obtained from the interpolation polynomial by
exact dierentiation. In general, the discrete derivative is approximate because
the interpolation polynomial is approximate. If (x) is a polynomial of degree
r l, the interpolation polynomial and the discrete derivative are exact.
The above expressions for the rst and second derivative are valid for non-
uniform grids. For uniform grids, for which we have x
i+j
x
i
= jh, equa-
tions (5.32) and (5.33) and equations (5.35) and (5.36) can be simplied to give

i
=
1
h
r

j=l

i+j
r

m=l
m=j
r

k=l
k=j,m
(i k)
r

k=l
k=j
(j k)
(5.37)
and

x
2

i
=
1
h
2
r

j=l

i+j
r

m=l
m=j
r

n=l
n=j,m
r

k=l
k=j,m,n
(i k)
r

k=l
k=j
(j k)
. (5.38)
Dierence Calculus. Equations (2.27), (2.31), (2.34) and (2.36) provide us
with four methods of computing D.
We are now in a position to derive forward nite-dierence approximations by
making use of equation (2.27) and equation (2.12) to give
xD = ln(1 +
+
) =
+

2
+
2
+

3
+
3


4
+
4
+. . . (5.39)
Approximations are derived by retaining as many terms on the right-hand side
as required by the desired order of accuracy. The rst neglected term determines
the order of the truncation error. For example, to derive a second-order accurate
Id
38 Overview of Discretization Methods
approximation, we retain only the rst two terms because
3
+
/x (x)
2
by
equation (2.28). This gives
D =
1
x
_

2
+
2
_
+O
_
(x)
2

(5.40)
Therefore, we obtain
D
i
=
1
x
_
(
i+1

i
)
1
2
(
i+2
2
i+1
+
i
)
_
+O
_
(x)
2

(5.41)
and collecting terms gives the same result as equation (5.23).
Backward nite-dierence approximations are derived by using equation (2.31)
together with equation (2.13) to get
xD = ln(1

) =

+

2

2
+

3

3
+

4

4
+. . . (5.42)
As with forward dierence approximations, backward dierence approximations
are derived by retaining as many terms on the right-hand side as required by the
desired order of accuracy.
Centered nite-dierence approximations are derived by expanding the right-
hand side of equation (2.36) to get
D =
1
x
sinh
1

0
=
1
x
_

3
0
6
+
3
40

5
0
. . .
_
(5.43)
This relation leads to the usual second-order accurate central dierence approx-
imation given by equation (5.12) if only the rst term in parentheses on the
right-hand side is retained. Retaining the second term in parentheses leads to a
fourth-order accurate approximation, but it involves function values at alternate
grid points,
D
i
=
1
48x
(
i+3
+ 27
i+1
27
i1
+
i3
) +O
_
(x)
4

(5.44)
Approximations that involve function values at alternate grid points admit spuri-
ous solutions, as was discussed in section 4.1.2.2. Before addressing this deciency,
we turn to equation (2.34), which can be written as
D =
2
x
sinh
1
_

2
_
=
1
x
_


3
24
+
3
640

5
. . .
_
(5.45)
This equation by itself is also not useful because it leads to expressions involving
function values at half-integer grid points. However, it can be made useful through
the identity expressed by equation (2.23) written as
1 =
_
1 +

2
4
_
1/2
=
_
1

2
8
+
3
128

5
1024

6
+. . .
_
(5.46)
Multiplying equation (5.45) by equation (5.46) gives
D =
_
1 +

2
4
_
1/2
2
x
sinh
1
_

2
_
=

x
_


3
6
+

5
30
. . .
_
(5.47)
and using equation (2.19) we can express this as
D =

0
x
_
1

2
6
+

4
30
. . .
_
(5.48)
Id
5.1. Finite-Difference Method 39
This expression will now involve only function values at integer grid points because
appears only raised to even powers. Consider including only the rst two terms
in parentheses in equation (5.48). Then we obtain
D
i
=

0
x
_
1

2
6
_

i
=
1
x
_

1
12

i+2
+
2
3

i+1

2
3

i1
+
1
12

i2
_
. (5.49)
This approximation is fourth-order accurate because we truncated the series ex-
pansion at the
4
term. Note that the term
0
/(x) does not inuence the order
of accuracy because
0
x.
So far, we have considered only the derivation of nite-dierence approxima-
tions for the rst derivative. One of the strengths of dierence calculus is that
higher-order derivatives can be derived in a straightforward way. Consider rst
forward-dierence approximations, for which we obtain from equation (5.39)
d
n
dx
n
= D
n
=
1
(x)
n
[ln(1 +
+
)]
n
(5.50)
Expanding the logarithm in a Taylor series gives
D
n
=
1
(x)
n
_

n
+

n
2

n+1
+
+
n(3n + 5)
24

n+2
+

n(n + 2)(n + 3)
48

n+3
+
+. . .
_
(5.51)
Evaluating this expression gives the formulae listed in table 5.3.
Conversely, backward-dierence approximations for higher-order derivatives
can be obtained from equation (5.42) as
d
n
dx
n
= D
n
=
1
(x)
n
[ln(1

)]
n
(5.52)
Expanding the logarithm in a Taylor series gives
D
n
=
1
(x)
n
_

+
n
2

n+1

+
n(3n + 5)
24

n+2

+
n(n + 2)(n + 3)
48

n+3

+. . .
_
(5.53)
Note the similarities between equations (5.51) and (5.53). Evaluating this expres-
sion gives the negative of the formulae listed in table 5.3.
Finally, centered-dierence approximations for higher-order derivatives can be
obtained from equation (2.34) as
D
n
=
2
n
(x)
n
_
sinh
1
_

2
__
n
(5.54)
Expanding the inverse hyperbolic sine in a Taylor series gives
D
n
=

n
(x)
n
_
1
n
24

2
+
n
64
5n + 22
90

4

n
4
5
_
5
7
+
n 1
5
+
(n 1)(n 2)
3
5
_

6
. . .
_
(5.55)
Because of the
n
operator outside the square brackets, this formula generates
dierence approximations with function values at integer (half-integer) points for
n even (uneven). Another expression for centered-dierence approximations can
be derived from equation (5.47) as
D
n
=
_
1 +

2
4
_
1/2
2
n
(x)
n
_
sinh
1
_

2
__
n
(5.56)
Id
40 Overview of Discretization Methods
Table 5.1: Weights and
truncation-error constant
C for centered nite-
dierence approximations
of derivatives of order n
and order of accuracy p.
After Fornberg (1998b, p.
16).
j
n p 4 3 2 1 0 1 2 3 4 C
1 2
1
2
0
1
2
1
6
4
1
12

2
3
0
2
3

1
12

1
30
6
1
60
3
20

3
4
0
3
4

3
20
1
60
1
140
8
1
280

4
105
1
5

4
5
0
4
5

1
5
4
105

1
280

1
630
2 2 1 2 1
1
12
4
1
12
4
3

5
2
4
3

1
12

1
90
6
1
90

3
20
3
2

49
18
3
2

3
20
1
90
1
560
8
1
560
8
315

1
5
8
5

205
72
8
5

1
5
8
315

1
560

1
3150
3 2
1
2
1 0 1
1
2
1
4
4
1
8
1
13
8
0
13
8
1
1
8

7
120
6
7
240
3
10

169
120
61
30
0
61
30
169
120

3
10
7
240
41
3024
4 2 1 4 6 4 1
1
6
4
1
6
2
13
2
28
3

13
2
2
1
6

7
240
6
7
240

2
5
169
60

122
15
91
8

122
15
169
60

2
5
7
240
41
7560
Note that because the product of the rst two terms on the right-hand side is
unity [cf. equation (2.23) or (5.46)], it does not need to be raised to power n.
Expanding the inverse hyperbolic sine in a Taylor series gives
D
n
=

n
(x)
n
_
1
n + 3
24

2
+
5n
2
+ 52n + 135
5760

4
. . .
_
(5.57)
Because of the
n
operator outside the square brackets, this formula generates
dierence approximations with function values at half-integer (integer) points for
n even (uneven).
Algorithm for Computing Finite-Dierence Approximation Weights.
Fornberg (1988, 1998a) presented an algorithm for computing weights of nite-
dierence approximations of arbitrary order of accuracy for derivatives of arbi-
trary order. With the denition
d
n

dx
n

i
=
1
x
n
r

j=l

j,n

i+j
+O[(x)
p
] (5.58)
we can encompass centered (l = r), biased (l = r, l = 0, and r = 0), and one-sided
(l = 0 and r > 0 or l > 0 and r = 0) approximations. The resulting weights are
presented in tables 5.1 and 5.3 for 1 n 4 and r + l 8. The tables also
list the truncation-error constant C, i.e., the numerical constant appearing in the
leading term of the truncation error,
C(x)
p
d
n+p

dx
n+p
.
Note that if the direction of biasing in the biased and one-sided approximations
is reversed, the weights change sign if the order of the derivative is odd.
5.1.2.2 Explicit Finite Dierences as Dierentiation Matrices
In the above, we have considered only the approximation to the derivative at a
specic point (i, j). We can also consider these approximations for an entire range
Id
5.1. Finite-Difference Method 41
j
n p 4 3 2 1 0 1 2 3 C
1 3
1
6
1
1
2
1
3
1
12
4
1
12
1
2

3
2
5
6
1
4
1
20
5
1
30
1
4
1
1
3
1
2

1
20

1
60
6
1
60

2
15
1
2

4
3
7
12
2
5

1
30

1
105
7
1
140

1
15
3
10
1
1
4
3
5

1
10
1
105
1
280
2 3
1
12
1
3
1
2

5
3
11
12
1
12
4
1
12

1
2
7
6

1
3

5
4
5
6
13
180
5
1
90

1
15
1
12
10
9

7
3
19
15

13
180

1
90
3 3
1
4

7
4
7
2

5
2
1
4
1
4
1
8
4
1
8
1
29
8
6
35
8
1
1
8
1
15
5
7
120
8
15

89
40
11
3

49
24

2
5
71
120

1
15
7
240
4 3
1
6
1
3
2

2
3
7
2
3
5
6
1
6
Table 5.2: Weights and
truncation-error con-
stant C for biased nite-
dierence approximations
of derivatives of order n
and order of accuracy p.
j
n p 0 1 2 3 4 5 6 7 8 C
1 1 1 1
1
2
2
3
2
2
1
2

1
3
3
11
6
3
3
2
1
3
1
4
4
25
12
4 3
4
3

1
4

1
5
5
137
60
5 5
10
3

5
4
1
5
1
6
6
49
20
6
15
2
20
3

15
4
6
5

1
6

1
7
7
363
140
7
21
2
35
3

35
4
21
5

7
6
1
7
1
8
8
761
280
8 14
56
3

35
2
56
5

14
3
8
7

1
8

1
9
2 1 1 2 1 1
2 2 5 4 1
11
12
3
35
12

26
3
19
2

14
3
11
12
5
6
4
15
4

77
6
107
6
13
61
12

5
6

137
180
5
203
45

87
5
117
4

254
9
33
2

27
5
137
180
7
10
6
469
90

223
10
879
20

949
18
41
201
10
1019
180

7
10

363
560
7
29531
5040

962
35
621
10

4006
45
691
8

282
5
2143
90

206
35
363
560
761
1260
3 1 1 3 3 1
3
2
2
5
2
9 12 7
3
2

7
4
3
17
4
71
4

59
2
49
2

41
4
7
4
15
8
4
49
8
29
461
8
62
307
8
13
15
8

29
15
5
967
120
638
15

3929
40
389
3

2545
24
268
5

1849
120
29
15
469
240
6
801
80
349
6

18353
120
2391
10

1457
6
4891
30

561
8
527
30

469
240

1791
917
4 1 1 4 6 4 1 2
2 3 14 26 24 11 2
17
6
3
35
6
31
137
2

242
3
107
2
19
17
6
7
2
4
28
3

111
2
142
1219
6
176
185
2
82
3

7
2

967
240
5
1069
80

1316
15
15289
60

2144
5
10993
24

4772
15
2803
20

536
15
967
240
89
20
Table 5.3: Weights and
truncation-error constant
C for one-sided nite-
dierence approximations
of derivatives of order n
and order of accuracy p.
After Fornberg (1998b, p.
17).
Id
42 Overview of Discretization Methods
of points, such as (1 i M, j). The approximate derivative then becomes a
matrix-derivative operator acting on
1iM,j
to produce

1iM,j
. Such matrix
operators may be thought of as nite-dimensional discrete analogues of innite-
dimensional continuous dierentiation operators. Interpreting nite dierences
as matrix-derivative operators is instructive because it allows us to point out
similarities with and dierences to spectral methods discussed in section 5.5.
Consider the second-order central nite-dierence approximation given by
equation (5.12) on a one-dimensional grid with M points. For simplicity, we
will rst assume periodic boundary conditions so that only M 1 grid points
need be considered. Then we can write
_

2
.
.
.

i
.
.
.

M2

M1
_

_
=
1
2x
_

_
0 1 1
1 0 1
.
.
.
.
.
.
.
.
.
1 0 1
.
.
.
.
.
.
.
.
.
1 0 1
1 1 0
_

_
_

2
.
.
.

i
.
.
.

M2

M1
_

_
(5.59)
where all entries that are not shown explicitly are zero. Note that the derivative
matrix is antisymmetric (because it represents an odd derivative), with zeroes
along the main diagonal (because it is based on a central-dierence approximation
of an odd derivative), and with constant entries along the diagonals (because the
grid spacing is constant). Matrices with constant entries along diagonals are
called Toeplitz matrices. In the case of periodic boundary conditions, the entries
on each row are given by a cyclic shift of the entries on the preceding row, and
such matrices are called circulant.
For non-periodic boundary conditions, one-sided approximations must be used
at the boundary points. To preserve the second-order accuracy at interior points,
we can use the approximation for n = 1 and p = 2 in table 5.3 to obtain,
_

2
.
.
.

i
.
.
.

M1

M
_

_
=
1
2x
_

_
3 4 1
1 0 1
.
.
.
.
.
.
.
.
.
1 0 1
.
.
.
.
.
.
.
.
.
1 0 1
1 4 3
_

_
_

2
.
.
.

i
.
.
.

M1

M
_

_
(5.60)
Analogously, we can construct derivative operators corresponding to second
derivatives. For periodic boundaries, we obtain
_

2
.
.
.

i
.
.
.

M2

M1
_

_
=
1
(x)
2
_

_
2 1 1
1 2 1
.
.
.
.
.
.
.
.
.
1 2 1
.
.
.
.
.
.
.
.
.
1 2 1
1 1 2
_

_
_

2
.
.
.

i
.
.
.

M2

M1
_

_
(5.61)
Note that the derivative matrix is symmetric (because it represents an even deriva-
tive), has non-zero entries along the main diagonal, and contains constant entries
along the diagonals (because the grid spacing is constant). For non-periodic
Id
5.1. Finite-Difference Method 43
boundary conditions, we choose a one-sided approximation for the boundary
points which preserves the order of accuracy of the centered interior approxi-
mation. Hence we choose n = 2 and p = 2 from table 5.3 and obtain
_

2
.
.
.

i
.
.
.

M1

M
_

_
=
1
(x)
2
_

_
2 5 4 1
1 2 1
.
.
.
.
.
.
.
.
.
1 2 1
.
.
.
.
.
.
.
.
.
1 2 1
1 4 5 2
_

_
_

2
.
.
.

i
.
.
.

M1

M
_

_
(5.62)
Furthermore, in general the derivative matrix is sparse because the stencil
width of nite-dierence approximations is nite. We will see in section 5.5 that
this is not true for spectral derivative operators.
5.1.3 Implicit Finite-Dierence Approximations
The increasing stencil width of explicit nite-dierence approximations as the
order of accuracy is increased brings two problems. First, near boundaries, one-
sided stencils must be adopted farther away than with lower-order approxima-
tions. Second, wider stencils include solution data that is farther removed from
the location at which the derivative is approximated. Since the derivative is a
local quantity, including remote solution data can actually reduce the accuracy
although the order of accuracy may be higher. For these reasons, it is common
to attempt to reduce the stencil width of nite-dierence approximations.
Implicit nite-dierence approximations provide a way to obtain higher-order
accuracy with narrower stencils than explicit approximations of the same order
of accuracy. For this reason, they are also referred to as compact nite-dierence
methods. An implicit nite-dierence approximation for the rst derivative at
point i can be expressed as
s

j=m

j
_
d
dx
_
i+j
=
r

j=l

i+j
+O[(x)
q
] (5.63)
Comparing to equation (5.58), we see that the crucial dierence is that a linear
combination of derivatives appears on the right-hand side, i.e., they appear im-
plicitly. To compute derivatives using equation (5.63), we are thus required to
solve a linear system. As a result, implicit nite-dierence approximations are
more costly than explicit nite-dierence approximations. However, as we will see
in section 6.3, implicit nite-dierence approximations oer improved accuracy
for a given stencil width or order of accuracy compared to explicit nite-dierence
approximations.
5.1.3.1 Derivation of Implicit Finite-Dierence Approximations
There are many ways in which implicit nite-dierence approximations can be
derived. Here we will illustrate three methods. We start by describing a method
which clearly demonstrates the relationship to explicit nite-dierence approxi-
mations and how the accuracy is improved.
Truncation-Error Cancellation. Consider the centered dierence approxima-
tion to the rst derivative,

i
=

i+1

i1
2x
=
d
dx

i
+
(x)
2
6
d
3

dx
3

i
+
(x)
4
120
d
5

dx
5

i
+. . . (5.64)
Id
44 Overview of Discretization Methods
We wish to raise the order of accuracy from second to fourth without increasing
the stencil size. We can achieve this by approximating the leading term in the
truncation error with at least second-order accuracy. Because we wish to compute
rst derivatives, we can approximate the third derivative appearing in the leading
term of the truncation error as the centered second dierence of centered rst
dierences. To do so, we take the rst derivative of the Taylor-series expansion
and evaluate it at i 1 to get
d
dx

i1
=
d
dx

i
x
d
2

dx
2

i
+
(x)
2
2
d
3

dx
3

(x)
3
6
d
4

dx
4

i
+
(x)
4
24
d
5

dx
5

i
+. . .
(5.65)
Hence we can approximate the third derivative as
1
6
_
d
dx

i+1
2
d
dx

i
+
d
dx

i1
_
=
(x)
2
6
d
3

dx
3

i
+
(x)
4
72
d
5

dx
5

i
+. . . (5.66)
and substituting this result back into equation (5.64) gives
1
6
_
d
dx

i+1
+ 4
d
dx

i
+
d
dx

i1
_
=

i+1

i1
2x
+
(x)
4
180
d
5

dx
5

i
+. . . (5.67)
Thus by computing derivatives from
1
6
_

i+1
+ 4

x

i
+

x

i1
_
=

i+1

i1
2x
(5.68)
we obtain fourth-order accuracy without having increased the stencil width com-
pared to the second-order accurate explicit centered dierence.
Similarly, we can take the explicit nite-dierence approximation to the second
derivative,

x
2

i
=

i+1
2
i
+
i1
(x)
2
=
d
2

dx
2

i
+
(x)
2
12
d
4

dx
4

i
+
(x)
4
360
d
6

dx
6

i
+. . . (5.69)
and approximate the leading term of the truncation error as the second derivative
of second derivatives as
1
12
_
d
2

dx
2

i+1
2
d
2

dx
2

i
+
d
2

dx
2

i1
_
=
(x)
2
12
d
4

dx
4

i
+
(x)
4
144
d
6

dx
6

i
+. . . (5.70)
and substituting back gives
1
12
_
d
2

dx
2

i+1
+ 10
d
2

dx
2

i
+
d
2

dx
2

i1
_
=

i+1
2
i
+
i1
(x)
2
+
(x)
4
240
d
6

dx
6

i
+. . .
(5.71)
so that a fourth-order accurate implicit nite-dierence approximation to the
fourth derivative can be computed from
1
12
_

x
2

i+1
+ 10

2

x
2

i
+

2

x
2

i1
_
=

i+1
2
i
+
i1
(x)
2
(5.72)
Taylor Series. We can follow the same procedure of using Taylor tables to derive
implicit nite-dierence approximations as outlined in section 5.1.2.1 for explicit
nite-dierence approximations. Here we will illustrate the use of Taylor tables
Id
5.1. Finite-Difference Method 45
Table 5.4: Taylor table for implicit centered nite-dierence approx-
imation for the rst derivative.

i

V
i

VII
i

IX
i

i
1 0 0 0 0

1
(

i+1
+

i1
) 2
1

1
(x)
2 2
4!

1
(x)
4 2
6!

1
(x)
6 2
8!

1
(x)
8

2
(

i+2
+

i2
) 2
2
4
2
(x)
2 22
4
4!

2
(x)
4 22
6
6!

2
(x)
6 22
8
8!

2
(x)
8
1
2x
(
i+1

i1
)
1
1
3!

1
(x)
2 1
5!

1
(x)
4 1
7!

1
(x)
6 1
9!

1
(x)
8
2
4x
(
i+2

i2
)
2
2
2
3!

2
(x)
2 2
4
5!

2
(x)
4 2
6
7!

2
(x)
6 2
8
9!

2
(x)
8
3
6x
(
i+3

i3
)
3
3
2
3!

3
(x)
2 3
4
5!

3
(x)
4 3
6
7!

3
(x)
6 3
8
9!

3
(x)
8
for a slightly modied form of equation (5.63) under the assumption of a centered
approximation on a uniform grid,

i
+
1
(

i+1
+

i1
)+
2
(

i+2
+

i2
) =
1

i+1

i1
2x
+
2

i+2

i2
4x
+
3

i+3

i3
6x
(5.73)
We can make the Taylor table more compact in this case by expanding the sum
and dierence terms appearing in equation (5.73) directly rather than each vari-
able separately and only showing the columns for the odd-derivative terms. Then
the Taylor table can be written as shown in table 5.4.
In the table, we have drawn an additional horizontal line to separate the terms
appearing on the left- and right-hand sides of equation (5.73). We thus extract
the following conditions from the Taylor table.
For second-order accuracy:
1 + 2
1
+ 2
2
=
1
+
2
+
3
(5.74)
For fourth-order accuracy: equation (5.74) and

1
+ 4
2
=
1
3!
_

1
+ 2
2

2
+ 3
2

3
_
(5.75)
For sixth-order accuracy: equations (5.74) and (5.75), and
2
4!
_

1
+ 2
4

2
_
=
1
5!
_

1
+ 2
4

2
+ 3
4

3
_
(5.76)
For eighth-order accuracy: equations (5.74) to (5.76), and
2
6!
_

1
+ 2
6

2
_
=
1
7!
_

1
+ 2
6

2
+ 3
6

3
_
(5.77)
For tenth-order accuracy: equations (5.74) to (5.77), and
2
8!
_

1
+ 2
8

2
_
=
1
9!
_

1
+ 2
8

2
+ 3
8

3
_
(5.78)
Rather than determining the general solution, we are interested in determining
families of solutions for a given stencil width and order of accuracy. By restricting
ourselves to only neighboring points, we have
2
=
2
=
3
= 0, and thus only
equations (5.74) and (5.75) can be fullled, giving
1 + 2
1
=
1
(5.79)

1
=
1
3!

1
(5.80)
Id
46 Overview of Discretization Methods
Table 5.5: Taylor table for implicit one-sided nite-dierence approx-
imation for the rst derivative.

1

IV
1

1
0 1 0 0 0

2
0 x
1
2
(x)
2 1
3!
(x)
3
1
x

1
1
x
0 0 0 0
2
x

2
2
x

2
1
2

2
x
1
3!

2
(x)
2 1
4!

2
(x)
3
3
x

3
3
x
2
3
2
2
2

3
x
2
3
3!

3
(x)
2 2
4
4!

3
(x)
3
4
x

4
4
x
3
4
3
2
2

4
x
3
3
3!

4
(x)
2 3
4
4!

4
(x)
3
with the solution
1
=
1
4
and
1
=
3
2
and hence we again obtain equation (5.68).
At boundaries, we will have to use either one-sided or biased approximations.
For a one-sided approximation at the rst grid point we replace equation (5.73)
by

1
+

2
=
1
x
(
1

1
+
2

2
+
3

3
+
4

4
) (5.81)
We produce a Taylor table in the same way as before to obtain that shown in
table 5.5. As before, an additional horizontal line appears to separate the terms
appearing on the left- and right-hand side of equation (5.81). Note that because
the approximation is not centered, the function value itself and odd derivatives
appear in the table. We extract the following conditions:
For zeroth-order accuracy:
0 =
1
+
2
+
3
+
4
(5.82)
For rst-order accuracy: equation (5.82) and
1 + =
2
+ 2
3
+ 3
4
(5.83)
For second-order accuracy: equations (5.82) and (5.83), and
=
1
2
_

2
+ 2
2

3
+ 3
2

4
_
(5.84)
For third-order accuracy: equations (5.82) to (5.84), and
1
2!
=
1
3!
_

2
+ 2
3

3
+ 3
3

4
_
(5.85)
For fourth-order accuracy: equations (5.82) to (5.85), and
1
3!
=
1
4!
_

2
+ 2
4

3
+ 3
4

4
_
(5.86)
To get a second-order accurate approximation with
3
=
4
= 0, we thus need
to solve the following system of equations,
0 =
1
+
2
(5.87)
1 + =
2
(5.88)
=
1
2

2
(5.89)
Id
5.1. Finite-Difference Method 47
Table 5.6: Taylor table for implicit centered nite-dierence approx-
imation for the second derivative.

i

IV
i

VI
i

VIII
i

X
i

i
1 0 0 0 0

1
(

i+1
+

i1
) 2
1

1
(x)
2 2
4!

1
(x)
4 2
6!

1
(x)
6 2
8!

1
(x)
8

2
(

i+2
+

i2
) 2
2
4
2
(x)
2 22
4
4!

2
(x)
4 22
6
6!

2
(x)
6 22
8
8!

2
(x)
8
1
(x)
2
(
i+1
2
i
+
i1
)
1
2
4!

1
(x)
2 2
6!

1
(x)
4 2
8!

1
(x)
6 2
10!

1
(x)
8
2
4(x)
2
(
i+2
2
i
+
i2
)
2
2
3
4!

2
(x)
2 2
5
6!

2
x
4 2
7
8!

2
(x)
6 2
9
10!

2
(x)
8
3
9(x)
2
(
i+3
2
i
+
i3
)
3
23
2
4!

3
(x)
2 23
4
6!

3
(x)
4 23
6
8!

3
(x)
6 23
8
10!

3
(x)
8
The solution is = 1,
1
= 2, and
2
= 2. Therefore, a second-order accurate
one-sided implicit nite-dierence approximation for the rst derivative is

1
+

2
=
2
x
(
1
+
2
) (5.90)
We can follow the same procedure to derive implicit approximations for the
second derivative,

i
+
1
(

i+1
+

i1
) +
2
(

i+2
+

i2
)
=
1

i+1
2
i
+
i1
(x)
2
+
2

i+2
2
i
+
i2
4(x)
2
+
3

i+3
2
i
+
i3
9(x)
2
(5.91)
The corresponding Taylor table is presented in table 5.6. In the table, we have
again drawn an additional horizontal line to separate the terms appearing on
the left- and right-hand sides of equation (5.91). We thus extract the following
conditions from the Taylor table.
For second-order accuracy:
1 + 2
1
+ 2
2
=
1
+
2
+
3
(5.92)
For fourth-order accuracy: equation (5.92) and

1
+ 4
2
=
2
4!
_

1
+ 2
2

2
+ 3
2

3
_
(5.93)
For sixth-order accuracy: equations (5.92) and (5.93), and
2
4!
_

1
+ 2
4

2
_
=
2
6!
_

1
+ 2
4

2
+ 3
4

3
_
(5.94)
For eighth-order accuracy: equations (5.92) to (5.94), and
2
6!
_

1
+ 2
6

2
_
=
2
8!
_

1
+ 2
6

2
+ 3
6

3
_
(5.95)
For tenth-order accuracy: equations (5.92) to (5.95), and
2
8!
_

1
+ 2
8

2
_
=
2
10!
_

1
+ 2
8

2
+ 3
8

3
_
(5.96)
Dierence Calculus. The dierence calculus introduced for explicit nite-
dierence approximations can also be used to derive implicit nite-dierence ap-
proximations. To illustrate this approach, we consider equation (5.48) expressed
Id
48 Overview of Discretization Methods
Table 5.7: Weights and
truncation-error constant
C for centered implicit
nite-dierence approx-
imations of derivatives
of order n and order of
accuracy p.
j j
n p 2 1 0 1 2 3 2 1 0 1 2 3 C
1 4
1
4
1
1
4

3
2
0
3
2
1
180
6
1
3
1
1
3

1
9

14
9
0
14
9
1
9
2 4
1
10
1
1
10
6
5

12
5
6
5
1
240
6
2
11
1
2
11
6
5

12
5
6
5
1
240
to fourth-order accuracy,
D =

0
x
_
1

2
6
_
+O
_
(x)
4

(5.97)
which we can recast as
D =

0
x
_
1 +

2
6
_
1
+O
_
(x)
4

(5.98)
to the same order of accuracy. When applied to a given function, the inverse
operator appearing in equation (5.98) is to be interpreted as
_
1 +

2
6
_
D
i
=
1
x

i
+O
_
(x)
4

(5.99)
which gives equation (5.68) on using equations (2.14) and (2.15).
A selection of centered implicit nite-dierence approximations are tabulated
in table 5.7.
5.1.3.2 Implicit Finite Dierences as Dierentiation Matrices
As indicated at the beginning of this section, implicit nite dierences require the
inversion of a matrix to compute derivatives.
As in section 5.1.2.2, we will rst consider the case of a periodic grid. Then
equation (5.68) can be written in the following form,
_

_
2
3
1
6
1
6
1
6
2
3
1
6
.
.
.
.
.
.
.
.
.
1
6
2
3
1
6
.
.
.
.
.
.
.
.
.
1
6
2
3
1
6
1
6
1
6
2
3
_

_
_

2
.
.
.

i
.
.
.

M2

M1
_

_
=
1
2x
_

_
0 1 1
1 0 1
.
.
.
.
.
.
.
.
.
1 0 1
.
.
.
.
.
.
.
.
.
1 0 1
1 1 0
_

_
_

2
.
.
.

i
.
.
.

M2

M1
_

_
(5.100)
which we can also write as
Ay = Bx (5.101)
Thus we require the inversion of a sparse circulant matrix to compute the deriva-
tive with an implicit nite-dierence approximation and periodic boundary con-
ditions, i.e.,
y = A
1
Bx = Dx (5.102)
If the matrix A is tridiagonal, the ecient Thomas algorithm can be used to
invert it. An ecient algorithm also exists for pentadiagonal matrices.
The inverse of a sparse matrix is generally dense, and multiplying a sparse
matrix by a dense matrix leads to a dense matrix. So the dierentiation matrix
Id
5.1. Finite-Difference Method 49
D associated with implicit nite-dierence approximations is generally dense, in
contrast to those associated with explicit nite-dierence approximations. Taking
equation (5.100) as an example with M = 9, we obtain
_

8
_

_
=
1
56x
_

_
0 45 12 3 0 3 12 45
45 0 45 12 3 0 3 12
12 45 0 45 12 3 0 3
3 12 45 0 45 12 3 0
0 3 12 45 0 45 12 3
3 0 3 12 45 0 45 12
12 3 0 3 12 45 0 45
45 12 3 0 3 12 45 0
_

_
_

8
_

_
(5.103)
Compare the weights with those in table 5.1 for n = 1 and p = 4. Note that the
weights would be dierent if we had chosen a dierent value of M, in contrast to
the weights of the explicit nite-dierence approach.
For non-periodic boundary conditions, we must use one-sided or biased ap-
proximations near boundaries. To preserve the tridiagonal structure of the matrix
A, we use a two-point approximation on the boundaries, namely equation (5.90).
On the right-hand side, we also use a one-sided approximation, namely that for
n = 1 and p = 2 in table 5.3 so that the second-order accuracy is preserved. Thus
we obtain
_

_
1 1
1
6
2
3
1
6
.
.
.
.
.
.
.
.
.
1
6
2
3
1
6
.
.
.
.
.
.
.
.
.
1
6
2
3
1
6
1 1
_

_
_

2
.
.
.

i
.
.
.

M1

M
_

_
=
1
2x
_

_
3 4 1
1 0 1
.
.
.
.
.
.
.
.
.
1 0 1
.
.
.
.
.
.
.
.
.
1 0 1
1 4 3
_

_
_

2
.
.
.

i
.
.
.

M1

M
_

_
(5.104)
The dierentiation matrix D would also be dense like that for the case of periodic
boundary conditions.
At this stage, it may be noted that although the implicit nite-dierence
approximations are sometimes referred to as compact nite-dierence approxi-
mations, this term is somewhat misleading. The approximations are only com-
pact when written in the form of equation (5.101), but not when written in the
form of equation (5.102), as exemplied by the example with periodic bound-
ary conditions. The global, i.e., non-compact, nature of implicit nite-dierence
approximations is the reason for their superior accuracy compared to explicit
nite-dierence approximations.
5.1.4 Multi-Dimensional Finite-Dierence Approximations
Finite-dierence approximations in multiple dimensions can be derived in the
same manner as in one dimension. Straight, i.e., non-mixed, derivatives are in
fact derived in exactly the same manner as in one dimension. Starting from the
denition of the partial derivative

x0,y0
= lim
x0
(x
0
+ x, y
0
) (x
0
, y
0
)
x
(5.105)
the nite-dierence method approximates the derivative by foregoing the limiting
process,

x0,y0
=
(x
0
+ x, y
0
) (x
0
, y
0
)
x
(5.106)
Id
50 Overview of Discretization Methods
y
s s s s
s s s s s
s
s s s s s
s s s s s
s s s s s
' E
T
c
E
T
i + 1, j
x i
y j
i, j
i, j 1
i, j + 1
i 1, j
x
s
Figure 5.4: Finite-dierence grid in two dimensions.
On the two-dimensional Cartesian grid depicted in gure 5.4 with constant but not
necessarily equal spacings x and y, vertices can be dened by indices (i, j) such
that the location (x
0
, y
0
) can be expressed as (ix, jy). Then equation (5.106)
can be expressed as

i,j
=

i+1,j

i,j
x
(5.107)
Similarly, we have for the y-derivative,

i,j
=

i,j+1

i,j
y
(5.108)
Equations (5.107) and (5.108) represent backward-dierence approximations. Forward-
and centered-dierence approximations can be derived in a way similar to that in
one dimension.
For the mixed second derivative, various nite-dierence approximations can
Id
5.1. Finite-Difference Method 51
Derivative Finite-dierence approximation TE

xy

i,j
1
x

i+1,j i+1,j1
y

i,j i,j1
y

O(x, y)

xy

i,j
1
x

i,j+1 i,j
y

i1,j+1 i1,j
y

O(x, y)

xy

i,j
1
x

i,j i,j1
y

i1,j i1,j1
y

O(x, y)

xy

i,j
1
x

i+1,j+1 i+1,j
y

i,j+1 i,j
y

O(x, y)

xy

i,j
1
x

i+1,j+1 i+1,j1
2y

i,j+1 i,j1
2y

O[x, (y)
2
]

xy

i,j
1
x

i,j+1 i,j1
2y

i1,j+1 i1,j1
2y

O[x, (y)
2
]

xy

i,j
1
2x

i+1,j+1 i+1,j1
2y

i1,j+1 i1,j1
2y

O[(x)
2
, (y)
2
]

xy

i,j
1
2x

i+1,j+1 i+1,j
y

i1,j+1 i1,j
2y

O[(x)
2
, y]

xy

i,j
1
2x

i+1,j i+1,j1
y

i1,j i1,j1
2y

O[(x)
2
, y]
Table 5.8: Finite-
dierence approximations
for mixed derivatives. Af-
ter Tannehill et al. (1997,
p. 53).
be derived from the two-dimensional Taylor series,

i+1,j+1
=
i,j
+ x

i,j
+ y

i,j
+
(x)
2
2

x
2

i,j
+ xy

2

xy

i,j
+
(y)
2
2

y
2

i,j
+. . .
(5.109)
by combining function values
i1,j1
in such a way that the rst and straight
second derivatives disappear. Possible approximations are shown in table 5.8.
Id