Numerical Methods Notes PDF

Numerical Methods
Summary notes of lectures on the subject of

Numerical Methods delivered by Asst. Prof.
Saqib Noman. Composed by:
Uzair Suria
Contents
1 Introduction ............................................................................................................................. 4
1.1 Motivation for Numerical Analysis ................................................................................. 4
1.1.1 When to Use Numerical Analysis ............................................................................. 4
1.2 Types of Numerical Analysis Problems .......................................................................... 4
1.2.1 Description for Types of Numerical Analysis .......................................................... 6
1.2.2 Characteristics of Numerical Method ....................................................................... 7
1.2.3 Iterative Method for Numerical Computation .......................................................... 8
1.3 Calculation Errors ............................................................................................................ 9
1.3.1 Accuracy and Precision............................................................................................. 9
1.3.2 Truncation and Round-off Error ............................................................................. 10
1.3.3 Error Definitions ..................................................................................................... 10
1.3.3.1 Stopping Criteria .............................................................................................. 11
1.3.4 Taylor Polynomial Error ......................................................................................... 12
2 Roots of Equations ................................................................................................................ 14
2.1 Bisection Method ........................................................................................................... 14
2.1.1 Bisection Computation Process .............................................................................. 14
2.1.1.1 Advantages of Bisection Method ..................................................................... 15
2.1.1.2 Drawbacks of Bisection Method...................................................................... 15
2.2 Regula Falsi Method ...................................................................................................... 16
2.2.1 Regula Falsi Computation Process ......................................................................... 16
2.2.1.1 Derivation of approximation ............................................................................ 17
2.3 Fixed Point Iteration Method ......................................................................................... 19
2.3.1 Convergence Criteria .............................................................................................. 19
2.3.2 Fixed Point Iteration Computation.......................................................................... 19
2.4 Newton Raphson Method .............................................................................................. 21
2.4.1 Computation with Newton-Raphson Method ......................................................... 21
2.4.1.1 Advantages of Newton-Raphson Method ........................................................ 22
2.4.1.2 Drawbacks/Limitations of Newton-Raphson Method ..................................... 22
3 Finite Differences and Interpolation ..................................................................................... 24
3.1 Finite Differences........................................................................................................... 24
3.1.1 Forward Differences ............................................................................................... 24
3.1.1.1 Forward Difference Tables .............................................................................. 24
3.1.2 Backward Differences ............................................................................................. 25
3.1.2.1 Backward Difference Tables............................................................................ 26
1|Page Uzair Suria

3.1.3 Central Differences ................................................................................................. 27
3.1.3.1 Central Difference Table.................................................................................. 27
3.1.4 Other Difference Operators..................................................................................... 28
3.1.4.1 Shift Operator ‘E’ ............................................................................................ 28
3.1.4.2 Averaging Operator ‘𝜇𝜇’ ................................................................................... 28
3.1.4.3 Differential Operator ‘D’ ................................................................................. 28
3.1.5 Relationship Between Operators............................................................................. 30
3.1.5.1 Forward Difference and Shift Operator ........................................................... 30
3.1.5.2 Backward Difference and Shift Operator ........................................................ 30
3.1.5.3 Central Difference and Shift Operator ............................................................. 30
3.1.5.4 Mean difference and Shift Operator ................................................................ 30
3.2 Interpolation with equal Intervals .................................................................................. 32
3.2.1 Finding One or more Missing Terms ...................................................................... 32
3.2.2 Newton’s Forward Interpolation Formula .............................................................. 34
3.2.3 Newton’s Backward Interpolation formula ............................................................ 35
3.3 Interpolation with Unequal Intervals ............................................................................. 37
3.3.1 Newton’s Divided Differences Formula ................................................................. 37
3.3.2 Lagrange’s Interpolation Formula .......................................................................... 38
3.3.2.1 General Expanded form of Lagrange’s Interpolation Formula........................ 39
3.3.2.2 Binomial Multiplication ................................................................................... 39
3.3.3 Piecewise/Spline Interpolation................................................................................ 41
3.3.3.1 Linear Splines .................................................................................................. 41
4 Numerical Calculus............................................................................................................... 43
4.1 Numerical Differentiation .............................................................................................. 45
4.1.1 Derivative using Forward Difference Formula ....................................................... 45
4.1.2 Derivative using Backward Difference Formula .................................................... 46
4.2 Numerical Integration .................................................................................................... 48
4.2.1 Newton – Cotes Quadrature Formula ..................................................................... 48
4.2.2 Trapezoidal Rule ..................................................................................................... 49
4.2.3 Simpson’s One-Third Rule ..................................................................................... 51
4.2.4 Simpson’s Three-Eighth Rule ................................................................................. 52
5 Numerical Solution of Differential Equations ...................................................................... 55
5.1 Ordinary Differential Equations of 1st Order ................................................................. 55
5.1.1 Types of Differential Equations .............................................................................. 55
5.1.2 Initial Value Problems ............................................................................................ 56
5.1.2.1 Euler’s Method................................................................................................. 56
2|Page Uzair Suria
5.1.2.1.1 Geometrical Interpretation ........................................................................ 56
5.1.2.2 Modified Euler’s Method (MEM) ................................................................... 58
5.1.2.3 4th Order Runge – Kutta Method (RK – 4 Method) ......................................... 59
5.2 Partial Differential Equations ........................................................................................ 62
5.2.1 Finite Differences Approximations......................................................................... 62
5.2.2 Solution of Laplace’s Equation............................................................................... 62
5.2.2.1 Solution of Dirichlet problems......................................................................... 64
6 System of linear equations .................................................................................................... 71
6.1 General Form of Linear System..................................................................................... 71
6.2 Types of Linear Systems ................................................................................................ 71
6.2.1 Consistent System ................................................................................................... 71
6.2.1.1 Homogenous System ....................................................................................... 72
6.2.2 Inconsistent System ................................................................................................ 72
6.3 Analytical Solution of Linear System ............................................................................ 72
6.3.1 Gauss Elimination Method ..................................................................................... 72
6.3.1.1 Steps for Solution through Gaussian Elimination............................................ 73
6.3.2 Gauss – Jordan Elimination Method ....................................................................... 74
6.3.2.1 Steps for Solution through Gauss – Jordan Elimination .................................. 75
6.4 LU Decomposition ......................................................................................................... 78
6.4.1 Doolittle Decomposition ......................................................................................... 78
6.4.2 Crout’s Decomposition ........................................................................................... 81
6.4.3 Cholesky Decomposition ........................................................................................ 83
6.5 Iterative Solution to Linear Systems .............................................................................. 84
6.5.1 Jacobi’s Iterative Method........................................................................................ 84
6.5.1.1 Steps to Solve a Linear System with Jacobi’s Iteration ................................... 84
6.5.2 Gauss Seidel Iterative Method ................................................................................ 85
6.5.2.1 Steps to Solve a Linear System with Gauss – Seidel Iteration ........................ 86
7 Inverse Interpolation And Curve Fitting ............................................................................... 89
7.1 Lagrange’s Inverse Interpolation ................................................................................... 89
7.2 Regression ...................................................................................................................... 90
7.2.1 Simple Linear Regression Model ........................................................................... 90
7.2.1.1 Least Square Model ......................................................................................... 90
7.2.2 Exponential Model Fitting ...................................................................................... 93
7.2.3 Power Curve Fitting ................................................................................................ 94
7.2.4 Polynomial Regression ........................................................................................... 96
3|Page Uzair Suria

1 INTRODUCTION
Numerical methods are techniques by which mathematical problems are formulated so that
they can be solved with arithmetic operations. The process of numerical analysis is very
tedious, time consuming and prone to error if done by a human manually. Therefore, it was not
before the development of fast and efficient computers that the method of numerical analysis
got popular.
1.1 Motivation for Numerical Analysis

Before the advent of computers, engineering problems were solved in following ways:
1. Solutions were obtained using analytical and/or exact methods. However, this method
could only be applied to limited types of problems and parameters.
2. Graphical solutions can also be used to characterize the behavior of the system and can
be represented by plots or nomographs. This method is very unprecise, and is only
limited three or lower dimensions
3. Application of numerical analysis manually can also give a solution to a complex
problem, but as discussed above, the results are inconsistent, prone to error due to
simple mistake that can be made while performing large manual calculations, and time-
consuming.
1.1.1 When to Use Numerical Analysis

Numerical Analysis is used when
1. Analytical methods to solve a problem does not exist
2. Information/Data available does not admit applicability of analytical method
3. Analytical solution exists, but it is quite time consuming due to large data or complex
functions such as: sin 𝑥𝑥 + 𝑥𝑥 2 = 0
1.2 Types of Numerical Analysis Problems

The following table shows common types of numerical analysis problems along with Part
Number of the book (Numerical Methods for Engineers, by Steven C. Chapra) in which the
type is discussed:
4|Page Uzair Suria

5|Page Uzair Suria
1.2.1 Description for Types of Numerical Analysis
a. Roots of Equation: These problems are concerned with the value of a variable or a
parameter that satisfies a single nonlinear equation. These problems are especially
valuable in engineering design contexts where it is often impossible to explicitly solve
design equations for parameters.
b. Systems of Linear Algebraic Equations: These problems are similar in spirit to roots of
equations in the sense that they are concerned with values that satisfy equations.
However, in contrast to satisfying a single equation, a set of values is sought that
simultaneously satisfies a set of linear algebraic equations. Such equations arise in a
variety of problem contexts and in all disciplines of engineering. They originate in the
mathematical modeling of large systems of interconnected elements such as structures,
electric circuits, and fluid networks. However, they are also encountered in other areas
of numerical methods such as curve fitting and differential equations.
c. Optimization: These problems involve determining a value or values of an independent

variable that correspond to a “best” or optimal value of a function. Thus, optimization
involves identifying maxima and minima. Such problems occur routinely in
engineering design contexts. They also arise in several other numerical methods. This
method addresses both single- and multi-variables unconstrained optimization and it
also describes constrained optimization with emphasis on linear programming
d. Curve Fitting: It is a process to fit curves to data points. The techniques developed for
this purpose can be divided into two general categories: regression and interpolation.
Regression is employed where there is a significant degree of error associated with the
data. Experimental results are often of this kind. For these situations, the strategy is to
derive a single curve that represents the general trend of the data without necessarily
matching any individual points. In contrast, interpolation is used where the objective is
to determine intermediate values between relatively error-free data points. Such is
usually the case for tabulated information. For these situations, the strategy is to fit a
curve directly through the data points and use the curve to predict the intermediate
values.
e. Integration: As depicted (above), a physical interpretation of numerical integration is

the determination of the area under a curve. Integration has many applications in
engineering practice, ranging from the determination of the centroids of oddly shaped
objects to the calculation of total quantities based on sets of discrete measurements. In
addition, numerical integration formulas play an important role in the solution of
differential equations.
f. Ordinary Differential Equations: Ordinary differential equations are of great

significance in engineering practice. This is because many physical laws are couched
in terms of the rate of change of a quantity rather than the magnitude of the quantity
itself. Examples range from population-forecasting models (rate of change of
population) to the acceleration of a falling body (rate of change of velocity). Two types
of problems are addressed: initial-value and boundary-value problems. In addition, the
computation of eigenvalues is covered.
6|Page Uzair Suria

g. Partial Differential Equations: Partial differential equations are used to characterize
engineering systems where the behavior of a physical quantity is couched in terms of
its rate of change with respect to two or more independent variables. Examples include
the steady-state distribution of temperature on a heated plate (two spatial dimensions)
or the time-variable temperature of a heated rod (time and one spatial dimension). Two
fundamentally different approaches are employed to solve partial differential equations
numerically. In this course finite-difference methods that approximate the solution in a
pointwise fashion will be focused. However, an introduction to finite-element methods,
which use a piecewise approach, will also be presented.
1.2.2 Characteristics of Numerical Method

 Numerical problem and numerical answer
 Several methods to solve each problem
 Fast computation (with computer’s aid)
 Direct and Indirect method
7|Page Uzair Suria

1.2.3 Iterative Method for Numerical Computation
Start
Input initial
estimate of
solution
Numerical method’s
process
Output
Improved Convergence
approximation
YES
Same value
NO up to YES Solution
required
accuracy
End
8|Page Uzair Suria

1.3 Calculation Errors
For many applied engineering problems, analytical solutions cannot be obtained. Therefore,
the errors associated with numerical methods cannot be computed exactly. In these cases, an
engineer must settle for approximations or estimates of the errors. Such errors are characteristic
of most of the techniques used in numerical analysis
1.3.1 Accuracy and Precision

The errors associated with both calculations and measurements can be characterized with
regards to their accuracy and precision.
• Accuracy refers to how closely a computed or measured value agrees with the true
value.
• Precision refers to how closely individual computed or measured values agree with each
other
• Collective term ‘error’ is used to represent both the inaccuracy and the imprecision of
the predictions
These concepts can be illustrated graphically using an analogy from target practice. The bullet
holes on each target in Fig. 1-1 below can be thought of as the predictions of a numerical
technique, whereas the bull’s-eye represents the truth. Inaccuracy (also called bias) is defined
as systematic deviation from the truth. Thus, although the shots in Fig. 1-1c are more tightly
grouped than those in Fig. 1-1a, the two cases are equally biased because they are both centered
on the upper left quadrant of the target. Imprecision (also called uncertainty), on the other hand,
refers to the magnitude of the scatter. Therefore, although Fig. 1-1b and d are equally accurate
(that is, centered on the bull’s-eye), the latter is more precise because the shots are tightly
grouped.
Figure 1-1 An example from marksmanship illustrating the concepts

of accuracy and precision. (a) Inaccurate and imprecise; (b) accurate
and imprecise; (c) inaccurate and precise; (d) accurate and precise
9|Page Uzair Suria

1.3.2 Truncation and Round-off Error
In numerical analysis and scientific computing, truncation error is the error made by truncating
an infinite sum and approximating it by a finite sum. For instance, if the sine function is
1
approximated by the first two non-zero term of its Taylor series, as in sin 𝑥𝑥 ≈ 𝑥𝑥 − 𝑥𝑥 3 (cubic
6
approximation) for small 𝑥𝑥, the resulting error is a truncation error.
A round-off error is the difference between the calculated approximation of a number and its
exact mathematical value due to rounding. This is a form of quantization error. One of the goals
of numerical analysis is to estimate errors in calculations, including round-off error, when using
approximation equations or algorithms, especially when using finitely many digits to represent
real numbers (which in theory have infinitely many digits).
1.3.3 Error Definitions

Numerical errors arise from the use of approximations to represent exact mathematical
operations and quantities. These include truncation errors, which result when approximations
are used to represent exact mathematical procedures, and round-off errors, which result when
numbers having limited significant figures are used to represent exact numbers. For both types,
the relationship between the exact, or true, result and the approximation can be formulated as:
• True error: A true error is defined as the difference between the true (exact) value and
an approximate value. This type of error is only measurable when the true value is
available
𝐸𝐸𝑡𝑡 = True Value − Approximation (2.1)
• Relative error: Relative true error, 𝜖𝜖𝑡𝑡 is defined as the ratio between the true error and
the true value. It can also be given in terms of percentage
𝐸𝐸𝑡𝑡
𝜀𝜀𝑡𝑡 = (2.2)
True value
or it can also be expressed in percentage
𝐸𝐸𝑡𝑡
𝜀𝜀𝑡𝑡 = × 100% (2.3)
True value
• Absolute error: The absolute error is the magnitude of the difference between the exact
value and the approximation
𝐸𝐸𝑎𝑎𝑎𝑎𝑎𝑎 = |True Value − Approximation| (2.4)
• Approximate error: Often the true value is unknown, which is usually the case in
numerical computing. In this case errors are quantified using approximate values only.
An iterative method produces an approximate value at the end of each iteration. The
approximate error 𝐸𝐸𝑎𝑎 , is defined as the difference between the present approximate
value and the previous approximation.
𝐸𝐸𝑎𝑎 = (Present approx − Previous approx) (2.5)
• Relative approximate error: The relative approximate error is the ratio of approximate
error with present approximation
10 | P a g e Uzair Suria
(Present approx − Previous approx)
𝜀𝜀𝑎𝑎 = (2.6)
Present approx
1.3.3.1 Stopping Criteria

Often, when performing computations, the sign of the error is not of any concern, instead the
point of interest is whether the percent absolute value is lower than a prespecified percent
tolerance 𝜀𝜀𝑠𝑠 . Therefore, it is often useful to employ the absolute value of Eqs. (2.1) through
(2.6). For such cases, the computation is repeated until
|𝜀𝜀𝑎𝑎 | = 𝜀𝜀𝑠𝑠
It is also convenient to relate these errors to the number of significant figures in the
approximation. It can be shown that if the following criterion is met, we can be assured that
the result is correct to at least n significant figures:
𝜀𝜀𝑠𝑠 = (0.5 × 102−𝑛𝑛 )%
Example 1-1
Evaluate 𝑒𝑒 −8.3 using following approximations:

𝑥𝑥 2 𝑥𝑥 3 𝑥𝑥 4 𝑥𝑥 24
i. 𝑒𝑒 −𝑥𝑥 = 1 − 𝑥𝑥 + − + + ⋯+
2! 3! 4! 24!
−1
𝑥𝑥 2 𝑥𝑥 3 𝑥𝑥 4 𝑥𝑥 24
ii. 𝑒𝑒 −𝑥𝑥 = �1 + 𝑥𝑥 + + + + ⋯+ �
2! 3! 4! 24!
and compare it with the true value ‘𝑒𝑒 −8.3 = 2.485168 × 10−4 ’.
Solution
i. The series must be expanded till 25th term: Individual terms and their sum is given below
Approx of 𝑒𝑒 −8.3 = 4.872486 × 10−3

𝐸𝐸𝑡𝑡 = 2.485168 × 10−4 − 4.872486 × 10−3 = −4.623969 × 10−3
|−4.623969 × 10−3 |
𝜀𝜀𝑡𝑡 = × 100% = 186%
2.485168 × 10−3
Example 1-1 continued…
ii. The second series must also be expanded till 25th term: Individual terms in denominator
and their sum is given as:
Then the approximate value of 𝑒𝑒 −8.3 is:

1
𝑒𝑒 −8.3 = = 2.485174 × 10−4
4.023863 × 103
The error in this approximation can be calculated as follows:
𝐸𝐸𝑡𝑡 = 2.485168 × 10−4 − 2.485174 × 10−4 = −6 × 10−10
|−6 × 10−10 |
𝜀𝜀𝑡𝑡 = × 100% = 2.41 × 10−5 %
2.485168 × 10−3
Comments: The first approximation is not very accurate and has a significant error of 186% ,
while the second approximation has negligible error 0.0000241%.
1.3.4 Taylor Polynomial Error

𝑓𝑓 ′′ (𝑥𝑥𝑖𝑖 ) 2 𝑓𝑓 (3) (𝑥𝑥𝑖𝑖 ) 3 𝑓𝑓 (𝑛𝑛) (𝑥𝑥𝑖𝑖 ) 𝑛𝑛
𝑓𝑓(𝑥𝑥𝑖𝑖+1 ) = 𝑓𝑓(𝑥𝑥𝑖𝑖 ) + 𝑓𝑓 ′ (𝑥𝑥𝑖𝑖 )ℎ + ℎ + ℎ + ⋯+ ℎ + 𝑅𝑅𝑛𝑛
2! 3! 𝑛𝑛!
where the remainder 𝑅𝑅𝑛𝑛 is given by
𝑓𝑓 (𝑛𝑛+1) (𝜉𝜉) 𝑛𝑛+1
𝑅𝑅𝑛𝑛 = ℎ
(𝑛𝑛 + 1)!
and
• 𝑥𝑥𝑖𝑖 < 𝜉𝜉 < 𝑥𝑥𝑖𝑖+1
• ℎ = 𝑥𝑥𝑖𝑖+1 − 𝑥𝑥𝑖𝑖
When series is convergent at 𝑥𝑥 → 0, then 𝑅𝑅𝑛𝑛 → 0.
Example 1-2
Use zero- through sixth-order Taylor series (𝑛𝑛 = 0 to 𝑛𝑛 = 6) to approximate 𝑓𝑓(𝑥𝑥) = cos 𝑥𝑥 at
𝑥𝑥𝑖𝑖+1 = 𝜋𝜋�3, given that the function 𝑓𝑓(𝑥𝑥) and all its derivatives exists at 𝑥𝑥𝑖𝑖 = 𝜋𝜋�4.
Solution
𝜋𝜋 𝜋𝜋 𝜋𝜋
ℎ = 𝑥𝑥𝑖𝑖+1 − 𝑥𝑥𝑖𝑖 = − =
3 4 12
Following table shows the partial sum of the series up to 6th order along with its associated
error.
Order 𝒇𝒇(𝒏𝒏) (𝒙𝒙) 𝝅𝝅 No. of terms Partial Sum 𝑬𝑬𝒂𝒂
𝒇𝒇(𝒏𝒏) � �
𝟒𝟒
0 - - 1 0.70710 -
1
1 − sin 𝑥𝑥 − 2 0.52196 0.354
√2
1
2 − cos 𝑥𝑥 − 3 0.49773 0.048
√2
1
3 sin 𝑥𝑥 4 0.49987 0.00243
√2
1
4 cos 𝑥𝑥 5 0.5000076 0.0002678
√2
1
5 − sin 𝑥𝑥 − 6 0.500000304 0.000014494
√2
1
6 − cos 𝑥𝑥 − 7 0.499999 0.000000632
√2
2 ROOTS OF EQUATIONS
2.1 Bisection Method
The bisection method, which is alternatively called binary chopping, interval halving, or
Bolzano’s method, is one type of incremental search method in which the interval is always
divided in half. It is used to compute roots for non-linear and transcendental equations.
2.1.1 Bisection Computation Process

The process is repeated to obtain refined estimates. A simple algorithm for the bisection
calculation is listed in below, and a graphical depiction of the method is provided in Fig. 2-1.
Step 1: Choose lower 𝑥𝑥𝑙𝑙 and upper 𝑥𝑥𝑢𝑢 guesses for the root such that the
function changes sign over the interval. This can be checked by ensuring
that 𝑓𝑓(𝑥𝑥𝑙𝑙 ) < 0 and 𝑓𝑓(𝑥𝑥𝑢𝑢 ) > 0.
Step 2: An estimate of the root 𝑥𝑥𝑟𝑟 is determined by
𝑥𝑥𝑙𝑙 + 𝑥𝑥𝑢𝑢
𝑥𝑥𝑟𝑟 =
2
Step 3: Make the following evaluations to determine in which subinterval
the root lies:
a) If 𝑓𝑓(𝑥𝑥𝑟𝑟 ) < 0, then the root lies between 𝑓𝑓(𝑥𝑥𝑟𝑟 ) and 𝑓𝑓(𝑥𝑥𝑢𝑢 ). Therefore
set 𝑥𝑥𝑙𝑙 = 𝑥𝑥𝑟𝑟 and return to step 2
b) If 𝑓𝑓(𝑥𝑥𝑟𝑟 ) > 0, then the root lies between 𝑓𝑓(𝑥𝑥𝑙𝑙 ) and 𝑓𝑓(𝑥𝑥𝑟𝑟 ). Therefore
set 𝑥𝑥𝑢𝑢 = 𝑥𝑥𝑟𝑟 and return to step 2
c) Terminate computation when the required precision is achieved OR
terminate when 𝑓𝑓(𝑥𝑥𝑙𝑙 ) = 𝑓𝑓(𝑥𝑥𝑢𝑢 ) = 0.
Figure 2-1 A graphical depiction of the bisection method
Example 2-1
Find the root of the equation, 𝑓𝑓(𝑥𝑥) = 𝑥𝑥𝑒𝑒 𝑥𝑥 − 5, by using bisection method. Computed result
must be precise up to three decimal places.
Solution
𝑓𝑓(0) = −5
𝑓𝑓(1) = −2.3
𝑓𝑓(2) = 0.4
From above, it is clear that the root lies between 𝑥𝑥 = 1 and 𝑥𝑥 = 2.
Following table simplifies and shows each iteration detail comprehensively:
Iteration
𝒙𝒙𝒍𝒍 𝒙𝒙𝒖𝒖 𝒙𝒙𝒓𝒓 𝒇𝒇(𝒙𝒙𝒓𝒓 )
#
1 1.00000 2.00000 1.50000 1.72253
2 1.00000 1.50000 1.25000 -0.63707
3 1.25000 1.50000 1.37500 0.43823
4 1.25000 1.37500 1.31250 -0.12347
5 1.31250 1.37500 1.34375 0.15112
6 1.31250 1.34375 1.32813 0.01229
7 1.31250 1.32813 1.32031 -0.05597
8 1.32031 1.32813 1.32422 -0.02193
9 1.32422 1.32813 1.32617 -0.00485
10 1.32617 1.32813 1.32715 0.00372
11 1.32617 1.32715 1.32666 -0.00057
12 1.32666 1.32715 1.32690 0.00158
13 1.32666 1.32690 1.32678 0.00050
2.1.1.1 Advantages of Bisection Method

1. It is always convergent to 𝑓𝑓(𝑥𝑥) = 0
2. It can be controlled
2.1.1.2 Drawbacks of Bisection Method

1. Convergence is generally slow
2. If one of the initial guesses is closer to the root, it will take larger number of iterations
to reach the root.
3. If a function 𝑓𝑓(𝑥𝑥) is such that it just touches the x-axis, such as 𝑓𝑓(𝑥𝑥) = 𝑥𝑥 2 = 0.
4. For functions 𝑓𝑓 (𝑥𝑥) where there is a singularity and it reverses sign at the singularity,
the bisection method may converge on the singularity. An example includes 1/𝑥𝑥.
Practice Questions
1. Compute a root of the function 𝑓𝑓(𝑥𝑥) = 2𝑥𝑥 − 3𝑥𝑥 in the interval using bisection method.
1
2. Find the root of the equation 𝑓𝑓(𝑥𝑥) = 𝑥𝑥 − sin 𝑥𝑥 − by using bisection method. The
2
result must be precise up to 2 decimal places.
2.2 Regula Falsi Method

Although bisection is a perfectly valid technique for determining roots, its “brute-force”
approach is relatively inefficient. False position is an alternative based on a graphical insight.
A shortcoming of the bisection method is that, in dividing the interval from 𝑥𝑥𝑙𝑙 to 𝑥𝑥𝑢𝑢 into equal
halves, no account is taken of the magnitudes of 𝑓𝑓(𝑥𝑥𝑙𝑙 ) and 𝑓𝑓(𝑥𝑥𝑢𝑢 ). For example, if 𝑓𝑓(𝑥𝑥𝑙𝑙 ) is
much closer to zero than 𝑓𝑓(𝑥𝑥𝑢𝑢 ), it is likely that the root is closer to 𝑥𝑥𝑙𝑙 than to 𝑥𝑥𝑢𝑢 . An alternative
method that exploits this graphical insight is to join 𝑓𝑓(𝑥𝑥𝑙𝑙 ) and 𝑓𝑓(𝑥𝑥𝑢𝑢 ) by a straight line. The
intersection of this line with the x-axis represents an improved estimate of the root.
The fact that the replacement of the curve by a straight line gives a “false position” of the root
is the origin of the name, method of false position, or in Latin, Regula Falsi. It is also called
the linear interpolation method.
2.2.1 Regula Falsi Computation Process

A simple algorithm for numerical analysis using Regula Falsi method is stated step by step
below:
Step 1: Choose lower 𝑥𝑥𝑙𝑙 and upper 𝑥𝑥𝑢𝑢 guesses for the root such that the
function changes sign over the interval. This can be checked by ensuring
that 𝑓𝑓(𝑥𝑥𝑙𝑙 ) < 0 and 𝑓𝑓(𝑥𝑥𝑢𝑢 ) > 0.
Step 2: An estimate of the root 𝑥𝑥𝑟𝑟 is determined by
𝑓𝑓(𝑥𝑥𝑢𝑢 )(𝑥𝑥𝑙𝑙 − 𝑥𝑥𝑢𝑢 )
𝑥𝑥𝑟𝑟 = 𝑥𝑥𝑢𝑢 −
𝑓𝑓(𝑥𝑥𝑙𝑙 ) − 𝑓𝑓(𝑥𝑥𝑢𝑢 )
Step 3: Make the following evaluations to determine in which subinterval
the root lies:
a) If 𝑓𝑓(𝑥𝑥𝑟𝑟 ) < 0, then the root lies between 𝑓𝑓(𝑥𝑥𝑟𝑟 ) and 𝑓𝑓(𝑥𝑥𝑢𝑢 ). Therefore
set 𝑥𝑥𝑙𝑙 = 𝑥𝑥𝑟𝑟 and return to step 2
b) If 𝑓𝑓(𝑥𝑥𝑟𝑟 ) > 0, then the root lies between 𝑓𝑓(𝑥𝑥𝑙𝑙 ) and 𝑓𝑓(𝑥𝑥𝑟𝑟 ). Therefore
set 𝑥𝑥𝑢𝑢 = 𝑥𝑥𝑟𝑟 and return to step 2
c) Terminate computation when the required precision is achieved OR
terminate when 𝑓𝑓(𝑥𝑥𝑙𝑙 ) = 𝑓𝑓(𝑥𝑥𝑢𝑢 ) = 0.
Figure 2-2 The graph shows numerous iterations and shows that after each iteration, the approximation gets closer to the
real value
2.2.1.1 Derivation of approximation
Consider three points:𝐴𝐴�𝑥𝑥𝑙𝑙 , 𝑓𝑓(𝑥𝑥𝑙𝑙 )�, 𝐵𝐵�𝑥𝑥𝑢𝑢 , 𝑓𝑓(𝑥𝑥𝑢𝑢 )� and 𝐶𝐶�𝑥𝑥𝑟𝑟 , 𝑓𝑓(𝑥𝑥𝑟𝑟 )�, where 𝑥𝑥𝑟𝑟 = 𝑥𝑥𝑟𝑟1
Using theory of similar triangles:
𝑓𝑓(𝑥𝑥𝑙𝑙 ) 𝑓𝑓(𝑥𝑥𝑢𝑢 )
=
𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑙𝑙 𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑢𝑢
𝑓𝑓(𝑥𝑥𝑙𝑙 )(𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑢𝑢 ) = 𝑓𝑓(𝑥𝑥𝑢𝑢 )(𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑙𝑙 )
𝑥𝑥𝑟𝑟 𝑓𝑓(𝑥𝑥𝑙𝑙 ) − 𝑥𝑥𝑢𝑢 𝑓𝑓(𝑥𝑥𝑙𝑙 ) = 𝑥𝑥𝑟𝑟 𝑓𝑓(𝑥𝑥𝑢𝑢 ) − 𝑥𝑥𝑙𝑙 𝑓𝑓(𝑥𝑥𝑢𝑢 )
𝑥𝑥𝑟𝑟 �𝑓𝑓(𝑥𝑥𝑙𝑙 ) − 𝑓𝑓(𝑥𝑥𝑢𝑢 )� = 𝑥𝑥𝑢𝑢 𝑓𝑓(𝑥𝑥𝑙𝑙 ) − 𝑥𝑥𝑙𝑙 𝑓𝑓(𝑥𝑥𝑢𝑢 )
𝑥𝑥𝑢𝑢 𝑓𝑓(𝑥𝑥𝑙𝑙 ) − 𝑥𝑥𝑙𝑙 𝑓𝑓(𝑥𝑥𝑢𝑢 )
𝑥𝑥𝑟𝑟 =
This can be further simplified:
𝑥𝑥𝑢𝑢 𝑓𝑓(𝑥𝑥𝑙𝑙 ) 𝑥𝑥𝑙𝑙 𝑓𝑓(𝑥𝑥𝑢𝑢 )
𝑥𝑥𝑟𝑟 = −
𝑓𝑓(𝑥𝑥𝑙𝑙 ) − 𝑓𝑓(𝑥𝑥𝑢𝑢 ) 𝑓𝑓(𝑥𝑥𝑙𝑙 ) − 𝑓𝑓(𝑥𝑥𝑢𝑢 )
𝑥𝑥𝑢𝑢 𝑓𝑓(𝑥𝑥𝑙𝑙 ) 𝑥𝑥𝑙𝑙 𝑓𝑓(𝑥𝑥𝑢𝑢 )
𝑥𝑥𝑟𝑟 = + 𝑥𝑥𝑢𝑢 − 𝑥𝑥𝑢𝑢 −
𝑓𝑓(𝑥𝑥𝑙𝑙 ) − 𝑓𝑓(𝑥𝑥𝑢𝑢 ) 𝑓𝑓(𝑥𝑥𝑙𝑙 ) − 𝑓𝑓(𝑥𝑥𝑢𝑢 )
𝑥𝑥𝑢𝑢 𝑓𝑓(𝑥𝑥𝑢𝑢 ) − 𝑥𝑥𝑙𝑙 𝑓𝑓(𝑥𝑥𝑢𝑢 )
= 𝑥𝑥𝑢𝑢 +
𝑓𝑓(𝑥𝑥𝑢𝑢 )(𝑥𝑥𝑙𝑙 − 𝑥𝑥𝑢𝑢 )
𝑥𝑥𝑟𝑟 = 𝑥𝑥𝑢𝑢 −
Example 2-2
Compute a real root of 𝑥𝑥 3 − 3𝑥𝑥 − 5 using Regula Falsi method precise up to 3 decimal places.
Solution
𝑓𝑓(2) = −3
𝑓𝑓(3) = 13
From above, it is clear that the root lies between 𝑥𝑥 = 2 and 𝑥𝑥 = 3.
13(2 − 3)
𝑥𝑥𝑟𝑟 = 3 − = 2.1875
−3 − 13
𝑓𝑓(𝑥𝑥𝑟𝑟 ) = −1.09497
Therefore, now 𝑥𝑥𝑙𝑙 = 𝑥𝑥𝑟𝑟 = 2.1875
Following table simplifies and shows each iteration detail comprehensively:
Iteration 𝒙𝒙𝒍𝒍 𝒙𝒙𝒖𝒖 𝒙𝒙𝒓𝒓 𝒇𝒇(𝒙𝒙𝒓𝒓 )
#
1 2.00000 3.00000 2.18750 -1.09497
2 2.18750 3.00000 2.25062 -0.35183
3 2.25062 3.00000 2.27037 -0.10836
4 2.27037 3.00000 2.27640 -0.03294
5 2.27640 3.00000 2.27823 -0.00997
6 2.27823 3.00000 2.27878 -0.00302
The root of the given equation is 2.278 precise up to 3 decimal places.
Practice Questions
1. Find the real root of the equation 𝑓𝑓(𝑥𝑥) = 𝑥𝑥𝑒𝑒 𝑥𝑥 − cos 𝑥𝑥 precise up to 4 decimal places
2. Find the real root of an equation 𝑓𝑓(𝑥𝑥) = 3𝑥𝑥 + sin 𝑥𝑥 − 𝑒𝑒 𝑥𝑥 , where 𝑥𝑥 ∈ [0, 1]. Perform
only five iterations. (Ans. 0.3604217)
2.3 Fixed Point Iteration Method
Open methods (such as bisection and regula falsi method) employ a formula to predict the root.
Such a formula can be developed for simple fixed-point iteration (or, as it is also called, one-
point iteration or successive substitution) by rearranging the function 𝑓𝑓(𝑥𝑥) = 0 so that 𝑥𝑥 is on
the left-hand side of the equation:
𝑥𝑥 = 𝑔𝑔(𝑥𝑥)
This transformation can be accomplished either by algebraic manipulation or by simply adding
𝑥𝑥 to both sides of the original equation. For example:
𝑥𝑥 2 − 2𝑥𝑥 + 3 = 0
can simply be rewritten as
𝑥𝑥 2 + 3
𝑥𝑥 =
2
while sin 𝑥𝑥 = 0 can be rewritten as
𝑥𝑥 = sin 𝑥𝑥 + 𝑥𝑥
2.3.1 Convergence Criteria

In complex problems, careful deliberation is required in the evaluation of 𝑔𝑔(𝑥𝑥) and selection
of 𝑥𝑥0 , so that approximations converge. Once 𝑔𝑔(𝑥𝑥) and 𝑥𝑥0 have been selected following
criteria can be used to determine the behavior of iterative approximations:
When
• |𝑔𝑔(𝑥𝑥0 )| < 1 then the approximations will converge
 0 < 𝑔𝑔(𝑥𝑥0 ) < 1, the approximations will converge
 𝑔𝑔(𝑥𝑥0 ) ≤ 0, the approximations will converge while oscillating
• 𝑔𝑔(𝑥𝑥0 ) > 1, the approximations will diverge
2.3.2 Fixed Point Iteration Computation

A simple algorithm for numerical analysis using Fixed Point Iteration method is stated step by
step below
Step 1: Rewrite 𝑓𝑓(𝑥𝑥) = 0 as 𝑥𝑥 = 𝑔𝑔(𝑥𝑥)

Step 2: Take 𝑥𝑥0 as the initial estimate of the root of 𝑓𝑓(𝑥𝑥). If |𝑔𝑔′(𝑥𝑥0 )| < 1,
then the condition to use iteration formula satisfies.
Step 3: Use following iteration formula until results up to the required
precision is obtained
𝑥𝑥𝑛𝑛+1 = 𝑔𝑔(𝑥𝑥𝑛𝑛 ) 𝑛𝑛 = 0, 1, 2, … , 𝑛𝑛
Figure 2-3 Graphical
representation of fixed-
point method. Graphs (a)
and (b) represents
convergence while (c)
and (d) represent
divergence
Example 2-3
Find the root of the equation 𝑓𝑓(𝑥𝑥) = cos 𝑥𝑥 + 1 − 3𝑥𝑥

Solution
cos 𝑥𝑥 + 1
𝑥𝑥 = (Iteration Formula)
3
𝑓𝑓(0) = 2
𝜋𝜋
𝑓𝑓 � � = −3.172
2
Therefore, the root lies between 0 and 𝜋𝜋/2.
Let 𝑥𝑥0 = 0, then
𝑔𝑔ʹ(𝑥𝑥0 ) = 𝑔𝑔ʹ(0) = 0
since 𝑔𝑔ʹ(𝑥𝑥0 ) < 1, the condition to use iteration formula is satisfied. Iterations are listed in
the table below:
𝑛𝑛 𝑥𝑥𝑛𝑛 𝑔𝑔(𝑥𝑥𝑛𝑛 )
1 0.0000000 0.6666667
2 0.6666667 0.5952958
3 0.5952958 0.6093276
4 0.6093276 0.6066777
𝑛𝑛 𝑥𝑥𝑛𝑛 𝑔𝑔(𝑥𝑥𝑛𝑛 )
5 0.6066777 0.6071822
6 0.6071822 0.6070863
7 0.6070863 0.6071046
8 0.6071046 0.6071011
9 0.6071011 0.6071018
10 0.6071018 0.6071016
The root of the equation, precise to five decimal places, is 0.607101
2.4 Newton Raphson Method

The most widely used of all root-locating formulas is the Newton-Raphson equation. If the
initial guess at the root is 𝑥𝑥𝑖𝑖 , a tangent can be extended from the point [𝑥𝑥𝑖𝑖 , 𝑓𝑓(𝑥𝑥𝑖𝑖 )]. The point
where this tangent cross the x-axis usually represents an improved estimate of the root.
2.4.1 Computation with Newton-Raphson Method
Step 1: Let 𝑥𝑥0 be any value of 𝑥𝑥 (in the domain where 𝑓𝑓(𝑥𝑥) changes sign),
or the initial approximate given.
Step 2: Use following iteration formula until results up to the required
precision is obtained
𝑓𝑓(𝑥𝑥𝑖𝑖 )
𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 − 𝑖𝑖 = 0, 1, 2, … , 𝑛𝑛
𝑓𝑓′(𝑥𝑥𝑖𝑖 )
Figure 2-4 Graphical depiction of the Newton-Raphson method.

A tangent to the function of 𝒙𝒙𝒊𝒊 [that is, 𝒇𝒇(𝒙𝒙𝒊𝒊 )] is extrapolated
down to the x-axis to provide an estimate of the root at 𝒙𝒙𝒊𝒊+𝟏𝟏 .
Example 2-4
Find the root of the equation 3𝑥𝑥 − cos 𝑥𝑥 − 1 = 0 using Newton-Raphson method, precise upto
4 decimal places
Solution:
𝑓𝑓(𝑥𝑥) = 3𝑥𝑥 − cos 𝑥𝑥 − 1
𝑓𝑓(0) = −2
𝑓𝑓(1) = 1.4596
Therefore 𝑥𝑥 ∈ [0, 1], and
𝑓𝑓(𝑥𝑥𝑛𝑛 ) = 3𝑥𝑥𝑛𝑛 − cos 𝑥𝑥𝑛𝑛 − 1
𝑓𝑓 ′(𝑥𝑥𝑛𝑛) = 3 + sin 𝑥𝑥𝑛𝑛
Using Newton-Raphson method:
𝑓𝑓(𝑥𝑥𝑛𝑛 )
𝑥𝑥𝑛𝑛+1 = 𝑥𝑥𝑛𝑛 −
𝑓𝑓 ′ (𝑥𝑥𝑛𝑛 )
3𝑥𝑥𝑛𝑛 − cos 𝑥𝑥𝑛𝑛 − 1
= 𝑥𝑥𝑛𝑛 −
3 + sin 𝑥𝑥𝑛𝑛
𝑥𝑥𝑛𝑛 sin 𝑥𝑥𝑛𝑛 + cos 𝑥𝑥𝑛𝑛 + 1
=
3 + sin 𝑥𝑥𝑛𝑛
Taking 𝑥𝑥0 = 0.5, then

𝑥𝑥1 = 0.6085
𝑥𝑥2 = 0.6071
𝑥𝑥3 = 0.6071
The root of the given function is 0.6071
2.4.1.1 Advantages of Newton-Raphson Method

1. Fast convergence (quadratic convergence), if it converges
2. Open method i.e. requires only one guess
2.4.1.2 Drawbacks/Limitations of Newton-Raphson Method

Newton-Raphson Method does not perform well when
1. Used with diverging sequence i.e., 𝑥𝑥𝑒𝑒 −𝑥𝑥
2. Used with cyclic sequence
3. Used with diverging oscillating sequence
4. The function diverges at the point of inflection
5. Zero gradient at any of the approximation, i.e., zero in denominator, will result in
divergence
Thus, there is no general convergence criterion for Newton-Raphson. Its convergence depends
on the nature of the function and on the accuracy of the initial guess. The only remedy is to
have an initial guess that is “sufficiently” close to the root. And for some functions, no guess
will work. Good guesses are usually predicated on knowledge of the physical problem setting
or on devices such as graphs that provide insight into the behavior of the solution
Figure 2-5 Graphical representation of limitations of Newton-Raphson
Method. (a) root Jumping, (b) oscillation near local maxima or minima,
(c) another form of root jumping, and (d) zero gradient at an
approximation (zero in denominator)
Practice Questions
Find the root of the following equations precise up to 4 decimal points:
i. 𝑥𝑥 − cos 𝑥𝑥 = 0; 𝑥𝑥 ∈ [0,1]
ii. 2𝑥𝑥 − ln 𝑥𝑥 − 7 = 0; 𝑥𝑥 ∈ [4, 5]
iii. 2𝑥𝑥 − log 𝑥𝑥 − 7 = 0; 𝑥𝑥 ∈ [3, 4]
iv. 𝑥𝑥 sin 𝑥𝑥 + cos 𝑥𝑥 = 0; 𝑥𝑥0 = 𝜋𝜋
Answers: (i)0.7390, (ii)4.2199, (iii)3.7893 (iv)2.7983
3 FINITE DIFFERENCES AND
INTERPOLATION
3.1 Finite Differences
Given that 𝑦𝑦 = 𝑓𝑓(𝑥𝑥) for a set of values of 𝑥𝑥, such that:
𝑥𝑥: 𝑥𝑥0 𝑥𝑥1 𝑥𝑥2 … 𝑥𝑥𝑛𝑛
𝑦𝑦: 𝑦𝑦0 𝑦𝑦1 𝑦𝑦2 … 𝑦𝑦𝑛𝑛
Then the process of finding the values of 𝑦𝑦 corresponding to any value of 𝑥𝑥 = 𝑥𝑥𝑖𝑖 between 𝑥𝑥0
and 𝑥𝑥𝑛𝑛 is called interpolation. Thus, interpolation is the technique of estimating the value of a
function for any intermediate value of the independent variable while the process of computing
the value of the function outside the given range is called extrapolation.
The study of the interpolation is based on the concept of differences of a function which is
discussed below
3.1.1 Forward Differences

The differences 𝑦𝑦1 − 𝑦𝑦0 , 𝑦𝑦2 − 𝑦𝑦1 , … , 𝑦𝑦𝑛𝑛 − 𝑦𝑦𝑛𝑛−1 are known as first forward differences and they
are denoted by Δ𝑦𝑦0 , Δ𝑦𝑦1 , … , Δ𝑦𝑦𝑛𝑛−1 . Therefore, the forward difference can be defined as Δ𝑦𝑦𝑟𝑟 =
𝑦𝑦𝑟𝑟+1 − 𝑦𝑦𝑟𝑟 .
Similarly, second forward differences are defined by Δ2 𝑦𝑦𝑟𝑟 = Δ𝑦𝑦𝑟𝑟+1 − Δ𝑦𝑦𝑟𝑟 .
In general, the 𝑝𝑝𝑡𝑡ℎ forward differences are defined as Δ𝑝𝑝 𝑦𝑦𝑟𝑟 = Δ𝑝𝑝−1 𝑦𝑦𝑟𝑟+1 − Δ𝑝𝑝−1 𝑦𝑦𝑟𝑟 .
3.1.1.1 Forward Difference Tables

The differences from 1𝑠𝑠𝑠𝑠 up to 𝑝𝑝𝑡𝑡ℎ can be systematically listed in a forward difference table.
In a difference table 𝑥𝑥 is called an argument, and 𝑦𝑦 is called the function or the entry. While
𝑦𝑦0 , the first term, is known as leading term, and Δ𝑦𝑦0 , Δ2 𝑦𝑦0 , Δ3 𝑦𝑦0 etc. are called leading
differences.
𝒙𝒙 𝒚𝒚 𝚫𝚫𝒚𝒚 𝚫𝚫𝟐𝟐 𝒚𝒚 𝚫𝚫𝟑𝟑 𝒚𝒚 𝚫𝚫𝟒𝟒 𝒚𝒚 𝚫𝚫𝟓𝟓 𝒚𝒚

𝒙𝒙𝟎𝟎 𝑦𝑦0
Δ𝑦𝑦0
𝒙𝒙𝟎𝟎 + 𝒉𝒉 𝑦𝑦1 Δ2 𝑦𝑦0
Δ𝑦𝑦1 Δ3 𝑦𝑦0
𝒙𝒙𝟎𝟎 + 𝟐𝟐𝟐𝟐 𝑦𝑦2 Δ2 𝑦𝑦1 Δ4 𝑦𝑦0
Δ𝑦𝑦2 Δ3 𝑦𝑦1 Δ5 𝑦𝑦0
𝒙𝒙𝟎𝟎 + 𝟑𝟑𝟑𝟑 𝑦𝑦3 Δ2 𝑦𝑦2 Δ4 𝑦𝑦1
Δ𝑦𝑦3 Δ3 𝑦𝑦2
𝒙𝒙𝟎𝟎 + 𝟒𝟒𝟒𝟒 𝑦𝑦4 Δ2 𝑦𝑦3
Δ𝑦𝑦4
𝒙𝒙𝟎𝟎 + 𝟓𝟓𝟓𝟓 𝑦𝑦5
Example 3-1
Construct a differences table for the function 𝑓𝑓(𝑥𝑥) = 𝑥𝑥 4 , where 𝑥𝑥 = −2(1) ⋅ 3
Solution
The description of domain of 𝑥𝑥 given in the question can be interpreted as follows: 𝑥𝑥 starts from
−2 and ascends to the value of 3 with steps of 1 unit.
−𝟐𝟐 16
−15
−𝟏𝟏 1 14
−1 −12
𝟎𝟎 0 2 24
1 12 0
𝟏𝟏 1 14 24
15 36
𝟐𝟐 16 50
65
𝟑𝟑 81
Note that the 𝑛𝑛𝑡𝑡ℎ differences column will have constant value and (𝑛𝑛 + 1)𝑡𝑡ℎ differences
column will have a value of zero for 𝑛𝑛𝑡𝑡ℎ degree polynomial.
3.1.2 Backward Differences

The differences 𝑦𝑦1 − 𝑦𝑦0 , 𝑦𝑦2 − 𝑦𝑦1 , … , 𝑦𝑦𝑛𝑛 − 𝑦𝑦𝑛𝑛−1 are known as first backward differences when
they are denoted by ∇𝑦𝑦0 , ∇𝑦𝑦1 , … , ∇𝑦𝑦𝑛𝑛−1 . Therefore, the forward difference can be defined as
∇𝑦𝑦𝑟𝑟 = 𝑦𝑦𝑟𝑟 − 𝑦𝑦𝑟𝑟−1 .
Similarly, second backward differences are defined by ∇2 𝑦𝑦𝑟𝑟 = ∇𝑦𝑦𝑟𝑟 − ∇𝑦𝑦𝑟𝑟−1 .
In general, the 𝑝𝑝𝑡𝑡ℎ backward differences are defined as ∇𝑝𝑝 𝑦𝑦𝑟𝑟 = ∇𝑝𝑝−1 𝑦𝑦𝑟𝑟 − ∇𝑝𝑝−1 𝑦𝑦𝑟𝑟−1 .
3.1.2.1 Backward Difference Tables
The differences can be systematically listed in a backward difference table. In a difference
table 𝑥𝑥 is called an argument, and 𝑦𝑦 is called the function or the entry.
𝒙𝒙 𝒚𝒚 𝛁𝛁𝒚𝒚 𝛁𝛁𝟐𝟐 𝒚𝒚 𝛁𝛁𝟑𝟑 𝒚𝒚 𝛁𝛁𝟒𝟒 𝒚𝒚 𝛁𝛁𝟓𝟓 𝒚𝒚

∇𝑦𝑦1
𝒙𝒙𝟎𝟎 + 𝒉𝒉 𝑦𝑦1 ∇2 𝑦𝑦2
∇𝑦𝑦2 ∇3 𝑦𝑦3
𝒙𝒙𝟎𝟎 + 𝟐𝟐𝟐𝟐 𝑦𝑦2 ∇2 𝑦𝑦3 ∇4 𝑦𝑦4
∇𝑦𝑦3 ∇3 𝑦𝑦4 ∇5 𝑦𝑦5
𝒙𝒙𝟎𝟎 + 𝟑𝟑𝟑𝟑 𝑦𝑦3 ∇2 𝑦𝑦4 ∇4 𝑦𝑦5
∇𝑦𝑦4 ∇3 𝑦𝑦5
𝒙𝒙𝟎𝟎 + 𝟒𝟒𝟒𝟒 𝑦𝑦4 ∇2 𝑦𝑦5
∇𝑦𝑦5
𝒙𝒙𝟎𝟎 + 𝟓𝟓𝟓𝟓 𝑦𝑦5
Alternative notations can also be used for backward difference tables:
𝒙𝒙 𝒚𝒚 𝛁𝛁𝒚𝒚 𝛁𝛁𝟐𝟐 𝒚𝒚 𝛁𝛁𝟑𝟑 𝒚𝒚 𝛁𝛁𝟒𝟒 𝒚𝒚 𝛁𝛁𝟓𝟓 𝒚𝒚

𝒙𝒙𝟎𝟎 − 𝟓𝟓𝟓𝟓 𝑦𝑦−5
∇𝑦𝑦−4
𝒙𝒙𝟎𝟎 − 𝟒𝟒𝟒𝟒 𝑦𝑦−4 ∇2 𝑦𝑦−3
∇𝑦𝑦−3 ∇3 𝑦𝑦−2
𝒙𝒙𝟎𝟎 − 𝟑𝟑𝟑𝟑 𝑦𝑦−3 ∇2 𝑦𝑦−2 ∇4 𝑦𝑦−1
∇𝑦𝑦−2 ∇3 𝑦𝑦−1 ∇5 𝑦𝑦0
𝒙𝒙𝟎𝟎 − 𝟐𝟐𝟐𝟐 𝑦𝑦−2 ∇2 𝑦𝑦−1 ∇4 𝑦𝑦0
∇𝑦𝑦−1 ∇3 𝑦𝑦0
𝒙𝒙𝟎𝟎 − 𝒉𝒉 𝑦𝑦−1 ∇2 𝑦𝑦0
∇𝑦𝑦0
Example 3-1
Construct a differences table for the function 𝑓𝑓(𝑥𝑥) = 3𝑥𝑥 3 − 2𝑥𝑥 2 + 𝑥𝑥 + 5; where 𝑥𝑥 = 0(1) ⋅ 4
Solution
The description of domain of 𝑥𝑥 given in the question can be interpreted as follows: 𝑥𝑥 starts from
0 and ascends to the value of 4 with steps of 1 unit.
𝒙𝒙 𝒚𝒚 𝛁𝛁𝒚𝒚 𝛁𝛁𝟐𝟐 𝒚𝒚 𝛁𝛁𝟑𝟑 𝒚𝒚 𝛁𝛁𝟒𝟒 𝒚𝒚
0 5
2
1 7 14
16 18
2 23 32 0
48 18
3 71 50
98
4 169
3.1.3 Central Differences

Sometimes it is convenient to employ another system of differences known as central
differences. In this system, the central difference operator o is defined by the relations:
𝑦𝑦1 − 𝑦𝑦0 = 𝛿𝛿𝑦𝑦1⁄2 , 𝑦𝑦2 − 𝑦𝑦1 = 𝛿𝛿𝑦𝑦3⁄2 , …
In general,
𝑦𝑦𝑛𝑛 − 𝑦𝑦𝑛𝑛−1 = 𝛿𝛿𝑦𝑦𝑛𝑛−1⁄2
Similarly, higher order central differences are defined as:
𝛿𝛿𝑦𝑦𝑛𝑛 − 𝛿𝛿𝑦𝑦𝑛𝑛−1 = 𝛿𝛿 2 𝑦𝑦𝑛𝑛−1⁄2
𝛿𝛿 2 𝑦𝑦𝑛𝑛 − 𝛿𝛿 2 𝑦𝑦𝑛𝑛−1 = 𝛿𝛿 3 𝑦𝑦𝑛𝑛−1⁄2
𝛿𝛿 𝑝𝑝 𝑦𝑦𝑛𝑛 − 𝛿𝛿 𝑝𝑝 𝑦𝑦𝑛𝑛−1 = 𝛿𝛿 𝑝𝑝+1 𝑦𝑦𝑛𝑛−1⁄2
3.1.3.1 Central Difference Table

Like previous difference operators, the central difference can also be represented by a table
(shown on the next page).
It should be noted that central differences on the same horizontal lines have same suffix.
Moreover, the difference columns of odd order have half values in the suffix, while columns
of even order have integral values for suffix.
𝒙𝒙 𝒚𝒚 𝛅𝛅𝒚𝒚 𝛅𝛅𝟐𝟐 𝒚𝒚 𝛅𝛅𝟑𝟑 𝒚𝒚 𝛅𝛅𝟒𝟒 𝒚𝒚 𝛅𝛅𝟓𝟓 𝒚𝒚
δ𝑦𝑦1/2
𝒙𝒙𝟎𝟎 + 𝒉𝒉 𝑦𝑦1 δ2 𝑦𝑦1
𝛿𝛿𝑦𝑦3/2 δ3 𝑦𝑦3/2
𝒙𝒙𝟎𝟎 + 𝟐𝟐𝟐𝟐 𝑦𝑦2 δ2 𝑦𝑦2 δ4 𝑦𝑦2
δ𝑦𝑦5/2 δ3 𝑦𝑦5/2 δ5 𝑦𝑦5/2
𝒙𝒙𝟎𝟎 + 𝟑𝟑𝟑𝟑 𝑦𝑦3 δ2 𝑦𝑦3 δ4 𝑦𝑦3
δ𝑦𝑦7/2 δ3 𝑦𝑦7/2
𝒙𝒙𝟎𝟎 + 𝟒𝟒𝟒𝟒 𝑦𝑦4 δ2 𝑦𝑦4
δ𝑦𝑦9/2
𝒙𝒙𝟎𝟎 + 𝟓𝟓𝟓𝟓 𝑦𝑦5
3.1.4 Other Difference Operators

Operators Δ, ∇ and 𝛿𝛿 have already been discussed. Apart from these three, there are three more
operators and those are: 𝜇𝜇, 𝐸𝐸 and 𝐷𝐷
3.1.4.1 Shift Operator ‘E’

In mathematics, and in particular functional analysis, the shift operator also known as
translation operator is an operator that takes a function 𝑥𝑥 ↦ 𝑓𝑓(𝑥𝑥) and increases its argument
by ℎ. Generally,
𝐸𝐸 𝑓𝑓(𝑥𝑥) = 𝑓𝑓(𝑥𝑥 + ℎ)
−1
𝐸𝐸 𝑓𝑓(𝑥𝑥) = 𝑓𝑓(𝑥𝑥 − ℎ)
𝐸𝐸 𝑛𝑛 𝑓𝑓(𝑥𝑥) = 𝑓𝑓(𝑥𝑥 + 𝑛𝑛ℎ)
3.1.4.2 Averaging Operator ‘𝜇𝜇’

The averaging operator, denoted by 𝜇𝜇, gives the average/mean value of two points, i.e.
1
𝜇𝜇𝑦𝑦𝑥𝑥 = �𝑦𝑦 ℎ + 𝑦𝑦 ℎ �
2 𝑥𝑥+2 𝑥𝑥− 2
3.1.4.3 Differential Operator ‘D’

Differential operators are a generalization of the operation of differentiation. The simplest
differential operator 𝐷𝐷 acting on a function 𝑓𝑓(𝑥𝑥), “returns” the first derivative of this function:
𝐷𝐷𝐷𝐷(𝑥𝑥) = 𝑓𝑓 ′ (𝑥𝑥)
𝐷𝐷𝑛𝑛 𝑓𝑓(𝑥𝑥) = 𝑓𝑓 𝑛𝑛 (𝑥𝑥)
Example 3-2
Prove that
i. Δ𝐸𝐸 −1 sin 𝑥𝑥 = ∇ sin 𝑥𝑥
Δ𝑓𝑓(𝑥𝑥)
ii. Δ log 𝑓𝑓(𝑥𝑥) = log �1 + �𝑓𝑓(𝑥𝑥)�
𝑥𝑥
iii. Δ2 𝐸𝐸 −1 𝑒𝑒 𝑥𝑥 ⋅ �𝐸𝐸 𝑒𝑒 �Δ2 𝑒𝑒 𝑥𝑥 � = 𝑒𝑒 𝑥𝑥
Solution
i. Taking left hand side (and ℎ = 1)
= Δ𝐸𝐸 −1 sin 𝑥𝑥
= Δ sin(𝑥𝑥 − ℎ)
= sin(𝑥𝑥 − ℎ + ℎ) − sin(𝑥𝑥 − ℎ)
= sin(𝑥𝑥) − sin(𝑥𝑥 − ℎ) = ∇ sin 𝑥𝑥
= ∇ sin 𝑥𝑥
Proved
ii. Taking left hand side (and ℎ = 1)
= Δ log 𝑓𝑓(𝑥𝑥)
= log 𝑓𝑓(𝑥𝑥 + ℎ) − log 𝑓𝑓(𝑥𝑥)
𝑓𝑓(𝑥𝑥 + ℎ)
= log � �
𝑓𝑓(𝑥𝑥)
𝑓𝑓(𝑥𝑥 + ℎ) − 𝑓𝑓(𝑥𝑥) + 𝑓𝑓(𝑥𝑥)
= log � �
𝑓𝑓(𝑥𝑥)
Δ𝑓𝑓(𝑥𝑥) + 𝑓𝑓(𝑥𝑥)
= log � �
𝑓𝑓(𝑥𝑥)
Δ𝑓𝑓(𝑥𝑥)
= log �1 + �
𝑓𝑓(𝑥𝑥)
Proved
iii. Taking left hand side (and ℎ = 1)
𝐸𝐸𝑒𝑒 𝑥𝑥
= Δ2 𝐸𝐸 −1 𝑒𝑒 𝑥𝑥
Δ2 𝑒𝑒 𝑥𝑥
2 𝑥𝑥−ℎ
𝑒𝑒 𝑥𝑥+ℎ
= Δ 𝑒𝑒
Δ2 𝑒𝑒 𝑥𝑥
Δ2 𝑒𝑒 𝑥𝑥−ℎ+𝑥𝑥+ℎ−𝑥𝑥
=
Δ2
𝑥𝑥
= 𝑒𝑒
Proved
Practice Questions
Evaluate:
i. Δ tan−1 𝑥𝑥
ii. Δ(𝑒𝑒 𝑥𝑥 log 2𝑥𝑥)
iii. Δ2 cos 2𝑥𝑥
iv. Δ𝑛𝑛 𝑒𝑒 𝑥𝑥
3.1.5 Relationship Between Operators

3.1.5.1 Forward Difference and Shift Operator
Δ𝑦𝑦𝑥𝑥 = 𝑦𝑦𝑥𝑥+ℎ − 𝑦𝑦𝑥𝑥
= 𝐸𝐸𝑦𝑦𝑥𝑥 − 𝑦𝑦𝑥𝑥
= (𝐸𝐸 − 1)𝑦𝑦𝑥𝑥
Then
Δ = 𝐸𝐸 − 1
3.1.5.2 Backward Difference and Shift Operator

∇𝑦𝑦𝑥𝑥 = 𝑦𝑦𝑥𝑥 − 𝑦𝑦𝑥𝑥−ℎ
= 𝑦𝑦𝑥𝑥 − 𝐸𝐸 −1 𝑦𝑦𝑥𝑥
= (1 − 𝐸𝐸 −1 )𝑦𝑦𝑥𝑥
Then
∇= 1 − 𝐸𝐸 −1
3.1.5.3 Central Difference and Shift Operator

𝛿𝛿𝑦𝑦𝑥𝑥 = 𝑦𝑦𝑥𝑥+ℎ⁄2 − 𝑦𝑦𝑥𝑥−ℎ⁄2
= 𝐸𝐸 1⁄2 𝑦𝑦𝑥𝑥 − 𝐸𝐸 −1⁄2 𝑦𝑦𝑥𝑥
= 𝑦𝑦𝑥𝑥 �𝐸𝐸 1⁄2 − 𝐸𝐸 −1⁄2 �
Then
𝛿𝛿 = 𝐸𝐸 1⁄2 − 𝐸𝐸 −1⁄2
3.1.5.4 Mean difference and Shift Operator

1
𝜇𝜇𝑦𝑦𝑥𝑥 =�𝑦𝑦 + 𝑦𝑦𝑥𝑥−ℎ⁄2 �
2 𝑥𝑥+ℎ⁄2
1
= �𝐸𝐸 1⁄2 𝑦𝑦𝑥𝑥 + 𝐸𝐸 −1⁄2 𝑦𝑦𝑥𝑥 �
2
1
= 𝑦𝑦𝑥𝑥 �𝐸𝐸 1⁄2 + 𝐸𝐸 −1⁄2 �
2
Then
1
𝜇𝜇 = �𝐸𝐸 1⁄2 + 𝐸𝐸 −1⁄2 �
2
Example 3-3
Prove whether the following relations are correct or not:
Δ+∇
i. 𝜇𝜇𝜇𝜇 ≡
2
ii. Δ∇≡ ∇ − Δ
𝛿𝛿 2
iii. 1 + 𝜇𝜇2 𝛿𝛿 2 ≡ �1 + �
2
Solution
i. Taking LHS
1
𝜇𝜇𝜇𝜇 = �𝐸𝐸 1⁄2 + 𝐸𝐸 −1⁄2 ��𝐸𝐸 1⁄2 − 𝐸𝐸 −1⁄2 �
2
1 2⁄2
= �𝐸𝐸 − 𝐸𝐸 −2⁄2 �
2
1
= (𝐸𝐸 − 𝐸𝐸 −1 )
2
1
= (𝐸𝐸 − 1 + 1−𝐸𝐸 −1 )
2
[(𝐸𝐸 − 1) + (1 − 𝐸𝐸 −1 )]
=
2
Δ+∇
=
2
Correct Relation
ii. Taking LHS
Δ∇= (𝐸𝐸 − 1)(1 − 𝐸𝐸 −1 )
= 𝐸𝐸 − 1 − 1 + 𝐸𝐸 −1
= 𝐸𝐸 − 2 + 𝐸𝐸 −1
Taking RHS
∇ − Δ = 1 − 𝐸𝐸 −1 − 𝐸𝐸 + 1
= 2 − 𝐸𝐸 −1 − 𝐸𝐸
LHS ≠ RHS
Incorrect Relation
iii. Taking LHS
2
𝐸𝐸 1⁄2 + 𝐸𝐸 −1⁄2 2
2 2
1 + 𝜇𝜇 𝛿𝛿 = 1 + � � ⋅ �𝐸𝐸 1⁄2 − 𝐸𝐸 −1⁄2 �
2
𝐸𝐸 + 2 + 𝐸𝐸 −1
=1+� � (𝐸𝐸 − 2 + 𝐸𝐸 −1 )
4
𝐸𝐸 2 − 2 − 𝐸𝐸 −2
=1+
4
𝐸𝐸 2 + 2 − 𝐸𝐸 −2
= … (1)
4
Now taking RHS
2 2
�𝐸𝐸 1⁄2 − 𝐸𝐸 −1⁄2 �
= �1 + �
2
2
𝐸𝐸 − 2 + 𝐸𝐸 −1
= �1 + �
2
2
2 + 𝐸𝐸 − 2 + 𝐸𝐸 −1
=� �
2
2
𝐸𝐸 + 𝐸𝐸 −1
=� �
2
𝐸𝐸 2 + 2 + 𝐸𝐸 −2
= … (2)
4
(1) = (2)
Correct Relation
Practice Questions
Prove that
Δ ∇
i. Δ + ∇= −
∇ Δ
ii. Δ3 𝑦𝑦2 = ∇3 𝑦𝑦5
iii. (1 + Δ)(1 − ∇) = 1
iv. Δ∇= ∇Δ = 𝛿𝛿 2
3.2 Interpolation with equal Intervals

3.2.1 Finding One or more Missing Terms
When one or more values of 𝑦𝑦 = 𝑓𝑓(𝑥𝑥) corresponding to the equidistant values of 𝑥𝑥 are
missing, we can find these using any of the following two methods:
First method: We assume the missing term or terms as 𝑎𝑎, 𝑏𝑏 etc. and form the difference table.
Assuming the last difference as zero, we solve these equations for 𝑎𝑎, 𝑏𝑏. These give the missing
term/terms.
Second method: If 𝑛𝑛 entries of 𝑦𝑦 are given, 𝑓𝑓(𝑥𝑥) can be represented by a (𝑛𝑛 − 1)th degree
polynomial i.e., Δ𝑛𝑛 𝑦𝑦 = 0 . Since Δ = 𝐸𝐸 − 1 , therefore (𝐸𝐸 − 1)𝑛𝑛 𝑦𝑦 = 0 . Now expanding
(𝐸𝐸 − 1)𝑛𝑛 and substituting the given values, we obtain the missing term/terms.
Example 3-4
Find the missing term in the table
𝑥𝑥: 2 3 4 5 6
𝑦𝑦: 45.0 49.2 54.1 … 67.4
Solution
Analytical Solution:
Since four entries are given, 𝑓𝑓(𝑥𝑥) can be represented by 3rd degree polynomial such that
Δ3 𝑦𝑦 = Constant
Then
Δ4 𝑦𝑦 = 0
= (𝐸𝐸 − 1)4 = 0
Using binomial theorem
(𝐸𝐸 − 1)4 𝑦𝑦0 = (𝐸𝐸 4 − 4𝐸𝐸 3 + 6𝐸𝐸 2 − 4𝐸𝐸 + 1)𝑦𝑦0
= 𝑦𝑦4 − 4𝑦𝑦3 + 6𝑦𝑦2 − 4𝑦𝑦1 + 𝑦𝑦0 = 0
Taking 𝑦𝑦3 = 𝑎𝑎 and substituting the values:

67.4 − 4𝑎𝑎 + 6 × 54.1 − 4 × 49.2 + 45.0 = 0
𝒂𝒂 = 𝒚𝒚𝟑𝟑 = 𝟔𝟔𝟔𝟔. 𝟎𝟎𝟎𝟎
Table Method:
Let the missing term be 𝑎𝑎, then the difference table is as follows:
𝒙𝒙 𝒚𝒚 𝚫𝚫𝒚𝒚 𝚫𝚫𝟐𝟐 𝒚𝒚 𝚫𝚫𝟑𝟑 𝒚𝒚 𝚫𝚫𝟒𝟒 𝒚𝒚

2 45.0(= 𝑦𝑦0 )
4.2
𝟑𝟑 49.2(= 𝑦𝑦1 ) 0.7
49.9 𝑎𝑎 −59.7
𝟒𝟒 54.1(= 𝑦𝑦2 ) 𝑎𝑎 −59.0 240.2−4𝑎𝑎
𝑎𝑎 −54.1 180.5−3𝑎𝑎
𝟓𝟓 𝑎𝑎(= 𝑦𝑦3 ) 121.5−𝑎𝑎
67.4−𝑎𝑎
𝟔𝟔 67.4(= 𝑦𝑦4 )
Since Δ4 𝑦𝑦 = 0, then
240.2 − 4𝑎𝑎 = 0
𝒂𝒂 = 𝟔𝟔𝟔𝟔. 𝟎𝟎𝟎𝟎
Practice Question
Find the missing values in the following data (refer to p955, Example 29.10, Higher
Engineering Math by BS Grewal 42e):
𝑥𝑥: 45 50 55 60 65
𝑦𝑦: 3.0 … 2.0 … -2.4
3.2.2 Newton’s Forward Interpolation Formula
Let the function 𝑦𝑦 = 𝑓𝑓(𝑥𝑥) take the values 𝑦𝑦0 , 𝑦𝑦1 , 𝑦𝑦2 , … corresponding to the values 𝑥𝑥0 , 𝑥𝑥0 +
ℎ, 𝑥𝑥0 + 2ℎ, … of 𝑥𝑥. Suppose it is required to evaluate 𝑓𝑓(𝑥𝑥) for 𝑥𝑥 = 𝑥𝑥0 + 𝑝𝑝ℎ, where 𝑝𝑝 is any
real number.
𝑥𝑥−𝑥𝑥0
For any real number 𝑝𝑝 �where 𝑝𝑝 = �, 𝐸𝐸 is defined such that
ℎ
𝐸𝐸 𝑝𝑝 𝑓𝑓(𝑥𝑥0 ) = 𝑓𝑓(𝑥𝑥0 + 𝑝𝑝ℎ)

𝑦𝑦𝑝𝑝 = 𝑓𝑓(𝑥𝑥0 + 𝑝𝑝ℎ) = 𝐸𝐸 𝑝𝑝 𝑓𝑓(𝑥𝑥0 ) = (1 + Δ)𝑝𝑝 𝑦𝑦0
Using binomial expansion

𝑝𝑝(𝑝𝑝 − 1) 2 𝑝𝑝(𝑝𝑝 − 1)(𝑝𝑝 − 2) 3
𝑦𝑦𝑝𝑝 = �1 + 𝑝𝑝Δ + Δ + Δ + ⋯ � 𝑦𝑦0
2! 3!
𝑝𝑝(𝑝𝑝 − 1) 2 𝑝𝑝(𝑝𝑝 − 1)(𝑝𝑝 − 2) 3
= 𝑦𝑦0 + 𝑝𝑝Δ𝑦𝑦0 + Δ 𝑦𝑦0 + Δ 𝑦𝑦0 + ⋯
2! 3!
It should be noted that Newton’s Forward Interpolation formula is used for interpolating the
values of 𝑦𝑦 near the beginning of a set of tabulated values and extrapolating values of 𝑦𝑦 a little
backward (i.e. to the left) of 𝑦𝑦0 .
Example 3-5
Find the cubic polynomial whose values are given in the table below. Also find 𝑓𝑓(1.5)
𝑥𝑥: 0 1 2 3 4 5
𝑦𝑦: -3 3 11 27 57 107
Solution
𝟎𝟎 −3
6
𝟏𝟏 3 2
8 6
𝟐𝟐 11 8 0
16 6 0
𝟑𝟑 27 14 0
30 6
𝟒𝟒 57 20
50
𝟓𝟓 107
𝑥𝑥 − 𝑥𝑥0 𝑥𝑥 − 0
𝑝𝑝 = = = 𝑥𝑥
ℎ 1
Newton’s Forward Difference formula gives
𝑝𝑝(𝑝𝑝 − 1) 2 𝑝𝑝(𝑝𝑝 − 1)(𝑝𝑝 − 2) 3
𝑓𝑓(𝑥𝑥) = 𝑓𝑓(𝑥𝑥0 ) + 𝑝𝑝Δ𝑓𝑓(𝑥𝑥0 ) + Δ 𝑓𝑓(𝑥𝑥0 ) + Δ 𝑓𝑓(𝑥𝑥0 )
2! 3!
substituting 𝑝𝑝 = 𝑥𝑥, 𝑓𝑓(𝑥𝑥0 ) = −3
𝑥𝑥(𝑥𝑥 − 1)(2) 𝑥𝑥(𝑥𝑥 − 1)(𝑥𝑥 − 2)(6)
𝑓𝑓(𝑥𝑥) = −3 + 6𝑥𝑥 + +
2 3×2
= −3 + 6𝑥𝑥 + 𝑥𝑥 2 − 𝑥𝑥 + (𝑥𝑥 2 − 𝑥𝑥)(𝑥𝑥 − 2)
= −3 + 6𝑥𝑥 + 𝑥𝑥 2 − 𝑥𝑥 + 𝑥𝑥 3 − 2𝑥𝑥 2 − 𝑥𝑥 2 + 2𝑥𝑥
𝒇𝒇(𝒙𝒙) = 𝒙𝒙𝟑𝟑 − 𝟐𝟐𝒙𝒙𝟐𝟐 + 𝟕𝟕𝟕𝟕 − 𝟑𝟑
For 𝑥𝑥 = 1.5
𝑓𝑓(𝑥𝑥 = 1.5) = 1.53 − 2 × 1.52 + 7 × 1.5 − 3
𝑓𝑓(1.5) = 𝟔𝟔. 𝟑𝟑𝟑𝟑𝟑𝟑
3.2.3 Newton’s Backward Interpolation formula

Let the function 𝑦𝑦 = 𝑓𝑓(𝑥𝑥) take the values 𝑦𝑦𝑛𝑛 , 𝑦𝑦𝑛𝑛−ℎ , 𝑦𝑦𝑛𝑛−2ℎ , … , 𝑦𝑦0 corresponding to the values
𝑥𝑥𝑛𝑛 , 𝑥𝑥𝑛𝑛 − ℎ, 𝑥𝑥𝑛𝑛 − 2ℎ, … , 𝑥𝑥0 of 𝑥𝑥 . Suppose it is required to evaluate 𝑓𝑓(𝑥𝑥) for 𝑥𝑥 = 𝑥𝑥𝑛𝑛 + 𝑝𝑝ℎ ,
where 𝑝𝑝 is any real number. Then
𝑦𝑦𝑝𝑝 = 𝑓𝑓(𝑥𝑥𝑛𝑛 + 𝑝𝑝ℎ) = 𝐸𝐸 𝑝𝑝 𝑓𝑓(𝑥𝑥𝑛𝑛 ) = (𝐸𝐸 −1 )−𝑝𝑝 𝑓𝑓(𝑥𝑥𝑛𝑛 ) = (1 − ∇)−𝑝𝑝 𝑓𝑓(𝑥𝑥𝑛𝑛 )
𝑝𝑝(𝑝𝑝 + 1) 2 𝑝𝑝(𝑝𝑝 + 1)(𝑝𝑝 + 2) 3
= �1 + 𝑝𝑝∇ + ∇ + ∇ + ⋯ � 𝑦𝑦𝑛𝑛
2! 3!
𝑝𝑝(𝑝𝑝 + 1) 2 𝑝𝑝(𝑝𝑝 + 1)(𝑝𝑝 + 2) 3
= 𝑦𝑦𝑛𝑛 + 𝑝𝑝∇𝑦𝑦𝑛𝑛 + ∇ 𝑦𝑦𝑛𝑛 + ∇ 𝑦𝑦𝑛𝑛 + ⋯
2! 3!
Example 3-6
Find the value of the function at 𝑥𝑥 = 1.45 by using following table
𝑥𝑥: 1 1.1 1.2 1.3 1.4 1.5
𝑦𝑦: 2 2.1 2.3 2.7 3.5 4.5
Solution
𝒙𝒙 𝒚𝒚 𝛁𝛁𝒚𝒚 𝛁𝛁𝟐𝟐 𝒚𝒚 𝛁𝛁𝟑𝟑 𝒚𝒚 𝛁𝛁𝟒𝟒 𝒚𝒚 𝛁𝛁𝟓𝟓 𝒚𝒚
𝟏𝟏 2
0.1
𝟏𝟏. 𝟏𝟏 2.1 0.1
0.2 0.1
𝟏𝟏. 𝟐𝟐 2.3 0.2 0.1
0.4 0.2 −0.5
𝟏𝟏. 𝟑𝟑 2.7 0.4 −0.4
0.8 −0.2
𝟏𝟏. 𝟒𝟒 3.5 0.2
1
𝟏𝟏. 𝟓𝟓 4.5
𝑥𝑥 − 𝑥𝑥𝑛𝑛 1.45 − 1.5
𝑝𝑝 = = = −0.5
ℎ 0.1
Newton’s Backward Difference formula gives
𝑝𝑝(𝑝𝑝 + 1) 2 𝑝𝑝(𝑝𝑝 + 1)(𝑝𝑝 + 2) 3
𝑦𝑦𝑝𝑝 = 𝑦𝑦𝑛𝑛 + 𝑝𝑝∇𝑦𝑦𝑛𝑛 + ∇ 𝑦𝑦𝑛𝑛 + ∇ 𝑦𝑦𝑛𝑛
2! 3!
substituting 𝑝𝑝 = −0.5, 𝑓𝑓(𝑥𝑥𝑛𝑛 ) = 4.5
−0.5 × 0.5 −0.5(0.5)(1.5)
𝑓𝑓(1.45) = 4.5 − 0.5 × 1 + (0.2) + (−0.2)
2 3×2
−(0.5)(0.5)(1.5)(2.5) −(0.5)(0.5)(1.5)(2.5)(3.5)
+ (−0.4) + (−0.5)
4×3×2 5×4×3×2
= 4.0167
It should be noted Newtons backward Interpolation formula is used for interpolating the values
of y near the end of a set of tabulated values and extrapolating the values of y a little ahead of
yn
Practice Problems
1. Calculate 𝑓𝑓(0.125) using the following table:
𝑥𝑥: 0 0.25 0.5 0.75 1.0
𝑦𝑦: 2 2.1 2.3 2.7 3.5
2. From the following table, estimate the number of students who obtained marks less than
or equal to 45
Marks 30-40 40-50 50-60 60-70 70-80
No. of Students 31 42 51 35 31
3. The sales in the departmental store for the last five years are given.
Year 2008 2010 2012 2014 2016
Sales (in Lacs) 40 43 48 52 57
Estimate the sales for the year 2013.
4. Using Newton’s Backward Formula, find the polynomial for the given data. Hence,
also evaluate 𝑓𝑓(2.5).
𝑥𝑥: 0 1 2 3
𝑦𝑦: 1 3 7 13
3.3 Interpolation with Unequal Intervals
The various interpolation formulae derived so far possess the disadvantages of being applicable
only to equally spaced values of the argument. It is, therefore, desirable to develop interpolation
formulae for unequally spaced values of 𝑥𝑥. Now two such formulae will be introduced:
3.3.1 Newton’s Divided Differences Formula

Newton’s divided-difference interpolating polynomial is among the most popular and useful
forms. Before presenting the general equation, the first- and second-order divided difference
will be introduced:
Let 𝑦𝑦0 , 𝑦𝑦1 , … , 𝑦𝑦𝑛𝑛 be the values of 𝑦𝑦 = 𝑓𝑓(𝑥𝑥) corresponding to the argument 𝑥𝑥0 , 𝑥𝑥1 , … , 𝑥𝑥𝑛𝑛 . Then:
1st divided difference:
𝑓𝑓(𝑥𝑥0 ) − 𝑓𝑓(𝑥𝑥1 )
𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 ] =
𝑥𝑥0 − 𝑥𝑥1
2nd divided difference
𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 ] − 𝑓𝑓[𝑥𝑥1 , 𝑥𝑥2 ]
𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 , 𝑥𝑥2 ] =
Generally, the 𝑛𝑛th divided difference can be represented as:
𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 , … , 𝑥𝑥𝑛𝑛−1 ] − 𝑓𝑓[𝑥𝑥1 , 𝑥𝑥2 , … , 𝑥𝑥𝑛𝑛 ]
𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 , … , 𝑥𝑥𝑛𝑛 ] =
𝑥𝑥0 − 𝑥𝑥𝑛𝑛
These divided differences can be used with Newton’s divided difference interpolating
polynomial to form a polynomial from a given data, and it is given by
𝑓𝑓𝑛𝑛 (𝑥𝑥) = 𝑓𝑓(𝑥𝑥0 ) + (𝑥𝑥 − 𝑥𝑥0 )𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 ] + (𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥1 )𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 , 𝑥𝑥2 ] + ⋯
+ (𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥1 ) … (𝑥𝑥 − 𝑥𝑥𝑛𝑛−1 )𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 , … , 𝑥𝑥𝑛𝑛 ]
𝑖𝑖 𝑥𝑥𝑖𝑖 𝑓𝑓(𝑥𝑥1 ) 1st DD 2nd DD
0 𝑥𝑥0 𝑓𝑓(𝑥𝑥0 )
𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 ] =
𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 ] − 𝑓𝑓[𝑥𝑥1 , 𝑥𝑥2 ]
1 𝑥𝑥1 𝑓𝑓(𝑥𝑥1 ) 𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 , 𝑥𝑥2 ] =
𝑓𝑓[𝑥𝑥1 , 𝑥𝑥2 ] =
𝑓𝑓[𝑥𝑥1 , 𝑥𝑥2 ] − 𝑓𝑓[𝑥𝑥2 , 𝑥𝑥3 ]
2 𝑥𝑥2 𝑓𝑓(𝑥𝑥2 ) 𝑓𝑓[𝑥𝑥1 , 𝑥𝑥2 , 𝑥𝑥3 ] =
𝑓𝑓[𝑥𝑥2 , 𝑥𝑥3 ] =
3 𝑥𝑥3 𝑓𝑓(𝑥𝑥3 )
Example 3-7
Find the polynomial using the following data, and hence find 𝑓𝑓(0.3)
𝑥𝑥: 0 1 3 4 7
𝑦𝑦: 1 3 49 129 813
Solution
𝑥𝑥𝑖𝑖 𝑓𝑓(𝑥𝑥𝑖𝑖 ) 1st DD 2nd DD 3rd DD 4th DD

0 1
1−3
=2
0−1
2 − 23
1 3 =7
0−3
3 − 49 7 − 19
= 23 =3
1−3 0−4
23 − 80
3 49 = 19 0
1−4
49 − 129 19 − 37
= 80 =3
3−4 1−7
80 − 228
4 129 3−7
= 37
129 − 813
= 228
4−7
7 813
Using the formula for divided difference interpolating polynomial:

𝑓𝑓(𝑥𝑥) = 𝑓𝑓(𝑥𝑥0 ) + (𝑥𝑥 − 𝑥𝑥0 )𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 ] + (𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥1 )𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 , 𝑥𝑥2 ]
+ (𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥1 )(𝑥𝑥 − 𝑥𝑥2 )𝑓𝑓[𝑥𝑥0 , 𝑥𝑥1 , 𝑥𝑥2 , 𝑥𝑥3 ]
𝑓𝑓(𝑥𝑥) = 1 + (𝑥𝑥 − 0)2 + (𝑥𝑥 − 0)(𝑥𝑥 − 1)7 + (𝑥𝑥 − 0)(𝑥𝑥 − 1)(𝑥𝑥 − 3)3
= 1 + 2𝑥𝑥 + 7𝑥𝑥 2 − 7𝑥𝑥 + 3𝑥𝑥 3 − 12𝑥𝑥 2 + 9𝑥𝑥
= 3𝑥𝑥 3 − 5𝑥𝑥 2 + 4𝑥𝑥 + 1
𝑓𝑓(0.3) = 1.831
3.3.2 Lagrange’s Interpolation Formula

The Lagrange interpolating polynomial is simply a reformulation of the Newton polynomial
that avoids the computation of divided differences. It can be represented concisely as
𝑛𝑛
𝑓𝑓𝑛𝑛 (𝑥𝑥) = � 𝐿𝐿𝑖𝑖 (𝑥𝑥)𝑓𝑓(𝑥𝑥𝑖𝑖 )

𝑖𝑖=0
where,
𝑛𝑛
𝑥𝑥 − 𝑥𝑥𝑗𝑗
𝐿𝐿𝑖𝑖 (𝑥𝑥) = �
𝑥𝑥𝑖𝑖 − 𝑥𝑥𝑗𝑗
𝑗𝑗=0
𝑗𝑗≠𝑖𝑖
where Π designates the “product of.” For example, in linear version

• 𝑛𝑛 = 1, then
• 𝑖𝑖 = 0, 1
• When 𝑖𝑖 = 0, 𝑗𝑗 = 1, 𝑗𝑗 ≠ 0
𝑥𝑥 − 𝑥𝑥1
𝐿𝐿0 (𝑥𝑥) =
• When 𝑖𝑖 = 1, 𝑗𝑗 = 0, 𝑗𝑗 ≠ 1
Similarly, in second order (Quadratic version)
• 𝑛𝑛 = 2, then
• 𝑖𝑖 = 0, 1, 2
• When 𝑖𝑖 = 0, 𝑗𝑗 = 1, 2 and 𝑗𝑗 ≠ 0
(𝑥𝑥 − 𝑥𝑥1 )(𝑥𝑥 − 𝑥𝑥2 )
(𝑥𝑥0 − 𝑥𝑥1 )(𝑥𝑥0 − 𝑥𝑥2 )
• When 𝑖𝑖 = 1, 𝑗𝑗 = 0, 2 and 𝑗𝑗 ≠ 1
(𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥2 )
(𝑥𝑥1 − 𝑥𝑥0 )(𝑥𝑥1 − 𝑥𝑥2 )
• When 𝑖𝑖 = 2, 𝑗𝑗 = 0,1 and 𝑗𝑗 ≠ 2
(𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥1 )
(𝑥𝑥2 − 𝑥𝑥0 )(𝑥𝑥2 − 𝑥𝑥1 )
3.3.2.1 General Expanded form of Lagrange’s Interpolation Formula

If 𝑦𝑦 = 𝑓𝑓(𝑥𝑥) takes value 𝑦𝑦0 , 𝑦𝑦1 , … , 𝑦𝑦𝑛𝑛 corresponding to 𝑥𝑥 = 𝑥𝑥0 , 𝑥𝑥1 , … , 𝑥𝑥𝑛𝑛 , then
(𝑥𝑥 − 𝑥𝑥1 )(𝑥𝑥 − 𝑥𝑥2 ) … (𝑥𝑥 − 𝑥𝑥𝑛𝑛 ) (𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥2 ) … (𝑥𝑥 − 𝑥𝑥𝑛𝑛 )
𝑓𝑓(𝑥𝑥) = 𝑦𝑦0 + 𝑦𝑦 + ⋯
(𝑥𝑥0 − 𝑥𝑥1 )(𝑥𝑥0 − 𝑥𝑥2 ) … (𝑥𝑥0 − 𝑥𝑥𝑛𝑛 ) (𝑥𝑥1 − 𝑥𝑥0 )(𝑥𝑥1 − 𝑥𝑥2 ) … (𝑥𝑥1 − 𝑥𝑥𝑛𝑛 ) 1
(𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥1 ) … (𝑥𝑥 − 𝑥𝑥𝑛𝑛−1 )
+ 𝑦𝑦
(𝑥𝑥𝑛𝑛 − 𝑥𝑥0 )(𝑥𝑥𝑛𝑛 − 𝑥𝑥1 ) … (𝑥𝑥𝑛𝑛 − 𝑥𝑥𝑛𝑛−1 ) 𝑛𝑛
is known as Lagrange’s interpolation formula for unequal intervals
3.3.2.2 Binomial Multiplication

Use of Lagrange’s interpolating polynomial formula inevitably leads to the need of simplifying
binomial terms multiplying with each other. Note that the binomials in the formula are similar
to the factors of a polynomial, therefore conversion of root of a polynomial to an equation of a
polynomial will provide a good starting base.
For two roots 𝛼𝛼 and 𝛽𝛽, such that 𝑔𝑔(𝑥𝑥) = (𝑥𝑥 − 𝛼𝛼)(𝑥𝑥 − 𝛽𝛽)
𝑔𝑔(𝑥𝑥) = 𝑥𝑥 2 − (𝛼𝛼 + 𝛽𝛽)𝑥𝑥 + 𝛼𝛼𝛼𝛼
Similarly, for three roots 𝛼𝛼, 𝛽𝛽 and 𝛾𝛾, such that 𝑔𝑔(𝑥𝑥) = (𝑥𝑥 − 𝛼𝛼)(𝑥𝑥 − 𝛽𝛽)(𝑥𝑥 − 𝛾𝛾)
𝑔𝑔(𝑥𝑥) = 𝑥𝑥 3 − (𝛼𝛼 + 𝛽𝛽 + 𝛾𝛾)𝑥𝑥 2 + (𝛼𝛼𝛼𝛼 + 𝛼𝛼𝛼𝛼 + 𝛽𝛽𝛽𝛽)𝑥𝑥 − (𝛼𝛼𝛼𝛼𝛼𝛼)
Example 3-8
Use the following data of the function 𝑓𝑓(𝑥𝑥) = 𝑥𝑥 log 𝑥𝑥, with Lagrange’s formula,
𝑥𝑥: 3 7 9 12
𝑦𝑦: 1.4313 5.9156 8.5881 12.9501
and find
i. Quadratic polynomial approximation and its value at 𝑥𝑥 = 5
ii. Cubic polynomial approximation and its value at 𝑥𝑥 = 5
iii. Absolute error for the results in (i) and (ii)
Solution
i. For quadratic polynomial, 𝑛𝑛 = 2
(𝑥𝑥 − 𝑥𝑥1 )(𝑥𝑥 − 𝑥𝑥2 ) (𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥2 ) (𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥1 )
𝑓𝑓2 (𝑥𝑥) = 𝑦𝑦0 + 𝑦𝑦1 + 𝑦𝑦
(𝑥𝑥0 − 𝑥𝑥1 )(𝑥𝑥0 − 𝑥𝑥2 ) (𝑥𝑥1 − 𝑥𝑥0 )(𝑥𝑥1 − 𝑥𝑥2 ) (𝑥𝑥2 − 𝑥𝑥0 )(𝑥𝑥2 − 𝑥𝑥1 ) 2
(𝑥𝑥 − 7)(𝑥𝑥 − 9) (𝑥𝑥 − 3)(𝑥𝑥 − 9)
𝑓𝑓2 (𝑥𝑥) = × 1.4313 + × 5.9156
(3 − 7)(3 − 9) (7 − 3)(7 − 9)
(𝑥𝑥 − 3)(𝑥𝑥 − 7)
+ × 8.5881
(9 − 3)(9 − 7)
𝑥𝑥 2 − 16𝑥𝑥 + 63 𝑥𝑥 2 − 12𝑥𝑥 + 27
= × 1.4313 + × 5.9156
24 −8
𝑥𝑥 2 − 10𝑥𝑥 + 21
+ × 8.5881
12
1.4313 5.9156 8.5881 2
=� − + � 𝑥𝑥
24 8 12
16 × 1.4313 12 × 5.9156 10 × 8.5881
+ �− + − � 𝑥𝑥
24 8 12
63 × 1.4313 27 × 5.9156 21 × 8.5881
+� − + �
24 8 12
𝑓𝑓2 (𝑥𝑥) = 0.03586𝑥𝑥 2 + 0.7624𝑥𝑥 − 1.178
𝑓𝑓2 (5) = 3.53
ii. For cubic polynomial, 𝑛𝑛 = 3

(𝑥𝑥 − 𝑥𝑥1 )(𝑥𝑥 − 𝑥𝑥2 )(𝑥𝑥 − 𝑥𝑥3 ) (𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥2 )(𝑥𝑥 − 𝑥𝑥3 )
𝑓𝑓3 (𝑥𝑥) = 𝑦𝑦0 + 𝑦𝑦
(𝑥𝑥0 − 𝑥𝑥1 )(𝑥𝑥0 − 𝑥𝑥2 )(𝑥𝑥0 − 𝑥𝑥3 ) (𝑥𝑥1 − 𝑥𝑥0 )(𝑥𝑥1 − 𝑥𝑥2 )(𝑥𝑥1 − 𝑥𝑥3 ) 1
(𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥1 )(𝑥𝑥 − 𝑥𝑥3 ) (𝑥𝑥 − 𝑥𝑥0 )(𝑥𝑥 − 𝑥𝑥1 )(𝑥𝑥 − 𝑥𝑥2 )
+ 𝑦𝑦2 + 𝑦𝑦
(𝑥𝑥2 − 𝑥𝑥0 )(𝑥𝑥2 − 𝑥𝑥1 )(𝑥𝑥2 − 𝑥𝑥3 ) (𝑥𝑥3 − 𝑥𝑥0 )(𝑥𝑥3 − 𝑥𝑥1 )(𝑥𝑥3 − 𝑥𝑥2 ) 3
(𝑥𝑥 − 7)(𝑥𝑥 − 9)(𝑥𝑥 − 12) (𝑥𝑥 − 3)(𝑥𝑥 − 9)(𝑥𝑥 − 12)
𝑓𝑓3 (𝑥𝑥) = (1.4313) + (5.9156)
(3 − 7)(3 − 9)(3 − 12) (7 − 3)(7 − 9)(7 − 12)
(𝑥𝑥 − 3)(𝑥𝑥 − 7)(𝑥𝑥 − 12) (𝑥𝑥 − 3)(𝑥𝑥 − 7)(𝑥𝑥 − 9)
+ (8.5881) + (12.9501)
(9 − 3)(9 − 7)(9 − 12) (12 − 3)(12 − 7)(12 − 9)
𝑥𝑥 3 − 28𝑥𝑥 2 + 255𝑥𝑥 − 756 𝑥𝑥 3 − 24𝑥𝑥 2 + 171𝑥𝑥 − 324
𝑓𝑓3 (𝑥𝑥) = (1.4313) + (5.9156)
−216 40
𝑥𝑥 3 − 22𝑥𝑥 2 + 141𝑥𝑥 − 252 𝑥𝑥 3 − 19𝑥𝑥 2 + 111𝑥𝑥 − 189
+ (8.5881) +
−36 135
⋅ 12.9501
𝑓𝑓3 (𝑥𝑥) = −0.00137𝑥𝑥 3 + 0.06186𝑥𝑥 2 + 0.61060𝑥𝑥 − 0.92025
𝑓𝑓3 (5) = 3.50811
iii. Since 𝑓𝑓(𝑥𝑥) = 𝑥𝑥 log 𝑥𝑥

Then
𝑓𝑓(5) = 5 log 5 = 3.49485
So, the absolute errors are as follows:
• For quadratic approximation, |3.49485 − 3.53| = 0.03515
• For cubic approximation, |3.49485 − 3.50811| = 0.01326
3.3.3 Piecewise/Spline Interpolation

In the previous sections, nth-order polynomials were used to interpolate between 𝑛𝑛 + 1 data
points. For example, for eight points, a perfect seventh-order polynomial can be derived. This
curve would capture all the meanderings (at least up to and including seventh derivatives)
suggested by the points. However, there are cases where these functions can lead to erroneous
results because of round-off error and overshoot. An alternative approach is to apply lower-
order polynomials to subsets of data points. Such connecting polynomials are called spline
functions.
3.3.3.1 Linear Splines

The simplest connection between two points is a straight line. The first-order splines for a group
of ordered data points can be defined as a set of linear functions.
Let 𝑦𝑦 = 𝑓𝑓(𝑥𝑥) be approximated by first degree polynomial for each interval (𝑥𝑥𝑖𝑖−1 , 𝑥𝑥𝑖𝑖 ), where
𝑖𝑖 = 1, 2, … , 𝑛𝑛. Then the function 𝑓𝑓𝑖𝑖 (𝑥𝑥) is obtained by using Lagrange’s interpolation. Then:
𝑥𝑥 − 𝑥𝑥𝑖𝑖 𝑥𝑥 − 𝑥𝑥𝑖𝑖−1
𝑓𝑓𝑖𝑖 (𝑥𝑥) = 𝑦𝑦𝑖𝑖−1 + 𝑦𝑦
𝑥𝑥𝑖𝑖−1 − 𝑥𝑥𝑖𝑖 𝑥𝑥𝑖𝑖 − 𝑥𝑥𝑖𝑖−1 𝑖𝑖
Therefore,
𝑓𝑓1 (𝑥𝑥) 𝑥𝑥0 ≤ 𝑥𝑥 ≤ 𝑥𝑥1
𝑓𝑓 (𝑥𝑥) 𝑥𝑥1 ≤ 𝑥𝑥 ≤ 𝑥𝑥2
𝑓𝑓(𝑥𝑥) = � 2
⋮ ⋮
𝑓𝑓𝑛𝑛 (𝑥𝑥) 𝑥𝑥2 ≤ 𝑥𝑥 ≤ 𝑥𝑥3
Example 3-9
Use the following table to form first order spline approximation of the data:
𝑥𝑥: 0 1 2
𝑦𝑦: 1 3 30
Solution
𝑥𝑥 − 𝑥𝑥𝑖𝑖 𝑥𝑥 − 𝑥𝑥𝑖𝑖−1
𝑓𝑓𝑖𝑖 (𝑥𝑥) = 𝑦𝑦𝑖𝑖−1 + 𝑦𝑦
𝑥𝑥𝑖𝑖−1 − 𝑥𝑥𝑖𝑖 𝑥𝑥𝑖𝑖 − 𝑥𝑥𝑖𝑖−1 𝑖𝑖
For 0 ≤ 𝑥𝑥 ≤ 1,
𝑥𝑥 − 1 𝑥𝑥 − 0
𝑓𝑓1 (𝑥𝑥) = ×1+ ×3
0−1 1−0
= −𝑥𝑥 + 1 + 3𝑥𝑥
= 2𝑥𝑥 + 1
For 1 ≤ 𝑥𝑥 ≤ 2
𝑥𝑥 − 2 𝑥𝑥 − 1
𝑓𝑓2 (𝑥𝑥) = ×3+ × 30
1−2 2−1
= −3𝑥𝑥 + 6 + 30𝑥𝑥 − 30
= 27𝑥𝑥 − 24
2𝑥𝑥 + 1 0 ≤ 𝑥𝑥 < 1
𝑓𝑓(𝑥𝑥) = �
27𝑥𝑥 − 24 1 ≤ 𝑥𝑥 ≤ 2
Practice Problem
Use the following table to form first order spline approximation of the data and hence find
𝑣𝑣(16)
𝑡𝑡 (𝑠𝑠) 0 10 15 20 22.5
𝑣𝑣 (𝑚𝑚𝑠𝑠 −1 ) 0 227.04 362.78 517.35 602.97
4 NUMERICAL CALCULUS
Calculus is the mathematics of change. Because engineers must continuously deal with systems
and processes that change, calculus is an essential tool of our profession. Standing at the heart
of calculus are the related mathematical concepts of differentiation and integration.
Mathematically, the derivative, which serves as the fundamental vehicle for differentiation,
represents the rate of change of a dependent variable with respect to an independent variable.
As shown in Fig. 4-1, the mathematical definition of the derivative begins with a difference
approximation:
Δ𝑦𝑦 𝑓𝑓(𝑥𝑥𝑖𝑖 + Δ𝑥𝑥) − 𝑓𝑓(𝑥𝑥𝑖𝑖 )
=
Δ𝑥𝑥 Δ𝑥𝑥
where 𝑦𝑦 and 𝑓𝑓(𝑥𝑥) are alternative representatives for the dependent variable and 𝑥𝑥 is the
independent variable. If Δ𝑥𝑥 approaches zero, as occurs in moving from Fig. 4-1(a) to (c), the
difference becomes a derivative
𝑑𝑑𝑑𝑑 𝑓𝑓(𝑥𝑥𝑖𝑖 + Δ𝑥𝑥) − 𝑓𝑓(𝑥𝑥𝑖𝑖 )
= lim
𝑑𝑑𝑑𝑑 Δ𝑥𝑥→0 Δ𝑥𝑥
where 𝑑𝑑𝑑𝑑/𝑑𝑑𝑑𝑑, (which can also be represented by 𝑦𝑦′ or 𝑓𝑓 ′ (𝑥𝑥), is the first derivative of 𝑦𝑦 with
respect to 𝑥𝑥 evaluated at 𝑥𝑥𝑖𝑖
Figure 4-1 The graphical definition of a derivative: as 𝚫𝚫𝒙𝒙 approaches zero in going from (a) to (c), the difference
approximation becomes a derivative.
Differentiation is commonplace in engineering because so much of our work involves

characterizing the changes of variables in both time and space. In fact, many of the laws and
other generalizations that figure so prominently in our work are based on the predictable ways
in which change manifests itself in the physical world. A prime example is Newton’s second
law, which is not couched in terms of the position of an object but rather in its change of
position with respect to time.
The inverse process to differentiation in calculus is integration. Mathematically, integration is
represented by:
𝑏𝑏
𝐼𝐼 = � 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑
𝑎𝑎
which stands for the integral of the function 𝑓𝑓(𝑥𝑥) with respect to the independent variable 𝑥𝑥,
evaluated between the limits 𝑥𝑥 = 𝑎𝑎 and 𝑥𝑥 = 𝑏𝑏.
Common numerical approach to evaluating

integral is to evaluate area of thin
rectangular strips under the curve and
summing them up. The errors could be
decreased by using trapezoidal strips
instead of rectangular to evaluate area.
Just as accurate estimates of derivatives are
important in engineering, the calculation of
integrals is equally valuable. Many
phenomenon and formulae relate directly to
the idea of the integral as the area under a
curve.
Figure 4-2 Approximation of area under the curve using

rectangular strips.
4.1 Numerical Differentiation
It is the process of calculating the value of the derivative of a function at some assigned value
of 𝑥𝑥 from the given set of values (𝑥𝑥𝑖𝑖 , 𝑦𝑦𝑖𝑖 ). The choice of the interpolation formula to be used,
will depend on the assigned value of 𝑥𝑥 at which 𝑑𝑑𝑑𝑑/𝑑𝑑𝑑𝑑 is desired.
If the values of 𝑥𝑥 are equispaced and 𝑑𝑑𝑑𝑑/𝑑𝑑𝑑𝑑 is required near the beginning of the table,
Newton's forward formula is used. If it is required near the end of the table, Newton's backward
formula is employed.
Consider a function 𝑓𝑓(𝑥𝑥) = 𝑦𝑦, tabulated for the values 𝑥𝑥𝑖𝑖 = 𝑥𝑥0 + 𝑖𝑖ℎ, 𝑖𝑖 = 0, 1, 2, … , 𝑛𝑛, then:
4.1.1 Derivative using Forward Difference Formula

𝑝𝑝(𝑝𝑝 − 1) 2 𝑝𝑝(𝑝𝑝 − 1)(𝑝𝑝 − 2) 3 𝑝𝑝(𝑝𝑝 − 1)(𝑝𝑝 − 2)(𝑝𝑝 − 3) 4
𝑦𝑦 = 𝑦𝑦0 + 𝑝𝑝Δ𝑦𝑦0 + Δ 𝑦𝑦0 + Δ 𝑦𝑦𝑜𝑜 + Δ 𝑦𝑦0
2! 3! 4!
+⋯
𝑝𝑝2 − 𝑝𝑝 2 𝑝𝑝3 − 3𝑝𝑝2 + 2𝑝𝑝 3 𝑝𝑝4 − 6𝑝𝑝3 + 11𝑝𝑝2 − 6𝑝𝑝 4
= 𝑦𝑦0 + 𝑝𝑝Δ𝑦𝑦0 + Δ 𝑦𝑦0 + Δ 𝑦𝑦𝑜𝑜 + Δ 𝑦𝑦0 + ⋯
2 6 24
Differentiating both side with respect to 𝑝𝑝 gives
𝑑𝑑𝑑𝑑 2𝑝𝑝 − 1 2 3𝑝𝑝2 − 6𝑝𝑝 + 2 3 4𝑝𝑝3 − 18𝑝𝑝2 + 22𝑝𝑝 − 6 4
= Δ𝑦𝑦0 + Δ 𝑦𝑦0 + Δ 𝑦𝑦0 + Δ 𝑦𝑦0 + ⋯
𝑑𝑑𝑑𝑑 2 6 24
since 𝑝𝑝ℎ = 𝑥𝑥 − 𝑥𝑥0 , 𝑑𝑑𝑑𝑑/𝑑𝑑𝑑𝑑 = 1/ℎ.
According to chain rule
𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑
= ×
then
𝑑𝑑𝑑𝑑 1 2𝑝𝑝 − 1 2 3𝑝𝑝2 − 6𝑝𝑝 + 2 3 2𝑝𝑝3 − 9𝑝𝑝2 + 11𝑝𝑝 − 3 4
= �Δ𝑦𝑦0 + Δ 𝑦𝑦0 + Δ 𝑦𝑦0 + Δ 𝑦𝑦0 + ⋯ �
𝑑𝑑𝑑𝑑 ℎ 2 6 12
Similarly,
𝑑𝑑 2 𝑥𝑥 1 2 3
6𝑝𝑝2 − 18𝑝𝑝 + 11 4
= �Δ 𝑦𝑦0 + (𝑝𝑝 − 1)Δ 𝑦𝑦0 + Δ 𝑦𝑦0 + ⋯ �
𝑑𝑑𝑦𝑦 2 ℎ2 12
and
𝑑𝑑 3 𝑥𝑥 1 3 2𝑝𝑝 − 3 4
= �Δ 𝑦𝑦0 + Δ 𝑦𝑦0 + ⋯ �
Example 4-1
Use the following table to evaluate 𝑓𝑓 ′ (0.25), 𝑓𝑓 ′′ (0.25), and 𝑓𝑓 ′′′ (0.25)
𝑥𝑥: 0 0.5 1 1.5 2.0
𝑦𝑦: 2.0286 2.4043 2.7637 3.1072 3.4350
Solution
𝒙𝒙 𝒚𝒚 𝚫𝚫𝒚𝒚 𝚫𝚫𝟐𝟐 𝒚𝒚 𝚫𝚫𝟑𝟑 𝒚𝒚 𝚫𝚫𝟒𝟒 𝒚𝒚
𝟎𝟎 2.0286
0.3757
𝟎𝟎. 𝟓𝟓 2.4043 −0.0163
0.3594 4 × 10−4
−2
𝟏𝟏. 𝟎𝟎 2.7637 −0.0159
× 10−4
0.3435 2 × 10−4
𝟏𝟏. 𝟓𝟓 3.1072 −0.0157
0.3278
𝟐𝟐. 𝟎𝟎 3.4350
𝑥𝑥 − 𝑥𝑥0 0.25 − 0
𝑝𝑝 = = = 0.5
ℎ 0.5
𝑑𝑑𝑑𝑑 1 2(0.5) − 1 3(0.5)2 − 6(0.5) + 2
= �0.3757 + (−0.0163) + 4 × 10−4
𝑑𝑑𝑑𝑑 0.5 2 6
2(0.5)3 − 9(0.5)2 + 11(0.5) − 3
+ (−2 × 10−4 ) + ⋯ �
12
= 0.7513
𝑑𝑑2 𝑥𝑥 1 −4
6(0.5)2 − 18(0.5) + 11
= �−0.0163 + (0.5 − 1)4 × 10 + (−2 × 10−4 )
𝑑𝑑𝑦𝑦 2 0.52 12
+ ⋯�
= −0.0664
𝑑𝑑 3 𝑥𝑥 1 2(0.5) − 3
3
= 3
�4 × 10−4 + (−2 × 10−4 ) + ⋯ �
𝑑𝑑𝑦𝑦 0.5 2
= 4.8 × 10−3
4.1.2 Derivative using Backward Difference Formula

𝑝𝑝(𝑝𝑝 + 1) 2 𝑝𝑝(𝑝𝑝 + 1)(𝑝𝑝 + 2) 3 𝑝𝑝(𝑝𝑝 + 1)(𝑝𝑝 + 2)(𝑝𝑝 + 3) 4
𝑦𝑦 = 𝑦𝑦𝑛𝑛 + 𝑝𝑝𝑝𝑝𝑦𝑦𝑛𝑛 + 𝛻𝛻 𝑦𝑦𝑛𝑛 + 𝛻𝛻 𝑦𝑦𝑛𝑛 + 𝛥𝛥 𝑦𝑦0
2! 3! 4!
+⋯
𝑝𝑝2 + 𝑝𝑝 2 𝑝𝑝3 + 3𝑝𝑝2 + 2𝑝𝑝 3 𝑝𝑝4 + 6𝑝𝑝3 + 11𝑝𝑝2 + 6𝑝𝑝 4
= 𝑦𝑦𝑛𝑛 + 𝑝𝑝𝑝𝑝𝑦𝑦𝑛𝑛 + 𝛻𝛻 𝑦𝑦𝑛𝑛 + 𝛻𝛻 𝑦𝑦𝑛𝑛 + 𝛥𝛥 𝑦𝑦0 + ⋯
2 6 24
Differentiating both sides with respect to 𝑝𝑝
𝑑𝑑𝑑𝑑 2𝑝𝑝2 + 1 2 3𝑝𝑝2 + 6𝑝𝑝 + 2 3 4𝑝𝑝3 + 18𝑝𝑝2 + 22𝑝𝑝 + 6 4
= 𝛻𝛻𝑦𝑦𝑛𝑛 + 𝛻𝛻 𝑦𝑦𝑛𝑛 + 𝛻𝛻 𝑦𝑦𝑛𝑛 + 𝛥𝛥 𝑦𝑦0 + ⋯
𝑑𝑑𝑑𝑑 2 6 24
since 𝑝𝑝ℎ = 𝑥𝑥 − 𝑥𝑥0 , 𝑑𝑑𝑑𝑑/𝑑𝑑𝑑𝑑 = 1/ℎ.
And according to the chain rule
= ×
𝑑𝑑𝑑𝑑 1 2𝑝𝑝 + 1 2 3𝑝𝑝2 + 6𝑝𝑝 + 2 3 2𝑝𝑝3 + 9𝑝𝑝2 + 11𝑝𝑝 + 3 4
= �∇𝑦𝑦𝑛𝑛 + ∇ 𝑦𝑦𝑛𝑛 + ∇ 𝑦𝑦𝑛𝑛 + Δ 𝑦𝑦0 + ⋯ �
𝑑𝑑𝑑𝑑 ℎ 2 6 12
Similarly,
𝑑𝑑 2 𝑥𝑥 1 2 3
6𝑝𝑝2 + 18𝑝𝑝 + 11 4
= �∇ 𝑦𝑦𝑛𝑛 + (𝑝𝑝 + 1)∇ 𝑦𝑦𝑛𝑛 + ∇ 𝑦𝑦𝑛𝑛 + ⋯ �
and
𝑑𝑑 3 𝑥𝑥 1 3 2𝑝𝑝 + 3 4
= �∇ 𝑦𝑦𝑛𝑛 + ∇ 𝑦𝑦𝑛𝑛 + ⋯ �
Example 4-2
Following table shows the distance travelled by an object with respect to time
𝑡𝑡 8 9 10 11 12
𝐷𝐷 17.453 21.460 25.752 30.301 35.084
a) Find the velocity at 𝑡𝑡 = 11.5 by using Newton Backward Differentiation method
b) The distance travelled by the object is given by 𝐷𝐷 = 7𝑡𝑡 + 70𝑒𝑒 −𝑡𝑡/10 − 70 . Use
analytical differentiation method to determine the velocity at 𝑡𝑡 = 11.5 and compare
it with the result of (a).
Solution
a) Using Backward Differentiation Method
𝒕𝒕 𝑫𝑫 𝛁𝛁𝛁𝛁 𝛁𝛁𝟐𝟐 𝐃𝐃 𝛁𝛁𝟑𝟑 𝐃𝐃 𝛁𝛁𝟒𝟒 𝐃𝐃
𝟖𝟖 17.453
4.007
𝟗𝟗 21.460 0.285
4.292 −0.028
𝟏𝟏𝟏𝟏 25.752 0.257 0.005
4.549 −0.023
𝟏𝟏𝟏𝟏 30.301 0.234
4.783
𝟏𝟏𝟏𝟏 35.084
11.5 − 12
𝑝𝑝 = = −0.5
1
𝑑𝑑𝑑𝑑 1 2(−0.5) + 1 3(−0.5)2 + 6(−0.5) + 2
= �4.783 + 0.234 + (−0.023)
𝑑𝑑𝑑𝑑 1 2 6
2(−0.5)3 + 9(−0.5)2 + 11(−0.5) + 3
+ 0.005 + ⋯ �
12
= 4.7665
b) Using analytical Method
𝐷𝐷ʹ (𝑡𝑡) = 7 − 7𝑒𝑒 𝑡𝑡⁄10
𝑣𝑣(11.5) = 4.7835
4.7835 − 4.7665
𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 = � � × 100 = 0.355%
4.7835
4.2 Numerical Integration

The process of evaluating a definite integral from a set of tabulated values of the integrand
𝑓𝑓(𝑥𝑥) is called numerical integration. This process when applied to a function of a single
variable, is known as quadrature.
The problem of numerical integration, like that of numerical differentiation, is solved by
representing 𝑓𝑓(𝑥𝑥) by an interpolation formula and then integrating it between the given limits.
In this way, we can derive quadrature formula for approximate integration of a function defined
by a set of numerical values only.
4.2.1 Newton – Cotes Quadrature Formula

Let
𝑦𝑦
𝑏𝑏
𝐼𝐼 = � 𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑
𝑎𝑎
𝑦𝑦 = 𝑓𝑓(𝑥𝑥)
where 𝑦𝑦 = 𝑓𝑓(𝑥𝑥) takes values 𝑦𝑦0 , 𝑦𝑦1 ,
𝑦𝑦2 , 𝑦𝑦3 , … , 𝑦𝑦𝑛𝑛 when 𝑥𝑥 = 𝑥𝑥0 , 𝑥𝑥1 , 𝑥𝑥2 ,
… , 𝑥𝑥3 , 𝑥𝑥𝑛𝑛 (Fig. 4-3) 𝑦𝑦0 𝑦𝑦1 𝑦𝑦2 𝑦𝑦𝑛𝑛
The interval (𝑎𝑎, 𝑏𝑏) is divided into 𝑛𝑛

sub-intervals of width ℎ so that 𝑥𝑥0 = 𝑥𝑥
𝑥𝑥0 𝑥𝑥0 + ℎ 𝑥𝑥0 + 2ℎ 𝑥𝑥0 + 𝑛𝑛ℎ
𝑎𝑎 , 𝑥𝑥1 = 𝑥𝑥0 + ℎ, 𝑥𝑥2 = 𝑥𝑥0 + 2ℎ, …,
𝑥𝑥𝑛𝑛 = 𝑥𝑥0 + 𝑛𝑛ℎ = 𝑏𝑏. Then Figure 4-3 Graphical representation of interval (a, b) and its sub-intervals
𝑥𝑥0 +𝑛𝑛ℎ
𝐼𝐼 = � 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑
𝑥𝑥0
where
𝑝𝑝(𝑝𝑝 − 1) 2 𝑝𝑝(𝑝𝑝 − 1)(𝑝𝑝 − 2) 3 𝑝𝑝(𝑝𝑝 − 1)(𝑝𝑝 − 2)(𝑝𝑝 − 3) 4
𝑦𝑦 = 𝑦𝑦0 + 𝑝𝑝Δ𝑦𝑦0 + Δ 𝑦𝑦0 + Δ 𝑦𝑦𝑜𝑜 + Δ 𝑦𝑦0
2! 3! 4!
+⋯
and
𝑝𝑝 =
ℎ
therefore
𝑑𝑑𝑑𝑑 1
= ⟹ 𝑑𝑑𝑑𝑑 = ℎ 𝑑𝑑𝑑𝑑
𝑑𝑑𝑑𝑑 ℎ
When
𝑥𝑥 = 𝑥𝑥0 ⟹ 𝑝𝑝 = 0
𝑥𝑥 = 𝑥𝑥𝑛𝑛 ⟹ 𝑝𝑝 = 𝑛𝑛
Now,
𝑥𝑥𝑛𝑛
𝑝𝑝(𝑝𝑝 − 1) 2 𝑝𝑝(𝑝𝑝 − 1)(𝑝𝑝 − 2) 3
𝐼𝐼 = � �𝑦𝑦0 + 𝑝𝑝Δ𝑦𝑦0 + Δ 𝑦𝑦0 + Δ 𝑦𝑦𝑜𝑜
𝑥𝑥0 2! 3!
𝑝𝑝(𝑝𝑝 − 1)(𝑝𝑝 − 2)(𝑝𝑝 − 3) 4
+ Δ 𝑦𝑦0 + ⋯ � 𝑑𝑑𝑑𝑑
4!
Substituting values
𝑝𝑝
𝑝𝑝(𝑝𝑝 − 1) 2 𝑝𝑝(𝑝𝑝 − 1)(𝑝𝑝 − 2) 3
𝐼𝐼 = ℎ � �𝑦𝑦0 + 𝑝𝑝Δ𝑦𝑦0 + Δ 𝑦𝑦0 + Δ 𝑦𝑦𝑜𝑜
0 2! 3!
𝑝𝑝(𝑝𝑝 − 1)(𝑝𝑝 − 2)(𝑝𝑝 − 3) 4
+ Δ 𝑦𝑦0 + ⋯ � 𝑑𝑑𝑑𝑑
4!
𝑝𝑝
𝑝𝑝2 − 𝑝𝑝 2 𝑝𝑝3 − 3𝑝𝑝2 + 2𝑝𝑝 3 𝑝𝑝4 − 6𝑝𝑝3 + 11𝑝𝑝2 − 6𝑝𝑝 4
= ℎ � �𝑦𝑦0 + 𝑝𝑝Δ𝑦𝑦0 + Δ 𝑦𝑦0 + Δ 𝑦𝑦𝑜𝑜 + Δ 𝑦𝑦0
0 2 6 24
+ ⋯ � 𝑑𝑑𝑑𝑑
𝑝𝑝2 𝑝𝑝3 𝑝𝑝2 Δ2 𝑦𝑦0 𝑝𝑝4 3𝑝𝑝3 2𝑝𝑝2 Δ3 𝑦𝑦0
= ℎ �𝑝𝑝𝑦𝑦0 + Δ𝑦𝑦0 + � − � +� − + �
2 3 2 2! 4 3 2 6
𝑛𝑛
𝑝𝑝5 6𝑝𝑝4 11𝑝𝑝3 6𝑝𝑝2 Δ4 𝑦𝑦0
+� − + − � +⋯�
5 4 3 2 24 0
Newton-Cotes Quadrature Formula up to 4th forward difference term is given as:

𝑛𝑛2 2𝑛𝑛3 − 3𝑛𝑛2 2 𝑛𝑛4 − 4𝑛𝑛3 + 4𝑛𝑛2 3
𝐼𝐼 = ℎ �𝑛𝑛𝑦𝑦0 + Δ𝑦𝑦0 + � � Δ 𝑦𝑦0 + � � Δ 𝑦𝑦0
2 12 24
6𝑛𝑛5 − 45𝑛𝑛4 + 110𝑛𝑛3 − 90𝑛𝑛2 4
+� � Δ 𝑦𝑦0 + ⋯ �
720
4.2.2 Trapezoidal Rule

Consider a linear curve through (𝑥𝑥0 , 𝑦𝑦0 ) and (𝑥𝑥1 , 𝑦𝑦1 ), i.e. a first order polynomial for which
differences of order higher than first become zero. For 𝑛𝑛 = 1, Quadrature formula becomes
𝑥𝑥0 +ℎ
1 1
� 𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑 = ℎ �𝑦𝑦0 + Δ𝑦𝑦0 � = ℎ �𝑦𝑦0 + (𝑦𝑦1 − 𝑦𝑦0 )�
𝑥𝑥0 2 2
ℎ
= (𝑦𝑦0 + 𝑦𝑦1 )
2
Similarly
𝑥𝑥0 +2ℎ
1 1
� 𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑 = ℎ �𝑦𝑦2 + Δ𝑦𝑦1 � = ℎ �𝑦𝑦1 + (𝑦𝑦2 − 𝑦𝑦1 )�
𝑥𝑥0+ℎ 2 2
ℎ
= (𝑦𝑦 + 𝑦𝑦2 )
2 1
and
ℎ
� 𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑 = (𝑦𝑦 + 𝑦𝑦𝑛𝑛 )
𝑥𝑥0+(𝑛𝑛−1)ℎ 2 𝑛𝑛−1
Adding these 𝑛𝑛 integrals results in:

ℎ
� 𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑 = [𝑦𝑦 + 𝑦𝑦𝑛𝑛 + 2(𝑦𝑦1 + 𝑦𝑦2 + 𝑦𝑦3 + ⋯ + 𝑦𝑦𝑛𝑛−1 )]
𝑥𝑥0 2 0
Alternatively,
𝑥𝑥0 +𝑛𝑛ℎ 𝑛𝑛−1
ℎ
� 𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑 = �𝑦𝑦0 + �2 � 𝑦𝑦𝑖𝑖 � + 𝑦𝑦𝑛𝑛 �
𝑥𝑥0 2
𝑖𝑖=1
Example 4-3
Evaluate the integral
1
1
𝑑𝑑𝑑𝑑 �
0 1 + 𝑥𝑥
i. using Trapezoidal rule and taking ℎ = 0.25
ii. using exact solution and finding error in (i).
Solution
𝑥𝑥 0.0 0.25 0.50 0.75 1.0
𝑓𝑓(𝑥𝑥) 1.00 0.800 0.667 0.5714 0.500
i. using trapezoidal rule:
1 3
ℎ
� 𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑 = �𝑦𝑦0 + �2 � 𝑦𝑦𝑖𝑖 � + 𝑦𝑦4 �
0 2
𝑖𝑖=1
0.25
= [1 + 2(0.8 + 0.667 + 0.5714) + 0.5]
2
= 0.6970238
ii. exact solution is given by:
1
1
� 𝑑𝑑𝑑𝑑 = |ln(𝑥𝑥 + 1)|10
0 1 + 𝑥𝑥
= 0.6931472
𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 = |𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸𝐸 − 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴| = 0.0038767
Example 4-3
Using Trapezoidal rule, evaluate (up to 5 d.p.) the integral with 𝑛𝑛 = 10

6
𝐼𝐼 = � �2 + sin�2√𝑥𝑥��𝑑𝑑𝑑𝑑
1
Solution
𝑏𝑏 − 𝑎𝑎 6 − 1
ℎ=
= = 0.5
𝑛𝑛 10
𝑥𝑥 1 1.5 2 2.5 3 3.5
𝑦𝑦 2.9093 2.63816 2.30807 1.97932 1.68305 1.4353
𝑥𝑥 4 4.5 5 5.5 6
𝑦𝑦 1.2432 1.10832 1.02872 1.00024 1.01736
0.5
𝐼𝐼 = [2.90930 + 2(2.63816 + 2.30807 + 1.97932 + 1.68305 + 1.43530 + 1.24320 + 1.10832
2
+ 1.02872 + 1.00024) + 1.01736]
𝐼𝐼 = 8.19384
Practice Questions
Evaluate following integrals
1
i. ∫0 x3 dx; take n = 5
1
ii. ∫0 dx/(1 + x2 ); h = 0.25
4.2.3 Simpson’s One-Third Rule

Putting 𝑛𝑛 = 2 in Newton-Cotes Quadrature formula and taking the curve through (𝑥𝑥0 , 𝑦𝑦0 ),
(𝑥𝑥1 , 𝑦𝑦1 ) and (𝑥𝑥2 , 𝑦𝑦2 )as a parabola i.e., a polynomial of second order so that differences of order
higher than second vanish. Therefore,
𝑥𝑥2
22 2(2)3 − 3(2)2 2
� 𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑 = ℎ �2𝑦𝑦0 + Δ𝑦𝑦0 + � � Δ 𝑦𝑦0 �
𝑥𝑥0 2 12
ℎ
= [𝑦𝑦 + 4𝑦𝑦1 + 𝑦𝑦2 ]
3 0
Similarly,
𝑥𝑥4
ℎ
� 𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑 = [𝑦𝑦 + 4𝑦𝑦3 + 𝑦𝑦4 ]
𝑥𝑥2 3 2
and generally,
𝑥𝑥2𝑛𝑛+2
ℎ
� 𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑 = [𝑦𝑦 + 4𝑦𝑦2𝑛𝑛+1 + 𝑦𝑦2𝑛𝑛+2 ]
𝑥𝑥2𝑛𝑛 3 2𝑛𝑛
Summing up all the results, Composite Simpson’s Rule is obtained (Where 𝑛𝑛 must always be
even.):
𝑥𝑥𝑛𝑛 𝑛𝑛−1 𝑛𝑛−2

ℎ
� 𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑 = �𝑦𝑦0 + 4 � 𝑦𝑦𝑖𝑖 + 2 � 𝑦𝑦𝑖𝑖 + 𝑦𝑦𝑛𝑛 �
𝑥𝑥0 3
𝑖𝑖>0, 𝑖𝑖>0,
𝑜𝑜𝑜𝑜𝑜𝑜 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒
Alternatively,
𝑥𝑥2𝑛𝑛 𝑛𝑛 𝑛𝑛−1
ℎ
� 𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑 = �𝑦𝑦0 + 4 � 𝑦𝑦2𝑖𝑖−1 + 2 � 𝑦𝑦2𝑖𝑖 + 𝑦𝑦2𝑛𝑛 �
𝑥𝑥0 3
𝑖𝑖=1 𝑖𝑖=1
Example 4-4
Evaluate the given integral using Simpson’s One Third Rule, taking 𝑛𝑛 = 6
4
𝐼𝐼 = � �1 + 𝑥𝑥 3
1
Solution
4−1
= 0.5 ℎ=
6
𝑥𝑥 1.0 1.5 2.0 2.5 3.0 3.5 4.0
𝑦𝑦 1.41421 2.09165 3.00000 4.07738 5.29150 6.62382 8.06226
ℎ
𝐼𝐼 = [𝑦𝑦 + 4(𝑦𝑦1 + 𝑦𝑦3 + 𝑦𝑦5 ) + 2(𝑦𝑦2 + 𝑦𝑦4 ) + 𝑦𝑦6 ]
3 0
0.5
= [1.41421 + 4(2.09165 + 4.07738 + 6.62382) + 2(3 + 5.29150) + 8.06226]
2
= 12.872
Practice Problem
Evaluate following integrals
𝜋𝜋
2
1. � sin(2𝑥𝑥) ⋅ 𝑒𝑒 −𝑥𝑥 𝑑𝑑𝑑𝑑 ; Using Simpson’s 1/3rd Rule. Take (a) 𝑛𝑛 = 2 and (b) 𝑛𝑛 = 10
0
4
2. � 𝑒𝑒 𝑥𝑥 𝑑𝑑𝑑𝑑 and compare with exact solution
0
5.2
3. � log 𝑥𝑥 𝑑𝑑𝑑𝑑 Using (a) Trapezoidal Rule, (b) Simpson’s 1/3rd Rule
4
4.2.4 Simpson’s Three-Eighth Rule

Putting 𝑛𝑛 = 3 in Newton-Cotes Quadrature formula and taking curve through (𝑥𝑥𝑖𝑖 , 𝑦𝑦𝑖𝑖 ), where
𝑖𝑖 = 0, 1, 2, 3, as a polynomial of third order so that differences above third order vanish, then:
𝑥𝑥0 +3ℎ
3 3 1
� 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 = 3ℎ �𝑦𝑦0 + Δ𝑦𝑦0 + Δ2 𝑦𝑦0 + Δ3 𝑦𝑦0 �
𝑥𝑥0 2 4 8
3ℎ
= (𝑦𝑦 + 3𝑦𝑦1 + 3𝑦𝑦2 + 𝑦𝑦3 )
8 0
Similarly,
𝑥𝑥0 +6ℎ
3ℎ
� 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 = (𝑦𝑦 + 3𝑦𝑦4 + 3𝑦𝑦5 + 𝑦𝑦6 )
𝑥𝑥0 +3ℎ 8 3
Adding all similar expressions from 𝑥𝑥0 to 𝑥𝑥0 + 𝑛𝑛ℎ, where 𝑛𝑛 is multiple of three, following
expression is obtained:
3ℎ
� 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 = �𝑦𝑦 + 𝑦𝑦𝑛𝑛 + 3(𝑦𝑦1 + 𝑦𝑦2 + 𝑦𝑦4 + 𝑦𝑦5 + ⋯ + 𝑦𝑦𝑛𝑛−1 )
8 0
𝑥𝑥0
+ 2(𝑦𝑦3 + 𝑦𝑦6 + ⋯ + 𝑦𝑦𝑛𝑛−3 )�
Alternatively,
𝑛𝑛−1 𝑛𝑛⁄3−1 𝑛𝑛⁄3−1
3ℎ
� 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 = �𝑦𝑦 + 3 �� 𝑦𝑦𝑖𝑖 − � 𝑦𝑦3𝑖𝑖 � + 2 � 𝑦𝑦3𝑖𝑖 + 𝑦𝑦𝑛𝑛 �
𝑥𝑥0 8 0
𝑖𝑖=1 𝑖𝑖=1 𝑖𝑖=1
The above expression is known as Simpson’s Three Eighth Rule. In this rule the number of sub-
intervals should be taken as multiple of 3.
Example 4-5
Evaluate the given integral using Simpson’s Three Eighth Rule, taking 𝑛𝑛 = 6
6
𝑑𝑑𝑑𝑑
𝐼𝐼 = � 2
0 1 + 𝑥𝑥
Solution
ℎ=1
𝑥𝑥 0 1 2 3 4 5 6
𝑦𝑦 1 0.5 0.2 0.1 0.0588 0.0385 0.027
3ℎ
𝐼𝐼 = [𝑦𝑦 + 3(𝑦𝑦1 + 𝑦𝑦2 + 𝑦𝑦4 + 𝑦𝑦5 ) + 2𝑦𝑦3 + 𝑦𝑦6 ]
8 0
3(1)
= [1 + 3(0.5 + 0.2 + 0.0588 + 0.0385) + 2(0.1) + 0.027]
8
= 1.3571 … 𝐀𝐀𝐀𝐀𝐀𝐀.
Example 4-6
Following table shows the velocity of a moving particle at a time at 𝑡𝑡 seconds

𝑡𝑡 0 2 4 6 8 10 12
𝑣𝑣 4 6 16 34 60 94 136
i. Find the distance travelled by the particle at 𝑡𝑡 = 12 by using 3/8th rule
ii. Also find acceleration at 𝑡𝑡 = 2
Solution
ℎ=2
i.
3(2)
= [4 + 3(6 + 16 + 60 + 94) + 2(34) + 136]
8
= 552
ii.
𝑑𝑑𝑑𝑑 1 2𝑝𝑝 − 1 2
𝑎𝑎 = = �Δ𝑦𝑦0 + Δ 𝑦𝑦0 �
𝑑𝑑𝑑𝑑 ℎ 2
1 2(1) − 1
= �2 + × 8�
2 2
= 3 … 𝐀𝐀𝐀𝐀𝐀𝐀.
5 NUMERICAL SOLUTION OF
DIFFERENTIAL EQUATIONS
The methods of solution so far presented are applicable to a limited class of differential
equations. Frequently differential equations appearing in physical problems do not belong to
any of these familiar types and one is obliged to resort to numerical methods. These methods
are of even greater importance when it is realized that computing machines are now available
which reduce numerical work considerably.
Several numerical methods are available for the solution of first order differential equations of
the form:
𝑑𝑑𝑑𝑑
= 𝑓𝑓(𝑥𝑥, 𝑦𝑦), given 𝑦𝑦(𝑥𝑥0 ) = 𝑦𝑦0
𝑑𝑑𝑑𝑑
These methods yield solutions either as a power series in 𝑥𝑥 from which the values of y can be
found by direct substitution, or as a set of values of 𝑥𝑥 and 𝑦𝑦.
Euler and Runge-Kutta methods are used for computing over a limited range of x-values
whereas Milne and Adams-Bashforth methods may be applied for finding y over a wider range
of x-values. These later methods require starting values which are found by Picard's or Taylor
series or Runge-Kutta methods.
The initial condition is specified at the point 𝑥𝑥0 . Such problems in which all the initial
conditions are given at the initial point only are called initial value problems. But there are
problems involving second and higher order differential equations in which the conditions may
be given at two or more points. These are known as boundary value problems. In this chapter,
methods for solving initial value problems will be explained first and then a method of solving
boundary value problems will be provided
5.1 Ordinary Differential Equations of 1st Order

A first-order differential equation is an equation
𝑑𝑑𝑑𝑑
= 𝑓𝑓(𝑥𝑥, 𝑦𝑦)
𝑑𝑑𝑑𝑑
in which 𝑓𝑓(𝑥𝑥, 𝑦𝑦) is a function of two variables defined on a region in the xy-plane. The equation
is of first order because it involves only the first derivative 𝑑𝑑𝑑𝑑/𝑑𝑑𝑑𝑑 (and not higher-order
derivatives).
5.1.1 Types of Differential Equations

1. Separable Differential Equation
A separable differential equation is any differential equation that can be written in the
following form.
𝑑𝑑𝑑𝑑
𝑁𝑁(𝑦𝑦) = 𝑀𝑀(𝑥𝑥)
𝑑𝑑𝑑𝑑
Note that for a differential equation to be separable all the 𝑦𝑦's in the differential equation
must be multiplied by the derivative and all the 𝑥𝑥's in the differential equation must be
on the other side of the equal sign.
2. Exact Differential Equations
A differential equation of the form:
𝑑𝑑𝑑𝑑
𝑀𝑀(𝑥𝑥, 𝑦𝑦) + 𝑁𝑁(𝑥𝑥, 𝑦𝑦) =0
𝑑𝑑𝑑𝑑
such that 𝑀𝑀𝑦𝑦 = 𝑁𝑁𝑥𝑥 is known as Exact differential equation
3. Linear Differential Equation

𝑑𝑑𝑑𝑑
+ 𝑃𝑃(𝑥𝑥)𝑦𝑦 = 𝑄𝑄(𝑥𝑥)
𝑑𝑑𝑑𝑑
where 𝑃𝑃 and 𝑄𝑄 are functions of only 𝑥𝑥. Then this type of differential equation is known
as Linear differential equation
4. Bernoulli Differential Equation
𝑑𝑑𝑑𝑑 𝑑𝑑𝑛𝑛 𝑦𝑦
+ 𝑃𝑃(𝑥𝑥)𝑦𝑦 = 𝑛𝑛 𝑄𝑄(𝑥𝑥)
𝑑𝑑𝑑𝑑 𝑑𝑑𝑥𝑥
where 𝑃𝑃 and 𝑄𝑄 are functions of only 𝑥𝑥. Then this type of differential equation is known
as Bernoulli differential equation.
5.1.2 Initial Value Problems

In the field of differential equations, an initial value problem is an ordinary differential equation
together with a specified value, called the initial condition, of the unknown function at a given
point in the domain of the solution.
5.1.2.1 Euler’s Method

Given that 𝑦𝑦 ′ = 𝑓𝑓(𝑥𝑥, 𝑦𝑦) and that the initially when 𝑥𝑥 = 𝑥𝑥0 , 𝑦𝑦 = 𝑦𝑦0 . If the value of h, the
increment in the value of x, is sufficiently small, the value of 𝑥𝑥𝑛𝑛 and 𝑦𝑦𝑛𝑛 can be approximated
from the following equation:
𝑦𝑦𝑛𝑛+1 = 𝑦𝑦𝑛𝑛 + ℎ 𝑓𝑓(𝑥𝑥𝑛𝑛 , 𝑦𝑦𝑛𝑛 )
5.1.2.1.1 Geometrical Interpretation
Given that the coordinate (𝑥𝑥0 , 𝑦𝑦0 ) is known, then to geometrically to find (𝑥𝑥𝑛𝑛 , 𝑦𝑦𝑛𝑛 ) following
steps are to be followed:
1. Evaluate gradient and draw the tangent at (𝑥𝑥0 , 𝑦𝑦0 ).
2. Find the value of rise in 𝑦𝑦 corresponding to the run of ℎ, and add it to 𝑦𝑦0 to evaluate 𝑦𝑦1 .
The rise can be calculated by multiplying the gradient with ℎ, i.e.
𝑦𝑦1 = 𝑦𝑦0 + ℎ𝑦𝑦 ′ (𝑥𝑥0 , 𝑦𝑦0 )
OR
𝑦𝑦1 = 𝑦𝑦0 + ℎ 𝑓𝑓(𝑥𝑥0 , 𝑦𝑦0 )
3. Find the value of gradient at newly calculated (𝑥𝑥1 , 𝑦𝑦1 ) and draw a line from previous
tangent at (𝑥𝑥1 , 𝑦𝑦1 ) with the same gradient as 𝑓𝑓(𝑥𝑥1 , 𝑦𝑦1 ). Repeat step 2 to find (𝑥𝑥2 , 𝑦𝑦2 ).
𝑦𝑦4
𝑦𝑦3
𝑦𝑦2
𝑦𝑦1
𝑦𝑦0
𝑥𝑥0 𝑥𝑥1 𝑥𝑥2 𝑥𝑥3 𝑥𝑥4

Figure 5-1 Graphical representation of Euler’s method and the error involved in this technique.
Example 5-1
Solve the differential equation 𝑦𝑦 ′ = (𝑦𝑦 − 𝑥𝑥)⁄(𝑦𝑦 + 𝑥𝑥), and find the value of 𝑦𝑦 in five steps
when 𝑥𝑥 = 0.1, given that when 𝑥𝑥 = 0, 𝑦𝑦 = 1.
Solution:
0.1 − 0
ℎ= = 0.02
5
1−0
𝑦𝑦1 = 𝑦𝑦𝑥𝑥=0.02 = 1 + 0.02 � � = 1.02
1+0
1.02 − 0.02
𝑦𝑦2 = 𝑦𝑦𝑥𝑥=0.04 = 1.02 + 0.02 � � = 1.039
1.02 + 0.02
Similarly, 𝑦𝑦3 = 1.058, 𝑦𝑦4 = 1.076 and 𝑦𝑦5 = 1.093
𝒙𝒙 = 𝟎𝟎. 𝟏𝟏, 𝒚𝒚 = 𝟏𝟏. 𝟎𝟎𝟎𝟎𝟎𝟎 … 𝐀𝐀𝐀𝐀𝐀𝐀.
5.1.2.2 Modified Euler’s Method (MEM)
Modified Euler’s method provides greater accuracy over the original Euler’s method. The core
idea is the use a gradient at (𝑥𝑥𝑛𝑛 , 𝑦𝑦𝑛𝑛 ) that is the average of gradient at (𝑥𝑥𝑛𝑛 , 𝑦𝑦𝑛𝑛 ) and
∗ ).
(𝑥𝑥𝑛𝑛+1 , 𝑦𝑦𝑛𝑛+1 ∗
The term 𝑦𝑦𝑛𝑛+1 represents initial approximation of 𝑦𝑦𝑛𝑛+1 through original Euler’s
method. This method is also known as Predictor – Corrector method.
Given that 𝑦𝑦 ′ = 𝑓𝑓(𝑥𝑥, 𝑦𝑦), then the corrector equation is given by
ℎ ∗ )]
𝑦𝑦𝑛𝑛+1 = 𝑦𝑦𝑛𝑛 + [𝑓𝑓(𝑥𝑥𝑛𝑛 , 𝑦𝑦𝑛𝑛 ) + 𝑓𝑓(𝑥𝑥𝑛𝑛+1 , 𝑦𝑦𝑛𝑛+1
2
where 𝑦𝑦 ∗ is predictor, and it is given by
∗
𝑦𝑦𝑛𝑛+1 = 𝑦𝑦𝑛𝑛 + ℎ ⋅ 𝑓𝑓(𝑥𝑥𝑛𝑛 , 𝑦𝑦𝑛𝑛 )
𝑙𝑙3
𝑙𝑙1 𝑙𝑙2
∗
𝑦𝑦𝑛𝑛+1
𝑦𝑦𝑛𝑛+1
𝑙𝑙4
𝑦𝑦𝑛𝑛
𝑥𝑥𝑛𝑛 𝑥𝑥𝑛𝑛+1
Figure 5-2 Graphical representation of Modified Euler’s Method, where 𝒍𝒍𝟏𝟏 is tangent line at 𝒙𝒙𝒏𝒏 , 𝒍𝒍𝟐𝟐 is tangent at 𝒙𝒙𝒏𝒏+𝟏𝟏 ,
𝒍𝒍𝟑𝟑 is the line whose gradient is the average of 𝒍𝒍𝟏𝟏 and 𝒍𝒍𝟐𝟐 , and 𝒍𝒍𝟒𝟒 is parallel to 𝒍𝒍𝟑𝟑 drawn from (𝒙𝒙𝒏𝒏 , 𝒚𝒚𝒏𝒏 ).
Example 5-2
Solve the differential equation 𝑦𝑦 ′ = 𝑥𝑥 + 𝑦𝑦, given that 𝑦𝑦 = 0 when 𝑥𝑥 = 0. Take ℎ = 0.2 and
perform 5 iterations.
Solution:
Taking 𝑥𝑥0 = 0, 𝑦𝑦0 = 0:
𝑦𝑦1∗ = 𝑦𝑦0 + ℎ 𝑓𝑓(𝑥𝑥0 , 𝑦𝑦0 )
= 0 + 0.2(0)
=0
ℎ
𝑦𝑦1 = 𝑦𝑦0 + [𝑓𝑓(𝑥𝑥0 , 𝑦𝑦0 ) + 𝑓𝑓(𝑥𝑥1 , 𝑦𝑦1∗ )]
2
0.2
=0+ (0 + 0.2)
2
= 0.02
Similarly, other values can also be found. Alternatively, CASIO fx – 991ES PLUS can also
be used to evaluate the values quickly.
Enter the following equation in the calculator:

0.2
𝑌𝑌 + [𝑋𝑋 + 𝐴𝐴 + 2𝑌𝑌 + 0.2(𝑋𝑋 + 𝑌𝑌)]
2
where
• 𝑋𝑋 = 𝑥𝑥𝑛𝑛
• 𝑌𝑌 = 𝑦𝑦𝑛𝑛
• 𝐴𝐴 = 𝑥𝑥𝑛𝑛+1
The evaluated values are as follows:
𝑥𝑥 0 0.2 0.4 0.6 0.8 1.0
𝒚𝒚 0 0.02 0.0884 0.215848 0.415335 0.70271
Practice Problem
Solve the given I.V.P up to 3 iterations by using modified Euler’s method with ℎ = 0.1
𝑦𝑦 ′ + 𝑦𝑦 tan 𝑥𝑥 = sin 2𝑥𝑥
given that 𝑦𝑦(0) = 1
(1.04305 Ans.)
5.1.2.3 4th Order Runge – Kutta Method (RK – 4 Method)

This method was developed by two German mathematicians Runge & Kutta. The core idea of
this method is to consider slope at multiple interior points and take their weighted average to
calculate the rise to be added to 𝑦𝑦𝑛𝑛 to get 𝑦𝑦𝑛𝑛+1 . There are several forms of this method. The
most common for is as follows:
ℎ
𝑦𝑦𝑛𝑛+1 = 𝑦𝑦𝑛𝑛 + (𝑘𝑘1 + 2𝑘𝑘2 + 2𝑘𝑘3 + 𝑘𝑘4 )
6
where,
𝑘𝑘1 = 𝑓𝑓(𝑥𝑥𝑛𝑛 , 𝑦𝑦𝑛𝑛 )
1 1
𝑘𝑘2 = 𝑓𝑓 �𝑥𝑥𝑛𝑛 + ℎ, 𝑦𝑦𝑛𝑛 + 𝑘𝑘1 ℎ�
2 2
1 1
𝑘𝑘3 = 𝑓𝑓 �𝑥𝑥𝑛𝑛 + ℎ, 𝑦𝑦𝑛𝑛 + 𝑘𝑘2 ℎ�
2 2
𝑘𝑘4 = 𝑓𝑓(𝑥𝑥𝑛𝑛 + ℎ, 𝑦𝑦𝑛𝑛 + 𝑘𝑘3 ℎ)
Example 5-3
Solve the differential equation 𝑦𝑦 ′ = 𝑦𝑦 − 𝑦𝑦 2 , given that 𝑦𝑦(0) = 0.5 and perform only three
iterations taking ℎ = 0.1. Also compare the approximation with exact solution.
Solution:
When 𝒏𝒏 = 𝟎𝟎, 𝒙𝒙 = 𝟎𝟎, 𝒚𝒚 = 𝟎𝟎. 𝟓𝟓
𝑘𝑘1 = 0.5 − 0.52 = 0.25
𝑘𝑘2 = [0.5 + 0.5(0.25)(0.1)] − [0.5 + 0.5(0.25)(0.1)]2 = 0.24984
𝑘𝑘3 = [0.5 + 0.5(0.2498)(0.1)] − [0.5 + 0.5(0.2498)(0.1)]2 = 0.24984
𝑘𝑘4 = [0.5 + (0.2498)(0.1)] − [0.5 + (0.2498)(0.1)]2 = 0.24937
then
0.1
𝑦𝑦1 = 0.5 + [0.25 + 2 × 0.24984 + 2 × 0.24984 + 0.24937] = 0.52498
6
When 𝒏𝒏 = 𝟏𝟏, 𝒙𝒙 = 𝟎𝟎. 𝟏𝟏, 𝒚𝒚 = 𝟎𝟎. 𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓
𝑘𝑘1 = 0.52498 − 0.524982 = 0.24938
𝑘𝑘2 = [0.52498 + 0.5(0.24938)(0.1)] − [0.52498 + 0.5(0.24938)(0.1)]2
= 0.24860
𝑘𝑘3 = [0.52498 + 0.5(0.24860)(0.1)] − [0.52498 + 0.5(0.24860)(0.1)]2
= 0.24860
𝑘𝑘4 = [0.52498 + 0.24860(0.1)] − [0.52498 + 0.24860(0.1)]2 = 0.24751
then
0.1
𝑦𝑦2 = 0.52498 + [0.24938 + 2(0.24860) + 2(0.24860) + 0.24751] = 0.54983
6
When 𝒏𝒏 = 𝟐𝟐, 𝒙𝒙 = 𝟎𝟎. 𝟐𝟐, 𝒚𝒚 = 𝟎𝟎. 𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓
𝑘𝑘1 = 0.54983 − 0.549832 = 0.24751
𝑘𝑘2 = [0.54983 + 0.5(0.24751)(0.1)] − [0.54983 + 0.5(0.24751)(0.1)]2
= 0.24613
𝑘𝑘3 = [0.54983 + 0.2(0.24613)(0.1)] − [0.54987 + 0.2(0.24613)(0.1)]2
= 0.24700
𝑘𝑘4 = [0.54983 + 0.24700(0.1)] − [0.54987 + 0.24700(0.1)]2 = 0.24444
then
0.1
𝑦𝑦3 = 0.54987 + [0.24751 + 2(0.24613) + 2(0.24700) + 0.24444] = 0.57451
6
𝑥𝑥 0 0.1 0.2 0.3
𝒚𝒚 𝟎𝟎. 𝟓𝟓 𝟎𝟎. 𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓 𝟎𝟎. 𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓 𝟎𝟎. 𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓𝟓
Exact Analysis:
𝑑𝑑𝑑𝑑
= 𝑦𝑦 − 𝑦𝑦 2
𝑑𝑑𝑑𝑑
1
𝑑𝑑𝑑𝑑 = 𝑑𝑑𝑑𝑑
𝑦𝑦(1 − 𝑦𝑦)
Applying partial fractions:
𝐴𝐴 𝐵𝐵 1
+ ≡
𝑦𝑦 1 − 𝑦𝑦 𝑦𝑦(1 − 𝑦𝑦)
𝐴𝐴(1 − 𝑦𝑦) + 𝐵𝐵𝐵𝐵 ≡ 1
When 𝑦𝑦 = 1, 𝐵𝐵 = 1, and when 𝑦𝑦 = 0, 𝐴𝐴 = 1
1 1 1
≡ +
𝑦𝑦(1 − 𝑦𝑦) 𝑦𝑦 1 − 𝑦𝑦
Now,
1 1
� 1 𝑑𝑑𝑑𝑑 = � � + � 𝑑𝑑𝑑𝑑
𝑦𝑦 1 − 𝑦𝑦
ln(1 − 𝑦𝑦)
𝑥𝑥 + 𝐶𝐶 = ln 𝑦𝑦 +
−1
𝑥𝑥 + 𝐶𝐶 = ln 𝑦𝑦 − ln(1 − 𝑦𝑦)
𝑦𝑦
𝑥𝑥 + 𝐶𝐶 = ln � �
1 − 𝑦𝑦
When 𝑥𝑥 = 0, 𝑦𝑦 = 0.5
0.5
𝐶𝐶 = ln � �=0
1 − 0.5
𝑦𝑦
𝑥𝑥 = ln � �
1 − 𝑦𝑦
Now,
𝑦𝑦
𝑒𝑒 𝑥𝑥 =
1 − 𝑦𝑦
𝑥𝑥 (1
𝑒𝑒 − 𝑦𝑦) = 𝑦𝑦
𝑒𝑒 𝑥𝑥 = 𝑦𝑦 + 𝑒𝑒 𝑥𝑥 𝑦𝑦
𝑒𝑒 𝑥𝑥 = 𝑦𝑦(1 + 𝑒𝑒 𝑥𝑥 )
𝑒𝑒 𝑥𝑥
𝑦𝑦 =
1 + 𝑒𝑒 𝑥𝑥
Comparing Results
𝑥𝑥 𝑦𝑦approx 𝑦𝑦exact 𝜖𝜖𝑡𝑡

0 − 0.5 −
0.1 0.52498 0.52497 0.00002 ≅ 0
0.2 0.54983 0.54983 0.00007 ≅ 0
0.3 0.57451 0.57444 0.00012 ≅ 0
Note: The difference of approximation with exact values is negligible, and therefore it is
considered/approximated as zero.
5.2 Partial Differential Equations
There are many boundary value problems which involve partial differential equations. Only a
few of these equations can be solved by analytical methods. In most cases, we depend on the
numerical solution of such partial differential equations. Of the various numerical methods
available for solving these equations, the method of finite differences is most commonly used.
In this method, the derivatives appearing in the equation and the boundary conditions are
replaced by their finite-difference approximations. Then the given equation is changed to a
difference equation which is solved by iterative procedures. This process is slow but gives good
results of boundary value problems. An added advantage of this method is that the computation
can be done by electronic computer.
5.2.1 Finite Differences Approximations

Given a function 𝑢𝑢 that depends on both 𝑥𝑥 and 𝑦𝑦, the partial derivative of 𝑢𝑢 with respect to 𝑥𝑥
at an arbitrary point (𝑥𝑥, 𝑦𝑦) is defined as:
𝜕𝜕𝜕𝜕 𝑢𝑢(𝑥𝑥 + ℎ, 𝑦𝑦) − 𝑢𝑢(𝑥𝑥, 𝑦𝑦)
= lim
𝜕𝜕𝜕𝜕 h→0 ℎ
And the partial derivative with respect to 𝑦𝑦 is defined as:
𝜕𝜕𝜕𝜕 𝑢𝑢(𝑥𝑥, 𝑦𝑦 + 𝑘𝑘) − 𝑢𝑢(𝑥𝑥, 𝑦𝑦)
= lim
𝜕𝜕𝜕𝜕 h→0 𝑘𝑘
Similarly, the second derivatives are as follows:
𝜕𝜕 2 𝑢𝑢 𝑢𝑢(𝑥𝑥 − ℎ, 𝑦𝑦) − 2𝑢𝑢(𝑥𝑥, 𝑦𝑦) + 𝑢𝑢(𝑥𝑥 + ℎ, 𝑦𝑦)
=
𝜕𝜕𝑥𝑥 2 ℎ2
𝜕𝜕 2 𝑢𝑢 𝑢𝑢(𝑥𝑥, 𝑦𝑦 − 𝑘𝑘) − 2𝑢𝑢(𝑥𝑥, 𝑦𝑦) + 𝑢𝑢(𝑥𝑥, 𝑦𝑦 + 𝑘𝑘)
=
𝜕𝜕𝑦𝑦 2 ℎ2
Replacing the derivatives in any partial differential equation by their corresponding difference
approximations, finite-difference analogues of 'the given equations can be obtained
5.2.2 Solution of Laplace’s Equation

The Laplace’s equation describes Dirichlet problem, and is given by:
2
𝜕𝜕 2 𝑢𝑢 𝜕𝜕 2 𝑢𝑢
∇ 𝑢𝑢 = 2 + 2 = 0
𝜕𝜕𝑥𝑥 𝜕𝜕𝑦𝑦
Poisson’s equation is a variation of the Laplace’s equation, and is given by:
𝜕𝜕 2 𝑢𝑢 𝜕𝜕 2 𝑢𝑢
∇2 𝑢𝑢 = + = 𝑓𝑓(𝑥𝑥, 𝑦𝑦)
𝜕𝜕𝑥𝑥 2 𝜕𝜕𝑦𝑦 2
Substituting the values of 1st and 2nd partial derivatives in to Laplace’s Equation gives:
1 1
∇2 𝑢𝑢 = 2
�𝑢𝑢𝑖𝑖−1,𝑗𝑗 − 2𝑢𝑢𝑖𝑖,𝑗𝑗 + 𝑢𝑢𝑖𝑖+1,𝑗𝑗 � + 2 �𝑢𝑢𝑖𝑖,𝑗𝑗−1 − 2𝑢𝑢𝑖𝑖,𝑗𝑗 + 𝑢𝑢𝑖𝑖+𝑗𝑗+1 � = 0
ℎ ℎ
Making 𝑢𝑢𝑖𝑖,𝑗𝑗 the subject of the equation gives:
1
𝑢𝑢𝑖𝑖,𝑗𝑗 = �𝑢𝑢 + 𝑢𝑢𝑖𝑖+1,𝑗𝑗 + 𝑢𝑢𝑖𝑖,𝑗𝑗+1 + 𝑢𝑢𝑖𝑖,𝑗𝑗−1 �
4 𝑖𝑖−1,𝑗𝑗
where
𝑢𝑢𝑖𝑖+1,𝑗𝑗 = 𝑢𝑢(𝑥𝑥 + ℎ, 𝑦𝑦)
𝑢𝑢𝑖𝑖−1,𝑗𝑗 = 𝑢𝑢(𝑥𝑥 − ℎ, 𝑦𝑦)
𝑢𝑢𝑖𝑖,𝑗𝑗−1 = 𝑢𝑢(𝑥𝑥, 𝑦𝑦 − 𝑘𝑘)
𝑢𝑢𝑖𝑖,𝑗𝑗+1 = 𝑢𝑢(𝑥𝑥, 𝑦𝑦 − 𝑘𝑘)
Consider a rectangular region 𝑹𝑹 for which 𝑢𝑢 (𝑥𝑥, 𝑦𝑦) is known at the boundary. Divide this
region into a network of square mesh of side ℎ.
𝑌𝑌
𝑏𝑏1,5 𝑏𝑏2,5 𝑏𝑏3,5 𝑏𝑏4,5 𝑏𝑏5,5
𝑢𝑢1,4 𝑏𝑏5,4
𝑢𝑢2,4 𝑢𝑢3,4 𝑢𝑢4,4
𝑏𝑏1,3 𝑏𝑏5,3
𝑢𝑢2,3 𝑢𝑢3,3 𝑢𝑢4,3
𝑏𝑏1,2 𝑏𝑏5,2
𝑢𝑢2,2 𝑢𝑢3,2 𝑢𝑢4,2
𝑋𝑋
𝑏𝑏1,1 𝑏𝑏2,1 𝑏𝑏3,1 𝑏𝑏4,1 𝑏𝑏5,1
Figure 5-3 Meshing of region R in to square sized mesh
The equation derived shows that the value of 𝑢𝑢𝑖𝑖,𝑗𝑗 at any interior mesh point is the average of
its values at four neighboring points to the left, right, above and below. This equation is called
the standard 5-point formula.
Alternatively, a diagonal 5-point formula can be used, and it is given by:
1
𝑢𝑢𝑖𝑖,𝑗𝑗 = �𝑢𝑢𝑖𝑖−1,𝑗𝑗+1 + 𝑢𝑢𝑖𝑖+1,𝑗𝑗−1 + 𝑢𝑢𝑖𝑖+1,𝑗𝑗+1 + 𝑢𝑢𝑖𝑖−1,𝑗𝑗−1 �
4
Diagonal 5-point formula is less accurate than Standard 5-point formula, yet it is a reasonably
good approximation for obtaining starting values at mesh points.
Usually, Diagonal 5-point formula is used find the value of 𝑢𝑢 at interior mesh points, then
further iterations are done using Standard 5-point formula. The method commonly employed
to solve the resulting simultaneous equations is Liebman’s method, also known as Gauss-Seidal
method (iterative)
When only the standard 5-point formula is used, then Gauss elimination method with
backwards substitution (exact) and Gauss-Jordan elimination method (also exact) are employed
to solve the resulting simultaneous equations.
𝑢𝑢𝑖𝑖,𝑗𝑗+1 𝑢𝑢𝑖𝑖−1,𝑗𝑗−1 𝑢𝑢𝑖𝑖+1,𝑗𝑗+1
𝑢𝑢𝑖𝑖−1,𝑗𝑗 𝑢𝑢𝑖𝑖,𝑗𝑗 𝑢𝑢𝑖𝑖+1,𝑗𝑗 𝑢𝑢𝑖𝑖,𝑗𝑗
𝑢𝑢𝑖𝑖,𝑗𝑗−1 𝑢𝑢𝑖𝑖−1,𝑗𝑗−1 𝑢𝑢𝑖𝑖+1,𝑗𝑗−1
(a) (b)
Figure 5-4 Representation of finite difference approximation of Laplace’s equation. (a) Standard 5-Point formula, (b)
Diagonal 5-Point formula
5.2.2.1 Solution of Dirichlet problems

1. Choose the value of ℎ and define grids
2. Get mesh points or nodes
3. Apply difference equations at mesh/nodes to get a system of linear equations:
𝑢𝑢(𝑅𝑅) + 𝑢𝑢(𝑈𝑈) + 𝑢𝑢(𝐿𝐿) + 𝑢𝑢(𝐷𝐷) − 4𝑢𝑢 = 0
where
𝑅𝑅 = 𝑅𝑅𝑅𝑅𝑅𝑅ℎ𝑡𝑡
𝑈𝑈 = 𝑈𝑈𝑈𝑈
𝐿𝐿 = 𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿
𝐷𝐷 = 𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷
Alternatively,
1
𝑢𝑢𝑖𝑖,𝑗𝑗 = �𝑢𝑢𝑖𝑖−1,𝑗𝑗+1 + 𝑢𝑢𝑖𝑖+1,𝑗𝑗−1 + 𝑢𝑢𝑖𝑖+1,𝑗𝑗+1 + 𝑢𝑢𝑖𝑖−1,𝑗𝑗−1 �
4
4. Solve the system of linear equations to obtain the solution to the partial differential
equation.
Example 5-4
A square plate of side 12 cm made of homogenous material is kept at constant temperature
of 0° C and 100° C as shown in the figure. Using a grid of 4 cm, find the steady state of
temperature at mesh points
𝑌𝑌
𝑢𝑢 = 0
𝑢𝑢 = 100
𝑢𝑢 = 100
𝑋𝑋
𝑢𝑢 = 100
Solution:
𝑌𝑌
𝑢𝑢1,3 𝑢𝑢2,3
𝑢𝑢0,3
𝑢𝑢3,3
𝑢𝑢0,2
𝑢𝑢1,2 𝑢𝑢2,2 𝑢𝑢3,2
𝑢𝑢0,1
𝑢𝑢1,1 𝑢𝑢2,1 𝑢𝑢3,1
𝑋𝑋
𝑢𝑢0,0 𝑢𝑢1,0 𝑢𝑢2,0 𝑢𝑢3,0
Applying Laplace equation:

𝑢𝑢21 + 𝑢𝑢12 + 𝑢𝑢01 + 𝑢𝑢10 − 4𝑢𝑢11 = 0
𝑢𝑢22 + 𝑢𝑢13 + 𝑢𝑢02 + 𝑢𝑢11 − 4𝑢𝑢12 = 0
𝑢𝑢31 + 𝑢𝑢22 + 𝑢𝑢11 + 𝑢𝑢20 − 4𝑢𝑢21 = 0
𝑢𝑢32 + 𝑢𝑢23 + 𝑢𝑢12 + 𝑢𝑢21 − 4𝑢𝑢22 = 0
Substituting boundary conditions:
𝑢𝑢21 + 𝑢𝑢12 − 4𝑢𝑢11 = −200
𝑢𝑢22 + 𝑢𝑢11 − 4𝑢𝑢12 = −100
𝑢𝑢22 + 𝑢𝑢11 − 4𝑢𝑢21 = −200
𝑢𝑢12 + 𝑢𝑢21 − 4𝑢𝑢22 = −100
The resulting simultaneous equation in augmented matrix form is
−4 1 1 0 −200
1 −4 0 1 −200
� � �
1 0 −4 1 −100
0 1 1 −4 −100
Applying Gauss-Jordan method and converting the above matric in reduced row echelon
form:
1 1
1 − − 0 | 50
1 ⎛ 4 4 ⎞
− 𝑅𝑅1 ⟶ ⎜1 −4 0 1 | −200⎟
4
1 0 −4 1 | −100
⎝0 1 1 −4 | −100⎠
1 1 |
1 − − 0 50
4 4 |
⎛ ⎞
15 1 |
⎜0 − 1 −250⎟
𝑅𝑅2 − 𝑅𝑅1 ⎜ 4 4 | ⎟
⟶⎜ ⎟
𝑅𝑅3 − 𝑅𝑅1 1 15 |
⎜0 − 1 −150⎟
⎜ 4 4 | ⎟
|
0 1 1 −4 −100
⎝ | ⎠
1 1 |
1 − − 0 50
4 4 |
⎛ ⎞
1 4 | 200
⎜0 1 − − ⎟
4 ⎜ 15 15 | 3 ⎟
− 𝑅𝑅2 ⟶ ⎜ ⎟
15 1 15 |
⎜0 − 1 −150⎟
⎜ 4 4 | ⎟
|
0 1 1 −4 −100
⎝ | ⎠
4 1 | 200
1 0 − −
⎛ 15 15 | 3 ⎞
1
R1 + R2 ⎜0 1 − 1 4 | 200 ⎟
4 −
⎜ 15 15 | 3 ⎟
1 ⟶⎜ ⎟
𝑅𝑅3 − 𝑅𝑅2 56 16 | 500
4 ⎜0 0 − − ⎟
𝑅𝑅4 − 𝑅𝑅2 ⎜ 15 15 | 3 ⎟
16 56 | 500
0 0 − −
⎝ 15 15 | 3 ⎠
4 1 | 200
1 0 − −
⎛ 15 15 | 3 ⎞
⎜0 1 − 1 4 | 200 ⎟
15 −
⎜ 15 15 | 3 ⎟
− R3 ⟶ ⎜
56 2 | 625 ⎟
⎜0 0 1 − ⎟
⎜ 7 | 14 ⎟
16 56 | 500
0 0 − −
⎝ 15 15 | 3 ⎠
1 | 550
1 0 0 −
4 ⎛ 7 | 7 ⎞
𝑅𝑅1 + 𝑅𝑅3 2 | 975
15 ⎜0 1 0 − − ⎟
1 ⎜ 7 | 14 ⎟
𝑅𝑅2 + 𝑅𝑅3 ⟶ ⎜ 2 | 625 ⎟
15 ⎜0 0 1 − ⎟
16 ⎜ 7 | 14 ⎟
𝑅𝑅4 − 𝑅𝑅3
15 24 | 1500
0 0 0 − −
⎝ 7 | 7 ⎠
1 | 550
1 0 0 −
7 | 7 ⎞
⎛
2 | 975
⎜0 1 0 − ⎟
7 ⎜ 7 | 14 ⎟
− 𝑅𝑅4 ⟶ ⎜
24 2 | 625⎟
⎜0 0 1 − ⎟
⎜ 7 | 14 ⎟
| 125
0 0 0 1
⎝ | 2 ⎠
| 175
1 0 0 0
1 | 2 ⎞
𝑅𝑅1 + 𝑅𝑅4 ⎛
7 | 175
⎜0 1 0 0 ⎟
2 ⎜ | 2 ⎟
𝑅𝑅2 + 𝑅𝑅4 ⟶ ⎜ 125⎟
7 |
⎜0 0 1 0 ⎟
2 ⎜ | 2 ⎟
𝑅𝑅3 + 𝑅𝑅4
7 | 125
0 0 0 1
⎝ | 2 ⎠
Therefore,
𝑢𝑢11 175/2
𝑢𝑢21 175/2
� 𝑢𝑢 � = � �
12 125/2
𝑢𝑢22 125/2
Alternatively, the resulting simultaneous equations can be solved using Gauss-Seidle

Method as follows:
The standard 5-point is arranged such that the required variables are the subject of the
relation.
𝑢𝑢11 = 0.25𝑢𝑢21 + 0.25𝑢𝑢12 + 50
𝑢𝑢21 = 0.25𝑢𝑢22 + 0.25𝑢𝑢11 + 50
𝑢𝑢12 = 0.25𝑢𝑢11 + 0.25𝑢𝑢22 + 25
𝑢𝑢22 = 0.25𝑢𝑢12 + 0.25𝑢𝑢21 + 25
Taking the value of adjacent border as initial guess:

(0) (0) (0) (0)
𝑢𝑢11 = 𝑢𝑢21 = 𝑢𝑢12 = 𝑢𝑢22 = 100
1st Iteration
(1)
𝑢𝑢11 = 0.25(100) + 0.25(100) + 50 = 100
(1)
𝑢𝑢21 = 0.25(100) + 0.25(100) + 50 = 100
(1)
𝑢𝑢12 = 0.25(100) + 0.25(100) + 25 = 75
(1)
𝑢𝑢22 = 0.25(75) + 0.25(100) + 25 = 68.75
2nd Iteration
(2)
𝑢𝑢22 = 0.25(65.625) + 0.25(90.625) + 25 = 64.064
3rd Iteration
(3)
𝑢𝑢11 = 0.25(90.625) + 0.25(65.625) + 50 = 89.0625
(3)
𝑢𝑢21 = 0.25(64.064) + 0.25(89.0625) + 50 = 88.2816
(3)
𝑢𝑢12 = 0.25(89.0625) + 0.25(64.064) + 25 = 63.2816
(3)
𝑢𝑢22 = 0.25(63.2816) + 0.25(88.2816) + 25 = 62.8908
4th Iteration
(4)
𝑢𝑢11 = 0.25(88.2816) + 0.25(63.2816) + 50 = 87.8908
(4)
𝑢𝑢21 = 0.25(62.8908) + 0.25(87.8908) + 50 = 87.6954
(4)
𝑢𝑢12 = 0.25(87.8908) + 0.25(62.8908) + 25 = 62.6954
(4)
𝑢𝑢22 = 0.25(62.6954) + 0.25(87.6954) + 25 = 62.5977
5th Iteration
(5)
𝑢𝑢11 = 0.25(87.6954) + 0.25(62.6954) + 50 = 87.5977
(5)
𝑢𝑢21 = 0.25(62.5977) + 0.25(87.5977) + 50 = 87.5489
(5)
𝑢𝑢12 = 0.25(87.5977) + 0.25(62.5977) + 25 = 62.5489
(5)
𝑢𝑢22 = 0.25(62.5489) + 0.25(87.5489) + 25 = 62.5244
6th Iteration
(6)
𝑢𝑢11 = 0.25(87.5489) + 0.25(62.5489) + 50 = 87.5244
(6)
𝑢𝑢21 = 0.25(62.5244) + 0.25(87.5244) + 50 = 87.5122
(6)
𝑢𝑢12 = 0.25(87.5244) + 0.25(62.5244) + 25 = 62.5122
(6)
𝑢𝑢22 = 0.25(62.5122) + 0.25(87.5122) + 25 = 62.5061
7th Iteration
(7)
𝑢𝑢11 = 0.25(87.5122) + 0.25(62.5122) + 50 = 87.5061
(7)
𝑢𝑢21 = 0.25(62.5061) + 0.25(87.5061) + 50 = 87.5030
(7)
𝑢𝑢12 = 0.25(87.5061) + 0.25(62.5061) + 25 = 62.5030
(7)
𝑢𝑢22 = 0.25(62.5030) + 0.25(87.5030) + 25 = 62.5015
Therefore, the internal values correct up to 2 d.p are
𝑢𝑢11 87.50
𝑢𝑢21 87.50
� 𝑢𝑢 � = � �
12 62.50
𝑢𝑢22 62.50
Example 5-5
The values of 𝑢𝑢(𝑥𝑥, 𝑦𝑦) on the boundary of a square is shown in the figure. Evaluate the
function 𝑢𝑢(𝑥𝑥, 𝑦𝑦) that satisfies the Laplace’s equation, ∇2 𝑢𝑢 = 0, at the pivotal point of this
figure
𝑌𝑌
1000 1000 1000

1000
2000 500
𝑢𝑢1 𝑢𝑢2
2000 𝑢𝑢3 𝑢𝑢4 0
1000 𝑋𝑋
500 0 0
Solution:
Since all the adjacent boundary values at 𝑢𝑢4 are 0, then taking the value of 𝑢𝑢4 = 0 is a good
approximation. Hence, the first approximation of the internal points are as follows:
(0) 1
𝑢𝑢1 = (1000 + 1000 + 𝑢𝑢4 + 2000)
4
1
= (1000 + 1000 + 0 + 2000) = 1000 (diag. )
4
(0) 1
𝑢𝑢2 = (500 + 1000 + 𝑢𝑢1 + 𝑢𝑢4 )
4
1
= (500 + 1000 + 1000 + 0) = 625 (stnd. )
4
(0) 1
𝑢𝑢3 = (500 + 𝑢𝑢4 + 𝑢𝑢1 + 2000)
4
1
= (500 + 0 + 1000 + 2000) = 875 (stnd. )
4
(0) 1
𝑢𝑢4 = (0 + 0 + 𝑢𝑢2 + 𝑢𝑢3 )
4
1
= (0 + 0 + 625 + 875) = 375 (stnd. )
4
Using Gauss-Seidle Method, the 5 – point standard iteration formulae can be written as
follows:
(𝑛𝑛+1) (𝑛𝑛) (𝑛𝑛)
𝑢𝑢1 = 0.25�1000 + 2000 + 𝑢𝑢3 + 𝑢𝑢2 �
(𝑛𝑛+1) (𝑛𝑛+1) (𝑛𝑛)
𝑢𝑢2 = 0.25�500 + 1000 + 𝑢𝑢1 + 𝑢𝑢4 �
(𝑛𝑛+1) (𝑛𝑛) (𝑛𝑛+1)
𝑢𝑢3 = 0.25�500 + 2000 + 𝑢𝑢4 + 𝑢𝑢1 �
(𝑛𝑛+1) (𝑛𝑛+1) (𝑛𝑛+1)
𝑢𝑢4 = 0.25�0 + 0 + 𝑢𝑢2 + 𝑢𝑢3 �
1st Iteration
(1)
𝑢𝑢3 = 0.25(500 + 2000 + 375 + 1125) = 1000
(1)
𝑢𝑢4 = 0.25(0 + 0 + 750 + 1000) = 438
2nd Iteration
(2)
𝑢𝑢1 = 0.25(1000 + 2000 + 1000 + 750) = 1188
(2)
𝑢𝑢2 = 0.25(500 + 1000 + 1188 + 438) = 782
(2)
𝑢𝑢3 = 0.25(500 + 2000 + 438 + 1188) = 1032
(2)
𝑢𝑢4 = 0.25(0 + 0 + 782 + 1032) = 454
3rd Iteration
(3)
𝑢𝑢1 = 0.25(1000 + 2000 + 1032 + 782) = 1204
(3)
𝑢𝑢2 = 0.25(500 + 1000 + 1204 + 454) = 790
(3)
𝑢𝑢3 = 0.25(500 + 2000 + 454 + 1204) = 1040
(3)
𝑢𝑢4 = 0.25(0 + 0 + 790 + 1040) = 458
4th Iteration
(4)
𝑢𝑢1 = 0.25(1000 + 2000 + 1040 + 790) = 1208
(4)
𝑢𝑢2 = 0.25(500 + 1000 + 1208 + 458) = 792
(4)
𝑢𝑢3 = 0.25(500 + 2000 + 458 + 1208) = 1042
(4)
𝑢𝑢4 = 0.25(0 + 0 + 792 + 1042) = 459
5th Iteration
(5)
𝑢𝑢1 = 0.25(1000 + 2000 + 1042 + 792) = 1209
(5)
𝑢𝑢2 = 0.25(500 + 1000 + 1209 + 459) = 792
(5)
𝑢𝑢3 = 0.25(500 + 2000 + 459 + 1209) = 1042
(5)
𝑢𝑢4 = 0.25(0 + 0 + 792 + 1042) = 459
Since there is no significant change between last two consecutive iterations, the values have
converged. Hence,
𝑢𝑢1 1209
𝑢𝑢2 792
�𝑢𝑢 � = � �
3 1042
𝑢𝑢4 459
6 SYSTEM OF LINEAR EQUATIONS
A system of linear equations consists of two or more linear equations made up of two or more
variables such that all equations in the system are considered simultaneously. To find the
unique solution to a system of linear equations, a numerical value for each variable in the
system must be found that will satisfy all equations in the system at the same time. Some linear
systems may not have a solution and others may have an infinite number of solutions. In order
for a linear system to have a unique solution, there must be at least as many equations as there
are variables. Even so, this does not guarantee a unique solution
6.1 General Form of Linear System

A system of linear equation is generally represented in matrix form as follows
𝐴𝐴𝐴𝐴 = 𝐵𝐵
where
𝑎𝑎11 ⋯ 𝑎𝑎1𝑛𝑛
𝐴𝐴 = � ⋮ ⋱ ⋮ �
𝑎𝑎𝑚𝑚1 ⋯ 𝑎𝑎𝑚𝑚𝑚𝑚
𝑚𝑚 = Number of rows, 𝑛𝑛 = Number of columns
𝑥𝑥1
𝑋𝑋 = � ⋮ �
𝑥𝑥𝑛𝑛
𝑏𝑏1
𝐵𝐵 = � ⋮ �
𝑏𝑏𝑛𝑛
When 𝐁𝐁 ≠ 𝟎𝟎, the system is said to be non-homogenous. While a homogenous system has 𝐁𝐁 =
𝟎𝟎.
6.2 Types of Linear Systems

6.2.1 Consistent System
A consistent system of equations has at least one solution. A consistent system is an
independent system if it has a single solution. For example, two non-parallel lines in the same
plane will intersect at only one point, and therefore will have only one solution.
For a system to be independent, following condition must be met
Rank of Matrix 𝐴𝐴 = Rank of Reduced Matrix 𝐴𝐴𝑏𝑏
• Rank is the number of non-zero rows
• 𝐀𝐀 𝒃𝒃 = Augmented Matrix, a matrix obtained by appending the columns of two given
matrices, usually for the purpose of performing the same elementary row operations on
each of the given matrices. For example, the linear system 𝐴𝐴𝐴𝐴 = 𝐵𝐵 can be represented
in form of augmented matrix as follows:
𝑎𝑎11 ⋯ 𝑎𝑎1𝑛𝑛 𝑏𝑏1
� ⋮ ⋱ ⋮ �⋮�
𝑎𝑎𝑚𝑚1 ⋯ 𝑎𝑎𝑚𝑚𝑚𝑚 𝑏𝑏𝑛𝑛
The separator line is used for visual reference and has no significance on the calculations or
the purpose of augmented matrix.
A consistent system is a dependent system if the equations have an infinite number of solutions.
For example, two lines with the same slope and the same y-intercepts will coincide and the two
equations represents the same line, and every point on the line represents a coordinate pair that
satisfies the system thereby giving infinite number of solutions. Alternatively, solution of two
equations representing non-parallel planes will also have infinite number of solutions since the
intersection of two planes results in a line.
For a system to be dependent, following condition must be met
Rank of Matrix 𝐴𝐴 < Rank of Reduced Matrix 𝐴𝐴𝑏𝑏
6.2.1.1 Homogenous System

A system of linear equations is homogeneous if all of the constant terms are zero. Every
homogeneous system has at least one solution, known as the zero solution (or trivial solution),
which is obtained by assigning the value of zero to each of the variables. If the system has a
non-singular matrix (det(A) ≠ 0) then it is also the only solution (trivial solution). If the system
has a singular matrix then there is a solution set with an infinite number of solutions (non-
trivial solution).
In case of homogenous system, since the zero vector is always a solution, a homogeneous
system of equations can never be inconsistent therefore it is always consistent.
6.2.2 Inconsistent System

Another type of system of linear equations is an inconsistent system, which is one in which the
equations represent two non-intersecting lines. For example, if two lines have the same slope
and different y-intercepts, then there are no points common to both lines; hence, there is no
solution to the system. Alternatively, two non-parallel line on two different parallel planes will
never intersect, hence there will be no solution to this system and the system will be considered
inconsistent. The reduced form of augmented matrix of an inconsistent system will have at
least one row where 0 = 𝐶𝐶, where 𝐶𝐶 is any non-zero constant.
6.3 Analytical Solution of Linear System

Simultaneous linear equations occur in various engineering problems. A given system of linear
equations can be solved by Cramer's rule or by Matrix method. But these methods become
tedious for large systems. However, there exist other numerical methods of solution which are
well suited for computing machines. Some of the analytical and iterative methods of solution
will be explored in this chapter.
6.3.1 Gauss Elimination Method

The fundamental idea is to add multiples of one row in augmented matrix (or equation) to the
others in order to eliminate a variable and to continue this process until only one variable is
left. Once this final variable is determined, its value is substituted back into the other equations
in order to evaluate the remaining unknowns. Applying Gauss elimination will convert the
matrix in row echelon form (REF).
Basically, all variables in the last row are eliminated except for one, all variables in the row
above are eliminated, except two, and so on and so forth to the top equation, which has all the
variables. Then back substitution is used to solve for one variable at a time by plugging the
known values into the equations from the bottom up.
A matrix is in row echelon form (ref) when it satisfies the following conditions.
• The first non-zero element in each row, called the leading entry, is 1.
• Each leading entry is in a column to the right of the leading entry in the previous row.
In case of consistent independent system, all the elements on principle diagonal will be
leading entry with a value of unity.
• Rows with all zero elements, if any, are below rows having a non-zero element.
6.3.1.1 Steps for Solution through Gaussian Elimination

The basic steps to solve a linear system through Gaussian elimination are as follows:
1. Write the system of equations in augmented matrix form
2. Perform elementary row operation to convert all the leading entries to unity and get
zeros below the leading entries. Elementary row operation is one of the following
• Multiply all the elements of the row by a non-zero constant
• Add or subtract a non-zero multiple of one row to another
• Switch two rows
3. Inspect the resulting matrix and re-interpret it as a system of equations
• If 0 = 𝑛𝑛𝑛𝑛𝑛𝑛 − 𝑧𝑧𝑧𝑧𝑧𝑧𝑧𝑧 𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞𝑞 , then the system is inconsistent and has no
solution
• If the number of equations obtained is less than the number of unknowns (e.g.
getting a row with all elements equaling zero), then the system has infinite
number of solutions
• If the number of equations obtained is equal to the number of unknowns, then
there is exactly one solution to the system
4. Use back substitution to solve for one variable at a time by plugging the known values
into the equations from the bottom up
Example 6-1
Solve the given system of equation by using Gauss Elimination method:
2𝑥𝑥 + 5𝑦𝑦 + 3𝑧𝑧 = 9
3𝑥𝑥 + 𝑦𝑦 + 2𝑧𝑧 = 3
𝑥𝑥 + 2𝑦𝑦 − 𝑧𝑧 = 6
Solution:
The augmented matrix form of the given system is:
2 5 3 | 9
�3 1 2 | 3�
1 2 −1 | 6
Applying gauss elimination:
1 2 −1 | 6
𝑅𝑅3 ⇆ 𝑅𝑅1 ⟶ �3 1 2 | 3�
2 5 3 | 9
1 2 −1 | 6
𝑅𝑅2 − 3𝑅𝑅1
⟶ �0 −5 5 | −15�
0 1 5 | −3
1 2 −1 | 6
1
− 𝑅𝑅2 ⟶ �0 1 −1 | 3�
5
0 1 5 | −3
1 2 −1 | 6
𝑅𝑅3 − 𝑅𝑅2 ⟶ �0 1 −1 | 3�
0 0 6 | −6
1 2 −1 | 6
1
𝑅𝑅 ⟶ �0 1 −1 | 3�
6 3
0 0 1 | −1
The form obtained above is REF, and
2 5 3 | 9 1 2 −1 | 6
�3 1 2 | 3� ≡ �0 1 −1 | 3�
1 2 −1 | 6 0 0 1 | −1
Using back substitution
𝑧𝑧 = −1
𝑦𝑦 − 𝑧𝑧 = 3
𝑦𝑦 = 3 + 𝑧𝑧
=3−1
=2
𝑥𝑥 + 2𝑦𝑦 − 𝑧𝑧 = 6
𝑥𝑥 = 6 + 𝑧𝑧 − 2𝑦𝑦
= 6 − 1 − 2(2)
=1
Therefore,
𝑥𝑥 1
�𝑦𝑦� = � 2�
𝑧𝑧 −1
6.3.2 Gauss – Jordan Elimination Method

Gauss – Jordan elimination method is a method of solving a linear system of equations. This is
done by transforming the system's augmented matrix into reduced row-echelon form (RREF)
by means of row operations.
A matrix is in reduced row echelon form (also called row canonical form) if it satisfies the
following conditions:
• It is in row echelon form
• The leading entry in each nonzero row is a 1 (called a leading 1)
• Each column containing a leading 1 has zeros everywhere else
6.3.2.1 Steps for Solution through Gauss – Jordan Elimination
The basic steps to solve a linear system through Gaussian elimination are as follows:
1. Write the system of equations in augmented matrix form
2. Perform elementary row operation to convert all the leading entries to unity and get
zeros below and above the leading entries. Elementary row operation is one of the
following
• Multiply all the elements of the row by a non-zero constant
• Add or subtract a non-zero multiple of one row to another
• Switch two rows
3. The final form of the augmented matrix will give the value of variables, without the
need of back substitution.
Example 6-2
Solve the given system of equation by using Gauss – Jordan Elimination method:
𝑥𝑥 + 2𝑦𝑦 + 3𝑧𝑧 = 6
3𝑥𝑥 − 2𝑦𝑦 + 𝑧𝑧 = 2
4𝑥𝑥 + 2𝑦𝑦 + 𝑧𝑧 = 7
Solution:
1 2 3 | 6
�3 −2 1 | 2�
4 2 1 | 7
Applying Gauss – Jordan elimination:
1 2 3 | 6
⟶ �0 −8 −8 | −16�
0 −6 −11 | −17
1 2 3 | 6
1
− 𝑅𝑅2 ⟶ �0 1 1 | 2�
8
0 −6 −11 | −17
1 0 1 | 2
⟶ �0 1 1 | 2�
𝑅𝑅3 + 6𝑅𝑅2
0 0 −5 | −5
1 0 1 | 2
1
− 𝑅𝑅2 ⟶ �0 1 1 | 2�
5
0 0 1 | 1
1 0 0 | 1
⟶ �0 1 0 | 1�
0 0 1 | 1
Hence,
𝑥𝑥 1
�𝑦𝑦� = �1�
𝑧𝑧 1
Example 6-3
Solve the given system of equation by using Elimination method:
𝑥𝑥 − 𝑦𝑦 + 𝑧𝑧 = 1
2𝑥𝑥 + 𝑦𝑦 − 𝑧𝑧 = 2
5𝑥𝑥 − 2𝑦𝑦 + 2𝑧𝑧 = 5
Solution:
1 −1 1 | 1
� 2 1 −1 | 2�
5 −2 2 | 5
Applying Gaussian elimination:
1 −1 1 | 1
⟶� 0 3 −3 | 0�
0 3 −3 | 0
1 −1 1 | 1
1
𝑅𝑅 ⟶ � 0 1 −1 | 0�
3 2
0 3 −3 | 0
1 −1 1 | 1
𝑅𝑅3 − 3𝑅𝑅2 ⟶ �0 1 −1 | 0�
0 0 0 | 0
Since 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟(𝐴𝐴𝑏𝑏 ) < number of unknowns, the system has infinite solutions. To obtain the
infinite solution, let 𝑧𝑧 = 𝑘𝑘, where 𝑘𝑘 ∈ ℝ, then
𝑦𝑦 − 𝑧𝑧 = 0
𝑦𝑦 = 𝑧𝑧
= 𝑘𝑘
𝑥𝑥 − 𝑦𝑦 + 𝑧𝑧 = 1
𝑥𝑥 = 1 − 𝑧𝑧 + 𝑦𝑦
= 1 − 𝑘𝑘 + 𝑘𝑘
=1
Hence,
𝑥𝑥 1
𝑦𝑦
� � = �𝑘𝑘 �
𝑧𝑧 𝑘𝑘
Example 6-4
Solve the given homogenous system of equation by using Elimination method:
𝑥𝑥1 + 𝑥𝑥2 + 2𝑥𝑥3 = 0
𝑥𝑥1 − 2𝑥𝑥2 + 𝑥𝑥3 = 0
3𝑥𝑥1 + 𝑥𝑥2 + 𝑥𝑥3 = 0
Solution:
1 1 2 | 0
�1 −2 1 | 0�
3 1 1 | 0
Applying Gaussian elimination:
1 1 2 | 0
⟶� 0 −3 −1 | 0�
0 −2 −5 | 0
|
1 1 2 0
|
⎛ ⎞
1 1 |
− 𝑅𝑅2 ⟶ ⎜0 1 0⎟
3 ⎜ 3 | ⎟
|
0 −2 −5 0
⎝ | ⎠
|
1 1 2 0
|
⎛ ⎞
1 |
⎜
𝑅𝑅3 + 2𝑅𝑅2 ⟶ ⎜0 1 0⎟
3 | ⎟
⎜ ⎟
13 |
0 0 − 0
⎝ 3 | ⎠
|
1 1 2 0
|
⎛ ⎞
3 1 |
− 𝑅𝑅3 ⟶ ⎜0 1 0⎟
13 ⎜ 3 | ⎟
|
0 0 1 0
⎝ | ⎠
Hence,
𝑥𝑥1 0
�𝑥𝑥2 � = �0�
𝑥𝑥3 0
6.4 LU Decomposition
Up till now, Gauss – elimination or its variation has been used to solve the system of linear
algebraic equation
Although it certainly represents a sound way to solve such systems, it becomes inefficient when
solving equations with the same coefficients 𝐴𝐴, but with different right-hand-side constants 𝐵𝐵.
Recall that Gauss elimination involves two steps: forward elimination and back substitution.
Of these, the forward-elimination step comprises the bulk of the computational effort. This is
particularly true for large systems of equations.
LU decomposition methods separate the time-consuming elimination of the matrix 𝐴𝐴 from the
manipulations of the right-hand side 𝐵𝐵. Thus, once 𝐴𝐴 has been “decomposed,” multiple right-
hand-side vectors can be evaluated in an efficient manner.
6.4.1 Doolittle Decomposition

Consider the equations
𝑎𝑎11 𝑥𝑥1 + 𝑎𝑎12 𝑥𝑥2 + 𝑎𝑎13 𝑥𝑥3 = 𝑏𝑏1
𝑎𝑎21 𝑥𝑥1 + 𝑎𝑎22 𝑥𝑥2 + 𝑎𝑎23 𝑥𝑥3 = 𝑏𝑏2
𝑎𝑎31 𝑥𝑥1 + 𝑎𝑎32 𝑥𝑥2 + 𝑎𝑎33 𝑥𝑥3 = 𝑏𝑏3
Where,
𝑎𝑎11 𝑎𝑎12 𝑎𝑎13 𝑥𝑥1 𝑏𝑏1
𝐴𝐴 = �𝑎𝑎21 𝑎𝑎22 𝑎𝑎23 � , 𝑋𝑋 = �𝑥𝑥2 � and 𝐵𝐵 = �𝑏𝑏2 �
Now, let
𝐴𝐴 = 𝐿𝐿𝐿𝐿
where,
1 0 0 𝑢𝑢11 𝑢𝑢12 𝑢𝑢13
𝑙𝑙
𝐿𝐿 = � 21 1 0� and 𝑈𝑈 = � 0 𝑢𝑢22 𝑢𝑢23 �
𝑙𝑙31 𝑙𝑙32 1 0 0 𝑢𝑢33
𝐿𝐿 is a lower triangular matrix with all the elements in the principle diagonal being unity. While
𝑈𝑈 is an upper triangular matrix with all the elements in the principle diagonal being real
numbers.
Now,
𝐿𝐿𝐿𝐿𝐿𝐿 = 𝐵𝐵
and
𝑈𝑈𝑈𝑈 = 𝑌𝑌
then
𝐿𝐿𝐿𝐿 = 𝐵𝐵
To compute matrices 𝐿𝐿 and 𝑈𝑈, following relation can be used:
1 0 0 𝑢𝑢11 𝑢𝑢12 𝑢𝑢13 𝑎𝑎11 𝑎𝑎12 𝑎𝑎13
�𝑙𝑙21 1 0� � 0 𝑢𝑢22 𝑢𝑢23 � = �𝑎𝑎21 𝑎𝑎22 𝑎𝑎23 �
𝑙𝑙31 𝑙𝑙32 1 0 0 𝑢𝑢33 𝑎𝑎31 𝑎𝑎32 𝑎𝑎33
Following steps are to be followed to solve a linear system using Crout’s method
1. Using 𝐴𝐴 = 𝐿𝐿𝐿𝐿, find the matrix 𝐿𝐿 and 𝑈𝑈
2. Using 𝐿𝐿𝐿𝐿 = 𝐵𝐵, find the matrix 𝑌𝑌
3. Using 𝑈𝑈𝑈𝑈 = 𝑌𝑌, find the matrix 𝑋𝑋
Example 6-5
Solve the given system of equations by using Doolittle method:
4𝑥𝑥1 + 3𝑥𝑥2 − 𝑥𝑥3 = −2
−2𝑥𝑥1 − 4𝑥𝑥2 + 5𝑥𝑥3 = 20
𝑥𝑥1 + 2𝑥𝑥2 + 6𝑥𝑥3 = 7
Solution:
4 3 −1 1 0 0 𝑢𝑢11 𝑢𝑢12 𝑢𝑢13
�−2 −4 5� = �𝑙𝑙21 1 0� � 0 𝑢𝑢22 𝑢𝑢23 �
1 2 6 𝑙𝑙31 𝑙𝑙32 1 0 0 𝑢𝑢33
4 3 −1 𝑢𝑢11 𝑢𝑢12 𝑢𝑢13
�−2 −4 𝑙𝑙
5� = � 21 11 𝑢𝑢 𝑙𝑙 𝑢𝑢
21 12 + 𝑢𝑢 22 𝑙𝑙21 13 + 𝑢𝑢23
𝑢𝑢 �
1 2 6 𝑙𝑙31 11𝑢𝑢 𝑙𝑙 𝑢𝑢
31 12 + 𝑙𝑙 𝑢𝑢
32 22 𝑙𝑙 𝑢𝑢
31 13 + 𝑙𝑙 𝑢𝑢
32 23 + 𝑢𝑢 33
Now,
𝑢𝑢11 = 4 𝑢𝑢12 = 3 𝑢𝑢13 = −1
𝑙𝑙21 𝑢𝑢11 = −2 𝑙𝑙21 𝑢𝑢12 + 𝑢𝑢22 = −4 𝑙𝑙21 𝑢𝑢13 + 𝑢𝑢23 = 5
2 1 𝑢𝑢22 = −4 − 𝑙𝑙21 𝑢𝑢12 𝑢𝑢23 = 5 − 𝑙𝑙21 𝑢𝑢13
𝑙𝑙21 = − = − 1 1
4 2
𝑢𝑢22 = −4 − �− � (3) 𝑢𝑢23 = 5 − �− � (−1)
2 2
5 9
𝑢𝑢22 = − 𝑢𝑢23 =
2 2
𝑙𝑙31 𝑢𝑢11 = 1 𝑙𝑙31 𝑢𝑢12 + 𝑙𝑙32 𝑢𝑢22 = 2 𝑙𝑙31 𝑢𝑢13 + 𝑙𝑙31 𝑢𝑢23 + 𝑢𝑢33 = 6
1 2 − 𝑙𝑙31 𝑢𝑢12 𝑢𝑢33 = 6 − 𝑙𝑙31 𝑢𝑢13 − 𝑙𝑙32 𝑢𝑢23
𝑙𝑙31 = 𝑙𝑙32 = 1 1 9
𝑢𝑢11 𝑢𝑢22 𝑢𝑢33 = 6 − � � (−1) − �− � � �
1 1 4 2 2
𝑙𝑙31 = 2 − � � (3) 17
4 𝑙𝑙32 = 4
5 𝑢𝑢33 =
− 2
2
1
𝑙𝑙32 = −
2
Since
Taking 𝐴𝐴 = 𝐿𝐿𝐿𝐿, therefore
Taking 𝑈𝑈𝑈𝑈 = 𝑌𝑌, hence
and
1 0 0
1 𝑦𝑦1 −2
⎛− 1 0⎞ 𝑦𝑦
⎜ 2 ⎟ � 2 � = � 20 �
1 1 𝑦𝑦3 7
− 1
⎝ 4 2 ⎠
𝑦𝑦1 = −2
1
− 𝑦𝑦1 + 𝑦𝑦2 = 20
2
1
𝑦𝑦2 = 20 + 𝑦𝑦1
2
1
= 20 + (−2)
2
= 19
1 1
𝑦𝑦1 − 𝑦𝑦2 + 𝑦𝑦3 = 7
4 2
1 1
𝑦𝑦3 = 7 − 𝑦𝑦1 + 𝑦𝑦2
4 2
1 1
= 7 − (−2) + (19)
4 2
= 17
But,
4 3 −1
5 9 𝑥𝑥 −2
⎛0 − ⎞ 1
⎜ 2 2 ⎟ � 𝑥𝑥2 � = � 19 �
17 𝑥𝑥3 17
0 0
⎝ 2⎠
𝑥𝑥3 = 2
5 9
− 𝑥𝑥2 + 𝑥𝑥3 = 19
2 2
9
19 − 𝑥𝑥3
𝑥𝑥2 = 2
5
−
2
9
19 − (2)
= 2
5
−
2
= −4
4𝑥𝑥1 + 3𝑥𝑥2 − 𝑥𝑥3 = −2
−2 − 3𝑥𝑥2 + 𝑥𝑥3
𝑥𝑥1 =
4
−2 − 3(−4) + (2)
=
4
=3
Therefore, 𝑥𝑥1 = 3, 𝑥𝑥2 − 4, 𝑥𝑥3 = 2
Practice Question
Use Doolittle’s method of matrix decomposition to solve the following linear system:
2𝑥𝑥1 + 𝑥𝑥2 + 3𝑥𝑥3 = 11
4𝑥𝑥1 + 3𝑥𝑥2 + 10𝑥𝑥3 = 28
2𝑥𝑥1 + 4𝑥𝑥2 + 17𝑥𝑥3 = 31
6.4.2 Crout’s Decomposition

The working of this method is similar to Doolittle method, but the conditions for 𝐿𝐿 and 𝑈𝑈
matrix are exchanged.
In Crout’s method, 𝐿𝐿 is a lower triangular matrix with all the elements in the principle diagonal
being real numbers. While 𝑈𝑈 is an upper triangular matrix with all the elements in the principle
diagonal being unity. Consider the equations
𝑎𝑎11 𝑥𝑥1 + 𝑎𝑎12 𝑥𝑥2 + 𝑎𝑎13 𝑥𝑥3 = 𝑏𝑏1
𝑎𝑎21 𝑥𝑥1 + 𝑎𝑎22 𝑥𝑥2 + 𝑎𝑎23 𝑥𝑥3 = 𝑏𝑏2
𝑎𝑎31 𝑥𝑥1 + 𝑎𝑎32 𝑥𝑥2 + 𝑎𝑎33 𝑥𝑥3 = 𝑏𝑏3
Where,
𝐴𝐴 = �𝑎𝑎21 𝑎𝑎22 𝑎𝑎23 � , 𝑋𝑋 = �𝑥𝑥2 � and 𝐵𝐵 = �𝑏𝑏2 �
Now, let
where,
𝑙𝑙11 0 0 1 𝑢𝑢12 𝑢𝑢13
𝐿𝐿 = �𝑙𝑙21 𝑙𝑙22 0 � and 𝑈𝑈 = �0 1 𝑢𝑢23 �
𝑙𝑙31 𝑙𝑙32 𝑙𝑙33 0 0 1
Therefore,
Let 𝑈𝑈𝑈𝑈 = 𝑌𝑌, then
To compute matrices 𝐿𝐿 and 𝑈𝑈, following relation can be used:
𝑙𝑙11 0 0 1 𝑢𝑢12 𝑢𝑢13 𝑎𝑎11 𝑎𝑎12 𝑎𝑎13
�𝑙𝑙21 𝑙𝑙22 0 � �0 1 𝑎𝑎
𝑢𝑢23 � = � 21 𝑎𝑎22 𝑎𝑎23 �
𝑙𝑙31 𝑙𝑙32 𝑙𝑙33 0 0 1 𝑎𝑎31 𝑎𝑎32 𝑎𝑎33
Following steps are to be followed to solve a linear system using Crout’s method
1. Using 𝐴𝐴 = 𝐿𝐿𝐿𝐿, find the matrix 𝐿𝐿 and 𝑈𝑈
3. Using 𝑈𝑈𝑈𝑈 = 𝑌𝑌, find the matrix 𝑋𝑋
Example 6-6
Solve the given system of equations by using Crout’s method:
2𝑥𝑥1 + 𝑥𝑥2 + 3𝑥𝑥3 = 11
4𝑥𝑥1 − 3𝑥𝑥2 + 10𝑥𝑥3 = 28
2𝑥𝑥1 + 4𝑥𝑥2 + 17𝑥𝑥3 = 7
Solution:
2 1 3 𝑙𝑙11 0 0 1 𝑢𝑢12 𝑢𝑢13
�4 −3 10� = �𝑙𝑙21 𝑙𝑙22 0 � �0 1 𝑢𝑢23 �
2 4 17 𝑙𝑙31 𝑙𝑙32 𝑙𝑙33 0 0 1
2 1 3 𝑙𝑙11 𝑙𝑙11 𝑢𝑢12 𝑙𝑙11 𝑢𝑢13
�4 −3 10� = �𝑙𝑙21 𝑙𝑙21 𝑢𝑢12 + 𝑙𝑙22 𝑙𝑙21 𝑢𝑢13 + 𝑙𝑙22 𝑢𝑢23 �
2 4 17 𝑙𝑙31 𝑒𝑒31 𝑢𝑢12 + 𝑒𝑒32 𝑒𝑒31 𝑢𝑢13 + 𝑒𝑒23 𝑢𝑢23 + 𝑒𝑒33
From above,
5 0 0 2 1
19 1 −
⎛7 0 ⎞ ⎛ 5 5⎞
𝐿𝐿 = ⎜ 5 ⎟ and 𝑈𝑈 = ⎜0 1 32⎟
41 327 19
3
⎝ 5 19 ⎠ ⎝0 0 1⎠
Now,
5 0 0
19 𝑦𝑦1 11
⎛7 0 ⎞ 𝑦𝑦
⎜ 5 ⎟ � 2 � = � 28�
41 327 𝑦𝑦3 7
3
⎝ 5 19 ⎠
4
𝑦𝑦1 ⎛ 5 ⎞
12
�𝑦𝑦2 � = ⎜ ⎟
𝑦𝑦3 ⎜ 19 ⎟
46
⎝327⎠
Using the relation 𝑈𝑈𝑈𝑈 = 𝑌𝑌
2 1
1 −
⎛ 5 5 ⎞ 𝑥𝑥1 4/5
⎜0 1 32⎟ � 2 � = � 12/19 �𝑥𝑥
19 𝑥𝑥3 46/327
⎝0 0 1⎠
122
𝑥𝑥1 ⎛109⎞
12
�𝑥𝑥2 � = ⎜ ⎟
𝑥𝑥3 ⎜ 19 ⎟
46
⎝327⎠
6.4.3 Cholesky Decomposition
Cholesky decomposition or Cholesky factorization is a decomposition of a Hermitian (a matrix
that is equal to its own conjugate transpose), positive-definite matrix into the product of a lower
triangular matrix and its conjugate transpose, which is useful for efficient numerical solutions.
When it is applicable, the Cholesky decomposition is roughly twice as efficient as the Doolittle
Decomposition for solving systems of linear equations.
Cholesky decomposition is given by
if 𝑈𝑈 = 𝐿𝐿𝑇𝑇 , then
𝐴𝐴 = 𝐿𝐿𝐿𝐿𝑇𝑇
Therefore
𝑎𝑎11 𝑎𝑎12 𝑎𝑎13
�𝑎𝑎21 𝑎𝑎22 𝑎𝑎23 �
𝑎𝑎31 𝑎𝑎32 𝑎𝑎33
Moreover,
𝐿𝐿𝐿𝐿𝑇𝑇 𝑋𝑋 = 𝐵𝐵
And let 𝐿𝐿𝑇𝑇 𝑋𝑋 = 𝑌𝑌, then
Following steps are to be followed to solve a linear system using Cholesky method
1. Using 𝐴𝐴 = 𝐿𝐿𝐿𝐿𝑇𝑇 , find the matrix 𝐿𝐿 and 𝐿𝐿𝑇𝑇
3. Using 𝐿𝐿𝑇𝑇 𝑋𝑋 = 𝑌𝑌, find the matrix 𝑋𝑋
Example 6-7
Solve the given system of equations by using Cholesky method:
4𝑥𝑥1 + 2𝑥𝑥2 + 14𝑥𝑥3 = 14
2𝑥𝑥1 + 17𝑥𝑥2 − 5𝑥𝑥3 = −101
14𝑥𝑥1 − 5𝑥𝑥2 + 83𝑥𝑥3 = 155
Solution:
𝐴𝐴 = 𝐿𝐿𝐿𝐿𝑇𝑇
4 2 14 𝑙𝑙11 0 0 𝑙𝑙11 𝑙𝑙21 𝑙𝑙31
� 2 17 −5� = �𝑙𝑙21 𝑙𝑙22 0 � � 0 𝑙𝑙22 𝑙𝑙32 �
14 −5 83 𝑙𝑙31 𝑙𝑙32 𝑙𝑙33 0 0 𝑙𝑙33
2
4 2 14 𝑙𝑙11 𝑙𝑙11 𝑙𝑙21 𝑙𝑙11 𝑙𝑙31
2 2
� 2 17 −5� = �𝑙𝑙21 𝑙𝑙11 𝑙𝑙21 + 𝑙𝑙22 𝑙𝑙21 𝑙𝑙31 + 𝑙𝑙22 𝑙𝑙32 �
14 −5 83 𝑙𝑙31 𝑙𝑙11 𝑙𝑙31 𝑙𝑙21 + 𝑙𝑙32 𝑙𝑙22 𝑙𝑙312
+ 𝑙𝑙322 2
+ 𝑙𝑙33
From above,
𝑙𝑙11 = 2, 𝑙𝑙21 = 1, 𝑙𝑙31 = 7, 𝑙𝑙22 = 4, 𝑙𝑙32 = −3 and 𝑙𝑙33 = 5
2 0 0 2 1 7
𝐿𝐿 = �1 4 0� and 𝐿𝐿𝑇𝑇 = �0 4 −3�
7 −3 5 0 0 5
Using 𝐿𝐿𝐿𝐿 = 𝐵𝐵
2 0 0 𝑦𝑦1 14
�1 4 0 � � 𝑦𝑦2 � = �−101�
7 −3 5 𝑦𝑦3 155
Then
𝑦𝑦1 7
�𝑦𝑦2 � = �−27�
𝑦𝑦3 5
Using the relation 𝐿𝐿𝑇𝑇 𝑋𝑋 = 𝑌𝑌
2 1 7 𝑥𝑥1 7
�0 4 −3 � � 𝑥𝑥2 � = � −27�
0 0 5 𝑥𝑥3 5
𝑥𝑥1 3
�𝑥𝑥2 � = �−6�
𝑥𝑥3 1
6.5 Iterative Solution to Linear Systems

Iterative method is a mathematical procedure that uses an initial guess to generate a sequence
of improving approximate solutions for a class of problems, in which the nth approximation is
derived from the previous ones. A specific implementation of an iterative method, including
the termination criteria, is an algorithm of the iterative method. An iterative method is called
convergent if the corresponding sequence converges for given initial approximations.
6.5.1 Jacobi’s Iterative Method

Jacobi method is an iterative algorithm for determining the solutions of a diagonally dominant
system of linear equations. Each diagonal element is solved for, and an approximate value is
plugged in. The process is then iterated until it converges.
In mathematics, a square matrix is said to be diagonally dominant if for every row of the matrix,
the magnitude of the diagonal entry in a row is larger than or equal to the sum of the magnitudes
of all the other (non-diagonal) entries in that row. More precisely, the matrix A is diagonally
dominant if
|𝑎𝑎𝑖𝑖𝑖𝑖 | ≥ ��𝑎𝑎𝑖𝑖𝑖𝑖 � , for all 𝑖𝑖

𝑗𝑗≠𝑖𝑖
6.5.1.1 Steps to Solve a Linear System with Jacobi’s Iteration

1. Arrange the given system in diagonally dominant form
2. In each row, make the dominant variable the subject of the equation
3. Input the initial approximate in to the equation
• If the initial approximations are provided, use them, else take the initial
(0) (0) (0)
approximate of every variable to be zero, i.e. 𝑥𝑥1 = 𝑥𝑥2 = ⋯ = 𝑥𝑥𝑛𝑛 = 0
4. Keep performing iteration until the results converge to the required accuracy.
Example 6-8
Solve the given system of equations by using Jacobi’s Iterative method:
5𝑥𝑥1 − 2𝑥𝑥2 + 3𝑥𝑥3 = −1
−3𝑥𝑥1 + 9𝑥𝑥2 + 𝑥𝑥3 = 2
2𝑥𝑥1 − 𝑥𝑥2 − 7𝑥𝑥3 = 3
Solution:
The given system is already in diagonally dominant form. Now, making dominant variable
subject of the equation:
(𝑛𝑛+1) 1 (𝑛𝑛) (𝑛𝑛)

𝑥𝑥1 = �−1 + 2𝑥𝑥2 − 3𝑥𝑥3 �
5
(𝑛𝑛+1) 1 (𝑛𝑛) (𝑛𝑛)
𝑥𝑥2 = �2 + 3𝑥𝑥1 − 𝑥𝑥3 �
9
(𝑛𝑛+1) 1 (𝑛𝑛) (𝑛𝑛)
𝑥𝑥3 = − �3 − 2𝑥𝑥1 + 𝑥𝑥2 �
7
(0) (0) (0)
Since no initial approximate is given, 𝑥𝑥1 = 𝑥𝑥2 = 𝑥𝑥3 = 0
When 𝑛𝑛 = 0
(1) 1 (0) (0)
𝑥𝑥1 = �−1 + 2𝑥𝑥2 − 3𝑥𝑥3 �
5
1
= �−1 + 2(0) − 3(0)�
5
1
=−
5
(1) 1 (0) (0)
𝑥𝑥2 = �2 + 3𝑥𝑥1 − 𝑥𝑥3 �
9
1
= (2 + 3(0) − 0)
9
2
=
9
(1) 1 (0) (0)
𝑥𝑥3 = − �3 − 2𝑥𝑥1 + 𝑥𝑥2 �
7
1
= − �3 − 2(0) + (0)�
7
3
=−
7
Similarly, the newly approximated values are substituted into the equation to obtain better
approximate. The results of further iterations are listed below:
𝑛𝑛 𝑥𝑥1 𝑥𝑥2 𝑥𝑥3

0 0 0 0
1 - 1/5 2/9 - 3/7
2 46/315 64/315 - 163/315
3 302/1575 133/405 - 131/315
4 1001/5534 673/2025 - 1886/4483
Therefore, 𝑥𝑥1 = 0.1808, 𝑥𝑥2 = 0.3323 and 𝑥𝑥3 = −0.4207
6.5.2 Gauss Seidel Iterative Method

Gauss–Seidel method, also known as the Liebmann method or the method of successive
displacement, is an iterative method used to solve a linear system of equations. It is named
after the German mathematicians Carl Friedrich Gauss and Philipp Ludwig von Seidel and is
an improvement to the Jacobi method. Though it can be applied to any matrix with non-zero
elements on the diagonals, convergence is only guaranteed if the matrix is either diagonally
dominant, or symmetric and positive definite.
6.5.2.1 Steps to Solve a Linear System with Gauss – Seidel Iteration

1. Arrange the given system in diagonally dominant form
2. In each row, make the dominant variable the subject of the equation
3. Input the initial approximate in to the equation
• If the initial approximations are provided, use them, else take the initial
(0) (0) (0)
approximate of every variable to be zero, i.e. 𝑥𝑥1 = 𝑥𝑥2 = ⋯ = 𝑥𝑥𝑛𝑛 = 0
4. Keep performing iteration until the results converge to the required accuracy
• Use the most recent approximation available for all variables, e.g., if 𝑥𝑥2 is
(𝑛𝑛)
dependent upon 𝑥𝑥3 and 𝑥𝑥1 and the most recent approximation available are 𝑥𝑥3
(𝑛𝑛+1) (𝑛𝑛+1) (𝑛𝑛) (𝑛𝑛)
and 𝑥𝑥1 , then 𝑥𝑥1 (NOT 𝑥𝑥1 ) along with 𝑥𝑥3 must be used to calculate
(𝑛𝑛+1)
𝑥𝑥2 .
Example 6-9
Solve the given system of equations by using Gauss – Seidel Iterative method:
𝑥𝑥1 + 𝑥𝑥2 + 54𝑥𝑥3 = 110
6𝑥𝑥1 + 15𝑥𝑥2 + 2𝑥𝑥3 = 72
27𝑥𝑥1 + 6𝑥𝑥2 − 𝑥𝑥3 = 85
Solution:
The given system can be represented in diagonally dominant form by rewriting:
27𝑥𝑥1 + 6𝑥𝑥2 − 𝑥𝑥3 = 85
6𝑥𝑥1 + 15𝑥𝑥2 + 2𝑥𝑥3 = 72
𝑥𝑥1 + 𝑥𝑥2 + 54𝑥𝑥3 = 110
Now, making dominant variable subject of the equation:
(𝑛𝑛+1) 1 (𝑛𝑛) (𝑛𝑛)
𝑥𝑥1 = �85 − 6𝑥𝑥2 + 𝑥𝑥3 �
27
(𝑛𝑛+1) 1 (𝑛𝑛+1) (𝑛𝑛)
𝑥𝑥2 = �72 − 6𝑥𝑥1 − 2𝑥𝑥3 �
15
(𝑛𝑛+1) 1 (𝑛𝑛+1) (𝑛𝑛+1)
𝑥𝑥3 = �110 − 𝑥𝑥1 − 𝑥𝑥2 �
54
(0) (0) (0)
Since no initial approximate is given, 𝑥𝑥1 = 𝑥𝑥2 = 𝑥𝑥3 = 0
When 𝑛𝑛 = 0
(1) 1 85 478
𝑥𝑥3 = �110 − − �
54 27 135
= 1.91316
The results of further iterations are listed below:

𝑛𝑛 𝑥𝑥1 𝑥𝑥2 𝑥𝑥3
0 0.0000 0.0000 0.0000
1 3.1481 3.5407 1.9132
2 2.4322 3.5720 1.9258
3 2.4257 3.5729 1.9260
4 2.4255 3.5730 1.9260
5 2.4255 3.5730 1.9260 ANS.
Practice Questions
1. Solve the following system of linear equations using Gauss Seidel method of iteration.
Perform only four iterations
10𝑥𝑥1 − 2𝑥𝑥2 − 𝑥𝑥3 − 𝑥𝑥4 = 3
−2𝑥𝑥2 + 10𝑥𝑥2 − 𝑥𝑥3 − 𝑥𝑥4 = 15
−𝑥𝑥1 − 𝑥𝑥2 + 10𝑥𝑥3 − 2𝑥𝑥4 = 27
−𝑥𝑥1 − 𝑥𝑥2 − 2𝑥𝑥3 + 10𝑥𝑥4 = −9
Ans.
(4)
𝑥𝑥1 = 0.9968
(4)
𝑥𝑥2 = 1.99818
(4)
𝑥𝑥3 = 2.99866
(4)
𝑥𝑥4 = −0.000709
2. Solve the following system of linear equations using Jacobi’s method of iteration.
Perform iterations till the results converge in two decimal places.
𝑥𝑥1 + 2𝑥𝑥2 + 10𝑥𝑥3 = 35
10𝑥𝑥1 + 3𝑥𝑥2 + 𝑥𝑥3 = 19
3𝑥𝑥1 + 10𝑥𝑥2 + 2𝑥𝑥3 = 29
Ans.
(13)
𝑥𝑥1 = 1.0000188886
(13)
𝑥𝑥2 = 2.000020935
(13)
𝑥𝑥3 = 3.000014781
7 INVERSE INTERPOLATION AND
CURVE FITTING
7.1 Lagrange’s Inverse Interpolation
Inverse interpolation is defined as the process of finding the value of the argument
corresponding to a given value of the function lying between two tabulated functional values.
In this method, the formula used is same as Lagrange’s Interpolation, except that the values of
𝑥𝑥 and 𝑦𝑦 are switched with each other. General form of Lagrange’s inverse interpolation
formula is given by:
(𝑓𝑓 − 𝑓𝑓1 )(𝑓𝑓 − 𝑓𝑓2 ) … (𝑓𝑓 − 𝑓𝑓𝑛𝑛 ) (𝑓𝑓 − 𝑓𝑓0 )(𝑓𝑓 − 𝑓𝑓2 ) … (𝑓𝑓 − 𝑓𝑓𝑛𝑛 )
𝑥𝑥 = 𝑥𝑥0 + 𝑥𝑥 + ⋯
(𝑓𝑓0 − 𝑓𝑓1 )(𝑓𝑓0 − 𝑓𝑓2 ) … (𝑓𝑓0 − 𝑓𝑓𝑛𝑛 ) (𝑓𝑓1 − 𝑓𝑓0 )(𝑓𝑓1 − 𝑓𝑓2 ) … (𝑓𝑓1 − 𝑓𝑓𝑛𝑛 ) 1
(𝑓𝑓 − 𝑓𝑓0 )(𝑓𝑓 − 𝑓𝑓1 ) … (𝑓𝑓 − 𝑓𝑓𝑛𝑛−1 )
+ 𝑥𝑥
(𝑓𝑓𝑛𝑛 − 𝑓𝑓0 )(𝑓𝑓𝑛𝑛 − 𝑓𝑓1 ) … (𝑓𝑓𝑛𝑛 − 𝑓𝑓𝑛𝑛−1 ) 𝑛𝑛
Example 7-1
Use Lagrange’s inverse interpolation to find the value of 𝑥𝑥 when 𝑦𝑦 = 85. Use the following
table in your calculations
𝑥𝑥 2 5 8 14
𝑦𝑦 94.8 87.9 81.3 68.7
Solution:
(𝑓𝑓 − 𝑓𝑓1 )(𝑓𝑓 − 𝑓𝑓2 )(𝑓𝑓 − 𝑓𝑓3 ) (𝑓𝑓 − 𝑓𝑓0 )(𝑓𝑓 − 𝑓𝑓2 )(𝑓𝑓 − 𝑓𝑓3 )
𝑥𝑥 = 𝑥𝑥0 + 𝑥𝑥
(𝑓𝑓0 − 𝑓𝑓1 )(𝑓𝑓0 − 𝑓𝑓2 )(𝑓𝑓0 − 𝑓𝑓3 ) (𝑓𝑓1 − 𝑓𝑓0 )(𝑓𝑓1 − 𝑓𝑓2 )(𝑓𝑓1 − 𝑓𝑓3 ) 1
(𝑓𝑓 − 𝑓𝑓0 )(𝑓𝑓 − 𝑓𝑓1 )(𝑓𝑓 − 𝑓𝑓2 )

+ 𝑥𝑥
(𝑓𝑓3 − 𝑓𝑓0 )(𝑓𝑓3 − 𝑓𝑓1 )(𝑓𝑓3 − 𝑓𝑓2 ) 3
(85 − 87.9)(85 − 81.3)(85 − 68.7)

𝑥𝑥 = ×2
(94.8 − 87.9)(94.8 − 81.3)(94.8 − 68.7)
(85 − 94.8)(85 − 81.3)(85 − 68.7)

+ ×5
(87.9 − 94.8)(87.9 − 81.3)(87.9 − 68.7)
(85 − 94.8)(85 − 87.9)(85 − 81.3)

+ × 14
(68.7 − 94.8)(68.7 − 87.9)(68.7 − 81.3)
𝑥𝑥 = 6.303830017
7.2 Regression
In statistical modeling, regression analysis is a set of statistical processes for estimating the
relationships among variables. It includes many techniques for modeling and analyzing several
variables, when the focus is on the relationship between a dependent variable and one or more
independent variables.
More specifically, regression analysis helps one understand how the typical value of the
dependent variable (or 'criterion variable') changes when any one of the independent variables
is varied, while the other independent variables are held fixed.
7.2.1 Simple Linear Regression Model

linear regression is a linear approach to modelling the relationship between a scalar response
(or dependent variable) and one or more explanatory variables (or independent variables). The
case of one explanatory variable is called simple linear regression.
Consider a model function
𝑦𝑦 = 𝛼𝛼 + 𝛽𝛽𝛽𝛽 + 𝜀𝜀
The above model describes a line with slope 𝛽𝛽 and 𝑦𝑦 -intercept 𝛼𝛼 . In general, such a
relationship may not hold exactly for the largely unobserved population of values of the
independent and dependent variables; we call the unobserved deviations from the above
equation the errors which is denoted by 𝜀𝜀. Alternatively, the model can also be represented by:
𝑌𝑌� = 𝑎𝑎 + 𝑏𝑏𝑏𝑏
where
𝑎𝑎 = regression constant
𝑏𝑏 = regression coefficient
7.2.1.1 Least Square Model
Let
𝑦𝑦� = 𝑎𝑎 + 𝑏𝑏𝑏𝑏, the curve fitting the given data
𝑦𝑦�𝑖𝑖 = value of 𝑦𝑦, obtained from the curve fitting equation 𝑦𝑦� = 𝑎𝑎 + 𝑏𝑏𝑏𝑏 corresponding to the
value of 𝑥𝑥𝑖𝑖
𝑦𝑦𝑖𝑖 = observed value of 𝑦𝑦 at 𝑥𝑥𝑖𝑖
𝜀𝜀𝑖𝑖 = residual (error), and is given by 𝜀𝜀𝑖𝑖 = 𝑦𝑦 − 𝑦𝑦�
Then the standard deviation of the error is given by
𝑠𝑠 = � 𝜀𝜀𝑖𝑖2
𝑠𝑠 = �(𝑦𝑦 − 𝑦𝑦�)2
The aim of method of least square is to produce a curve such that the standard deviation of the
error is minimized.
Simple Linear Regrssion
𝑦𝑦� = 𝑎𝑎 + 𝑏𝑏𝑏𝑏
𝜀𝜀𝑖𝑖 = 𝑦𝑦𝑖𝑖 − 𝑦𝑦�𝑖𝑖

y
𝑦𝑦�𝑖𝑖 = 𝑎𝑎 + 𝑏𝑏𝑥𝑥𝑖𝑖
� represents the
Figure 7-1 represents the parameters of method of least squares. 𝜺𝜺𝒊𝒊 is the error (or residual), 𝒚𝒚
�𝒊𝒊 is the value of 𝒚𝒚 from the curve fitting corresponding to 𝒙𝒙𝒊𝒊 and 𝒚𝒚𝒊𝒊 is the observed
linear curve fitting, 𝒚𝒚
value of 𝒚𝒚 at 𝒙𝒙𝒊𝒊 .
The value of 𝑠𝑠 will be minimum when the rate of change of 𝑠𝑠 w.r.t 𝑎𝑎 and 𝑏𝑏 will be zero.
Therefore,
𝑠𝑠min = 𝑠𝑠 = �(𝑦𝑦 − 𝑎𝑎 − 𝑏𝑏𝑏𝑏)2
Differentiating w.r.t 𝑎𝑎:

𝑑𝑑𝑑𝑑
= −2 �(𝑦𝑦 − 𝑎𝑎 − 𝑏𝑏𝑏𝑏) = 0
𝑑𝑑𝑑𝑑
Σ𝑦𝑦 = Σ𝑎𝑎 + 𝑏𝑏Σ𝑥𝑥
Since 𝑎𝑎 is constant, Σ𝑎𝑎 = 𝑛𝑛𝑛𝑛.
Σ𝑦𝑦 = 𝑛𝑛𝑛𝑛 + 𝑏𝑏Σ𝑥𝑥 (7.1)
Σ𝑦𝑦 Σ𝑥𝑥
𝑎𝑎 = − × 𝑏𝑏
𝑛𝑛 𝑛𝑛
𝑎𝑎 = 𝑦𝑦� − 𝑏𝑏𝑥𝑥̅ (7.2)
Differentiating w.r.t 𝑏𝑏:
𝑑𝑑𝑑𝑑
= 2 �(𝑦𝑦 − 𝑎𝑎 − 𝑏𝑏𝑏𝑏) (−𝑥𝑥) = 0
𝑑𝑑𝑑𝑑
Σ𝑥𝑥𝑥𝑥 − 𝑎𝑎Σ𝑥𝑥 − 𝑏𝑏Σ𝑥𝑥 2 = 0
Σ𝑥𝑥𝑥𝑥 = 𝑎𝑎Σ𝑥𝑥 + 𝑏𝑏Σ𝑥𝑥 2 (7.3)
Equation 7.1 and 7.3 are known as normal equations. Normal equations are analytical
approach to reduce the least square error to its minimum value.
Solving the normal equations simultaneously:
(7.1) can be rewritten as
Σ𝑦𝑦 = 𝑛𝑛𝑛𝑛 + 𝑏𝑏Σ𝑥𝑥
Σ𝑦𝑦 − 𝑏𝑏Σ𝑥𝑥
𝑎𝑎 = (7.4)
𝑛𝑛
and (7.3) can be rewritten as
Σ𝑥𝑥𝑥𝑥 = 𝑎𝑎Σ𝑥𝑥 + 𝑏𝑏Σ𝑥𝑥 2
Σ𝑥𝑥𝑥𝑥 − 𝑏𝑏Σ𝑥𝑥 2
𝑎𝑎 = (7.5)
Σ𝑥𝑥
Now equating equations 7.4 and 7.5
Σ𝑦𝑦 − 𝑏𝑏Σ𝑥𝑥 Σ𝑥𝑥𝑥𝑥 − 𝑏𝑏Σ𝑥𝑥 2
=
𝑛𝑛 Σ𝑥𝑥
Σ𝑥𝑥Σ𝑦𝑦 − 𝑏𝑏(Σ𝑥𝑥)2 = 𝑛𝑛Σ𝑥𝑥𝑥𝑥 − 𝑛𝑛𝑛𝑛Σ𝑥𝑥 2
𝑛𝑛Σ𝑥𝑥𝑥𝑥 − Σ𝑥𝑥Σ𝑦𝑦
𝑏𝑏 = (7.6)
𝑛𝑛Σ𝑥𝑥 2 − (Σ𝑥𝑥)2
Example 7-2
The following table gives the age of cars of certain make and annual maintenance costs.
Obtain the regression equation for the cost related to age. Also give a prediction of
maintenance cost of a car that is aged at 12 years.
Age 2 4 6 8
Cost 10 20 25 30
Solution:
Since age is independent, it is denoted by 𝑥𝑥, while annual maintenance cost is represented
by 𝑦𝑦. Now, calculating required data:
Sr. # 𝑥𝑥 𝑦𝑦 𝑥𝑥 2 𝑦𝑦 2 𝑥𝑥𝑥𝑥
1 2 10 4 100 20
2 4 20 16 400 80
3 6 25 36 625 150
4 8 30 64 900 240
𝛴𝛴 20 85 120 2025 490
Avg 5 21.25
4(490) − (20)(85)
𝑏𝑏 =
4(120) − (20)2
= 3.25
𝑎𝑎 = 21.25 − 5(3.25)
=5
Therefore,
𝑦𝑦 = 5 + 3.25𝑥𝑥
If a car is 12 years old, its annual maintenance cost is given by
𝑦𝑦 = 5 + 3.25(12)
= 44
Practice Questions
1. Fit a straight line for the data given by using least square method:
𝑥𝑥 1.01 1.27 1.85 2.38 2.83 3.13 3.96 4.91
𝑦𝑦 0 0.19 0.58 0.96 1.26 1.47 2.07 2.75
Ans. 𝒂𝒂 = −𝟎𝟎. 𝟕𝟕𝟕𝟕𝟕𝟕𝟕𝟕𝟕𝟕; 𝒃𝒃 = 𝟎𝟎. 𝟕𝟕𝟕𝟕𝟕𝟕𝟕𝟕𝟕𝟕
2. Fit a straight line for the data given by using least square method. Also estimate the
force when velocity is 45m/s.
𝑥𝑥 1.01 1.27 1.85 2.38 2.83 3.13 3.96 4.91
𝑦𝑦 0 0.19 0.58 0.96 1.26 1.47 2.07 2.75
7.2.2 Exponential Model Fitting

Let
𝑦𝑦 = 𝛼𝛼𝑒𝑒 𝛽𝛽𝛽𝛽
so that
ln 𝑦𝑦 = ln 𝛼𝛼 + 𝛽𝛽𝛽𝛽
Comparing the above equation with simple linear regression equation, following conclusions
can be made:
𝑦𝑦 ≡ ln 𝑦𝑦
𝑎𝑎 ≡ ln 𝛼𝛼
𝑏𝑏 ≡ 𝛽𝛽
𝛼𝛼 ≡ 𝑒𝑒 𝑎𝑎
then,
��
𝑎𝑎 = ln 𝑦𝑦 − 𝑏𝑏𝑥𝑥̅
and
𝑛𝑛Σ𝑥𝑥 ln 𝑦𝑦 − Σ𝑥𝑥Σ ln 𝑦𝑦
𝑏𝑏 =
After evaluating 𝑎𝑎 and 𝑏𝑏, following relations can be used to determine 𝛽𝛽 and 𝛼𝛼
𝛽𝛽 = 𝑏𝑏
𝛼𝛼 = 𝑒𝑒 𝑎𝑎
Example 7-3
Fit an exponential curve through the given set of data:
𝑥𝑥 0.4 0.8 1.2 1.6 2.0 2.3

𝑦𝑦 800 975 1500 1950 2900 3600
Solution:
From the given data following values are evaluated:
Σ ln 𝑦𝑦 = 44.617, Σ𝑥𝑥 ln 𝑦𝑦 = 63.855, Σ𝑥𝑥 = 8.3, Σ𝑥𝑥 2 = 14.09
Now, Example 7-3

𝑛𝑛Σ𝑥𝑥 ln 𝑦𝑦 − Σ𝑥𝑥Σ ln 𝑦𝑦
𝑏𝑏 = 4000
6(63.855) − 8.3(44.617) y = 546.59e0.8187x
3500
=
6(14.09) − (8.3)2
= 0.819 3000
��
𝑎𝑎 = ln 𝑦𝑦 − 𝑏𝑏𝑥𝑥̅
44.617 8.3 2500
= − 0.818651 � �
6 6 2000
= 6.303
From the values of 𝑎𝑎 and 𝑏𝑏, 1500
𝛼𝛼 = 𝑒𝑒 𝑎𝑎
= 𝑒𝑒 6.303 1000
= 546.59
𝛽𝛽 = 𝑏𝑏 500
= 0.819
Thus, the exponential function fitting the given 0
0 0.5 1 1.5 2 2.5
data is given by:
𝑦𝑦 = 𝛼𝛼𝑒𝑒 𝛽𝛽𝛽𝛽 Data Points
𝑦𝑦 = 546.59𝑒𝑒 0.819𝑥𝑥 Exponential Fitting Curve
The figure shows the data provided in

the question along with the derived
fitting curve.
7.2.3 Power Curve Fitting

This fitting is very similar to the exponential fitting discussed above. The only difference is,
instead of using natural logarithms, base-10 logarithms are used. Consider the equation:
𝑦𝑦 = 𝛼𝛼𝑥𝑥 𝛽𝛽
so that
log 𝑦𝑦 = log 𝛼𝛼 + 𝛽𝛽 log 𝑥𝑥
Comparing the above equation with simple linear regression equation, following conclusions
can be made:
𝑦𝑦 ≡ log 𝑦𝑦
𝑥𝑥 ≡ log 𝑥𝑥
𝑎𝑎 ≡ log 𝛼𝛼
𝑏𝑏 ≡ 𝛽𝛽
𝛼𝛼 ≡ 10𝑎𝑎
then,
𝑎𝑎 = ��
log 𝑦𝑦 − 𝑏𝑏 ��
log 𝑥𝑥
𝑛𝑛Σ(log 𝑥𝑥 ⋅ log 𝑦𝑦) − Σ log 𝑥𝑥 Σ log 𝑦𝑦
𝑏𝑏 =
𝑛𝑛Σ(log 𝑥𝑥)2 − (Σ log 𝑥𝑥)2
After evaluating 𝑎𝑎 and 𝑏𝑏, following relations can be used to determine 𝛽𝛽 and 𝛼𝛼
𝛽𝛽 = 𝑏𝑏
𝛼𝛼 = 10𝑎𝑎
Example 7-4
Fit a power curve through the given set of data:
𝑥𝑥 10 20 30 40 50
𝑦𝑦 25 70 380 550 610
Solution:
From the given data following values are evaluated:
Σ log 𝑥𝑥 = 7.0791, Σ log 𝑦𝑦 = 11.3485,
Power Fitting
Σ(log 𝑥𝑥)2 = 10.3277, Σ(log 𝑥𝑥 ⋅ log 𝑦𝑦) = 16.7315 900
Then, 800
5(16.7351) − (7.0791)(11.3485) y = 0.153x2.1789
𝑏𝑏 = 700
5(10.3277) − (7.0791)2
= 2.1789 600
11.3485 7.0791
𝑎𝑎 = − 2.1789 ×
5 5 500
= −0.8152
400
Therefore,
𝛽𝛽 = 𝑏𝑏 300
= 2.1789
𝛼𝛼 = 10𝑎𝑎 200
= 10−0.8152
100
= 0.1530
Thus, the power equation fitting the given data is: 0
𝑦𝑦 = 0.1530𝑥𝑥 2.1789 0 20 40 60
Data Points
Power Fitting Curve
The figure shows the data

provided in the question along
with the derived fitting curve.
7.2.4 Polynomial Regression
Previously, a procedure was developed to derive the equation of a straight line using the least-
squares criterion. This procedure can be readily extended to fit the data to a higher-order
polynomial. For example, consider a second-order polynomial or quadratic:
𝑦𝑦 = 𝑎𝑎 + 𝑏𝑏𝑏𝑏 + 𝑐𝑐𝑥𝑥 2 + 𝜀𝜀𝑖𝑖
and
𝑦𝑦� = 𝑎𝑎 + 𝑏𝑏𝑏𝑏 + 𝑐𝑐𝑥𝑥 2
then
𝑠𝑠 = �(𝑦𝑦 − 𝑦𝑦�)2
𝑠𝑠 = �(𝑦𝑦 − 𝑎𝑎 − 𝑏𝑏𝑏𝑏 − 𝑐𝑐𝑥𝑥 2 )2
Following the procedure employed in simple linear regression model, the derivative of the
standard deviation of error with respect to each of the unknown coefficients of the polynomial,
is taken and rearranged to obtain following set of normal equations
Σ𝑦𝑦 = 𝑛𝑛𝑛𝑛 + 𝑏𝑏Σ𝑥𝑥 + 𝑐𝑐Σ𝑥𝑥 2
Σ𝑥𝑥𝑥𝑥 = 𝑎𝑎Σ𝑥𝑥 + 𝑏𝑏Σ𝑥𝑥 2 + 𝑐𝑐Σ𝑥𝑥 3
Σ𝑥𝑥 2 𝑦𝑦 = 𝑎𝑎Σ𝑥𝑥 2 + 𝑏𝑏Σ𝑥𝑥 3 + 𝑐𝑐Σ𝑥𝑥 4
The above normal equations can be solved simultaneously to evaluate the values of unknown
coefficients.
Example 7-5
Fit a second order polynomial through the given set of data:
𝑥𝑥 -3 -2 -1 0 1 2 3
𝑦𝑦 4.63 2.11 0.67 0.09 0.63 2.15 4.58
Solution:
The given data along with evaluated values is tabulated below:
Sr. # 𝑥𝑥 𝑥𝑥 2 𝑥𝑥 3 𝑥𝑥 4 𝑦𝑦 𝑥𝑥𝑥𝑥 𝑥𝑥 2 𝑦𝑦
1 -3 9 -27 81 4.63 -13.89 41.67
2 -2 4 -8 16 2.11 -4.22 8.44
3 -1 1 -1 1 0.67 -0.67 0.67
4 0 0 0 0 0.09 0 0
5 1 1 1 1 0.63 0.63 0.63
6 2 4 8 16 2.15 4.3 8.6
7 3 9 27 81 4.58 13.74 41.22
SUM 0 28 0 196 14.86 -0.11 101.23
Thus, normal equations are:
14.86 = 7𝑎𝑎 + 28𝑐𝑐
−0.11 = 28𝑏𝑏
101.23 = 28𝑎𝑎 + 196𝑐𝑐
Solving the above equations simultaneously will yield
𝑏𝑏 = 3.9285 × 10−3
𝑎𝑎 = 0.132857
𝑐𝑐 = 0.4975
Therefore, the polynomial fitting the data provided is given as

𝑦𝑦 = 0.1328 + 3.9285 × 10−3 𝑥𝑥 + 0.4975𝑥𝑥 2
Polynomial Fitting
5
y = 0.4975x2 - 0.0039x + 0.1329
4.5
3.5
2.5 Data Points

Poly. Curve Fitting
2
1.5
0.5
0
-4 -3 -2 -1 0 1 2 3 4
The figure shows the data provided in the question along with the derived fitting curve.

Numerical Methods Notes PDF

Încărcat de

Informații document

Descriere originală:

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Numerical Methods Notes PDF

Încărcat de

Drepturi de autor:

Formate disponibile

Numerical Methods

Summary notes of lectures on the subject of

1|Page Uzair Suria

3|Page Uzair Suria

1.1 Motivation for Numerical Analysis

1.1.1 When to Use Numerical Analysis

1.2 Types of Numerical Analysis Problems

4|Page Uzair Suria

c. Optimization: These problems involve determining a value or values of an independent

e. Integration: As depicted (above), a physical interpretation of numerical integration is

f. Ordinary Differential Equations: Ordinary differential equations are of great

6|Page Uzair Suria

1.2.2 Characteristics of Numerical Method

7|Page Uzair Suria

8|Page Uzair Suria

1.3.1 Accuracy and Precision

Figure 1-1 An example from marksmanship illustrating the concepts

9|Page Uzair Suria

1.3.3 Error Definitions

1.3.3.1 Stopping Criteria

Evaluate 𝑒𝑒 −8.3 using following approximations:

Approx of 𝑒𝑒 −8.3 = 4.872486 × 10−3

Then the approximate value of 𝑒𝑒 −8.3 is:

1.3.4 Taylor Polynomial Error

2.1.1 Bisection Computation Process

Figure 2-1 A graphical depiction of the bisection method

2.1.1.1 Advantages of Bisection Method

2.1.1.2 Drawbacks of Bisection Method

2.2 Regula Falsi Method

2.2.1 Regula Falsi Computation Process

2.2.1.1 Derivation of approximation

2.3.1 Convergence Criteria

2.3.2 Fixed Point Iteration Computation

Step 1: Rewrite 𝑓𝑓(𝑥𝑥) = 0 as 𝑥𝑥 = 𝑔𝑔(𝑥𝑥)

Find the root of the equation 𝑓𝑓(𝑥𝑥) = cos 𝑥𝑥 + 1 − 3𝑥𝑥

The root of the equation, precise to five decimal places, is 0.607101

2.4 Newton Raphson Method

2.4.1 Computation with Newton-Raphson Method

Figure 2-4 Graphical depiction of the Newton-Raphson method.

Taking 𝑥𝑥0 = 0.5, then

2.4.1.1 Advantages of Newton-Raphson Method

2.4.1.2 Drawbacks/Limitations of Newton-Raphson Method

3.1.1 Forward Differences

3.1.1.1 Forward Difference Tables

𝒙𝒙 𝒚𝒚 𝚫𝚫𝒚𝒚 𝚫𝚫𝟐𝟐 𝒚𝒚 𝚫𝚫𝟑𝟑 𝒚𝒚 𝚫𝚫𝟒𝟒 𝒚𝒚 𝚫𝚫𝟓𝟓 𝒚𝒚

Construct a differences table for the function 𝑓𝑓(𝑥𝑥) = 𝑥𝑥 4 , where 𝑥𝑥 = −2(1) ⋅ 3

3.1.2 Backward Differences

𝒙𝒙 𝒚𝒚 𝛁𝛁𝒚𝒚 𝛁𝛁𝟐𝟐 𝒚𝒚 𝛁𝛁𝟑𝟑 𝒚𝒚 𝛁𝛁𝟒𝟒 𝒚𝒚 𝛁𝛁𝟓𝟓 𝒚𝒚

Alternative notations can also be used for backward difference tables:

𝒙𝒙 𝒚𝒚 𝛁𝛁𝒚𝒚 𝛁𝛁𝟐𝟐 𝒚𝒚 𝛁𝛁𝟑𝟑 𝒚𝒚 𝛁𝛁𝟒𝟒 𝒚𝒚 𝛁𝛁𝟓𝟓 𝒚𝒚

3.1.3 Central Differences

𝛿𝛿 2 𝑦𝑦𝑛𝑛 − 𝛿𝛿 2 𝑦𝑦𝑛𝑛−1 = 𝛿𝛿 3 𝑦𝑦𝑛𝑛−1⁄2

𝛿𝛿 𝑝𝑝 𝑦𝑦𝑛𝑛 − 𝛿𝛿 𝑝𝑝 𝑦𝑦𝑛𝑛−1 = 𝛿𝛿 𝑝𝑝+1 𝑦𝑦𝑛𝑛−1⁄2

3.1.3.1 Central Difference Table

3.1.4 Other Difference Operators

3.1.4.1 Shift Operator ‘E’

3.1.4.2 Averaging Operator ‘𝜇𝜇’

3.1.4.3 Differential Operator ‘D’

3.1.5 Relationship Between Operators

3.1.5.2 Backward Difference and Shift Operator

3.1.5.3 Central Difference and Shift Operator

3.1.5.4 Mean difference and Shift Operator