NUMERICAL METHODS

Part of the Numerical Methods and Statistic Course

KNS 2723
by
Prof. DR. F.J.Putuhena
February 2015 & 2016
Department of Civil Engineering
Faculty of Engineering University Malaysia Sarawak

NUMERICAL METHODS

Table of Contents:
1. Introduction
2. Linear Algebraic Equation
3. Non Linear Equation
4. Interpolation and Extrapolation (Curve
Fitting)
5. Numerical Integration
6. The Numerical Solution of Ordinary
Differential Equation
7. Application
2

NUMERICAL METHODS
1. Introduction
WHAT ARE NUMERICAL METHODS
Numerical methods are a class of methods for
solving a wide variety of mathematical problems
This class of methods in unusual in that only
arithmetic operations and logics are employed
The methods came to age with the introduction of
the electronic digital computers

NUMERICAL METHODS
1. Introduction

Introduction to the course topics, which include:

Course Plan KNS 2723_Old version
Course Plan KNS 2723_New version

NUMERICAL METHODS

2. Linear Algebraic Equations

Introduction
Solutions of a set of Linear Equations
Direct Solution (Gaussian Elimination,
Pivoting)
Basic Matrix Terminology and Operations
Matrix Representation and Formal Solution of a
set of Linear Equations
Gauss Elimination and Gauss-Jordan
Elimination
Matrix Inversion by Gauss-Jordan Elimination
Ill-Conditioned Matrices and Sets of Equations

Gauss-Siedel Method (Iteration) and

Concepts of Relaxation
5

2. Linear Algebraic Equations

BASIC MATRIX TERMINOLOGY AND OPERATIONS
Matrix Terminology
Matrix size: n * m ;
n = number of row and
m = number of column
Element: aij ; i = row ; j = column;

Vector:
Identity or Unit matrix
1
0
0

0
1

0
0

0
0

0
0

2. Linear Algebraic Equations

BASIC MATRIX TERMINOLOGY AND OPERATIONS
Banded Matrix
a11 a12
a21 a22 a23 0

a32 a33 a34

0

a43 a44 a45

a54 a55

Symmetric matrix :

ij

= a

ji

Triangular matrix :
Upper triangular matrix
Lower triangular matrix
7

2. Linear Algebraic Equations

BASIC MATRIX TERMINOLOGY AND OPERATIONS
Matrix Operation
Addition & Subtraction: S = A B
bij

=>

sij = aij

(A + B) + C = A + (B + C)
A+B=B+A;
A, B have the same n * m
n

Multiplication:

P = AB

=>

k=1

Pij = aik bkj

If AB exists then conformable in general AB = BA

Scalar operation: A = aij
Inverse: I = AA-1 ; I = Identical Matrix; A-1 =
Inverse M

2. Linear Algebraic Equations

BASIC MATRIX TERMINOLOGY AND OPERATIONS
Transpose : AT
Anm => AT ; aij = ajiT
i.e.: exchange rows and columns
Symetric:

A = AT

Orthogonal:
AT = A-1
Determinant:
The determinant of a matrix A is a scalar
quantity, denoted as Det (A) or |A|;
The determinant of a 1 * 1 matrix is the
number within the matrix itself;
To each element of a matrix with higher
dimensions, aij, we assign a number called its
minor, denoted as Mij;

2. Linear Algebraic Equations

BASIC MATRIX TERMINOLOGY AND OPERATIONS

Mij is a determinant of the matrix that arises

by eliminating the elements in the ith row and
jth column of matrix A, and the result is a
reduced matrix with dimensions (n-1)*(m-1);
The corresponding cofactor is defined as
i+j
nCij = (-1) n Mij
The determinant of A is defined as:
J=1
i=1
Det (A) = aij Cij
= aik bik ;
where the value of i and k are arbitrary
The determinant is equal to the sum of the
product of the elements of the matrix with the
associated cofactors, where the sum is

10

2. Linear Algebraic Equations

BASIC MATRIX TERMINOLOGY AND OPERATIONS

Properties of determinant:
Det = 0, if all elements of any row and
column equal to zero;
Value of determinant unchanged if any
2 rows (or columns) are interchanged
(Sign changes);
Det (AT) = Det (A);
If all elements in row (or column) are
multiplied by scalar k, then
determinant multiplied by k;
If 2 rows (or columns) are equal or
proportional, the determinant will be
equal to zero; and
Etc.

11

2. Linear Algebraic Equations

Matrix Representation and Formal Solution of a set of Linear

Equations

A set of linear equations:

C11 x1 + C12 x2+ C13 x3+ C14 x4 = r1
C21 x1 + C22 x2+ C23 x3+ C24 x4 = r2
C31 x1 + C32 x2+ C33 x3+ C34 x4 = r3
C41 x1 + C42 x2+ C43 x3+ C44 x4 = r4
An equivalent representation in matrix form is:
C11 C12 C13 C14
x1 = r1
Or

C21 C22 C23 C24

x2 =

r2

C31 C32 C33 C34

x3 =

r3

C41 C42 C43 C44

x4 =

r4

CX = R

12

2. Linear Algebraic Equations

Matrix Representation and Formal Solution of a set of Linear Equations

Cramers Rule:
det Ck

Where Ck is the matrix C, with

xk =
det C its kth column replaced by R

Ci1 Ci2

Cik-1 ri Cik+1

Cin

Ck =

If det C = 0, no unique solution can be obtained, in this event

the matrix C is term Singular (See Example)
13

2.

Linear Algebraic Equations

Matrix Representation and Formal Solution of a set of Linear Equations

R , can also be solved formally as X = C -1 R, however most of

me the solution is done by finding X directly, without solving for
intermediate step, since this is usually the most efficient appro

ertain situation, solutions must be obtained for many different s

uations in which C stays the same and only R is changed. Thus o
found, new solution vector X can be obtained with little addition
.
2 (two) approaches or methods:
Direct
Indirect
-a small matrix (densely pact) -a large matrix and sparse
-large banded matrices
-Accuracy, function of # of iterations
-exact solution (round off error
only)-Siedel
-Gauss
-Cramers rule
-Relaxation
-Gauss, Gauss Jordan elimination
14

2.

Linear Algebraic Equations

Gauss Elimination and Gauss- Jordan Elimination

Gauss elimination approach is to obtain triangular

matrix
C11 C12

C1n

x1

C21 C22

C2n

Cm1 Cm2

Cmn

r1

x2

= r2
Assume |C| = 0

xm =

C12/ C11

CX = R

rm

C1n/ C11

0 [C22 C21C12/ C11] . ..

x1

= r1/ C11

x2 =
15

2.

Linear Algebraic Equations

Gauss Elimination and Gauss- Jordan Elimination

Gauss elimination approach is to obtain triangular

matrix
1

Cij

x1

x2

=
xm r
=1 rm
=xm-1
r+
2 Cm-1,n xm =
rm-1

xm = xm-1
rm = rm-1 - Cm-1,n
xm
Etc.

In term of computation, Gauss Elimination

reduce the Cramers computation from O(n4) to
O(n3)
16

2.

Linear Algebraic Equations

Gauss Elimination and Gauss- Jordan

Elimination
Gauss Jordan elimination is further to Gauss
Elimination by making C12 , C13 , C14 , C23 ,C24 , C34
are equal to zero

x1

x2
1

x3
1

x4

r1
x1 =
r1
= r
x22 = r2
= r3x3 = r3
x44 = r4
= r

In term of computation, Gauss Jordan

Elimination O(n3),
while the Cramers rule O(n4)
See Illustrative Problem

17

2.

Linear Algebraic Equations

Gauss Elimination and Gauss- Jordan Elimination

Gauss Elimination for Banded Matrix

b1 c1

a2

b2

c2

a3

1
0

x 1 = r1

x2 =

r2

b 3 c3

x3 =

r3

a 4 b4

x4 =

r4

c1/b1
[b2- a2c1/b1]

0
c2

0
0

x1
x2

= r1/b1
=

r2- a2r1/b1

a3

b3 c3

x3

r3

a4

x4

r4

b4

18

2.

Linear Algebraic Equations

Gauss Elimination and Gauss- Jordan Elimination

Gauss Elimination for Banded Matrix

c1/b1
0
0
x1 = r1/b1

0
1
a2r1/b1)/[
0
[ ]
0
[ ]

c2 / [ ]

x2

= ( r 2-

c 3/ [ ]

x3

)/

x4

)/

19

2. Linear Algebraic Equations

Pivoting

By the time any given row becomes the pivot row,

the diagonal element in that row will have been
modified from its original value, with the element
in the lower rows being recomputed the most
times.
Under certain circumstances, the diagonal
elements can become very small in magnitude
compared to the rest of the elements in the pivot
rows.
For various reasons, this can create a very
unfavorable situation in terms of round off error
problem which can result in an inaccurate
solution vector.
The problem can be effectively treated by
interchanging column or row of the matrix to shift
the largest element in the pivot row to the
diagonal position.
20
Strategies to maximize pivot elements are

2. Linear Algebraic Equations

Pivoting
Row Operation for pivoting
3
6
9
6
2
1
1
4
3 =
1
4
3
6 2
1
3
6
9
Shift column for pivoting
9

x3

10 4

x2

x1
change place
21

2. Linear Algebraic Equations

Pivoting
Solve the following set of equations with
Gauss- Jordan Elimination:
a) without maximization
b) With maximization

4 3 -1 x16
7 -2 3 x2= 9
5 -18 13x33
Solution