CpE- 310B

Engineering Computation
and Simulation

Dr. Manal Al-Bzoor
Chapter 2:
Solving Sets of Equations
Yarmouk University Computer Engineering Department
What is this course about?
This is not a course to teach you to
code.

This is a course to teach you
computer
algorithms for analyzing and solving
science and engineering problems in
numerical ways.
Analytical vs. Numerical
Analytical
Exact
Typically only solvable in
cases of simple models
Numerical
Approximate
Allow for imperfect and
complex models to be
approximated
Can account for more
variables and dimensions
than would be solvable
when using analytical
methods
What we will Learn?
Solving Sets of Equations

Interpolation and Curve Fitting

Numerical Differentiation and Numerical
Integration

Numerical Solution of Ordinary Differential
Equations

Simulation of some basic engineering
problems (if time Permits)
Chapter2: Solving Set of
Equations
Matrix Notation
The Elimination Method ( Gauss
Elimination)
Pathology in Linear Systems, Singular
Matrices
Determinants and Matrix Inversion
Iterative Methods (Jacobi and Gauss-
Seidel Methods)

Notation
Matrix
A =
a11 ,., a1n
a21 ,., a2n
. . . .
am1 ,., amn





(

(
(
(
= Aij { }
A matrix is any doubly subscripted array of
elements arranged in rows and columns.
Row Vector
[1 x n] matrix


| | { } j n a a a a A , , 2 1 = .
Column Vector
{ } i
m
a
a
a
a
A
2
1
=
(
(
(
(

.
=
[m x 1] matrix
Square Matrix
B =
5 4 7
3 6 1
2 1 3




(

(
(
Same number of rows and columns
Identity Matrix
I =
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1





(

(
(
(
Square matrix with ones on the
diagonal and zeros elsewhere.
Transpose Matrix
A' =
a11 a21 ,., am1
a12 a22 ,., am2
. . . . .
a1n a2n ,., amn





(

(
(
(
Rows become columns and
columns become rows
Matrix Addition and
Subtraction
A new matrix C may be defined as the
additive combination of matrices A and B
where: C = A + B
is defined by:


Cij { } = Aij { } + Bij { }
Note: all three matrices are of the same dimension
Addition
A =
a11 a12
a21 a22



(

(
B =
b11 b12
b21 b22



(

(
C =
a11 +b11 a12 + b12
a21 +b21 a22 + b22



(

(
I
f
and
then
Matrix Addition Example
A + B =
3 4
5 6



(

(
+
1 2
3 4



(

(
=
4 6
8 10



(

(
= C
Matrix Subtraction
C = A - B
Is defined by

Cij { } = Aij { } Bij { }
Scalar Multiplication
If a matrix A is multiplied by a scalar k
the product is a matrix, each element of
which is the scalar times the original
element. We can write

Matrix Multiplication
Matrices A and B have these dimensions:
[r x c] and [s x d]
Matrix Multiplication
Matrices A and B can be multiplied if:
[r x c] and [s x d]
c = s
Matrix Multiplication
The resulting matrix will have the dimensions:
[r x c] and [s x d]
r x d
Computation: A x B = C
A =
a11 a12
a21 a22



(

(
B =
b11 b12 b13
b21 b22 b23



(

(
(

+ + +
+ + +
=
23 22 13 21 22 22 12 21 21 22 11 21
23 12 13 11 22 12 12 11 21 12 11 11



b a b a b a b a b a b a
b a b a b a b a b a b a
C
[2 x 2]
[2 x 3]
[2 x 3]
Computation: A x B = C

A =
2 3
1 1
1 0




(

(
(
and B =
1 1 1
1 0 2



(

(
[3 x 2] [2 x 3]
A and B can be multiplied
(
(
(

=
(
(
(

= + = + = +
= + = + = +
= + = + = +
=
1 1 1
3 1 2
8 2 5

1 2 * 0 1 * 1 1 0 * 0 1 * 1 1 1 * 0 1 * 1
3 2 * 1 1 * 1 1 0 * 1 1 * 1 2 1 * 1 1 * 1
8 2 * 3 1 * 2 2 0 * 3 1 * 2 5 1 * 3 1 * 2
C
[3 x 3]
Computation: A x B = C

(
(
(

=
(
(
(

= + = + = +
= + = + = +
= + = + = +
=
1 1 1
3 1 2
8 2 5

1 2 * 0 1 * 1 1 0 * 0 1 * 1 1 1 * 0 1 * 1
3 2 * 1 1 * 1 1 0 * 1 1 * 1 2 1 * 1 1 * 1
8 2 * 3 1 * 2 2 0 * 3 1 * 2 5 1 * 3 1 * 2
C
A =
2 3
1 1
1 0




(

(
(
and B =
1 1 1
1 0 2



(

(
[3 x 2] [2 x 3]
[3 x 3]
Result is 3 x 3
Matrix Inversion
B
1
B = BB
1
= I
Like a reciprocal
in scalar math
Like the number one
in scalar math
Upper and Lower Triangular
A matrix is called lower triangular If all
elements above the diagonal are zeros and
an upper triangular if all elements below the
diagonal are zeros
Determinant of a matrix
For a 2 x 2 matrix, the determinant is computed by
subtracting the product of the elements on the minor diagonal
(from upper right to lower left) from the product of terms on
the major diagonal. For example,
In general the determinant is found by adding the product of each
term in any row or column by the determinant of its
minor, with signs alternating + and -. We expand each of the
determinants of the minor until we reach 2 x 2 matrices
Determinant of a matrix
Can be written in matrix notation as
where :
Linear System of Simultaneous
Equations
Elimination Method
Consider
Eliminate x1 : Multiply row 1 by 3 and row 2 by 4 and add. Multiply row 3 by 4 and row 1 by
-1 and add we get
Eliminate x2 : Multiply the second row by 2 and the third row by -10 and adding to get

We use back substitution to find x3 , then x2 and x1

Elimination Method
In Matrix notation
Only the coefficient and constant terms are affected
by arithmetic operations
Perform elementary row transformation

Augmented Coefficient
matrix
Back substitution
The back-substitution step can be performed quite mechanically by solving the
Above equations in reverse order:
Summary of Elimination Method
- We transform the augmented coefficient matrix by elementary row
transformations until an upper triangular matrix is created on the left.
- We then are ready to back-substitute using the elements in the rightmost
column.
Gauss Elimination
We eliminate the first coefficient in the ith row by subtracting
a
i1
/a
11
times the first equation from the ith equation.
Gauss Elimination
Zeros below the main diagonal show that we solving an upper-
triangular system of equations. If we store the ratio of coefficients
in place of zero we get :
The original matrix A
Gauss Elimination
Produces LU decomposition
A=L*U
Thus for the example given
Pivoting : rearrange the equations so as to put the
coefficient of largest magnitude on the diagonal at
each step to guard against dividing by zero.
Example :

Gauss Elimination
Partial Pivoting
Gauss Elimination
Partial Pivoting
A zero on first column first row is not permitted, we
interchange row1 by row 6
Make all elements in column 1 zero by subtracting multiples of row1 and we
interchange to put largest magnitude on the diagonal
Interchange
row1 by row4
Interchange
row2 by row3
Reduce second column
Gauss Elimination
Partial Pivoting
No interchange needed, reduce third column
Back substitution
Gauss Elimination
The correct answers are - 2, 1/3, 1, and -1/2.
Replace zeroes with the ratios of the coefficient
LU decomposition
Gauss Elimination
The LU product Will give a permutation of original matrix A because we did row
interchange.
The determinant of Original matrix A is :
Transforms the coefficient matrix into the identity matrix.
Gauss-Jordan Method
Interchange row 4 by 1, divide row1 by 6 and reduce first
column
Interchange row 2 and 3 and divide row2 by 3.6667, reduce
column 2 by operating above and below the diagonal element
Gauss-Jordan Method
Divide the third row by 15 and reduce column 3.
Divide the fourth row by 1.5599 and reduce column 4 by
creating zeros above the diagonal element
Gauss-Jordan Method
Pathology in Linear Systems-
Singular Matrices
An arbitrary set of equations may not have a unique
solution If:
- Number of equations less than number of variables


Pathology in Linear Systems- Singular
Matrices

An arbitrary set of equations may not have a unique solution If:
Number of equations less than number of variables
Number of equations greater than number of variables
Redundant equations exists in a set of equations where n
equations for n variables condition applies


Inconsistence exists, ex.


If n x n systems have a unique solution and can be triangularized
without having zeros on the diagonal the matrix is nonsingular.


Pathology in Linear Systems- Singular
Matrices
Pathology in Linear Systems- Singular
Matrices
If the columns or rows of the coefficient matrix is linearly
dependent, the matrix is singular

We say that vectors x1, x2, . . . ,xn are linearly dependent if we can find
scalar coefficients, a1, a2, . . . , an (with not all the as simultaneously zero),
for which
Similarly for the rows :