Solve AX=B for X with two steps

1. Forward Elimination of Unknowns.

2. Back Substitution
Assume a system of n equations and n unknowns
1 1 3 13 2 12 1 11
... b x a x a x a x a
n n
= + + + +
2 2 3 23 2 22 1 21
... b x a x a x a x a
n n
= + + + +
n n nn n n n
b x a x a x a x a = + + + + ...
3 3 2 2 1 1
. .
. .
. .
Transform to an Upper Triangular Matrix
Step 1: Eliminate x
1
in 2
nd
through nth equation, by
adding the appropriate scalar multiple of equation 1 to
the other equations.
For example, to eliminate the x
1
in the 2
nd
equation, use
1
11
21
2 1
11
21
2 2 12
11
21
22
... b
a
a
b x a
a
a
a x a
a
a
a
n n n
=
|
|

\
|
+ +
|
|

\
|

) (
1
2
21
11
a
a
Eqn
Eqn
(

Which makes the new second equation:

After the elimination of x
1
, the system is:
1 1 3 13 2 12 1 11
... b x a x a x a x a
n n
= + + + +
'
2
'
2 3
'
23 2
'
22
... b x a x a x a
n n
= + + +
'
3
'
3 3
'
33 2
'
32
... b x a x a x a
n n
= + + +
' '
3
'
3 2
'
2
...
n n nn n n
b x a x a x a = + + +
. . .
. . .
. . .
Step 2: Eliminate x
2
in the 3
rd
through n
th
equations by adding the appropriate scalar
multiple of equation 2.
For example, for eliminating x
2
in Equation 3:
) (
2
3
32
22
a
a
Eqn
Eqn
(

Continue this procedure. At the end of all steps, the system of

equations be in upper triangular form.
Actual operations are performed on the augmented matrix
instead of the actual system of equations.
'
2
'
2 3
'
23 2
'
22
... b x a x a x a
n n
= + + +
"
3
"
3
"
33
... b x a x a
n n
= + +
( )
( ) 1
1

=
n
n n
n
nn
b x a
. .
. .
. .
1 1 3 13 2 12 1 11
... b x a x a x a x a
n n
= + + + +
(
(
(
(
(
(

=
(
(
(
(
(
(

(
(
(
(
(
(

) (n-
n
"
'
n
) (n
nn
"
n
"
'
n
' '
n
b
b
b
b
x
x
x
x
a
a a
a a a
a a a a
1
3
2
1
3
2
1
1
3 33
2 23 22
1 13 12 11
0 0 0 0
0 0
0
M M M L M M M
L
L
L
When during the process of Gaussian
Elimination, a zero is on a pivot position, rows
are swapped to achieve a nonzero pivot.
Example We would need to swap row 2 and
row 3 to achieve a zero in row 2, column 2
(
(
(

(
(
(

4 2 0 0
1 3 2 0
6 4 3 2
1 3 2 0
4 2 0 0
6 4 3 2
Any time a computer is used, an operation
can produce round-off error.
This can be significant in Gaussian
Elimination when the pivot value is small
compared to the other values in the pivot
column below the pivot. This occurs since we
divide by this pivot value.
Example 3.17, page 132. The augmented
matrix form for this system is
If we add (-24.14/1.133)* Row 1 to Row 2, and
assume 4 digit rounding at each step, we get
(

93 . 22 210 . 1 14 . 24
414 . 6 281 . 5 133 . 1
(
(

8 . 113 7 . 113
414 . 6 281 . 5 133 . 1
Using back substitution (again with four digit
rounding) the solution is
The true solution is x
1
= x
2
= 1
The problem is the small pivot value relative to the
other entries in the column.
This leads to division of a large number by a small
number, causing the round off error.
9956 . 0
113 . 1
286 . 5 414 . 6
001 . 1
7 . 113
8 . 113
1
2
=

=
=

=
x
x
If the rows are swapped (example 3.18), then
the larger pivot value 24.14 leads to division
of a smaller number by a larger (1.133/24.14),
and a corresponding reduction in round-off
error. The matrix after Gaussian elimination
becomes:
(

338 . 5 338 . 5
93 . 22 219 . 1 14 . 24
For the current pivot position in row k,
column k, check the elements in the pivot
column k that are in row k or below. Choose
the element with the largest absolute value.
This is the next pivot do a row swap with
the kth row to bring this pivot into row k
column k.
(Repeat this strategy for all pivots).
First Step: Determine the absolute value of the largest element in
row Store this in a Scale matrix S.
For each pivot position for p, column p: Choose the next pivot as the
one that column that has the largest scaled magnitude -- i.e. the
maximum for k= p,p+1, .n of
Then swap rows to bring this element into the pivot position in row p,
column p.
Also swap the entries in the scale matrix S to maintain the
appropriate scaling choices.
This can be an improvement in round off error over partial pivoting,
but takes significantly more computations.
] [
,
p S
a
p k