Gauss-Siedel Method

Major: All Engineering Majors

Authors: Autar Kaw

http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM
Undergraduates

12/7/21 http://numericalmethods.eng.usf.edu 1
 Gauss-Seidel Method

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method
An iterative method.

Basic Procedure:
-Algebraically solve each linear equation for xi
-Assume an initial guess solution array
-Solve for each xi and repeat
-Use absolute relative approximate error after each iteration
to check if error is within a pre-specified tolerance.
 Gauss-Seidel Method
Why?
The Gauss-Seidel Method allows the user to control round-off error.

Elimination methods such as Gaussian Elimination and LU

Decomposition are prone to prone to round-off error.

Also: If the physics of the problem are understood, a close initial

guess can be made, decreasing the number of iterations needed.
 Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:
If: the diagonal elements are
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1 non-zero
a21x1  a22 x2  a23 x3  ...  a2n xn  b2 Rewrite each equation solving
. .
. . for the corresponding unknown
. .
ex:
an1 x1  an 2 x2  an 3 x3  ...  ann xn  bn
First equation, solve for x1
Second equation, solve for x2
 Gauss-Seidel Method
Algorithm
Rewriting each equation
c1  a12 x 2  a13 x3   a1n x n From Equation 1
x1 
a11

c2  a21 x1  a23 x3   a2 n xn
x2  From equation 2
a22
  
cn 1  an 1,1 x1  an 1, 2 x2   an 1,n  2 xn  2  an 1,n xn From equation n-1
xn 1 
an 1,n 1
cn  an1 x1  an 2 x2    an ,n 1 xn 1
xn  From equation n
ann
 Gauss-Seidel Method
Algorithm
General Form of each equation
n n

c1   a1 j x j cn 1  a
j 1
n 1, j xj
j 1
j  n 1
x1 
j 1 xn 1 
a11 an 1,n 1
n
c n   a nj x j
n
c2   a2 j x j
j 1 j 1
j n
x2 
j 2
xn 
a 22 a nn
 Gauss-Seidel Method
Algorithm
General Form for any row ‘i’
n
ci   aij x j
j 1
j i
xi  , i  1,2, , n.
aii

How or where can this equation be used?

 Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X] Use rewritten equations to solve for
each value of xi.
Important: Remember to use the
 x1  most recent value of xi. Which
x  means to apply values calculated to
 2 the calculations remaining in the
  current iteration.
 
 xn -1 
 xn 
 Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error
x new
x old
a i  i
new
i
 100
x i

So when has the answer been found?

The iterations are stopped when the absolute relative

approximate error is less than a prespecified tolerance for all
unknowns.
Gauss-Seidel Method: Example 1
The upward velocity of a rocket
is given at three different times
Table 1 Velocity vs. Time data.
t  s v  m/s 

The velocity data is approximated by a polynomial as:

v t   a1t 2  a2t  a3 , 5  t  12.
 Gauss-Seidel Method: Example 1
Using a Matrix template of the form t12 t1 1  a1   v1 
2 
t 2 t 2 1 a2   v2 
t32 t3 1  a3  v3 

 25 5 1  a1  106.8 
The system of equations becomes  64 8 1 a   177.2 
   2  
144 12 1  a3  279.2

 a1  1
 a    2
Initial Guess: Assume an initial guess of
 2  
 a3  5
 Gauss-Seidel Method: Example 1
Rewriting each equation
106.8  5a 2  a3
a1 
25
 25 5 1  a1  106.8 
 64 8 1 a   177.2  177.2  64a1  a 3
   2   a2 
144 12 1  a3  279.2 8

279.2  144a1  12a 2

a3 
1
 Gauss-Seidel Method: Example 1
Applying the initial guess and solving for ai
 a1  1  106.8  5(2)  (5)
a1   3.6720
 a    2 25
 2  
 a3  5 177.2  64 3.6720   5
a2   7.8510
Initial Guess 8
279.2  144 3.6720   12  7.8510 
a3   155.36
1

When solving for a2, how many of the initial guess values were used?
 Gauss-Seidel Method: Example 1
Finding the absolute relative approximate error
xinew  xiold At the end of the first iteration
a i  new
100
xi
 a1   3.6720 
a    7.8510
3.6720  1.0000  2  
a 1  x100  72.76%
3.6720  a3    155.36 

 7.8510  2.0000 The maximum absolute

a  x100  125.47% relative approximate error is
2
 7.8510
125.47%

 155.36  5.0000
a 3  x100  103.22%
 155.36
 Gauss-Seidel Method: Example 1
Iteration #2
Using
 a1   3.6720  the values of ai are found:
a    7.8510 106.8  5  7.8510   155.36
 2   a1   12.056
 a3    155.36  25

from iteration #1
177.2  6412.056   155.36
a2   54.882
8

279.2  14412.056  12  54.882 

a3   798.34
1
 Gauss-Seidel Method: Example 1
Finding the absolute relative approximate error
12.056  3.6720 At the end of the second iteration
a 1  x100  69.543%
12.056  a   12.056 
1
a     54.882
 2  
 54.882    7.8510   a3   798.54
a 2  x100  85.695%
 54.882
The maximum absolute
 798.34    155.36  relative approximate error is
a 3  x100  80.540% 85.695%
 798.34
 Gauss-Seidel Method: Example 1
Repeating more iterations, the following values are obtained
a 1 % a 2 % a 3 %

Notice – The relative errors are not decreasing at any significant rate
 a1  0.29048
Also, the solution is not converging to the true solution of    
a
 2   19 .690 
a 3   1.0857 
 Gauss-Seidel Method: Pitfall
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gauss-Siedel method: not all
systems of equations will converge.

Is there a fix?
One class of system of equations always converges: One with a diagonally
dominant coefficient matrix.

Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:

n n
aii   aij for all ‘i’ and aii   aij for at least one ‘i’
j 1 j 1
j i j i
 Gauss-Seidel Method: Pitfall
Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other coefficients in that row and at least one row
with a diagonal coefficient greater than the sum of the other coefficients
in that row.
Which coefficient matrix is diagonally dominant?

 2 5.81 34 124 34 56 

 A   45 43 1  [B]   23 53 5 
123 16 1   96 34 129

Most physical systems do result in simultaneous linear equations that

have diagonally dominant coefficient matrices.
 Gauss-Seidel Method: Example 2
Given the system of equations The coefficient matrix is:
12 x1  3x2 - 5 x3  1 12 3  5
x1  5 x2  3x3  28  A   1 5 3 
3 x1  7 x2  13 x3  76  3 7 13 

With an initial guess of Will the solution converge using the

Gauss-Siedel method?
 x1  1
 x   0 
 2  
 x3  1
 Gauss-Seidel Method: Example 2
Checking if the coefficient matrix is diagonally dominant
a11  12  12  a12  a13  3   5  8
12 3  5
 A   1 5 3  a 22  5  5  a 21  a 23  1  3  4
 3 7 13 
a33  13  13  a31  a32  3  7  10

The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method
 Gauss-Seidel Method: Example 2
Rewriting each equation With an initial guess of
12 3  5  a1   1   x1  1
 1 5 3  a   28  x   0 
   2    2  
 3 7 13  a3  76  x3  1

1  3 x 2  5 x3 1  3 0   51
x1  x1   0.50000
12 12

28  x1  3x3 28   0.5  31

x2  x2   4.9000
5 5

76  3x1  7 x 2 76  3 0.50000   7 4.9000 

x3  x3   3.0923
13 13
 Gauss-Seidel Method: Example 2
The absolute relative approximate error
0.50000  1.0000
a 1  100  100.00%
0.50000

4.9000  0
a 2
 100  100.00%
4.9000

3.0923  1.0000
a 3
  100  67.662%
3.0923

The maximum absolute relative error after the first iteration is 100%
 Gauss-Seidel Method: Example 2
After Iteration #1
 x1  0.5000
 x   4.9000
 2  
 x3  3.0923

Substituting the x values into the After Iteration #2

equations
 x1  0.14679
1  3 4.9000   5 3.0923  x    3.7153 
x1   0.14679
12  2  
 x3   3.8118 
28   0.14679   3 3.0923
x2   3.7153
5

76  3 0.14679   7 4.900 
x3   3.8118
13
 Gauss-Seidel Method: Example 2
Iteration #2 absolute relative approximate error
0.14679  0.50000
a 1  100  240.61%
0.14679
3.7153  4.9000
a 2
 100  31.889%
3.7153
3.8118  3.0923
a 3
  100  18.874%
3.8118
The maximum absolute relative error after the first iteration is 240.61%

This is much larger than the maximum absolute relative error obtained in
iteration #1. Is this a problem?
 Gauss-Seidel Method: Example 2
Repeating more iterations, the following values are obtained
a 1 % a 2 % a 3 %

 x1  0.99919  x1  1 
The solution obtained  x    3.0001  is close to the exact solution of  x    3 .
 2    2  
 x3   4.0001   x3  4
 Gauss-Seidel Method: Example 3
Given the system of equations

3x1  7 x2  13x3  76 Rewriting the equations

x1  5 x2  3x3  28 76  7 x2  13 x3
x1 
12 x1  3x2  5 x3  1 3
28  x1  3 x3
With an initial guess of x2 
5
 x1  1
 x   0  1  12 x1  3 x 2
 2   x3 
 x3  1 5
 Gauss-Seidel Method: Example 3
Conducting six iterations, the following values are obtained

a 1 % a 2 % a 3 %

The values are not converging.

Does this mean that the Gauss-Seidel method cannot be used?
 Gauss-Seidel Method
The Gauss-Seidel Method can still be used
 3 7 13 
The coefficient matrix is not
diagonally dominant
 A   1 5 3 
12 3  5
But this is the same set of 12 3  5
equations used in example #2,
which did converge.
 A   1 5 3 
 3 7 13 

If a system of linear equations is not diagonally dominant, check to see if

rearranging the equations can form a diagonally dominant matrix.
 Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.
Observe the set of equations

x1  x 2  x3  3
2 x1  3 x 2  4 x3  9
x1  7 x 2  x3  9

Which equation(s) prevents this set of equation from having a

diagonally dominant coefficient matrix?
 Gauss-Seidel Method
Summary

-Advantages of the Gauss-Seidel Method

-Algorithm for the Gauss-Seidel Method
-Pitfalls of the Gauss-Seidel Method
Gauss-Seidel Method

Questions?
 Additional Resources
For all resources on this topic such as digital audiovisual
lectures, primers, textbook chapters, multiple-choice
tests, worksheets in MATLAB, MATHEMATICA, MathCad
and MAPLE, blogs, related physical problems, please
visit

http://numericalmethods.eng.usf.edu/topics/gauss_seid
el.html
 THE END

http://numericalmethods.eng.usf.edu