Numerical Computing Summer ’10 MATH/CSCI 4800-01

Homework-3 Solutions

PROBLEMS

1. (Pencil-and-paper) Text page 90, Exercises 2.2, Problem 4(a).

The matrix A is given by  
3 1 2
A= 6 3 4 .
3 1 5
We proceed with Gaussian elimination.
Step (i) We note that the multipliers for the first column are m21 = 6/3 = 2 and m31 = 3/1 = 1.
The row operations R2 → R2 − 2R1 and R3 → R3 − R1 yield
 
3 1 2
U =  0 1 0 .
0 0 3

No other steps in the elimination process are required as the matrix has been rendered upper triangular,
and can rightly be designated as U . The corresponding lower triangular matrix L is obtained by
starting with the identity matrix and inserting the multipliers computed above in the appropriate
locations dictated by their indices. Thus,
 
1 0 0
L =  2 1 0 .
1 0 1

It is a simple matter to check that the product LU does indeed generate A. Then the original
problem Ax = b can be written first as LU x = b and then split into two as Lc = b and U x = c,
which are solved in turn to obtain the solution x.
Now, Lc = b is     
1 0 0 c1 0
 2 1 0   c2  =  1  .
1 0 1 c3 3
Forward substitution yields    
c1 0
 c2  =  1  .
c3 3
Then U x = c, given by     
3 1 2 x1 0
 0 1 0   x2  =  1 
0 0 3 x3 3
can be solved by backward substitution to obtain
   
x1 −1
 x2  =  1  .
x3 1

It is easily seen that the solution satisfies Ax = b.

 2. (Pencil-and-paper) Text page 100, Exercises 2.3, Problems 6(a) and 6(e).
The system is Ax = b with    
1 2 3
A= , b= .
2 4.01 6.01
(a) The approximate solution is x1c = −10, x2c = 6. The exact solution, found easily by Gaussian
elimination (or even by inspection) is x1 = x2 = 1. Therefore the forward error, computed by
using the infinity norm, is
||ef || = ||x − xc || = ||11, −5|| = 11.
Then, since the infinity norm of the exact solution is 1, the relative forward error is
||ef || 11
= = 11.
||x|| 1
1
The backward error, given by the residual , is

||eb || = ||Axc − b||.

Now,
    
2 1 −10 3
Axc − b = −
4.01 2 6 6.01
   
2 3
= −
4.06 6.01
 
−1
= .
−1.95

Therefore,
||eb || = 1.95
and the relative backward error is
||eb || 1.95
= .
||b|| 6.01
The error magnification factor M is the ratio
relative forward error
M =
relative backward error
11
= = 33.90.
1.95/6.01

(b) Condition number requires computation of the inverse matrix of A For the 2 × 2 matrix it is a
simple computation and we get
 
4.01 −2
A−1 = 100
−2 1

Then the condition number of A is given by

κ = ||A||||A−1 || = 6.01 × 100(6.01) = 3612.

As we would expect, M ≤ κ.
1 For the system Ax = b and the computed solution x we had defined in class the backward error as ||E||, where E is
c
the perturbation to the input matrix A such that the perturbed problem is satisfied exactly by the computed solution, i.e.,
{A + E}xc = b. However, one could equally well assign a perturbation r to b and define the backward error as the norm of
the perturbation, ||r||, where the computed solution now satisfies exactly the alternate perturbed problem Axc = b + r. Note
that r is just the residual since r = Axc − b. Thus the backward error defined in this way is simply the norm of the residual.
This is the approach that the Text takes, and we have retained it in this homework problem.
3. (MATLAB) Text page 101, Computer Problems 2.3, Problem 4.
The MATLAB script and the numerical output are given below.

% script HW3P3.m
for n=1:5
n=100*n;
for i=1:n
for j=1:n
A(i,j)=sqrt((i-j)^2+n/10);
end
end
x=ones(n,1);
b=A*x;
xc=A\b;
% Forward Error and Relative Forward Error
FE=norm(x-xc,Inf);
RFE=FE/norm(x,Inf);
% Backward Error and Relative Backward Error
BE=norm(b-A*xc,Inf);
RBE=BE/norm(b,Inf);
% Magnification Factor
M=RFE/RBE;
% Estimated condition number
kappa=condest(A);
fprintf(’n=%4.0g Mag=%6.2e Est Kappa=%6.2e \n’,n,M,kappa)
end
%%%%%%
NUMERICAL OUTPUT
n=1e+02 Mag=4.30e+06 Est Kappa=6.18e+07
n=2e+02 Mag=8.21e+08 Est Kappa=1.29e+10
n=3e+02 Mag=3.87e+10 Est Kappa=6.20e+11
n=4e+02 Mag=5.82e+11 Est Kappa=1.48e+13
n=5e+02 Mag=9.78e+12 Est Kappa=2.28e+14

We observe that in each case the magnification factor is smaller than the condition number, as expected.

4. (Pencil-and-paper and MATLAB)

(a) (Pencil-and-paper) Show that the following matrix has determinant zero and hence is singular:
 
0.1 0.2 0.3
A =  0.4 0.5 0.6 
0.7 0.8 0.9
Describe the set of solutions to the system Ax = b for
 
0.1
b =  0.3  .
0.5

(b) (Pencil-and-paper) If we were to use Gaussian elimination with partial pivoting to solve this
system using exact arithmetic, at what point would the process fail?
(c) (MATLAB) Because some of the entries of A are not exactly representable in a finite-precision
binary floating-point system, the matrix is no longer exactly singular when entered into the
computer. Thus, solving the system by Gaussian elimination will not necessarily fail. Solve this
system on MATLAB by using the backslash function
 A\b
discussed in class. Compare the computed solution with your description of the solution set in
part (a) above. Estimate the condition number of A using the MATLAB condest or rcond
commands. How many digits of accuracy would the estimated value of the condition number lead
you to expect?

(a) Expanding from the first row the determinant of A is

0.1(0.45 − 0.48) − 0.2(0.36 − 0.42) + 0.3(0.32 − 0.35) = 0.

Since the matrix is singular, the number of solutions would be zero or infinite, depending upon
b. Further progress demands that we attempt to solve the problem, say by Gaussian elimination.
We consider the augmented matrix
 
0.1 0.2 0.3 0.1
A|b =  0.4 0.5 0.6 0.3  .
0.7 0.8 0.9 0.5

The steps are detailed below.

Step 1. The operations R2 → R2 − 4R1 and R3 → R3 − 7R1 lead to
 
0.1 0.2 0.3 0.1
 0 −0.3 −0.6 −0.1  .
0 −0.6 −1.2 −0.2

Step 2. The operation R3 → R3 − 2R2 yields

 
0.1 0.2 0.3 0.1
 0 −0.3 −0.6 −0.1  .
0 0 0 0

The appearance of zeros in the first three columns of the third row is in accordance with the
singular character of A. The appearance of a zero entry in the last column of the augmented
matrix indicates that A and b of the same rank. The system is therefore consistent and
has an infinity of solutions. We can take x3 to be arbitrary, and backward substitution
determines x2 and x1 . The result is
   
1/3 1
x =  1/3  + x3  −2  .
0 1

(b) As seen above, the process fails due to the appearance of zeros in the last row.
(c) The relevant MATLAB commands and the output are given below.
>> A=[0.1 0.2 0.3;0.4 0.5 0.6;0.7 0.8 0.9];
>> b=[0.1;0.3;0.5];
>> xc=A\b;
Warning: Matrix is close to singular or badly
scaled.
Results may be inaccurate. RCOND =
1.029820e-17.
>> disp(xc)
0.0526
0.8948
-0.2807
 Since the condition number is of the order of 1017 , and machine epsilon is of the order of 10−16 ,
no significant digit is expected. This is an excellent example that demonstrates that if possible, it
is useful to know in advance whether the problem has a solution, because computation can always
provide a ‘solution’ that is meaningless.

5. (MATLAB) Lorenz derived a simple system of ordinary differential equations describing buoyant con-
vection in a fluid as a crude model for atmospheric circulation. At steady state the convective speed
x, temperature gradient y and heat flow z satisfy the following system of nonlinear equations:

σ(y − x) = 0,
rx − y − xz = 0,
xy − bz = 0,

where σ (the Prandtl number), r (the Rayleigh number) and b are positive constants that depend
on the properties of the fluid, the applied temperature gradient and the geometry of the flow. Let
σ = 10, r = 28 and b = 8/3. Write a program using Newton’s method for systems to solve this
problem. You should be able to find three different solutions by varying the initial guesses.
The MATLAB code and the numerical results are displayed below. Three different solutions are found.
In each case, convergence is swift. In the third solution all entries are of the order of machine epsilon,
suggesting that the exact entries may well be zero. It is easily checked that [0 0 0] is indeed a solution
of the system.

% Script HW3P5.m
% Calling program for Newton3d
tol=1e-4; % Tolerance in the increment to the solution
m=20; % Max number of iterations
x0=[1;1;1]; % Initial guess
fprintf(’ x0 \n’);
fprintf(’%8.4e %8.4e %8.4e \n’,x);
x=Newton3d(’f3d’,x0,tol,m);
%%%%%%%%%%
%function x= Newton3d(fun,x0,tol,m)
%
function x = Newton3d(fun,x0,tol,m)
% Matlab function Newton3d.m
%
% Solves a nonlinear system f(x)=0 by Newton’s method
%
% Inputs:
% x0 = initial guess, a column vector
% tol = tolerance for stopping
% m = maximum number of iterations
%
% Outputs:
% x = solution, a column vector
%
% Required m-files
% fun.m for the function and its jacobian matrix
%
x=x0; k = 0;
while k <= m
k=k+1;
[f,Jf]=feval(fun,x);
s = -Jf\f;
x=x+s;
if norm(s)<tol,
fprintf(’k = %2.0f \n’,k);
fprintf(’ x \n’);
fprintf(’%8.4e %8.4e %8.4e \n’,x);
return;end
end
warning(sprintf(’root not found within tolerance after %d iterations\n’,k));
%%%%%%%%%%%%%
function [F,JF]=f3d(x)
%
% Input
% x: column vector (x1;x2;x3)
%
% Output:
% F: function (F1;F2;F3)
% JF: jacobian of F with respect to x
% Define function F
sigma=10; r=28; b=8/3;
x1=x(1);
x2=x(2);
x3=x(3);
F1=sigma*(x2-x1);
F2=r*x1-x2-x1*x3;
F3=x1*x2-b*x3;
F=[F1;F2;F3];
% Define jacobian
J11=-sigma;
J12=sigma;
J13=0;
J21=r-x3;J22=-1;J23=-x1;
J31=x2;J32=x1;J33=-b;
JF=[J11,J12,J13;J21,J22,J23;J31,J32,J33];
%%%%%%%%%%%%%
COMPUTED RESULTS
x0
8.0000e+00 8.0000e+00 1.0000e+01
k = 4
x
8.4853e+00 8.4853e+00 2.7000e+01

x0
-8.0000e+00 -8.0000e+00 1.0000e+01
k = 4
x
-8.4853e+00 -8.4853e+00 2.7000e+01

x0
1.0000e+00 1.0000e+00 1.0000e+00
k = 4
x
-3.1006e-16 -3.1006e-16 -4.0577e-16
%%%%%%%%%%