Eigen Calculations Using the Jacobi Iteration Method

How are eigenvalues and eigenvectors calculated in practice?

Look at simple case when A is symmetric.
Jacobi method circa 1845.
Orthogonal matrices are essential. Q is orthogonal if Q T = Q 1
Examples: Q = sc cs when c 2 + s s = 1

c s 0
Q0
Q 1 = s c 0 =
0 1
0 0 1

use block matrix calculations

Suppose Q is orthogonal and P is a permutation matrix

Then P T QP is an orthogonal matrix.

1 0 0
Q 2 = 0 c s obtained from Q1 using a permutation matrix
0 s c
Basic ideas from earlier.
If A is symmetric then Q TAQ is also.
If Q is orthogonal, then Q TAQ and A have the same eigenvalues.

The basic idea is to choose special orthogonal matrices that zero

out specified off-diagonal elements. Givens rotations are one
choice( see page 104, problem 23).

5 2 0
A = 2 5 0
0 0 3
1/ 2 1/ 2 0
c s 0
and Q 1 = s c 0 = 1/ 2 1/ 2 0 is a Givens rotation
0 0 1
0
01
designed to zero out the (2,1) element.

300
A1 =
= 070
003
The columns of Q are eigenvectors and the eigenvalues are on the
diagonal.
Q T1 AQ 1

Rotations that zero out a specified element for this note are called
Givens rotations after the scientist at Oak Ridge who pioneerred
their use more than fifty years ago.
Related orthogonal transformations that zero out a portion of a
column are called Householder transformations.

312
Start with A = 1 2 1 and construct a Givens rotation Q 1that
214
zeros out the (2,1) element in then sense that the (2,1) element of
A 1 = Q T1 AQ 1 is zero.
Then find a Givens rotation Q 2 that zeros out the (3,1) element of
A1 in the same manner A 2 = Q T2 A 1Q 2 . Proceed with another
rotation Q 3 that zeros out the (3,2) element of A2 and set

A 3 = Q T3 A 2Q 3 . Start over again with the (2,1) location for the

latest iterate and continue the process.
The off-diagonal elements go to zero after several iterations
leaving an essentially diagonal matrix of eigenvalues.
After p iterations, we have

A p = Q Tp A p1Q p or A p = Q TAQ
for the orthogonal matrix Q = Q 1Q 2

Q p1 Q p .

After several sweeps, A p is almost diagonal. This means that A is

orthogonally similar to a diagonal matrix A p which holds the
eigenvalues of A. Q holds the corresponding eigenvectors of A.

The symmetric matrix A is

3
1
2

1
2
1

2
1
4

Begin the sweep process.

Zero out spot (2,1)
Givens rotation
0.8507 -0.5257
0.0000
0.5257
0.8507
0.0000
0.0000
0.0000
1.0000
*** *** *** *** *** ***
Q'*A*Q
|
3.6180
0.0000
2.2270
|
0.0000
1.3820 -0.2008
|
2.2270 -0.2008
4.0000
|

Q
0.8507 -0.5257
0.5257
0.8507
0.0000
0.0000

0.0000
0.0000
1.0000

*** ***

Zero out spot (3,1)

Givens rotation
0.7367 0.0000 0.6762
0.0000 1.0000 0.0000
-0.6762 0.0000 0.7367
*** *** *** *** *** ***
Q'*A*Q
1.5738 0.1358 0.0000
0.1358 1.3820 -0.1479
0.0000 -0.1479
6.0442
*** ***

|
|
|
|

current product
of Givens rotations
|
\|/
Q
0.6267 -0.5257 0.5752
0.3873 0.8507 0.3555
-0.6762
0.0000
0.7367

Notice how pushing an element to zero lets a

previously zeroed element to reappear.

Zero out spot (3,2)

Givens rotation
1.0000
0.0000
0.0000
0.0000
0.9995 -0.0317
0.0000
0.0317
0.9995
*** *** *** *** *** ***
Q'*A*Q
1.5738 0.1357 -0.0043
0.1357 1.3773 0.0000
-0.0043 0.0000 6.0489
*** ***

|
|
|
|

Q
0.6267 -0.5072
0.3873 0.8615
-0.6762 0.0233

0.5916
0.3284
0.7363

One sweep is complete.

Absolute sum of off-diagonal elements = 2.800552e-01
Another sweep?y/n-->y
Zero out spot (2,1)
Givens rotation
0.8906 -0.4547
0.0000
0.4547
0.8906
0.0000
0.0000
0.0000
1.0000
*** *** *** *** *** ***
Q'*A*Q
1.6431 0.0000 -0.0038
0.0000 1.3080 0.0020
-0.0038 0.0020 6.0489
*** ***
Zero out spot (3,1)
Givens rotation
1.0000
0.0000 -0.0009
0.0000
1.0000
0.0000
0.0009
0.0000
1.0000
*** *** *** *** *** ***
Q'*A*Q
1.6431 0.0000 0.0000
0.0000 1.3080 0.0020
0.0000 0.0020 6.0489
*** ***

|
|
|
|

|
|
|
|

Q
0.3275 -0.7367
0.7367 0.5911
-0.5917 0.3283

Q
0.3280 -0.7367
0.7370 0.5911
-0.5910 0.3283

0.5916
0.3284
0.7363

0.5913
0.3277
0.7368

Zero out spot (3,2)

Givens rotation
1.0000
0.0000
0.0000
0.0000
1.0000
0.0004
0.0000 -0.0004
1.0000
*** *** *** *** *** ***
Q'*A*Q
1.6431 0.0000 0.0000
0.0000 1.3080 0.0000
0.0000 0.0000 6.0489
*** ***

|
|
|
|

Q
0.3280 -0.7370
0.7370 0.5910
-0.5910 0.3280

0.5910
0.3280
0.7370

2 sweeps are complete.

Absolute sum of off-diagonal elements = 3.404415e-06
Another sweep?y/n-->y
Zero out spot (2,1)
Givens rotation
1.0000 -0.0000
0.0000
0.0000
1.0000
0.0000
0.0000
0.0000
1.0000
*** *** *** *** *** ***
Q'*A*Q
1.6431 0.0000 0.0000
0.0000 1.3080 -0.0000
0.0000 -0.0000
6.0489
*** ***
Zero out spot (3,1)
Givens rotation
1.0000
0.0000
0.0000
0.0000
1.0000
0.0000
-0.0000
0.0000
1.0000
*** *** *** *** *** ***
Q'*A*Q
1.6431 0.0000 0.0000
0.0000 1.3080 -0.0000
0.0000 -0.0000
6.0489
*** ***

|
|
|
|

Q
0.3280 -0.7370
0.7370 0.5910
-0.5910
0.3280

0.5910
0.3280
0.7370

|
|
|
|

Q
0.3280 -0.7370
0.7370 0.5910
-0.5910
0.3280

0.5910
0.3280
0.7370

Zero out spot (3,2)

Givens rotation
1.0000
0.0000
0.0000
0.0000
1.0000 -0.0000
0.0000
0.0000
1.0000
*** *** *** *** *** ***
Q'*A*Q
1.6431 0.0000 -0.0000
0.0000 1.3080 0.0000
-0.0000 0.0000 6.0489
*** ***

|
|
|
|

Q
0.3280 -0.7370
0.7370 0.5910
-0.5910 0.3280

0.5910
0.3280
0.7370

3 sweeps are complete.

Absolute sum of off-diagonal elements = 1.136271e-24
Another sweep?y/n-->n
Diagonal matrix D of approximate eigenvalues
1.6431
0.0000
-0.0000

0.0000
1.3080
0

-0.0000
0
6.0489

Matrix Q of corresponding approximate eigenvectors

0.3280
0.7370
-0.5910

-0.7370
0.5910
0.3280

0.5910
0.3280
0.7370

*****************************************************
*** *** ***
Summary Information
*** *** ***
*****************************************************
At termination, 3 sweeps were completed.
D should be diagonal and it is
1.6431
0.0000
-0.0000
0.0000
1.3080
0
-0.0000
0
6.0489
Orthogonal Q is
0.3280
-0.7370
0.7370
0.5910
-0.5910
0.3280

0.5910
0.3280
0.7370

What about MATLAB?

[Q,D] = eig(A);
disp(Q)
0.7370
-0.3280
-0.5910
-0.7370
-0.3280
0.5910

0.5910
0.3280
0.7370

disp(D)
1.3080
0
0

0
0
6.0489

0
1.6431
0

The results are basically the same but the Qs are

different. This is acceptable because MATLAB used a
somewhat different algorithm.