Basis

A vector x can be written as a linear combination of its basis elements:

x=

3 2

=3

1 0

+2

0 1

= 3e1 + 2e2

Multiplying x by a matrix A can be interpreted as a linear combination of Ae1 and

Ae2. Ax = A 3 2 = A [3e1 + 2e2] = 3Ae1 + 2Ae2

Imagine that there is a basis v1 and v2 such that Av1 = 1v1 and Av2 = 2v2 (where

1 and 2 are scalars) and that x = c1v1 + c2v2. Then: Ax = A [c1v1 + c2v2] = c1Av1 + c2Av2 = c11v1 + c22v2
If Av = v, call an eigenvalue of A and v an eigenvector of A.

Power Method
Suppose a matrix A has two distinct, real eigenvalues 1 and 2 with eigenvectors x1 and x2. If x1 and x2 are not parallel, you can write any vector z(0) as a linear combination of x1 and x2:

z(0) = c1x1 + c2x2

Consider xed point iteration z(k+1) = Az(k) so that z(k) = Ak z(0):

z(1) = Az(0) = c1Ax1 + c2Ax2 = c11x1 + c22x2 z(2) = Az(1) = c11Ax1 + c22Ax2 = c12x1 + c22x2 1 2 z(k) = Az(k1) = c1k1Ax1 + c2k1Ax2 = c1k x1 + c2k x2 1 2 1 2

Now, if |1| > |2|, then

2 k 1

0 as k . Therefore: c1k x1 as k 1

2 k z(k) = k c1x1 + c2 x2 1 1
But if z(k)
(k) k , then |z | 1 |z(k1)|

1. One way to look at the ratio of z(k) to z(k1) is to

evaluate the Rayleigh coefcient:

r(k) =

z(k1) z(k) z(k1) z(k1)

1 as k

This method can be extended to look for the other eigenvalues of A and for complex eigenvalues as well. It works most effectively in nding the extreme eigenvalues those with the largest and smallest magnitude.

Eigenvalues and Eigenvectors

To nd the eigenvalues and eigenvectors of a matrix A, we look for solutions to the equation Ax = x. (The matrix A comes from an equations for the evolution of a mass-spring damper system where K represents the stiffness of the spring.)

Ax = x Ax = Ix (A I) x = 0

2 K 1 0 2 K 1 0

x = x x= 0 0 x=0 x

2 K 1

Since the last equation is satised for a non-zero x, the matrix (A I) is singular:

det

2 K 1

= 0 and (2 )() + K = 0

This polynomial is called the characteristic polynomial of the matrix A and the roots

of the characteristic polynomial are the eigenvalues of the matrix A. The vector x is an eigenvector of the matrix A.

We have the polynomial 2 + 2 + K = 0. First consider the case where K = 0: couple of cases:

2 4 K=0 = = 1 1 = 0, 2 2 The eigenvalues for K = 0 are = 2, 0. Remember that the eigenvalue equation may be written Ax = x. Look for the eigenvector x corresponding to each eigenvalue : Ax1 = 1x1 2x1 = 2x1 Ax2 = 2x2 2x1 = 0 2 0 1 0 x1 x2 = 2 x1 x2

and x1 = 2x2

x1 = x1 x2

1 0.5 x1 x2

2 0 1 0

=0 0 1

and x1 = 0

x2 =

Note that the eigenvectors are only determined up to a constant: A(3x) = (3x).

Eigenvalues of real matrices come in two different avors:

Purely real: correspond to stretching/contraction or to growth/decay in time-dependent

problems.

Complex conjugate pairs (i.e. r 1i): correspond to rotation or oscillation in

time-dependent problems. The real part can also lead to stretching/contraction and to growth/decay in time-dependent problems. Look at different values of K in our example:

K=1 K=2

2 0 = = 1 0 = 1, 1 2 2 4 = 1 1 = 2

The real part of the eigenvalues in these case is negative, so that solutions that evolve as exp(t) will decay with time.

Examples/Applications
Error analysis of iterative methods: The initial error is multiplied repeatedly by a matrix T:

e(1) = T e(0) = T (c1x1 + c2x2) = c1T x1 + c2T x2 = c11x1 + c22x2 e(k) = Tk e(0) = c1(1)k x1 + c2(2)k x2
Therefore, if |1| < 1 and |2| < 1 then the error will go to zero as k . Ordinary differential equations: Consider a system of ordinary differential equations:

v=Av
If x1 and x2 are the eigenvectors of A (i.e. Ax1 = 1x1 and Ax2 = 2x2), then:

v(t) = c1(t)x1 + c2(t)x2

and

v = c1x1 + c2x2 = A v = c1Ax1 + c2Ax2 = c11x1 + c22x2

This implies that:

[c1 1c1] x1 + [c2 2c2] x2 = 0

For this to be true for all values of c1 and c2 (including zero), we need:

c1 = 1c1 c2 = 2c2

c1(t) = c1(t = 0) exp(1t) c2(t) = c2(t = 0) exp(2t)

The solution to the (system of) ordinary differential equations is:

v(t) = c1(t = 0) exp(1t) x1 + c2(t = 0) exp(2t) x2 If the real part of 1 and 2 is negative, the solutions will decay with time. If the real part of 1 and 2 is positive, the solutions will grow with time. If the eigenvalues are complex ( = r + 1i), the solution will evolve like: exp(t) = exp(rt) cos(it) + 1sin(it)