Eigen-analysis

and the
Power Method
Module Goals

Power Method
Shift technique (optional)
Inverse Method
Accelerated Power Method
Power method
The special advantage of the power method is
that the eigenvector corresponds to the dominant
eigenvalue and is generated at the same time. The
inverse power method solves for the minimal
eigenvalue/vector pair.

The disadvantage is that the method only supplies
obtains one eigenvalue
Power Method
Readers Digest Version
Eigenvalues can be ordered in magnitude and the largest is
called the dominant eigenvalue or spectral radius.
Think about how eigenvalues are a reflection of the nature
of a matrix. Now if we multiply by that matrix over and
over again..eventually the biggest eigenvalue will make
everyone else have eigen-envy.
One to rule them all, One to find them, One to bring them all
and in the darkness bind them.
Power Method
In general continue the multiplication:
A AA A =
k
where,
{ } { } { } { } { }
3
k
n n 3
k
3 3 2
k
2 2 1
k
1 1
k
| o | o | o | o + + + + = x A
Power Method
Factor the large value term
As you continue to multiply the vector by [A]
{ } { } { } { }
(
(

|
|
.
|

\
|
+ +
|
|
.
|

\
|
+ =
3
k
1
n
n 2
k
1
2
2 1 1
k
1
k
|

o |

o | o x A
{ } { } = k as
1 1
k
1
k
| o x A
Power Method
The basic computation of the power method is
summarized as
= = = =


k as
1
1
1
k
1
k
1
k
1
k
0
k
0
k
k
|
|
| o
| o
| o
| o
x A
x A
u
Power Method
The basic computation of the power method is
summarized as
lim and
1 k
k
1 - k
1 - k
k
| = =

u
Au
Au
u
The equation can be written as:
1 - k
1 - k
1 1 - k 1 1 - k
u
Au
u Au = =
The Power Method Algorithm
(algorithm 3.3.1 pg 107)
y=nonzero random vector
Initialize x = A*y vector
for k =1,2,n
y=x/||x||
x =Ay (x is the approximate eigenvector)

approximate eigenvalue

= (yT*x)/(yT*y)
r=y-x
k++




Example of Power Method
Consider the follow matrix A
(
(
(

=
1 0 0
1 2 0
0 1 4
A
Assume an arbitrary vector x
0
= { 1 1 1}
T

Example of Power Method
Multiply the matrix by the matrix [A] by {x}

(
(
(

1
3
5
1
1
1
1 0 0
1 2 0
0 1 4
Normalize the result of the product

2 . 0
6 . 0
1
5
1
3
5
Example of Power Method

(
(
(

0435 . 0
4783 . 0
2174 . 4
0435 . 0
217 . 0
1
1 0 0
1 2 0
0 1 4

0435 . 0
217 . 0
1
6 . 4
2 . 0
1
6 . 4

(
(
(

2 . 0
1
6 . 4
2 . 0
6 . 0
1
1 0 0
1 2 0
0 1 4

0183 . 0
1134 . 0
1
2174 . 4
0435 . 0
4783 . 0
2174 . 4
Example of Power Method

(
(
(

0103 . 0
2165 . 0
1134 . 4
0183 . 0
1134 . 0
1
1 0 0
1 2 0
0 1 4

0025 . 0
0526 . 0
1
1134 . 4
0103 . 0
2165 . 0
1134 . 4
As you continue to multiple each successive vector
= 4 and the vector u
k
={1 0 0}
T

Shift method
(optional)
It is possible to obtain another eigenvalue from the
set equations by using a technique known as
shifting the matrix.
| |{ } { } x x A =
Subtract the a vector from each side, thereby
changing the maximum eigenvalue
| |{ } | |{ } ( ){ } x s x I s x A =
Shift method
The eigenvalue, s, is the maximum value of the
matrix A. The matrix is rewritten in a form.
| | | | | | I A B
max
=
Use the Power method to obtain the largest
eigenvalue of [B].
Example of Power Method
Consider the follow matrix A
(
(
(

=
(
(
(

(
(
(

=
5 0 0
1 2 0
0 1 0
1 0 0
0 1 0
0 0 1
4
1 0 0
1 2 0
0 1 4
B
Assume an arbitrary vector x
0
= { 1 1 1}
T

Example of Power Method
Multiply the matrix by the matrix [A] by {x}

(
(
(

5
1
1
1
1
1
5 0 0
1 2 0
0 1 0
Normalize the result of the product

1
6 . 0
2 . 0
5 -
5
1
1
Example of Power Method

1
12 . 0
04 . 0
5
5
6 . 0
2 . 0

(
(
(

5
6 . 0
2 . 0
1
2 . 0
2 . 0
5 0 0
1 2 0
0 1 0
Continue with the iteration and the final value is
= -5. However, to get the true you need to shift
back by:
1 4 5
max
= + = +
'
=
Inverse Power Method
The inverse method is similar to the power method,
except that it finds the smallest eigenvalue. Using
the following technique.
| |{ } { } x x A = | | | |{ } | | { } x A x A A
1 1
=
{ } | | { } x A x
1
1

=

{ } | |{ } x B x =
Inverse Power Method
The algorithm is the same as the Power method and
the eigenvector is not the eigenvector for the
smallest eigenvalue. To obtain the smallest
eigenvalue from the power method.

1

1
= =
Inverse Power Method
The inverse method is similar to the power method,
except that it finds the smallest eigenvalue. Using
the following technique.
| |{ } { } x x A = | | | |{ } | | { } x A x A A
1 1
=
{ } | | { } x A x
1
1

=

{ } | |{ } x B x =
Inverse Power Method
The algorithm is the same as the Power method and
the eigenvector is not the eigenvector for the
smallest eigenvalue. To obtain the smallest
eigenvalue from the power method.

1

1
= =
Inverse Power Method
The inverse algorithm use the technique avoids
calculating the inverse matrix and uses a LU
decomposition to find the {x} vector.
| |{ } { } x x A = | || |{ } { } x x U L =

1
Example
| |
(
(
(

=
5 1 2
1 3 1
0 2 4
A
The matrix is defined as:

=
82 . 1
2535 . 5
9264 . 4

Accelerated Power Method

The Power method can be accelerated by using the
Rayleigh Quotient instead of the largest w
k
value.


The Rayeigh Quotient is defined as:

| |{ }
1 1
= z A
z z
w z
'
'
1
=
Accelerated Power Method
The values of the next z term is defined as:



The Power method is adapted to use the new value.

1
2

w
z =
Example of Accelerated Power
Method
Consider the follow matrix A
(
(
(

=
1 0 0
1 2 0
0 1 4
A
Assume an arbitrary vector x
0
= { 1 1 1}
T

Example of Power Method
Multiply the matrix by the matrix [A] by {x}

(
(
(

1
3
5
1
1
1
1 0 0
1 2 0
0 1 4
{ }
{ }
333 . 2
3
7
1
1
1
1 1 1
1
3
5
1 1 1
1
= =

= =
4286 . 0
2857 . 1
1429 . 2
1
1
2

w
z
Example of Accelerated Power
Method
Multiply the matrix by the matrix [A] by {x}

(
(
(

4286 . 0
1429 . 2
8571 . 9
4286 . 0
2857 . 1
1429 . 2
1 0 0
1 2 0
0 1 4
{ }
{ }
6857 . 3
429 . 0
2857 . 1
143 . 2
429 . 0 286 . 1 143 . 2
429 . 0
143 . 2
857 . 9
429 . 0 2857 . 1 142 . 2
2
=

= =
1163 . 0
5814 . 0
6744 . 2
2
2
3

w
z
Example of Accelerated Power
Method

(
(
(

1163 . 0
2791 . 1
2791 . 11
1163 . 0
5814 . 0
6744 . 2
1 0 0
1 2 0
0 1 4
1171 . 4
3
=

= =
0282 . 0
3107 . 0
7396 . 2
3
3
4

w
z
Example of Accelerated Power
Method

(
(
(

0282 . 0
5931 . 0
2689 . 11
0282 . 0
3107 . 0
7396 . 2
1 0 0
1 2 0
0 1 4
0849 . 4
4
=

= =
0069 . 0
1452 . 0
7587 . 2
4
4
5

w
z
And so on ...