Documente Academic
Documente Profesional
Documente Cultură
2/28/11
nP =
Pij
jk
j Pjk
Thus
is a steady-state vector for the Markov
chain, i.e.,
=
[P ] and
0.
where e = (1, 1, . . . , 1)T
In matrix terms, limn[P n] = e
is a column vector and
is a row vector.
[PT ]
[P ] =
[0]
[PT R]
[PR]
where
P11
[PT ] =
Pt1 . . .
P1t
Ptt
such that
[P ] =
, but we already know there is
such a probability vector (i.e., a steady-state vector)
if [P ] is the matrix of an ergodic unichain.
We show later that there is a steady-state vector
for all Markov chains.
Ai,(i)
i=1
P111 + P122 = 1
.
P211 + P222 = 2
right eigenvector
2 = 1 P12 P21
1P11 + 2P21 = 1
1P12 + 2P22 = 2
1 = 1;
P111 + P122 = 1
.
P211 + P222 = 2
2 = 1 P12 P21
P12
21
c
(1) = P12P+P
(1) = (1
P
+P
21
12
21
, 1)
c
P12
(2) = (1, 1)
(2) =
, P21
P12 +P21
P12 +P21
Note that
(i)
(j) = ij . In general, if
(i)[P ] = i
(i)
and [P ]
(i) = i
(i) for i = 1, . . . , M, then
(i)
(j) = 0
if i = j . To see this,
i
(i)
(j) =
(i)[P ]
(j) =
(i)(j
(j)) = j
(i)
(j)
so if
i or
i
j = 0. Normalization (of either
i = j , then
i) can make
i
i = 1 for each i.
(i)
(i)
(i)
P111 + P122 = i1 ;
(i)
(i)
P211 + P222 = i2
1 0
0 2
where
(1)
(2)
[U ] = 1(1) 1(2) ,
2
2
and
Since
(i)
(j ) = ij , we see that
(1)
(1)
2
1
(2)
(2)
1
2
(1)
(2)
= [I],
(1)
(2)
2
2
so [U ] is invertible and [U 1] has
(1) and
(2) as
rows. Thus [P ] = [U ][][U 1] and
10
[P n] = [U ][n][U 1]
(1)
where
n
[n
i ] has i in position (i, i) and has zeros elsewhere.
Then
[P n] =
n
(i)
(i)
i
i=1
11
[P n] =
(1) =
(2)
P21
, P12
P12 +P21 P12 +P21
= (1, 1)
M
n
(i)
(i)
i=1 i
(1) = (1,
1)
12
(2) = P12P+P
,
21
P21
P12 +P21
(1)
=
1 2
1 2
[P n] =
(2)
(2) =
1 + 2n
2
1 1n
2
2 2
1
1
2 2n
2
n
2 + 12
12
to
.
Then [V ][U ] = I, so [V ] = [U 1]. Thus the eigenvec
tors
(1) to
(M) are linearly independent and span
M space. Same with
(1) to
(M).
13
i=1
n
(i)
(i)
i
14
(2)
|k |Pkj
n
j
mizing j,
|n
|
n M
Pij
i
n
Thus | | M for all n, so || 1.
16
n
(i)
(i)
i
i=1
17
n
(i)
(i)
i
i=1
18
19
1 0 0 0 0
0 1 0 0 0
[J] = 0 0 2 1 0 .
0 0 0 0
2
0 0 0 0 2
The eigenvalues are on the main diagonal and ones
are on the next diagonal up where needed for de
cient eigenvectors.
20
Example:
1/2 1/2 0
[P ] = 0 1/2 1/2 .
0
0
1
21
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.