MarkovExcel Module

by Germán Riaño
Grupo COPA
griano@uniandes.edu.co

Ver esta página en español ...

This workbook has a series of macros that can be used to analyze medium size Markov
Chain Problems. Its accuracy for large model has not been tested, and anyway Excel
cannot handle more than 256 columns
For really large models use:
jMarkov

You have to enable macros in Excel in order to use this tool.

Go to tools/Macros/Security to enable them

Discrete Time Markov Chains

DTMC

Continuous Time Markov Chains

CTMC

Reference Manual
Reference

MarkovExcel can be used free of charge for academic purposes.

No warranty is given or implied.
I am extremely grateful if you point to me any mistakes in the program. But I cannot give
personalized customer support by e-mail. In particular I canot teach you stochastics by
e-mail, nor will I help you with homework (except my own students, of course)
 Discrete Time Markov Chains
Go back to Main Menu
Let's define P and call it "TheP", via Insert/Name/Define

0.2 0.7 0.1

P= 1 0 0
0.2 0.3 0.5

For steady state probabilities we mark the destination range and write
""=DTMCSteadyState(theP)" and then hit ctrl-shft-enter at the same time.
Attention: For all functions you have to hit Ctrl shift y enter at the same time.

p= #VALUE! #VALUE! #VALUE!

To obtain powers of P we use:

' =MatPower(Matrix,power)

#VALUE! #VALUE! #VALUE!

P(4) = #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

To find the occupancy matrix:

M(n) = I + P + P2 +...+Pn
usamos "=DTMCMatOcup(Matriz, n)"

#VALUE! #VALUE! #VALUE!

M(1)= #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

Reducible Chains

1 2 3 4 5 6
1 0.1 0.2 0.1 0.2 0.2 0.2
2 0.2 0.3 0.1 0.3 0.1 0
3 0.1 0.3 0.2 0.2 0.1 0.1
4 0 0 0 0.2 0.8 0
5 0 0 0 0.8 0.2 0
6 0 0 0 0 0 1

If we try DTMCSteadyState we obviously get an error:

#VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!

The former chain must be decomposed as:

 T = {1,2,3} Transient states
C1 ={4,5} First closed communicating class
C2 = {6} Second communicating class

Absortion Probabilities

Find R y Q, that correspond to transitions among transient and from transient the the closed comunicating class
1 2 3
1 0.1 0.2 0.1
A= 2 0.2 0.3 0.1 R=
3 0.1 0.3 0.2

Absortion Probabilities are computed through

DTMCAbsor(A,R)
C1 C2
1 #VALUE! #VALUE!
F= 2 #VALUE! #VALUE!
3 #VALUE! #VALUE!

Expected time before absortion is computed through

DTMCExpected(A)
1 #VALUE!
m= 2 #VALUE!
3 #VALUE!

Limit probability exists, since there are no periodic states, and is given by

1 2 3 4 5 6
1 0 0 0 #VALUE! #VALUE! #VALUE!
2 0 0 0 #VALUE! #VALUE! #VALUE!
3 0 0 0 #VALUE! #VALUE! #VALUE!
4 0 0 0 #VALUE! #VALUE! 0
5 0 0 0 #VALUE! #VALUE! 0
6 0 0 0 0 0 1

Lets compute a big power to check the result:

1 2 3 4 5 6
1 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
2 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
3 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
4 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
5 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
6 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!

It works!
e closed comunicating classes
C1 C2
1 0.4 0.2
2 0.4 0
3 0.3 0.1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
 Continuous Time Markov Chains
Go back to Main Menu
Define generator matrix Q y and call it "Generateor", via Insert/Name/Define

-9 4 5
Q= 10 -17 7
7 7 -14

For steady state probabilities

mark destination range and write "=CTMCSteadyState(Generator)"
and hit ctrl-shift-enter at the same time.
Attention: For all functions you have to hit Ctrl shift y enter at the same time.

p= #VALUE! #VALUE! #VALUE!

To find P(t)=exp(Q t) we use:

=CTMCTransient(Matrix, time)

#VALUE! #VALUE! #VALUE!

P(0.01) = #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

To find the occupancy matrix

t
M (t )=∫0 P(s)ds
we use CTMCMatOccup(Generator, n)

#VALUE! #VALUE! #VALUE!

M(0.1)= #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

Reducible Chains
Assume Q is given as
1 2 3 4 5 6
1 -9 2 1 2 2 2
2 12 -17 1 3 1 0
3 1 3 -8 2 1 1
4 0 0 0 -8 8 0
5 0 0 0 8 -8 0
6 0 0 0 0 0 0

If we try CTMCSteadyState we get, obviously, an error:

#VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
 The previous chain must be decomposed as

T = {1,2,3}
C1 ={4,5}
C2 = {6}

Absortion Probabilities:

Lets compute R and Q

1 2 3
1 -9 2 1
A= 2 12 -17 1 R=
3 1 3 -8

Absortion Probabilities:
DTMCAbsor(A,R)
C1 C2
1 #VALUE! #VALUE!
F= 2 #VALUE! #VALUE!
3 #VALUE! #VALUE!

Expected time before absortion:

DTMCExpected(A)
1 #VALUE!
m= 2 #VALUE!
3 #VALUE!

The limit matrix exists and is given by:

1 2 3 4 5 6
1 0 0 0 #VALUE! #VALUE! #VALUE!
2 0 0 0 #VALUE! #VALUE! #VALUE!
3 0 0 0 #VALUE! #VALUE! #VALUE!
4 0 0 0 #VALUE! #VALUE! 0
5 0 0 0 #VALUE! #VALUE! 0
6 0 0 0 0 0 1

As a test, we compute P(t) for a large t

1 2 3 4 5 6
1 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
2 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
3 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
4 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
5 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
6 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!

and it works!
 C1 C2
1 4 2
2 4 0
3 3 1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
Reference Manual Go back to Main Menu
The following functions are defined in the Macros.
To use this functions in another worksheet save this workboos as an add-in (menu: File/Save
as, choose the last option)
If you plan to use the functions frequenlt you can install the add-in permanently (menu:
Tools/Addins)
General Matrix Functions
MatPower(A, n As Integer)
Computes the n-th power of the given matrix

Function MatSum(A, B, Optional minus = False)

Adds the given matrices. If minus is TRUE then it computes A-B

Function MatTimes(A, alpha)

Computes alpha*A

Function MatIden(n As Integer)

Returns an nxn identity matrix

Function MaxRate(Q) As Double

Assuming Q is a generator matrix this computes the biggest rate, i.e., the largest entry in the
diagonal:

λ=max (−qii )
Function UniformProb(Q, Optional EnsureGenerator As Boolean = True,
Optional lda = -1#)
Assuming Q is a generator matrix, computes the associated uniformized DTMC transition
matrix. If lda is not given it is internally computed. If EnsureGenerator is true then Q is
converted to a generator rather than a rates matrix
Q
P= +I
λ
DTMC Functions
MatExpo(A, t)
Computes exp(A x t)

DTMCMatOccup(A, n As Integer)
Computes the occupancy matrix for a Discrete Markov Chain

Function DTMCSteadyState(P)
Computes the steady state probability for DTMC with matix P. If P is not irreducible (or
unichain) this returns an error.

Function DTMCAbsor(A, R)
This computes the absorving probabilities for a markov chain with transient to transient
probabilities Q, and transient to absorving probabilities R, through the formula

( I − A )−1 R
 I−
( (I−Q )−1 RA )−1 R
Function DTMCExpected(A)
Computes the expected number of steps before absorption, starting from each transient state,
through the formula
−1−1
( II−Q
− A) ) R1
1
CTMC Functions
Function CTMCSteadyState(Q)
Computes the steady state probabilities for the given generator matrix

Function CTMCTransient(Q, t, Optional Cum = False)

Assuming that Q is a generator matrix this computes exp(Q t). If cum is true then it rathers
computes the occupancy matrix. The algorithm used is uniformization.

Function CTMCMatOccup(Q, t)
Computed teh occupancy matrix
t
M (t )=∫0 P(s)ds
Function CTMCAbsor(A, R)
Computes teh absortion probabilities where A is the generator rates matrix from transient to
transient states, and R from transient to the absorving classes

Function CTMCExpected(A)
Computes the expected value of the time to absortion starting from each of the given states.
 MarkovExcel Module
por Germán Riaño
Grupo COPA
griano@uniandes.edu.co

See this page in English...

Esta herramienta de Excel tiene una serie de macros para el análisis de Cadenas de
Markov de tamaño intermedio. Su precisión para modelos muy grandes no has sido
probada y, en todo caso, Excel no maneja más de 256 columnas.
Para nodelos grandes use:
jMarkov

Para usar esta herramienta se deben habilitar los macros de Excel.

Vaya al manu herrmiantas/Macro/Seguridad.

Cadenas de Markov de Tiempo Discreto

CMTD

Cadenas de markov de Tiempo Continuo

CMTC

Manual de Referencia.
Referencia

MarkovExcel se puede usar sin costo para labores académicas.

No se da garantía alguna ni explícita ni implícita
Le agardezco que me avise si encuentra errores. Pero desafortunadamente no puedo
dar "soporte al cliente" personalizado. En particular no puedo enseñarle procesos
estocásticos por e-mail ni ayudarle en sus tareas (excepto a mis estudiantes claro).
 Cadenas de Markov de Tiempo Discreto
Regresar al Menu Principal
Definimos P y le damos el nombre "laPe", via Insertar/Nombre/Definir

0.2 0.7 0.1

P= 1 0 0
0.2 0.3 0.5

Para Probabilidades de estado estable

marcamos el destino, escribimos "=DTMCSteadyState(laPe)"
y luego ctrl-shft-enter al tiempo.
OJO: Para todas las funciones damos Ctrl shift y enter al tiempo.

p= #VALUE! #VALUE! #VALUE!

Para potencias de P usamos

MatPower(Matriz,potencia)

#VALUE! #VALUE! #VALUE!

P(4) = #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

Para Hallar la matriz de ocupación:

M(n) = I + P + P^2 +...+P^n
usamos DTMCMatOcup(Matriz, n)

#VALUE! #VALUE! #VALUE!

M(1)= #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

Cadenas Reducibles

1 2 3 4 5 6
1 0.1 0.2 0.1 0.2 0.2 0.2
2 0.2 0.3 0.1 0.3 0.1 0
3 0.1 0.3 0.2 0.2 0.1 0.1
4 0 0 0 0.2 0.8 0
5 0 0 0 0.8 0.2 0
6 0 0 0 0 0 1

Si intentamos DTMCSteadyState obviamente generamos un error:

#VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!

La anterior cadena se DEBE descomponer como

 T = {1,2,3}
C1 ={4,5}
C2 = {6}

Probabilidades de Absorción:

Calculamos A y R
1 2 3
1 0.1 0.2 0.1
A= 2 0.2 0.3 0.1 R=
3 0.1 0.3 0.2

Las probabilidades de absroción son

DTMCAbsor(A,R)
C1 C2
1 #VALUE! #VALUE!
F= 2 #VALUE! #VALUE!
3 #VALUE! #VALUE!

Los tiempos esperados antes de absorción son

DTMCExpected(A)
1 #VALUE!
m= 2 #VALUE!
3 #VALUE!

La matriz límite existe, pues no hay estados periódicos y es:

1 2 3 4 5 6
1 0 0 0 #VALUE! #VALUE! #VALUE!
2 0 0 0 #VALUE! #VALUE! #VALUE!
3 0 0 0 #VALUE! #VALUE! #VALUE!
4 0 0 0 #VALUE! #VALUE! 0
5 0 0 0 #VALUE! #VALUE! 0
6 0 0 0 0 0 1

A modo de revisión calculamos una potencia grande:

1 2 3 4 5 6
1 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
2 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
3 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
4 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
5 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
6 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!

y si lo es!
 C1 C2
1 0.4 0.2
2 0.4 0
3 0.3 0.1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
 Cadenas de Markov de Tiempo Continuo
Regresar al Menu Principal
Definimos la matriz generadora Q y le damos el nombre "Generateor", via Inserter/Nombre/Definir

-9 4 5
Q= 10 -17 7
7 7 -14

Para Probabilidades de estado estable

marcamos el destino, escribimos CTMCSteadyState
y luego ctrl-shft-enter al tiempo.
OJO: Para todas las funciones damos Ctrl shift y enter al tiempo.

p= #VALUE! #VALUE! #VALUE!

Para hallar P(t)=exp(Q t) usamos

=CTMCTransient(Matriz, tiempo)

#VALUE! #VALUE! #VALUE!

P(0.01) = #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

Para Hallar la matriz de ocupación:

t
M (t )=∫0 P(s)ds
usamos CTMCMatOccup(Generator, tiempo)

#VALUE! #VALUE! #VALUE!

M(0.1)= #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

Cadenas Reducibles
Suponga que Q viene dada por
1 2 3 4 5 6
1 -9 2 1 2 2 2
2 12 -17 1 3 1 0
3 1 3 -8 2 1 1
4 0 0 0 -8 8 0
5 0 0 0 8 -8 0
6 0 0 0 0 0 0

Si intentamos SteadyState obviamente generamos un error:

#VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
 La anterior cadena se DEBE descomponer como

T = {1,2,3}
C1 ={4,5}
C2 = {6}

Probabilidades de Absorción:

Calculamos A y R
1 2 3
1 -9 2 1
A= 2 12 -17 1 R=
3 1 3 -8

Las probabilidades de absroción son

=DTMCAbsor(A,R)
C1 C2
1 #VALUE! #VALUE!
F= 2 #VALUE! #VALUE!
3 #VALUE! #VALUE!

Los tiempos esperados antes de absorción son

=DTMCExpected(A)
1 #VALUE!
m= 2 #VALUE!
3 #VALUE!

La matriz límite existe y es:

1 2 3 4 5 6
1 0 0 0 #VALUE! #VALUE! #VALUE!
2 0 0 0 #VALUE! #VALUE! #VALUE!
3 0 0 0 #VALUE! #VALUE! #VALUE!
4 0 0 0 #VALUE! #VALUE! 0
5 0 0 0 #VALUE! #VALUE! 0
6 0 0 0 0 0 1

A modo de revisión calculamos P(t) para un valor grande de t:

1 2 3 4 5 6
1 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
2 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
3 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
4 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
5 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
6 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!

y si lo es!
Nombre/Definir
 C1 C2
1 4 2
2 4 0
3 3 1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
 Manual de Referencia Regresar al Menu Principal
(Lo siento, no tengo traducción.. Quizás Ud. quiera ayudarme..)
The following functions are defined in the Macros.
To use this functions in another worksheet save this workboos as an add-in (menu: File/Save
as, choose the last option)
If you plan to use the functions frequenlt you can install the add-in permanently (menu:
Tools/Addins)
General Matrix Functions
MatPower(A, n As Integer)
Computes the n-th power of the given matrix

Function MatSum(A, B, Optional minus = False)

Adds the given matrices. If minus is TRUE then it computes A-B

Function MatTimes(A, alpha)

Computes alpha*A

Function MatIden(n As Integer)

Returns an nxn identity matrix

Function MaxRate(Q) As Double

Assuming Q is a generator matrix this computes the biggest rate, i.e., the largest entry in the
diagonal:

λ=max (−qii )
Function UniformProb(Q, Optional EnsureGenerator As Boolean = True,
Optional lda = -1#)
Assuming Q is a generator matrix, computes the associated uniformized DTMC transition
matrix. If lda is not given it is internally computed. If EnsureGenerator is true then Q is
converted to a generator rather than a rates matrix
Q
P= +I
λ
DTMC Functions
MatExpo(A, t)
Computes exp(A x t)

DTMCMatOccup(A, n As Integer)
Computes the occupancy matrix for a Discrete Markov Chain

Function DTMCSteadyState(P)
Computes the steady state probability for DTMC with matix P. If P is not irreducible (or
unichain) this returns an error.

Function DTMCAbsor(A, R)
This computes the absorving probabilities for a markov chain with transient to transient
probabilities Q, and transient to absorving probabilities R, through the formula

( I − A )−1 R
 ( I − A )−1 R
Function DTMCExpected(A)
Computes the expected number of steps before absorption, starting from each transient state,
through the formula
−1
( I− A ) 1
CTMC Functions
Function CTMCSteadyState(Q)
Computes the steady state probabilities for the given generator matrix

Function CTMCTransient(Q, t, Optional Cum = False)

Assuming that Q is a generator matrix this computes exp(Q t). If cum is true then it rathers
computes the occupancy matrix. The algorithm used is uniformization.

Function CTMCMatOccup(Q, t)
Computed teh occupancy matrix
t
M (t )=∫0 P(s)ds
Function CTMCAbsor(A, R)
Computes teh absortion probabilities where A is the generator rates matrix from transient to
transient states, and R from transient to the absorving classes

Function CTMCExpected(A)
Computes the expected value of the time to absortion starting from each of the given states.
Principal