Modelling System Reliability Using Continuous-

Time Markov Chains

Raffi Paichuk

Introduction maintenance. In this discussion continuous – time Markov

The reliability of a component or system is defined as the
probability of failure not occurring over some specified time
under specified conditions. Thus the time to failure for a
component is a random variable. We can obtain a description
of the failure time distribution for a particular component by
observing the failure times of many such components operating
under normal conditions.
This document starts with a discussion of these issues and
then goes on to consider the modeling of the reliability of
multicomponent systems. This is followed by a description of
reliability models for systems that are subject to repair and
chains are introduced for modeling the reliability and we
generalize the corresponding formulae.
RELIABILITY MODELLING
A component or system is reliable if it has a high probability
of functioning correctly during a period of time. The reliability
is the probability that the system does not fail during this
period. Failure rate generally follows a bathtub curve comprising
three periods of Infant Mortality, useful life and wear out
phase (Fig 1).

64
7 û²~ï¼ - ñö® í~¸

Fig 1. Bathtub Curve

 If T is the time to failure, then the reliability function is defined: The distributions of minimum and maximum of a set of
(1) random variables are called extreme value distributions.
The distribution function for the minimum is :
(10)

(2)
The minimum is greater than t only if all the random
For example if the time to failure is exponentially distributed variables are greater than t. Assuming that the Tk are
with mean t (the mean time to failure or MTTF) then the independent, the probability of this event is
reliability function is : (11)
(3)

Therefore,
(12)
A clever result in probability theory permits the expected
system life (mean time to failure) to be calculated from the
reliability function :
(4) The distribution function for the maximum is
(13)

(5) Assuming that the Tk are independent, the probability of

this event is,
14)

SERIES AND PARALLEL SYSTEMS

A system which functions only if all of its components
function may be regarded, for modeling purposes, as a series Therefore,
connection of components. For a system with n independent 15)
components, the probability that the system is "up" is :
(6)

Using the extreme value distributions, we can calculate the

reliability functions of the series and parallel system .
On the other hand a system which functions if any of its (16)
components function may be regarded as a parallel
connection of components . As above :
(7)

(17)
A series system is only as reliable as its weakest link. If Tk is
the time to failure of the Kth component, then the time to
failure of the system is :
(8)
63

On the other hand, a parallel system does not fail until the
7 û²~ï¼ - ñö® í~¸

last component fails. Its time to failure is

(9) This may be valid for combination of series/parallel systems
and architectures e.g. 1002D or 2004D or 2003 etc.
 Fig 2. 2004D System Architecture

Continuous – Time Markov Chains And if we let Dt 0 we obtain :

We have seen that when equipment or components, are (20)
operating as required (after the early failure stage) failures
tend to be due to random events and the associated failure
rate is exponentially distributed. It turns out that the
exponential distribution is also a reasonable model for repair and for the other two states, we obtain similarly
times so, if failure times and repair times are both assumed (21)
exponential, the continuous- time Markov chain provides
the basis for analysis.
(22)

Fig 3. State Diagram Of Simple Chain

In Matrix form, these three equations can be written
The probability of being in state (0) at time t + Dt is the (23)
probability of being in state (0) at time t multiplied by the
probability of not having moved out in time Dt plus the
probability of being in state (1) at time t and repair being
completed in time Dt, that is:
(18)
Generalizing the analysis to more complicated architectures
(e.g. TMR) yields to more complex state diagrams (Fig 4)

Where the o (Dt) term simply covers the higher order

probability of the system going from state (0) to state (2) in 62
time Dt, equation (18) can be written : Fig 4. State Diagram Of A TMR System
(19) Also these equations can be written in the form
7 û²~ï¼ - ñö® í~¸

(24)

Which is a general form of the Markov state equations for
repairable systems, where the subscript W indicates working
states and subscript F indicates failed states, in terms of
equation (24) submatrix E is 2x2 , because there are two
working states and H is 1x1 because there is just one failed
state.
Mean Time To Failure (MTTF)
Mean time to failure (MTTF) is a standard measure of
equipment reliability that allows products from different
manufactures to be compared. The MTTF can be calculated
from the matrix in equation (24). Submatrices G and H in
(24) contain entries that equal the rates at which transitions
(by repair) are made from failed states to working states. If
we wish to calculate the mean time to failure from an initial
working state, then these repair rates play no role in the
calculation, and sub matrices G and H have zero entries.
Consider the case of an n-state system where the sub matrix
E in (24) in k x k and H is (n-k) x (n-k). And if G and H are
null, equation (24) becomes :
(25)
(26)
And taking Laplace transforms , we have
(27)
(28)
Rearranging (27) gives
(29)
Which, if we assume that the system is initially in a working
61 state (so that PF(0) = 0) can be substituted into the (28) to give
(30)
7 û²~ï¼ - ñö® í~¸
Now, the probability that system is in a failed state at time t
is hn-kPF(t), where hn-k is a row vector with n-k entries, each
equal to unity. And so, if the system failure time is t, we
have
(31)
And this of course is the commutative density of failure
(cdf) of the failure time . Let's denote this cdf by A(t) not to
confuse with sub matrix F.
As the failure time density function is the derivative of the
cdf so its Laplace transform is given by sA* (s) – A(0) and
A(0) = 0 if the system is known to be operational at time
zero. Therefore the Laplace transform of the failure time
density is given by
(32)
And form 30 we can write
(33)
The columns in (24) always add to zero, see for example the
matrix in (23), so that hn-k F = -h k E, substituting this in
(34)
Remembering the definition of Laplace transform for the
case, where the density function is equal to zero for t < 0
(35)
The above is simply the Laplace transform in which s in
replaced by–s . Consequently, the calculation of moments
from the Laplace transform f*(s) is slightly different from
when using the MGF. The rth moment is given by
(36)
Using this, and differentiating (34) we will have
(37)
As required, higher moments of the failure time can be
obtained similarly .
References :
1. Beasley, Michael, "Reliability for Engineers", 1991
2. W.R. Blischke and D.N. P. Murthy, "Reliability : Modelling, Prediction
and optimization", John Wiley, 2000.
3. Roush L. Marvin, Webb M. Willie, "Applied Reliability Engineering".
Center for reliability engineering. The university of Maryland, 1998.