Sunteți pe pagina 1din 36

Probability and Statistics with

Reliability, Queuing and


Computer Science Applications:
Chapter 6 on Stochastic Processes

Kishor S. Trivedi
Visiting Professor
Dept. of Computer Science and
Engineering
Indian Institute of Technology, Kanpur

What is a Stochastic Process?

Stochastic Process: is a family of random


variables {X(t) | t T} (T is an index set; it
may be discrete or continuous)
Values assumed by X(t) are called states.
State space (I): set of all possible states
Sometimes called a random process or a
chance process

Stochastic Process Characterization

At a fixed time t=t1, we have a random variable


X(t1). Similarly, we have
X(t2), .., X(tk).
X(t1) can be characterized by its distribution
function,

We can also consider the joint distribution


function,

Discrete and continuous cases:


States X(t) (i.e. time t) may be
discrete/continuous
State space I may be discrete/continuous

Classification of Stochastic Processes

Four classes of stochastic processes:

discrete-state process chain


discrete-time process stochastic sequence {Xn |
n T} (e.g., probing a system every 10 ms.)

Example: a Queuing System


Queue (waiting station)
Random arrivals
Inter arrival time
distribution fn. FY

m
servers

Service time
distribution fn. FS

Interarrival times Y1, Y2, (common dist. Fn. FY)

Service times: S1, S2,

(iid with a common cdf FS)

Notation for a queuing system: FY /FS/m

Some interarrival/service time distributions types are:


M: Memoryless (i.e., EXP)
D: Deterministic
E : k-stage Erlang etc.
k
H : k-stage Hyper exponential distribution
k
G: General distribution
GI: General independent inter arrival times

Discrete/Continuous Stochastic
Processes
Nk: Number of jobs waiting in the system at the time of k th
jobs departure Stochastic process {Nk| k=1,2,}:

Discrete time, discrete state


Nk

Discret
e

Discrete

Continuous Time, Discrete Space


X(t): Number of jobs in the system at time t. {X(t) | t T}
forms a continuous-time, discrete-state stochastic process,
with,
X(t)

Discrete

Continuo

Discrete Time, Continuous Space


Wk: waiting time for the kth job. Then {Wk | k T} forms a
Discrete-time, Continuous-state stochastic process, where,
Wk

Continuo
us

Discrete

Continuous Time, Continuous Space

Y(t): total service time for all jobs in the system at time t. Y(t) forms a continuous-time,
continuous-state stochastic process, Where,

Y(t)

Further Classification

(1st order distribution)

(2nd order distribution)

Similarly, we can define nth order distribution:

Formidable task to provide nth order distribution


for all n.

Further Classification (contd.)

Can the nth order distribution be simplified?


Yes. Under some simplifying assumptions:

Independence

As example, we have the Renewal Process

Discrete time independent process {Xn | n=1,2,} (X1, X2, ..


are iid, non-negative rvs), e.g., repair/replacement after a
failure.

Markov process introduces a limited form of dependence

Markov Process

Stochastic proc. {X(t) | t T} is Markov if for any t 0 < t1<


< tn< t, the conditional distribution satisfies the Markov
property:

Markov Process

We will only deal with discrete state Markov


processes i.e., Markov chains
In some situations, a Markov chain may also exhibit
time-homogeneity
Future of process (probabilistically) determined by
its current state, independent of how it reached this
particular state; but in a non homogeneous case,
current time can also determine the future.
For a homogeneous Markov chain current time is
also not needed to determine the future.
Let Y: time spent in a given state in a hom. CTMC

Homogeneous CTMC-Sojourn
time

Since Y, the sojourn time, has the memoryless prop.

This result says that for a homogeneous continuous time Markov chain,
sojourn time in a state follows EXP( ) distribution (not true for non-hom
CTMC)
Hom. DTMC sojourn time dist. Is geometric.
Semi-Markov process is one in which the sojourn time in a state is generally
distributed.

Bernoulli Process

A sequence of iid Bernoulli rvs, {Yi | i=1,2,3,..}, Yi =1 or 0

{Yi} forms a Bernoulli Process, an example of a renewal


process.
Define another stochastic process , {Sn | n=1,2,3,..},
where Sn = Y1 + Y2 ++ Yn (i.e. Sn :sequence of partial
sums)
S = S
n
n-1+ Yn (recursive form)

P[Sn = k | Sn-1= k] = P[Yn = 0] = (1-p) and,

P[Sn = k | Sn-1= k-1] = P[Yn = 1] = p

{Sn |n=1,2,3,..}, forms a Binomial process, an


example of a homogeneous DTMC

Renewal Counting Process

Renewal counting process: # of


renewals (repairs, replacements,
arrivals) by time t: a continuous time
process:
If time interval between two renewals
follows EXP distribution, then Poisson
Process

Note:
For a fixed t, N(t) is a random
variable (in this case a discrete
random variable known as the Poisson
random variable)
The family {N(t), t 0} is a stochastic
process, in this case, the
homogeneous Poisson process
{N(t), t 0} is a homogeneous CTMC
as well

Poisson Process

A continuous time, discrete state process.


N(t): no. of events occurring in time (0, t]. Events may be,
1.
# of packets arriving at a router port
2.
# of incoming telephone calls at a switch
3.
# of jobs arriving at file/compute server
4.
Number of component failures
Events occurs successively and that intervals between
these successive events are iid rvs, each following EXP( )
1.
2.

: arrival rate (1/ : average time between arrivals)


: failure rate (1/ : average time between failures)

Poisson Process (contd.)

1.

N(t) forms a Poisson process provided:


1.
N(0) = 0
2.
Events within non-overlapping intervals are
independent
3.
In a very small interval h, only one event may occur
(prob. p(h))
Letting, pn(t) = P[N(t)=n],

For a Poisson process, interarrival times follow EXP( )


(memoryless) distribution.
E[N(t)] = Var[N(t)] = t ; What about E[N(t)/t], as t
infinity?

Merged Multiple Poisson Process


Streams

Consider the system,


+

Proof: Using z-transform. Letting, = t,

Decomposing a Poisson Stream

Decompose a Poisson process using a prob. switch

N arrivals decomposed into {N1, N2, .., Nk}; N= N1+N2, ..,


+Nk

Cond. pmf

Since,

The uncond. pmf

Generalizing the Poisson


Process
Poisson Process

Non-Homogeneous Poisson
Process (NHPP)

Non-Homogeneous Poisson
Process (NHPP)
If the expected number of events per unit time, , changes
with age (time), we have a non-homogeneous Poisson model.
We assume that:

1.
If 0 t, the pmf of N(t) is given by:

P N t k m t / k!e m t
k

k 0, 1, 2, ...

where m(t) 0 is the expected number of events in the


time period [0, t]

are

2.

Counts of events in non-overlapping time periods

mutually independent.
m(t) : the mean value function. (x) :the time-dependent
rate of occurrence of events or time-dependent failure rate

m(t ) 0 (x) dx
t

NHPP(cont.)

Generalizing Poisson
Process
Poisson Process

Non-Homogeneous Poisson
Process (NHPP)

Renewal Counting
Process

Renewal Counting Process

Poisson process EXP( ) distributed interarrival times.


What if the EXP( ) assumption is removed renewal proc.
Renewal proc. : {Xi | i=1,2,} (Xis are iid non-EXP rvs)
Xi : time gap between the occurrence of (i-1) st and ith event

Sk = X1 + X2 + .. + Xk time to occurrence of the kth


event.

N(t)- Renewal counting process is a discrete-state,


continuous-time stochastic process. N(t) denotes no. of
renewals in the interval (0, t].

Renewal Counting Processes (contd.)

Sn

For N(t), what is P(N(t) = n)?


tn

More arrivals
possible

Renewal Counting Process


Expectation

Let, m(t) = E[N(t)]. Then, m(t) = mean


no. of arrivals in time (0,t]. m(t) is
called the renewal function.

Renewal Density Function

Renewal density function:

For example, if the renewal interval X is


EXP(), then

d(t) = , t >= 0 and m(t) = t , t >= 0.


P[N(t)=n] = e t ( t)n/n! i.e Poisson pmf
Fn(t) will turn out to be n-stage Erlang

Alternating Renewal
Process
I(t)

Operating

Restoration

0
Time

Where:
Failure times T , T , are mutually independent with a
1
2
common distribution function W
Restoration times D , D , are mutually independent with
1
2
a common distribution function G
The sequences {T } and {D } are independent
n
n

Availability Analysis

Availability: is defined is the ability of a system to


provide the desired service.
If no
repair/replacement,Availability(t)=Reliability(t)
If repairs are possible, then above is pessimistic.
MTBF

T1

D1

T2

D2

T3

D3

T4

D4 .

MTBF = E[Di+Ti+1] = E[Ti+Di]=E[Xi]=MTTF+MTTR

Availability Analysis (contd.)


renewal

Repair is completed with in this interval

Two mutually exclusive situations:


1. System does not fail before time t A(t)
= R(t)
2. System fails, but the repair is completed
before time t
Therefore, A(t) = sum of these two
probabilities

Availability Expression

dA(x) : Incremental availability


Repair is completed with in this interval

x x+dx

Renewed life time >= (t-x)

dA(x) = Prob(that after renewal, life time is >


(t-x) & that the renewal occurs in the interval
(x,x+dx])

Availability Expression (contd.)

A(t) can also be expressed in the Laplace domain.

Since, R(t) = 1-W(t) or LR(s) = 1/s LW(s) = 1/s


Lw(s)/s

What happens when t becomes very large?

However,

Availability, MTTF and MTTR

Steady state availability A is:

Taking the expression of sLA(s) and


taking the limit via LHospital rule and
using the moment generating property
of the LT, we get the required result for
the steady-state
A=MTTF/(MTTF+MTTR)

Availability Example

Assuming EXP( ) density fn for g(t) and


w(t)

Generalizing Poisson
Process
Poisson Process
Renewal Counting
Process

Compound Poisson
Process
Non-Homogeneous Poisson
Process (NHPP)

Non-Homogeneous
Continuous Time
Markov Chain

Bernoulli Process
Homogeneous
Continuous Time
Markov Chain
Homogeneous
Discrete Time
Markov Chain

Semi-Markov
Process
Markov Regenerative
Process

S-ar putea să vă placă și