Renewal theory and its applications

Renewal theory and its

applications
Stella Kapodistria and Jacques Resing
September 11th, 2012
ISP

Renewal theory and its applications

Definition of a Renewal process

Renewal theory and its applications

If we substitute the Exponentially distributed inter-arrival times of the
Poisson process by any arbitrary sequence of iid r.v. {X1 , X2 , . . .} we can
generalize the definition of the counting process.
Definition
If the sequence of nonnegative random variables {X1 , X2 , . . .} is
independent and identically distributed, then the counting process
{N(t), t 0} is said to be a renewal process.
For any sequence of iid r.v. we can define a counting process as
N(t) = max{n : Sn t} with

n
X

Xj = Sn

j=1

The
(i)
(ii)
(iii)
(iv)

definition implies:
N(t) 0
N(t) is integer valued
If s < t, then N(s) N(t)
For s < t, N(t) N(s) equals the number of events in (s, t].

Renewal theory and its applications

Definition of a Renewal process

Poisson process
Definition
The counting process {N(t), t 0} is called a Poisson process with rate
, if {X1 , X2 , . . .} are iid having the exponential distribution at rate .
In this case we know that:
(i) N(t) Poisson(t) and m(t) = E [N(t)] = t
Pn
(ii)
j=1 Xj Erlang (, n) and E [Sn ] = n/
(iii) By the Strong Law of Large Numbers (SLLN) it follows that,
1
Sn
, n (w.p.1)
n

Renewal theory and its applications

Distribution of N(t)

Distribution of N(t)
The distribution of N(t) can be obtained, at least in theory, by noting
that
N(t) n Sn t
Then,
P[N(t) = n] = P[N(t) n] P[N(t) n + 1]
= P[Sn t] P[Sn+1 t]
Now since the random
Pnvariables Xi , i 1, are iid having a distribution F ,
it follows that Sn = j=1 Xj is distributed as Fn , the n-fold convolution
of F with itself (Section 2.5). Therefore, we obtain
P[N(t) = n] = Fn (t) Fn+1 (t)

Renewal theory and its applications

Distribution of N(t)
The mean-value function

The mean-value function

The mean-value function can be obtained by noting that
N(t) n Sn t
Then,
m(t) = E [N(t)] =
=
=

X
n=1

X
n=1

P[N(t) n]
P[Sn t]
Fn (t)

n=1

The function m(t) is known as the mean-value or the renewal function.

Example
Suppose m(t) = 2t. What is the distribution P[N(10) = n] =?

Renewal theory and its applications

Distribution of N(t)
The mean-value function

The mean-value function

There is one-to-one correspondence between the renewal process and its
mean-value function!
We define m(s)

to be the Laplace-Stieltjes transform of m(t)

Z
m(s)

=
e st m0 (t)dt
0

Then we can prove that

m(s)

(s)
1 (s)

with (s) = E [e sX ] we have denoted the Laplace-Stieltjes transform of

the inter-arrival times {X1 , X2 , . . .}.

Renewal theory and its applications

Distribution of N(t)
The mean-value function

The mean-value function

There is one-to-one correspondence between the renewal process and its
mean-value function!
We want to determine m(t) for t 1. We will prove a basic renewal
equation as follows
Z
m(t) = E [N(t)] =
E [N(t)|X1 = x]f (x)dx
0
Z t
Z
=
E [N(t)|X1 = x]f (x)dx +
E [N(t)|X1 = x]f (x)dx
0
t
Z t
Z
[1 + m(t x)]f (x)dx +
0f (x)dx
=
0
t
Z t
= F (t) +
m(t x)f (x)dx
0

This last equation is called the renewal equation and can sometimes be
solved to obtain the mean-value function.

Renewal theory and its applications

Distribution of N(t)
The mean-value function

Proposition 7.2 E [SN(t)+1 ] = (m(t) + 1)

Proposition (7.2)
E [SN(t)+1 ] = (m(t) + 1)
Remark (Walds Equation)
For any sequence {X1 , X2 , . . .} of iid r.v. with mean E [X ] = and a r.v.
N independent from the sequence {X1 , X2 , . . .} we can easily prove that
N
X
E[
Xi ] = E [N]E [X ] = E [N]
i=1

Renewal theory and its applications

Distribution of N(t)
The mean-value function

Proposition 7.2 E [SN(t)+1 ] = (m(t) + 1)

Proposition (7.2)
E [SN(t)+1 ] = (m(t) + 1)
Proof.
We define E [SN(t)+1 ] = g (t), then
Z

E [SN(t)+1 |X1 = x]f (x)dx

g (t) =
0

Z
E [SN(t)+1 )|X1 = x]f (x)dx +
E [SN(t)+1 |X1 = x]f (x)dx
0
t
Z
Z t
[x + g (t x)]f (x)dx +
xf (x)dx
=
t
0
Z t
=+
g (t x)f (x)dx
Z

Renewal theory and its applications

Distribution of N(t)
The mean-value function

Proposition 7.2 E [SN(t)+1 ] = (m(t) + 1)

Proposition (7.2)
E [SN(t)+1 ] = (m(t) + 1)
Proof.
We define E [SN(t)+1 ] = g (t), then
Z

g (t x)f (x)dx

g (t) = +
0

If we substitute g (t) = (m(t) + 1) yields

Z
0

which completes the proof.

m(t x)f (x)dx

m(t) = F (t) +

Renewal theory and its applications

Distribution of N(t)
The mean-value function

Proposition 7.2 E [SN(t)+1 ] = (m(t) + 1)

Proposition (7.2)
E [SN(t)+1 ] = (m(t) + 1)
Proof.
We define E [SN(t)+1 ] = g (t), then
Z t
g (t) = +
g (t x)f (x)dx
|{z}
0
k(t)

For any known function k(t) the renewal type equation has a unique
solution:
Z t
g (t)= k(t) +
k(t x)m0 (x)dx
0
Z t
(m(t) = F (t) +
m(t x)F (x)dx)
0

Then by setting k(t) = immediately yields the result.

Renewal theory and its applications

Limit Theorems

Limit Theorems
Let {X1 , X2 , . . .} be a sequence of iid r.v. and we define the renewal
process {N(t), t 0} as
N(t) = max{n : Sn t} with

n
X

Xj = Sn

j=1

Let E [Xj ] = .
By the SLLN it follows that,
Sn
, n (w.p.1)
n
Hence, Sn as n . Thus, Sn t for at most a finite number of
values of n, and hence by definition N(t) must be finite. However,
though N(t) < for each t it is true that, with probability 1,
N() = lim N(t) =
t

Renewal theory and its applications

Limit Theorems

Limit Theorems
Proposition ( 7.1)
With probability 1,
N(t)
1
as t
t

Proof.
First of all recall that E [SN(t) ] = E [N(t)]E [X ], hence by the SLLN
SN(t) /N(t) E [X ] = t (w.p.1)
Secondly,
SN(t) t < SN(t)+1
SN(t)+1
SN(t)
t

<
N(t)
N(t)
N(t)
PN(t)
j=1

Xj

N(t)

PN(t)+1
Xj N(t) + 1
SN(t)+1 N(t) + 1
t
j=1

<
=
N(t)
N(t) + 1 N(t)
N(t) + 1
N(t)

Renewal theory and its applications

Limit Theorems

Limit Theorems
Proposition ( 7.1)
With probability 1,
N(t)
1
as t
t

Proof.
First of all recall that E [SN(t) ] = E [N(t)]E [X ], hence by the SLLN
SN(t) /N(t) E [X ] = t (w.p.1)
Secondly,
SN(t) t < SN(t)+1
SN(t)+1
SN(t)
t

<
N(t)
N(t)
N(t)

PN(t) 
PN(t)+1 
>

*

*1
X
X
S

t
N(t)
+
1
N(t)
+
1
j
j
N(t)+1
j=1
j=1



<
= 


N(t)
N(t) + 1 N(t)
N(t) + 1  N(t)
 N(t)

Renewal theory and its applications

Limit Theorems

Limit Theorems
Theorem (Elementary Renewal Theorem)
m(t)
1
as t
t

Theorem (Central Limit Theorem for Renewal Process)

"

#
Z x
2
N(t) t/
1
p

e x /2 dx
lim P
<x =
2
3
t
2
t /
with = E [X ] and 2 = Var [X ].
Remark
2
Var [N(t)]
= 3
t
t

lim

Renewal theory and its applications

Limit Theorems
Example 7.7

Example 7.7
Suppose that potential customers arrive at a single-server bank in
accordance with a Poisson process having rate . Furthermore, suppose
that the potential customer will enter the bank only if the server is free
when he arrives; if upon arrival the customer sees the bank teller
occupied he will immediately leave. If we assume that the amount of
time spent in the bank by an entering customer is a random variable
having distribution G , then
(a) what is the rate at which customers enter the bank?
(b) what proportion of potential customers actually enter the bank?
In answering these questions, suppose that at time 0 a customer has just
entered the bank.

Renewal theory and its applications

Limit Theorems
Example 7.9

Example 7.9
Consider the renewal process whose inter-arrival distribution is the
convolution of two exponentials; that is, F = F1 F2 , where
Fi (t) = 1 e i t , i = 1, 2. Imagine that each renewal corresponds to a
new machine being put in use, and suppose that each machine has two
componentsinitially component 1 is employed and this lasts an
exponential time with rate 1 , and then component 2, which functions
for an exponential time with rate 2 , is employed. When component 2
fails, a new machine is put in use (that is, a renewal occurs). Now
consider the process {X (t), t 0} where X (t) is i if a type i component
is in use at time t. Calculate
(a) the probability that the machine in use at time t is using its first
component.
(b) the expected excess time E [Y (t)] := E [SN(t)+1 t].
(c) the mean-value function.
In answering these questions, suppose that at time 0 a component 1 has
just been employed.

Renewal theory and its applications

Limit Theorems
Example 7.10

Example 7.10
Two machines continually process an unending number of jobs. The time
that it takes to process a job on machine 1 is a Gamma random variable
with parameters n = 4, = 2, whereas the time that it takes to process
a job on machine 2 is Uniformly distributed between 0 and 4.
Approximate the probability that together the two machines can process
at least 90 jobs by time t = 100.

Renewal theory and its applications

Renewal Reward Processes

Renewal Reward Processes

Consider a renewal process {N(t), t 0} having inter-arrival times
{X1 , X2 , . . .} and suppose that each time a renewal occurs we receive a
reward. We denote by Rn , the reward earned at the time of the n-th
renewal. We assume that the Rn , n 1, are iid r.v. If we let
N(t)

R(t) =

Rn

n=1

then R(t) represents the total reward earned by time t. Let

E [R] = E [Rn ], E [X ] = E [Xn ]
Proposition (7.3)
If E [R] < and E [X ] < , then
E [R]
a) w.p.1 lim R(t)
b) lim
t = E [X ]
t

E [R(t)]
t

E [R]
E [X ]

Renewal theory and its applications

Renewal Reward Processes
Example 7.11

Example 7.11
Suppose that potential customers arrive at a single-server bank in
accordance with a Poisson process having rate . However, suppose that
the potential customer will enter the bank only if the server is free when
he arrives. That is, if there is already a customer in the bank, then our
arriver, rather than entering the bank, will go home. If we assume that
the amount of time spent in the bank by an entering customer is a
random variable having distribution G , and
that each customer that enters makes a deposit and that the amounts
that the successive customers deposit in the bank are iid r.v. having a
common distribution H, then the rate at which deposits accumulate
that is, limt (total deposits by the time t)/t is given by
H
E [deposits during a cycle]
=
E [time of cycle]
G + 1/
where G + 1/ is the mean time of a cycle, and H is the mean of the
distribution H.

Renewal theory and its applications

Renewal Reward Processes
Example 7.16

Example 7.16 (The Average Age of a Renewal Process)

Consider a renewal process having inter-arrival distribution F and define
A(t) := t SN(t) at time t. We are interested in
Rs
A(t)dt
= average value of age
lim 0
s
s
Rs
Assume that 0 A(t)dt represents our total earnings by time s:
Rs
A(t)dt
E [reward during a renewal cycle]
lim 0

s
s
E [time of a renewal cycle]
Now since the age of the renewal process a time t into a renewal cycle is
just t, we have
Z X
reward during a renewal cycle =
tdt
0

where X F is the time of the renewal cycle. Hence, we have that

Rs
A(t)dt
E [X 2 ]/2
average value of age = lim 0
=
s
s
E [X ]

Renewal theory and its applications

Regenerative Processes

Regenerative Processes
Consider a stochastic process {X (t), t 0} with state space
{0, 1, 2, . . .}, having the property that there exist time points at which
the process (probabilistically) restarts itself. That is, suppose that with
probability one, there exists a time T1 , such that the continuation of the
process beyond T1 is a probabilistic replica of the whole process starting
at 0. Note that this property implies the existence of further times
T2 , T3 , . . ., having the same property as T1 . Such a stochastic process is
known as a regenerative process.
From the preceding, it follows that T1 , T2 , . . ., constitute the arrival
times of a renewal process, and we shall say that a cycle is completed
every time a renewal occurs.
Example
(1) A renewal process is regenerative, and T1 represents the time of the
first renewal.
(2) A recurrent Markov chain is regenerative, and T1 represents the time
of the first transition into the initial state.

Renewal theory and its applications

Regenerative Processes

Regenerative Processes
We are interested in determining the long-run proportion of time that a
regenerative process spends in state j.
To obtain this quantity, let us imagine that we earn a reward at a rate 1
per unit time when the process is in state j and at rate 0 otherwise. That
is, if I (s) represents the rate at which we earn at time s, then
(
1, if X (s) = j
I (s) =
0, if X (s) 6= j
and
Z
total reward earned by t =

I (s)ds
0

Proposition (7.4)
For a regenerative process, the long-run proportion of time in state j
=

E [amount of time in j during a cycle]

E [time of a cycle]

Renewal theory and its applications

Regenerative Processes
Example 7.18

Example 7.18
Consider a positive recurrent continuous time Markov chain that is
initially in state i. By the Markovian property, each time the process
reenters state i it starts over again. Thus returns to state i are renewals
and constitute the beginnings of new cycles. By Proposition 7.4, it
follows that
the long-run proportion of time in state j
=

E [amount of time in j during an ii cycle]

E [Ti,i ]

where E [Ti,i ] represents the mean time to return to state i. If we take j

to equal i, then we obtain
proportion of time in state i =

1/vi
E [Ti,i ]

Renewal theory and its applications

Regenerative Processes
Example 7.19

Example 7.19 (A Queueing System with Renewal Arrivals)

Consider a waiting time system in which customers arrive in accordance
with an arbitrary renewal process and are served one at a time by a single
server having an arbitrary service distribution. If we suppose that at time
0 the initial customer has just arrived, then {X (t), t 0} is a
regenerative process, where X (t) denotes the number of customers in the
system at time t. The process regenerates each time a customer arrives
and finds the server free.

Renewal theory and its applications

Regenerative Processes
Alternating Renewal Processes

Alternating Renewal Processes

Consider a system that can be in one of two states: on or off. Initially it
is on, and it remains on for a time Z1 ; it then goes off and remains off for
a time Y1 . It then goes on for a time Z2 ; then off for a time Y2 ; then on,
and so on.
We suppose that the random vectors (Zn , Yn ), n 1 are iid; but we allow
Zn and Yn to be dependent. In other words, each time the process goes
on, everything starts over again, but when it then goes off, we allow the
length of the off time to depend on the previous on time.
We are concerned with Pon , the long-run proportion of time that the
system is on
E [on]
E [Z ]
=
Pon =
E [Y ] + E [Z ]
E [off] + E [on]

Renewal theory and its applications

Regenerative Processes
Example 7.23

Example 7.23 (The Age of a Renewal Process)

Suppose we are interested in determining the proportion of time that the
age of a renewal process is less than some constant c. To do so, let a
cycle correspond to a renewal, and say that the system is on at time t if
the age at t is less than or equal to c, and say it is off if the age at t is
greater than c. In other words, the system is on the first c time units of
a renewal interval, and off the remaining time. Hence, letting X denote a
renewal interval, we have
E [min(X , c)]
E [X ]
R
P[min{X , c} > x]dx
= 0
E [X ]
Rc
P[X > x]dx
= 0
E [X ]
Rc
(1 F (x))dx
= 0
E [X ]

proportion of time age is less than c =

Renewal theory and its applications

Regenerative Processes
Example 7.23

Summary Renewal theory

1 Definition of a Renewal process
2 Distribution of N(t)

The mean-value function

3 Limit Theorems

Example 7.7
Example 7.9
Example 7.10
4 Renewal Reward Processes

Example 7.11
Example 7.16
5 Regenerative Processes

Example 7.18
Example 7.19
Alternating Renewal Processes
Example 7.23

Renewal theory and its applications

Regenerative Processes
Example 7.23

Exercises
Introduction to Probability Models
Harcourt/Academic Press, San Diego, 9th ed., 2007
Sheldon M. Ross
Chapter 7
Sections 7.1, 7.2, 7.3, 7.4, 7.5
Exercises: 2, 4, 5, 10, 11, 12, 15, 19, 26, 32, 37, 44