1.

A downtown car service station has facilities for a maximum of 4 cars being serviced or
waiting for service on its premises, Past experience indicates that no potential customers
to join the queue once these 4 places are filled. The arrival rate of customers is 24 per
hour during off-peak hours, and the input process is approximately Poisson. The service
times are exponential with a mean of 3 minutes. Assume customers balk in the following
manner: All customers join fewer than 2 cars are in the service station. If 2 cars are in the
service station, only 75% of total customers decided to join. If 3 cars are in the service
station, only 50% join, and if 4 cars are in the station, no additional customers join.
a) Derive the state balance equations on the basis of this diagram. Find the
steady state probabilities.
b) What is the average idle time of the attendant? What is the average
number of customers lost?
c) What is the number of customers waiting for the service? What is the
average waiting time per customer?
2. Barges arrive at a river dam lock system at an average rate of 4 per hour. Each lock deals
with traffic in only one direction to reduce the danger of collisions. The arrival pattern is
approximately Poisson. The time to enter a barge into the lock, raise the barge, vacate the
lock, and the lower the water level in the lock approximately normally distributed, with a
mean of 10 minutes and standard deviation of 3 minutes.
a) Find the average fraction of time the lock is busy.
b) Find the average number of barges waiting to be raised and the average
waiting time per request.
c) If the standard deviation is 1 minute, how would this tend to change your
answer in b?
3. An operator supervised 2 machines. Each has an arrival rate of per hour, with negative
exponential interarrival times. Service times follow an Erlang distribution with k=2
phases and a mean service time of 1/.
a) What is the average idle time per operator? What is the busy time?
b) What is the average machine down (=being serviced or waiting for
service)? What is the average arrival rate? What is the average downtime
per service request? What is the total downtime per hour for both
machines?
4. An average of 10 machinists per hour arrive seeking tools. At present, the tool center is
staffed by a clerk who is paid $6 per hour and who takes an average of 5 minutes to
handle each request for tools. Since each machinist produces $10 worth of goods per
hour, each hour that a machinist spends at the tool center costs the company $10. The
company is deciding whether or not it is worthwhile to hire (at $4 per hour) a helper for
the clerk. If the helper is hired the clerk will take an average of only 4 minutes to process
requirements for tools. Assume that service and arrival times are exponential. Should the
helper be hired?
5. Consider a game ofladder climbing. There are 5 levels in the game, level 1 is the lowest
(bottom) and level 5 is the highest (top). A player starts at the bottom. Each time, a fair
coin is tossed. If it turns up heads, the player moves up one rung. If tails, the player
moves down to the very bottom. Once at the top level, the player moves to the very
bottom if a tail turns up.
a) Find the transition probability matrix.
b) Find the two-step transition probability matrix.
c) Find the steady-state distribution of the Markov chain.

6. A system has three possible states, 0, 1 and 2. Every hour it makes a transition to a
different state, which is determined by a coin flip. For example, from state 0, it makes a
transition to state 1 or state 2 with probabilities 0.5 and 0.5.
a) Find the transition probability matrix.
b) Find the three-step transition probability matrix.
c) Find the steady-state distribution of the Markov chain.

7. A computer system can operate in two different modes. Every hour, it remains in the
same mode or switches to a different mode according to the transition probability matrix

P = 0:4 0:6
0:6 0:4
a) Compute the 2-step transition probability matrix.
b) If the system is in Mode I at 5:30 pm, what is the probability that it will be in Mode I
at 8:30 pm on the same day?

8. The pattern of sunny and rainy days on the planet Rainbow is a homogeneous Markov
chain with two states. Every sunny day is followed by another sunny day with
probability 0.8. Every rainy day is followed by another rainy day with probability 0.6.
a) Today is sunny on Rainbow. What is the chance of rain the day after
tomorrow?
b) Compute the probability that April 1 next year is rainy on Rainbow.

Markov Chains are frequently