PROBLEM # 1 (35 points)

Assume that 30% of those who eat in the Chinese restaurant go to the pizza place next time and 20% eat at
home. From those who eat at the pizza place, 40% go to the Chinese restaurant and 30% eat at home next time.
From those who eat at home, 20% go to the Chinese restaurant and 40% to pizza place next time. We call this
situation a system. This system can be modeled as a discrete-time Markov chain with three states.
(a) (5 points) Define the three states, the time unit, and write down the one-step transition probability matrix.
States: 1 = Chinese restaurant; 2 = pizza place; 3 = Home.
Time unit = 1 day.
0.5 0.3 0.2

P 0.4 0.3 0.3

0.2 0.4 0.4

One-step transition probability matrix:

(b) (5 points) If a family has decided to eat dinner in the pizza place today, what is the probability that they will
eat at home in three days?
0.5 0.3 0.2

0.5 0.3 0.2

0.41 0.32 0.27

0.2 0.4 0.4

0.2 0.4 0.4

0.34 0.34 0.32

P 2 P P 0.4 0.3 0.3 0.4 0.3 0.3 0.38 0.33 0.29

,

0.2

3
p23
row 2 of P 2 column 3 of P 0.38 , 0.33 , 0.29 0.3 0.38 0.2 0.33 0.3 0.29 0.4 0.291
0.4

(c) (12 points) We are interested in the success of these restaurants in terms of their business. If 900 people live in
this town, on the average, how many people go to the pizza place? How many people go to the Chinese
restaurant?
0.5 0.3 0.2

1, 2 , 3 1, 2 , 3 0.4 0.3 0.3

0.2 0.4 0.4

1 0.5 1 0.4 2 0.2 3

Equation (1):

2 0.3 1 0.3 2 0.4 3

Equation (2):
2 1 2

2 1 2 0.7 1 0.5 2

1.3 1 1.5 2

13
1 0.2 3
15

0.5 1 0.4

Equation (1):

1 2 3 1

0.2 3

13
1
15

7.5 5.2
23
1
1
15
150

13
23
30 26 23
1
1
1 1
15
30
30

Normalization:

23
1
30
.

30
79

2
,

26
79

3
,

23
79
.

900 1 341 .77 342

Avg. # of families eating Chinese food for dinner =
families per day,
900 2 296 .2 296
Avg. # of families eating pizza for dinner =
families per day.
(d) (8 points) Ms. Smith is a school teacher living in this town. She likes all types of food. If she is having pizza
today, on the average, how long will it take for her to have pizza again?
m 22

1
79

3.0385
2 26
days.

(e) (10 points) If a family plans to eat at home today, what is the probability that they will eat pizza in two days for
the first time?

1 p 0.4
f 32
32

,
2 0.34
p32
, (from matrix

P 2

 2 p 2 f 1 p1 0.34 0.4 0.3 0.22

f32
32
32
22

PROBLEM # 2 (35 points)

Consider the discrete Markov Chain represented in the following state-transition diagram:

(a) (10 points) Identify the classes and states in each class, and classify the classes and states. If a state is
periodic, indicate its period.
There are 4 classes:
Class 1 = { 1, 2 } : class and states are transient,
Class 2 = { 3 }

: class and state are transient,

Class 3 = { 4 }

: class and states are absorbing,

Class 4 = { 5, 6 } : class is absorbing and states are positive recurrent.

In addition, all classes and states are aperiodic.

(b) (10 points) Reduce the class containing state 5 to a single (aggregate) state, called state 5. Then, compute
the absorption probability from state 3 to state 5.

a 35' p35' p31 a15' p32 a 25' p33 a 35'

3
1
a 35'
4
2

a 35'

1 1
a 35'
2 4

2
3
.

a 34 1 a 35' 1

2 1

3 3

Also,

(c) (15 points) Using the solution of part (b), compute the long-run distribution given that the process starts in
state 3.
For transient states:
1 2 3 0
.
For the absorbing state:
4 a 34

1
3
.

For recurrent states: (considering class 5 separately, also call3d class 4 in part (a)):

P'

5
6

1
3

2
3

5' , '6 5' , '6 P' 5' , '6 10

5'

2
3

1 '
6
3

Equation 1:
1
4
5' '6 '6 '6 '6 1
3
3
Normalization:

'6

3
4

5' 1 '6 1

3 1

4 4
.

5 a 35' 5'

2 1 1

3 4 6
,

6 a 35' '6

2 3 1

3 4 2
.

PROBLEM # 3 (30 points)

Glueco is planning to introduce a new product in three different regions. Present estimates are that the product will
sell well in each region with respective probabilities 0.4, 0.3, and 0.2. The firm has available four top sales
representatives that it can send to any of the three regions. The estimated probabilities that the product will sell
well in each region when 0, 1, 2, 3, and 4 additional sales representatives are assigned to a region are given in the
table below. Glueco wants to maximize the probability that its new product will sell well in all three regions; where
should the company assign its sales representatives? It can be assumed that the sales in the three regions are
independent.
Probability of selling well

No. of additional
sales representatives

Region 1

Region 2

Region 3

0
1
2
3
4

0.4
0.6
0.7
0.85
0.9

0.3
0.4
0.6
0.65
0.75

0.2
0.5
0.7
0.9
0.95

STAGE 3 (INITIALIZATION):
State
s3

Optimal value

Optimal decision

f 3* s 3

x*3 s3

Resulting state
s 4 s3 x*3

0
1
2
3
4

0.2
0.5
0.7
0.9
0.95

0
1
2
3
4

0
0
0
0
0

STAGE 2:
p 2 x 2 f3* s 2 x 2

s2

f 2* s 2

x *2 s 2

s3 s 2 x*2

x2 0

x2 1

x2 2

x2 3

x2 4

0.30.2

0.06

0.30.5

0.40.2

0.15

0.30.7

0.40.5

0.60.2

0.21

0.30.9

0.40.7

0.60.5

0.650.2

0.30

0.30.95

0.40.9

0.60.7

0.650.5

0.750.2

0.42

Place a - in cells corresponding to infeasible decisions.

STAGE 1:
p1 x1 f 2* s1 x1

s1

x1 0

x1 1

x1 2

x1 3

x1 4

0.40.42

0.60.3

0.70.21

0.850.15

0.90.06

f1* s1

x1* s1

s2 s1 x1*

0.18

OPTIMAL SOLUTION: (provide numerical arguments and optimal values for the objective function and decision
variables)

x1* 4 1
,

x*2 3 2
,

x*3 1 1
,

f1* 4 0.18
(maximum probability that the new product will sell well in all three regions).