Modeling and Simulation (408360)

Eng. Ghada Al-Mashaqbeh

Output Analysis
Case 1: Terminating Simulation
Consider the simulation example found in Chapter 1 (single server service system). We have made 10 runs
of the system simulation with stopping criteria of computing the queuing delay of the first 9 customers
entering the system. We are interested in measuring the queuing delay of customers (discrete quantity). The
obtained results are as shown below (where D1 is the queuing delay of customer 1 and so on):

Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10

D1
10.8
11.4
9.4
10.5
6
12.3
8
7.6
9.6
8.7

D2
8
9.2
8.7
8.6
4.7
13.5
9.2
9.9
10.7
9.5

D3
5.6
7.6
9.9
13.5
8.9
9.8
8.7
5.5
8.5
12.3

D4
11.9
10.5
6.7
6
10.8
8
8.6
11.4
10.4
6.6

D5
12.1
8.6
7.8
8.8
9.2
7.6
7.7
10.2
11.2
9

D6
4.5
13.3
8.9
11.5
5.6
12.1
11.7
9.8
10.8
10.3

D7
3
6
4.7
12.1
11.8
8.6
9.9
11.5
12.5
8.3

D8
10.2
8.8
14.3
11.9
8.4
7.8
10.1
13.5
10.2
12.9

D9
9.8
11.5
5.8
5.6
10.5
8.8
9.1
8.6
9.6
8.5

1. Compute the mean and variance of the output using :

a. Point estimator.
b. Quantile estimator with p = 0.6 (i.e. 60%).
2. Compute the 90% and 95% confidence interval (CI) for both:
a. Point estimator.
b. Quantile estimator with p = 0.6 (i.e. 60%).
3. Based on the point estimator results and with = 0.05 compute the needed number of simulation
runs (or replications) to get the following:
a. An absolute error of 0.3
b. A relative error of 0.04
c. The sequential procedure with relative error of 0.04

Solution:
1 - a) Mean and variance computation using point estimator:
i) Compute the average of each simulation run:
1 n
1 9
10.8 + 8 + 5.6 + 11.9 + 12.1 + 4.5 + 3 + 10.2 + 9.8
Run 1 1 = Yi = Di =
= 8.43
n i =1
9 i =1
9
1 n
1 9
11.4 + 9.2 + 7.6 + 10.5 + 8.6 + 13.3 + 6 + 8.8 + 11.5
Run 2 2 = Yi = Di =
= 9.66
n i =1
9 i =1
9
Complete for all runs you will obtain the following results:

Modeling and Simulation (408360)

Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10

Eng. Ghada Al-Mashaqbeh

Average Delay
8.43
9.66
8.47
9.83
8.43
9.83
9.22
9.78
10.39
9.57

ii) Compute the average of the average which is the mean of obtained results for all runs:
8.43 + 9.66 + 8.47 + 9.83 + 8.43 + 9.83 + 9.22 + 9.78 + 10.39 + 9.57
1 n
1 10
= Yi = i =
= 9.36
n i =1
10
10 i =1

iii) Now compute the variance of the output (Note that R is the number of simulation runs):
1 R
S2 =
(Yi. Y.. ) 2

R 1 i =1

S2 =

1 10
(Yi. Y.. ) 2
10 1 i =1

2
2
2
2
2
2
1 (8.43 9.36) + (9.66 9.36) + (8.47 9.36) + (9.83 9.36) + (8.43 9.36) + (9.83 9.36)
S =

9 + (9.22 9.36) 2 + (9.78 9.36) 2 + (10.39 9.36) 2 + (9.57 9.36) 2

S 2 = 0.484

1 - b) Mean and variance computation using 60% quantile estimator:

i) Compute the 60% quantile mean of each simulation run:
First: Sort the results of each run in an ascending order as follows:
Rank
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10

1
3
6
4.7
5.6
4.7
7.6
7.7
5.5
8.5
6.6

2
4.5
7.6
5.8
6
5.6
7.8
8
7.6
9.6
8.3

3
5.6
8.6
6.7
8.6
6
8
8.6
8.6
9.6
8.5

4
8
8.8
7.8
8.8
8.4
8.6
8.7
9.8
10.2
8.7

5
9.8
9.2
8.7
10.5
8.9
8.8
9.1
9.9
10.4
9

6
10.2
10.5
8.9
11.5
9.2
9.8
9.2
10.2
10.7
9.5

7
10.8
11.4
9.4
11.9
10.5
12.1
9.9
11.4
10.8
10.3

8
11.9
11.5
9.9
12.1
10.8
12.3
10.1
11.5
11.2
12.3

9
12.1
13.3
14.3
13.5
11.8
13.5
11.7
13.5
12.5
12.9
2

Modeling and Simulation (408360)

Eng. Ghada Al-Mashaqbeh

Second: The mean for each run will be (R*p)th smallest value which is for each run = 9*0.6 = 5.4. Lets
take it as the 5th smallest value.
So, 60% mean for each run is:

Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10

60% mean
9.8
9.2
8.7
10.5
8.9
8.8
9.1
9.9
10.4
9

ii) Compute the average of the average which is the 60% mean of obtained results for all runs:
First: Sort the means of all runs in an ascending order as follows:
Rank
1
2
3
4
5
6
7
8
9
10

60% mean
8.7
8.8
8.9
9
9.1
9.2
9.8
9.9
10.4
10.5

Second: The mean for each run will be (R*p)th smallest value which is for each run = 10*0.6 = 6. So, we
will take it as the 6th smallest value.
So, 60% mean for our output = 9.2
iii) Now compute the variance of the output (Note that R is the number of simulation runs):
1 R
S2 =
(Yi. Y.. ) 2
R 1 i =1
1 10
(Yi. Y.. ) 2
S2 =

10 1 i =1
2
2
2
2
2
2
1 (9.8 9.2) + (9.2 9.2) + (8.7 9.2) + (10.5 9.2) + (8.9 9.2) + (8.8 9.2)
S =

9 + (9.1 9.2) 2 + (9.9 9.2) 2 + (10.4 9.2) 2 + (9. 9.2) 2

S 2 = 0.503

Modeling and Simulation (408360)

Eng. Ghada Al-Mashaqbeh

2 - a) 90% and 95% CI computation using point estimator:

i) CI is computed as follows:

X(n) t n 1,1 / 2

s 2 ( n)
,
n

where X(n) is
n (number of runs) = 10
t is the criritcal values of the normal distribution found at Table T.1 at page 716 of the textbook
n-1 is the degree of freedom = 9
First: 95% CI:

95% CI = 0.05
0.484
9.36 t 9,0.975
10

9.36 2.262 *

0.484
10

9.36 0.498
95% CI is [8.862, 9.858]
First: 90% CI:

90% CI = 0.1
0.484
9.36 t 9,0.95
10
9.36 1.833 *

0.484
10

9.36 0.403
90% CI is [8.957, 9.763]

2 - b) 90% and 95% CI computation using 60% quantile estimator:

i) CI is computed as follows:
p l = p z1 / 2

p (1 p )
R 1

pu = p + z1 / 2

p (1 p )
R 1

where p = 0.6 (from the 60%)

R (number of runs) = 10
4

Modeling and Simulation (408360)

Eng. Ghada Al-Mashaqbeh

z is the critical values of the normal distribution found at Table T.1 at page 716 of the textbook when degree
of freedom in infinity.
First: 95% CI:

95% CI = 0.05
p l = p z1 / 2

p (1 p )
R 1

p l = 0.6 z 0.975

0.6(1 0.6)
10 1

p l = 0.6 1.96 *

0 .6 * 0 .4
10 1

p l = 0.28
pu = p + z1 / 2

p (1 p )
R 1

pu = 0.6 + z 0.975

0.6(1 0.6)
10 1

pu = 0.6 + 1.96 *

0 .6 * 0 .4
10 1

pu = 0.92

Then multiply P-lower and P-upper with R to get: 2.8 and 9.2
The 95% CI is the values between the 3rd and 9th smallest values which is [8.9, 10.4] from the table in part
1-b.
Second: 90% CI:

95% CI = 0.1
p l = p z1 / 2

p (1 p )
R 1

p l = 0.6 z 0.95

0.6(1 0.6)
10 1

p l = 0.6 1.645 *

0 . 6 * 0 .4
10 1

p l = 0.33

Modeling and Simulation (408360)

pu = p + z1 / 2

p (1 p )
R 1

pu = 0.6 + z 0.95

0.6(1 0.6)
10 1

pu = 0.6 + 1.645 *

Eng. Ghada Al-Mashaqbeh

0 .6 * 0 .4
10 1

pu = 0.87

Then multiply P-lower and P-upper with R to get: 3.3 and 8.7
The 90% CI is the values between the 3rd and 9th smallest values which is [8.9, 10.4] from the table in part
1-b.

3 - a) Number of runs with an absolute error = 0.3:

The absolute error means that = 0.3 , to find n we use the following equation:

S 2 ( n)
na* ( ) = min i n : t i 1,1 / 2

i

To use this equation (remember that = 0.05 ):

i) Start with n runs and compute from the results both the mean and then the variance. Remember that in our
question we already have the results of 10 runs and we have already computed the variance S 2 (n) = 0.484 .
ii) set i = n = 10 and apply the above equality relation:
t i 1,1 / 2
t 9, 0.975

S 2 ( n)

i
0.484
0 .3
10

0.484
0 .3
10
0.497 0.3 No
2.262 *

So, we must increase i and repeat the above step.

iii) set i = 11 and apply the above equality relation:
t i 1,1 / 2

S 2 ( n)

t10,0.975

0.484
0.3
11

0.484
0.3
11
0.467 0.3 No
2.228 *

Modeling and Simulation (408360)

Eng. Ghada Al-Mashaqbeh

So, we must increase i and repeat the above step.

iv) set i = 14 and apply the above equality relation:
t i 1,1 / 2

S 2 ( n)

t13,0.975

0.484
0.3
14

0.484
0.3
14
0.401 0.3 No
2.16 *

So, we must increase i and repeat the above step.

iv) set i = 18 and apply the above equality relation:
t i 1,1 / 2

S 2 ( n)

t17 ,0.975

0.484
0.3
18

0.484
0 .3
18
0.346 0.3 No
2.11 *

So, we must increase i and repeat the above step.

v) set i = 20 and apply the above equality relation:
t i 1,1 / 2

S 2 ( n)

t19,0.975

0.484
0 .3
20

0.484
0 .3
20
0.326 0.3 No
2.093 *

So, we must increase i and repeat the above step.

v) set i = 23 and apply the above equality relation:
t i 1,1 / 2

S 2 ( n)

t 22, 0.975

0.484
0 .3
23

0.484
0 .3
23
0.3 0.3 Yes
2.074 *

Modeling and Simulation (408360)

Eng. Ghada Al-Mashaqbeh

So, we will stop here and the needed number of runs to obtain an absolute error of 0.3 is:
n * = 23

3 - b) Number of runs with a relative error = 0.04:

The relative error means that = 0.04 , to find n we use the following equation:

S 2 ( n)
t

i 1,1 / 2

nr* ( ) = min i n :

1+
X ( n)

To use this equation (remember that = 0.05 ):

i) Start with n runs and compute from the results both the mean and then the variance. Remember that in our
question we already have the results of 10 runs and we have already computed the mean X ( n) = 9.36 the
variance S 2 (n) = 0.484 .
ii) set i = n = 10 and apply the above equality relation:
S 2 ( n)

1+
X ( n)

t i 1,1 / 2

0.484
0.04
10

9.36
1 + 0.04
0.484
2.262 *
10
0.038
9.36
0.053 0.038 No
t 9, 0.975

So, we must increase i and repeat the above step.

ii) set i = 12 and apply the above equality relation:

Modeling and Simulation (408360)

Eng. Ghada Al-Mashaqbeh

S 2 ( n)

1+
X ( n)

t i 1,1 / 2

0.484
12 0.04
9.36
1 + 0.04
0.484
2.201 *
12 0.038
9.36
0.047 0.038 No
t11,0.975

So, we must increase i and repeat the above step.

iii) set i = 15 and apply the above equality relation:
S 2 ( n)

1+
X ( n)

t i 1,1 / 2

0.484
0.04
15

9.36
1 + 0.04
0.484
2.145 *
15
0.038
9.36
0.041 0.038 No
t14, 0.975

So, we must increase i and repeat the above step.

iv) set i = 18 and apply the above equality relation:
S 2 ( n)

1+
X ( n)

t i 1,1 / 2

0.484
0.04
18

9.36
1 + 0.04
0.484
2.11 *
18
0.038
9.36
0.037 0.038 Yes
t17 , 0.975

Modeling and Simulation (408360)

Eng. Ghada Al-Mashaqbeh

So, we will stop here and the needed number of runs to obtain a relative error of 0.04 is:
n * = 18

3 - c) Number of runs using sequential procedure with a relative error = 0.04:

The relative error means that = 0.04 , to find n we use the following algorithm:
1. Make c runs of the simulation and set n = c.

S 2 ( n)
2. Compute X (n) and b = t n 1,1 / 2
from the results of the c runs.
n
3. If b / X (n) stop and return n, else increment n by 1, make an additional simulation run
and return to step 2.
To use this algorithm (remember that = 0.05 ):
i) Start with c runs and compute from the results both the mean and then the variance. Remember that in our
question we already have the results of 10 runs and we have already computed the mean X ( n) = 9.36 the
variance S 2 (n) = 0.484 .
ii) set n = c = 10 and apply the rest of the algorithm steps:

b = t n 1,1 / 2

S 2 ( n)
0.484
0.484
= t 9,0.975
= 2.262 *
= 0.498
n
10
10

Then
b

X ( n)
0.498
0.04
9.36
0.053 0.04 No
Increment n by 1 and repeat.
ii) set n = 11 and apply the rest of the algorithm steps:
make another simulation run and assume that we have obtained
Run 11 11 = 9.4
Now recompute the mean and the variance using the point
estimator:
8.43 + 9.66 + 8.47 + 9.83 + 8.43 + 9.83 + 9.22 + 9.78 + 10.39 + 9.57 + 9.4
1 n
1 11
= X (n) = Yi = i =
= 9.364
n i =1
11
11 i =1
1 R
S2 =
(Yi. Y.. ) 2

R 1 i =1
S2 =

1 11
(Yi. Y.. ) 2

11 1 i =1

S2 =

2
2
2
2
2
2
1 (8.43 9.364) + (9.66 9.364) + (8.47 9.364) + (9.83 9.364) + (8.43 9.364) + (9.83 9.364)

10 + (9.22 9.364) 2 + (9.78 9.364) 2 + (10.39 9.364) 2 + (9.57 9.364) 2 + (9.4 9.364) 2

S 2 = 0.436
10

Modeling and Simulation (408360)

b = t n 1,1 / 2

Eng. Ghada Al-Mashaqbeh

0.436
S 2 ( n)
0.436
= t10,0.975
= 2.228 *
= 0.443
11
n
11

Then
b

X ( n)
0.443
0.04
9.364
0.047 0.04 No
Increment n by 1 and repeat.
ii) set n = 12 and apply the rest of the algorithm steps:
make another simulation run and assume that we have obtained
Run 12 12 = 9.1
Now recompute the mean and the variance using the point estimator:
= X (n) = 9.34
S 2 = 0.401

b = t n 1,1 / 2

S 2 ( n)
0.401
0.401
= t11,0.975
= 2.201 *
= 0.402
n
12
12

Then
b

X ( n)
0.402
0.04
9.34
0.043 0.04 No
Increment n by 1 and repeat.
ii) set n = 13 and apply the rest of the algorithm steps:
make another simulation run and assume that we have obtained
Run 13 13 = 9.2
Now recompute the mean and the variance using the point estimator:
= X ( n) = 9.34
S 2 = 0.368

b = t n 1,1 / 2

S 2 ( n)
0.368
0.368
= t12,0.975
= 2.179 *
= 0.367
n
13
13

Then

11

Modeling and Simulation (408360)

Eng. Ghada Al-Mashaqbeh

X ( n)
0.367
0.04
9.34
0.039 0.04 Yes
So, we will stop here and the needed number of runs to obtain a relative error of 0.04 is:
n * = 13

12