9th IEEE International Workshop on Performance and Management of Wireless and Mobile Networks, 2013

QoS Analysis and Evaluations: Improving Cellularbased Distance Education

Farnaz Farid, Seyed Shahrestani, and Chun Ruan
School of Computing, Engineering and Mathematics
University of Western Sydney
Sydney, Australia
The coverage and limited throughput issues are mostly
experienced in rural areas and developing regions. These issues
can be partially addressed via efficient utilization of different
technologies such as deploying WiFi hotspots in a 3G network.
These issues can also be dealt with by deploying different
traffic models. For example, allowing multimedia application
in an off-peak hour would help to improve the performance of
applications up to a certain level. The QoS analysis of these
traffic models is necessary to evaluate and enhance the
performance of multimedia applications. To address this
necessity, several studies have been conducted to examine the
multimedia centric performance of wireless and cellular-based
remote services in terms of QoS [2, 4]. However, they have
failed to provide a unified metric for QoS measurement of
wireless and cellular based services in the presence of different
traffic models and applications.
Different studies have already established the importance of
QoS evaluations for network configuration selection purposes
[5, 6]. More specifically, most of these studies aim to provide
users with the ability to use the network configuration with the
best QoS levels at lowest costs. This approach is useful when
users have options to choose from different technologies,
which is the case for most situations in developed countries.
However, the situation in developing countries, or some remote
areas in developed nations, is quite different. Considering this,
our work aims to evaluate QoS of different traffic models
engaging multimedia traffic. It also investigates the constraints
in available communication and networking technologies in
those regions to meet the requirements for these applications.
More specifically, the work aims to address these constraints in
context of distance education scenarios. With this aim we
concentrate specifically on cellular technology based distance
education model. We propose an application-based QoS
scheme to analyze the performance of this education model.
The QoS scheme combines the key QoS parameters such as
packet loss and delay to calculate an application-based QoS
value.
To carry out this analysis, we simulate a UMTS-based
distance education platform. The effects of number of users and
different traffic types on the network performance are
examined. Our application-based QoS evaluation scheme that
takes all these effects into account is then used to investigate

Abstract Mobile broadband technologies are now an important

part of the communication infrastructure even for most of the
developing world. These technologies can potentially play an
important role in improving the socioeconomic status of rural
areas. However, adaptation of these technologies for provision of
relevant multimedia services faces major challenges. Perhaps,
Quality of Service (QoS) issues still tops the list of such
challenges. In this work we study how to evaluate the QoS of
cellular-based systems focusing on an application perspective. We
quantify the QoS levels for different traffic models using
application and network related parameters to identify the most
suitable configuration for running multimedia-based services.
More specifically, our analysis is based on considering a unified
measure combining key QoS metrics such as packet loss, and
delay. For evaluation purposes, we also investigate the QoS issues
of deploying a distance education platform running over UMTS
cellular systems. The QoS issues related to the deployment of
multimedia services in cellular technologies are then considered
and analyzed in detail through simulation studies. The results
show that by inclusion of the communication technology and
application related parameters along with the number of users in
QoS evaluations, better performing network configurations can
be readily selected. This is achieved, through our proposed
application-based QoS evaluation scheme that is based on
combining various related measures. The proposed scheme is
shown to be particularly beneficial for evaluating and improving
QoS for multimedia-based heterogeneous networks.
Keywords- QoS; UMTS; 3G; rural area; distance education

I.

INTRODUCTION

The transmission of multimedia-based services over

cellular and wireless technologies is on a very steep rise.
Multimedia applications such as streaming video,
videoconferencing, and IPTV are widely used in different
socioeconomic remote services. As a consequence of this
growing demand for multimedia-based services, the
communication networks are experiencing a massive increase
in the traffic they carry. This is particularly the case for cellular
and heterogeneous networks. According to some forecasts, by
2015, 66 percent of the global mobile data flow will consist of
video traffic [1]. However, QoS provisions for multimedia
applications over wireless and cellular technologies are still
challenging the growth of these applications [2]. This is mainly
due to the underlying characteristics, for instance, the channel
capacity variations, limited throughput, or area coverage [3].

978-1-4799-0540-9/13/$31.00 2013 IEEE

17

9th IEEE International Workshop on Performance and Management of Wireless and Mobile Networks, 2013

the performance of various network configurations. The

suitability of this QoS scheme is also scrutinized and reported.
The rest of the paper is organized as follows: section II
presents an analysis of the QoS requirements for various
potential services. It also presents the architecture of the
UMTS-based distance education model used in this work.
Section III outlines the simulation studies along with analysis
of their results. The final section gives the concluding remarks
and the potential future works.
II.

TABLE II. BENCHMARKS OF DELAY FOR DIFFERENT APPLICATIONS

Services

Delay (msec)

ITU
CISCO
Conversational based services
3GPP

DISTANCE EDUCATION MODEL AND QOS REQUIREMENTS

ITU

It can be quite challenging to provide high QoS levels when

the network is to support several services that differ in terms of
their required performance metrics. Although previous research
works have addressed these issues, there are still many
discrepancies in determining acceptable benchmarks of QoS
parameters [8]. For instance, ITU-T, 3GPP and CISCO have
recommended different benchmarks for key QoS metrics for
the same service [8]. Table I shows example of such variant
benchmarks for packet loss recommended for different
services.
After considering the above mentioned benchmarks, for this
work, application-oriented lower and upper bound values for
packet loss are identified. These are 1% and 3% for
conversation, and 1% and 5% for streaming. For
videoconferencing, the consensus is that around 1 to 3% of
packet loss can be tolerated. There is also discrepancy in
regards to acceptable values for end-to-end delay. Table II
shows these values. In general, conversation and
videoconferencing based services require the same upper
bound and lower bound delay which is between 150 msec and
400 msec. However, the lower bound value refers to a longterm achievable value [8]. There are different benchmarks for
jitter or delay variations as well. For instance, Cisco requires
the delay variation for an audio call to be less than 30 msec,
while ITU-T and 3GPP refers the preferable delay variation as
less than 1 msec.
Similarly, there are recommendations for different levels of
required bandwidth for the same application. For instance,
ITU-T recommends data rates of 16 to 384 Kbps for video

Videoconferencing

Streaming based services

Services

Conversational based services

Streaming based services

Videoconferencing based services

ITU
CISCO
3GPP
PingER

3
1
3
2.5

CISCO
3GPP
ITU-T

150
150-400
10 seconds

CISCO

5 seconds

Date rate

QoS indicators

Highly
Interactive
Lectures/Discussions
(Videoconferencing)

16-384 Kbps

Delay, packet loss,

jitter

Interactive
Lectures/Discussions
(Voice applciations)

4-320 Kbps

Delay, packet loss,

jitter

16-384 Kpbs

Packet loss

Non-interactive
Lectures (Streaming)

streaming services, while a rate of 384 Kbps is recommended

by Cisco. Additionally, performance parameters may vary
depending on the environmental settings. It is apparent that
network-based applications have specific data rates and QoS
indicators. Table III shows the mapping of the distance
education oriented services to their QoS requirements. The
values and indicators from this analysis are used in the later
section for the application-based QoS scheme.
Fig. 1 illustrates the 3G-based distance education model
from a QoS point of view. In this multiple-layered model,
technologies and content are presented in parallel. From
networking perspectives, the delay, delay variation and packet
loss are the main concerns for QoS. On the other hand, from
user perspectives, parameters like response time, image quality,
download time, and upload time are of great concern.
Fig. 2 shows the architecture of the 3G-based distance
education model. This architecture is to facilitate a rural-urban
cooperative model. The model offers different types of learning
services, including those based on multimedia streaming, voice
applications and videoconferencing. IP based calls are used for
voice communications. 7.4 kbit/s mode AMR speech codec is
used for these calls with an activity factor of 0.5. A typical
AMR packet runs for 20 msec with a payload size of 224 bits.

Packet loss (%)

3
1
3

15-400

REQUIREMENTS

APPLICATIONS

ITU
CISCO
3GPP
Study
based on a
developing
country [7]
ITU
CISCO

150-400

TABLE III. DISTANCE EDUCATION APPLICATIONS AND QOS

TABLE I. BENCHMARKS OF PACKET LOSS FOR DIFFERENT

Services

150-400
150

5
1
5

18

9th IEEE International Workshop on Performance and Management of Wireless and Mobile Networks, 2013

Figure 1. Layer based distance education model

Figure 2. Architecture of 3G-based distance education model

Streaming-based video applications are popular mediums for

distance education. The video codec used in this case are the
H.263 and MPEG-4 as these two are the most popular video
codec for mobile devices according to a study published in [9].
Two video trace files encoded with H.263 and MPEG-4 are
used for simulations.
The trace files are collected from [10]. The parameters for
video trace files are specified in Table IV. Group of Picture
(GoP) structure is denoted using GgBb where g defines the
total number of frames in a GoP and b indicates the number of
B frames between successive I or P frames. B frames are the
bi-directionally predictive coded frames, I frames are intracoded frames and P are predictive coded frames.
Videoconferencing is used with a rate of 15 fps. Packet size
used for videoconferencing is 576 bytes. We have considered

FDD version of UMTS for this model. A vehicular path-loss

model is used which is the most suitable model for a rural
environment. The throughput-based admission control
algorithm is used and in the downlink, and other-cell
interference factor is set to 0.65.
III.

SIMULATION STUDIES AND QOS ANALYSIS

This section presents the simulation designs for 3G-based

distance education platform. Then we analyze the QoS of this
model using an application-based QoS scheme which is to be
discussed. The platform involves different types of multimedia
traffic. It is assumed that there is a network with n number
of application flows, where

n = { voice , strea ming , videoconferencing ,........}

19

9th IEEE International Workshop on Performance and Management of Wireless and Mobile Networks, 2013

designed involving different number of users and applications.

OPNET has been used as the simulator since its UMTS model
is well established. Table V illustrates the simulator parameters
of different layers. In the first phase of the simulations, a
UMTS network in the context of a rural area is simulated with
various numbers of voice application users starting from 8 to
20. Values of packet loss and delay are evaluated in each
scenario. Each scenario is run for 13 times with different seed
values and 95% confidence interval.
In the second phase of the simulations, a streaming client is
added to the voice service users. Two different video codecs,
H.263 and MPEG-4, are used for the streaming client. The
performance of the voice application is evaluated with each
type of the streaming codec running. This is done in this
fashion to facilitate identifying the optimal number of voice
and streaming users for this platform. In the third stage of
simulations, a videoconferencing client is added.
Fig. 3 shows the end-to-end delay experienced by voice
users after adding a MPEG-4 or H.263 codec based streaming
client. It shows that an H.263 streaming client influences the
performance of voice clients less compared to a MPEG-4 client
does. Fig. 4 shows the percentage of packet loss for voice
service users after adding a MPEG-4 or H.263 based streaming
client. The simulation results clearly indicate that voice clients
experience less packet loss in the presence of an H.263
streaming client.
Table VI presents the performance of a streaming client
using H.263 and MPEG-4 codec. In terms of packet loss,
H.263 codec shows a better performance under the UMTSbased environment than MPEG-4 codec does. In case of delay,
the results show the same behavior. The simulation results also
show that the platform experiences better performance with
one H.263 codec based streaming client and twelve voice users
in terms of packet loss.
However, in terms of delay, the platform shows an
acceptable performance with one H.263 based streaming client
and twenty voice users. On the other hand, the platform shows
a different behavior with the presence of the MPEG-4 based

TABLE IV. STREAMING TRACE SPECIFICATION OF H.263 AND MPEG-4

Performance metrics
codec

H.263

MPEG-4

Metric

Value

QP

Resolution

QCIF (177*144)
tmn encoder (Version
2.0/3.2)
16 kbit/s
ON
3603.28 sec
19436
Single Layer
MOMUSYS MPEG-4
QCIF 176x144
89998
7499
12,
IBBPBBPBBP
BB
5

Encoder
Bit rate
USE of PB-frames
Length
No. Frames
Layer
Encoder
Frame Size
No. Frames
Number of GoP
INTRA PERIOD
QP

TABLE V. SIMULATION PARAMETER DETAILS

Layers

Application

RLC

Parameters

Voice calls
Streaming
video
Videoconfe
-rencing
Mode
Timer
MRW
(msec)
Timer
Discard
(msec)
MAX
MRW
MAX DAT

QoS class
Conversational

Payload size
224 bits

Background

Variant

Conversational

576 bytes

Unacknowledged
900
7500
6
4
DCH

PHY

Channel
type

Conversational
Bit
TTI
rate
(msec)
(Kbps)

Background
Bit rate
(Kbps)

TTI
(msec)

64

144

20

10

The model is a multi-variable based system, where

QoS = f (user , application , technol ogy )

We analyze the QoS of this model based on the relationship
among these variables. Several simulation scenarios are
Figure 3. Comparison of end-to-end delay of voice application after
adding a streaming client with different codec

20

9th IEEE International Workshop on Performance and Management of Wireless and Mobile Networks, 2013

TABLE VIII. APPLICATION REQUIREMENTS

Application

Packet loss
(%)

Li

Ui

Delay (msec)

Li

Ui

Voice application

150

400

Streaming
application
Videoconferencing
application

N/A

N/A

150

400

TABLE IX. WEIGHT FOR DIFFERENT QOS METRICS

QoS metric
Packet loss

Delay
Figure 4. Comparison of packet loss of voice application after adding a
streaming client with different codec

DIFFERENT NUMBERS OF VOICE CLIENTS

Number of
streaming
client

Packet loss for

different codec
MPEG-4

H.263

10

12

14

16

18

20

5.58

1.82

5.56

1.85

5.75

1.89

5.93

1.89

5.79

1.91

5.62

1.94

5.75

2.02

streaming client. Table VII shows the percentage of packet loss

in the presence of eight voice users, one streaming client and
one videoconferencing client. The results show that the
presence of the videoconferencing and the streaming client at
the same time affects the performance of voice application.
This type of uncertainty in the network behavior creates
needs to combine values of different QoS parameters and
evaluate the network performance utilizing one unified QoS
TABLE VII. PACKET LOSS FOR DIFFERENT APPLICATIONS
Type of application
Voice application

Packet loss
(%)
13.55

Streaming application

1.0287

Videoconferencing
application

4.02

0.4
0.8

Weight

Videoconferencing
Voice
Streaming
Videoconferencing

0.4
0.6
0.2
0.6

value. Our work has been inspired by the QoS analysis method
proposed in [6]. The authors in their work have used two
performance points of an application for QoS metrics under
consideration. They have used these points to calculate the QoS
of an application on the fly. First, they calculate the normalized
value of each QoS metric using the performance metric of the
serving network. Then they provide a weight for each QoS
metric according to its relevant importance.
In our case, the application performance measurement is
collected from the average value of a certain QoS parameter
after the application is run for a certain period of time in the
considered network. These measured values are used with the
acceptable QoS parameter values which are defined earlier to
calculate the application QoS value. Then this application QoS
value is used to measure the network QoS value. Although, in
this work the network QoS is evaluated engaging one network
technology, the aim of this scheme is to involve different
network technologies which will be stated in our future work.
This scheme helps to find the best available combination in
context of networking technologies, applications and number
of users.
This approach also unfolds a way for bridging network QoS
with Quality of Experience (QoE). User satisfactions of a
network are closely related to the application performance. For
instance, a voice based service with a lower delay, normally
results in higher user satisfactions. So, if the application
performs as the user expects the user usually gives a better
satisfaction rating for the network. From the analysis of section
II, it is apparent that each service has its own QoS
requirements. To find out the performance of a particular
network, the performances of application-based traffic flows
present in that network are analyzed. We define the lower and
the upper bound values of QoS metrics for each service from
the analysis in the previous section. Table VIII illustrates these
values in the context of packet loss and delay. The bound for
packet loss of voice application is from 1% to 3%. For
streaming based service, the range is from 1% to 5% and for

TABLE VI. PACKET LOSS FOR STREAMING CLIENT IN THE PRESENCE OF

Number of
Voice users

Application
Voice
Streaming

21

9th IEEE International Workshop on Performance and Management of Wireless and Mobile Networks, 2013

is assigned to delay for voice applications. For packet loss, the

assigned value for the weight is 0.4. This is because voice
applications are more sensitive to delay; yet they are more loss
tolerable. As a result, a larger delay affects voice quality. On
the other hand, streaming applications are less sensitive to
delay. So a value of 0.2 for delay and 0.8 for packet loss is
assigned for this type of application. Upon calculation of
normalized values for each of the metrics, an overall QoS value
corresponding to each application is calculated using Equation
(2). In this equation, w is the weight for each QoS metric and
n is the application under consideration.

TABLE XI. QOS VALUES FROM ANALYSIS OF VOICE APPLICATION

Number
of voice
calls

Delay
(msec)

Packet
loss (%)

md,n m pl ,n

8
10
14
18

216
224
203
215

2.06
2.92
2.94
3.23

0.74
0.70
0.79
0.74

QoSn

0.47
0.04
0.03
0

0.63
0.50
0.48
0.44

videoconferencing the range is from 0% to 1%. Similarly,

lower and upper bound values for delay are different for
different services.
For each QoS metric i under each application n , there is

QoS n = mi ,n wi
i =1

an upper and lower bound value U i and Li . These upper and

As in this work, end-to-end delay and packet loss are

considered for QoS measurement of each application, the
equation should be as follows:

lower bound values for the specific QoS metric and the average
for the same QoS metric value derived from each application
flow in the network are put into Equation (1) to calculate the
normalization value for each QoS metric.

mi,n=

QoSVOIP = mdelay ,VOIP wdelay + mPL,VOIP wPL

(3)

To come up with a unified metric for a network that

involves different types of traffic, the effects are combined in
Equation (3). This combination quantifies the QoS level taking
into account the various services running over the network.
Extending this to network
with k being the numbers of
services over , the overall QoS level for the network can be
defined as

Umi
Lmi
Umi

0,
1,

Umi Lmi

(2)

, Umi > > Lmi

(1)

QoS = QoSn

After defining the acceptable value for each QoS metric, a

weight is assigned to each metric. This weight depends on the
relevant importance of a particular QoS metric to the
application performance as presented in Table III. Table IX
illustrates the weight for different QoS metrics. A value of 0.6

n =1

(4)

TABLE X. QOS VALUES FOR MIXED TRAFFIC

Number of
different
application user(s)

Delay (msec)

Packet loss (%)

md,n

QoSn

m pl ,n

QoS

Voice

Streaming

Voice

Streaming

Voice

Streaming

Voice

Streaming

Voice

Streaming

Voice

Streaming

202

176

2.92

1.82

0.79

0.04

0.49

1.49

10

221

179

2.95

1.85

0.72

0.025

0.43

1.43

14

208

180

4.41

1.89

0.79

0.47

1.47

18

219

180

6.46

1.94

0.72

0.43

1.43

22

9th IEEE International Workshop on Performance and Management of Wireless and Mobile Networks, 2013

Table X and Table XI illustrates the calculation results from

the first and the second phase of simulations. For eight
simultaneous calls, the users experience an average delay of
216 msec and an average packet loss of 2.06%. The
normalization factor of the delay for this flow is calculated
as md ,voice = 400 216 400 150 = 0.74 . Similarly, the factor for packet
loss is calculated as m pl ,voice = 3 2.06

= 0.47 . Then the

3 1
QoS value for application such as voice is calculated using
Equation (3). QoSvoice = 0.74*0.6 + 0.47 *0.4 = 0.63 .The
network QoS value for this network is analyzed using Equation
(4). For example, in one of the simulation scenario, there are
both voice and video streaming services. The application QoS
values for these two services are 0.49 and 1 respectively. So the
network QoS value using Equation (4) is 1.49. Fig. 5 shows the
comparison of the QoS levels of voice applications in the
UMTS education platform with and without streaming client.
The comparison shows that the presence of a streaming client
in the network affects the QoS of voice applications.
IV.

CONCLUSIONS AND FUTURE WORKS

Figure 5. Effect of streaming application on QoS of voice application

In this paper, we analyze a 3G-based distance education

platform. The analysis takes into account the variations in
number of users and different types of services. An overarching
application-based QoS evaluation scheme is also proposed. The
scheme is based on evaluating a unifying QoS metric. The
metric is a measurement of the performance of applications and
combines the effects of key QoS parameters such as packet loss
and delay. The scheme then combines the performance effects
of different applications to arrive at the overall unified network
QoS metric. Our simulation studies show that this is a
convenient and yet a resourceful approach for evaluating the
performance of a multimedia-based network. More
specifically, we have performed detailed simulations to
evaluate the performance of a 3G-based distance education
platform. One of our aims has been methodical identification of
the suitable configurations that best accommodate the
requirements for this platform. The results clearly indicate the
importance of taking into account the number of users and the
nature of utilized applications in evaluating the QoS of the
platform. In turn, this can be used to improve the performance
of the distance education platform. In our future work, we
intend to focus on WiFi and LTE technologies as the potential
candidates for distance education with high QoS levels. In this
work although a fixed weight has been considered for each
parameter based on its relevant importance to a specific
application, in the future work the opinions of different sources
will be considered to assign a more appropriate weight.
Moreover, more parameters such as throughput and energy
consumption will be taken into account for the QoS evaluation
scheme.

REFERENCES
[1]

Cisco, "Cisco Visual Networking Index: Global Mobile Data Traffic

Forecast Update, 20122017," Cisco2013.
[2] H. Luo and M.-L. Shyu, "Quality of service provision in mobile
multimedia - a survey," Human-centric Computing and Information
Sciences, vol. 1, pp. 1-15, 2011/11/22 2011.
[3] H. Moustafa, Mare, x, N. chal, and S. Zeadally, "Mobile Multimedia
Applications: Delivery Technologies," IT Professional, vol. 14, pp. 1221, 2012.
[4] N. De Cristofaro, G. McGill, A. Sallahi, M. Davis, A. Alsibai, and M.
St-Hilaire, "QoS evaluation of a voice over IP network with video: A
case study," in Electrical and Computer Engineering, 2009. CCECE '09.
Canadian Conference on, 2009, pp. 288-292.
[5] X. Liu, L.-g. Jiang, C. He, and H.-w. Liao, "An intelligent vertical
handoff algorithm in heterogeneous wireless networks," in Neural
Networks and Signal Processing, 2008 International Conference on,
2008, pp. 550-555.
[6] C. Wen-Tsuen and S. Yen-Yuan, "Active application oriented vertical
handoff in next-generation wireless networks," in Wireless
Communications and Networking Conference, 2005 IEEE, 2005, pp.
1383-1388 Vol. 3.
[7] Z. H. a. D. T. E. Sedoyeka, "Analysis of QoS Requirements in
Developing Countries," International Journal of Computing and ICT
Research, vol. 3, pp. 18-31, 2009.
[8] F. Farid, S. Shahrestani, and C. Ruan, "Quality of Service requirements
in Wireless and Cellular Networks: Application-based Analysis," in
International Business Information Management Association
Conference, Bercelona, Spain, 2012.
[9] L. Yao, "Measurement and Analysis of an Internet Streaming Service to
Mobile Devices," IEEE Transactions on Parallel and Distributed
Systems, vol. 99, pp. 1-1, 2012.
[10] TKN. MPEG-4 and H.263 Video Traces for Network Performance
Evaluation. Available: http://trace.eas.asu.edu/TRACE/ltvt.html

ACKNOWLEDGMENT
We would like to thank OPNET for providing us with
Modeler software license.

23