Fourth International Conference on Networked Computing and Advanced Information Management

The QoE Evaluation Method through the QoS-QoE Correlation Model

Hyun-Jong Kim1, Dong-Hyeon Lee1, Jong-Min Lee1, Kyoung-Hee Lee2, Won Lyu2, Seong-Gon Choi1 1 School of Electrical & Computer Engineering, Chungbuk National University (CBNU), 12 Gaeshin-dong, Heungduk-gu, Cheongju-city, Chungbuk, 361 763, Republic of Korea 2 Electronics and Telecommunications Research Institute (ETRI), 161 Gajeong-dong, Yuseong-gu, Daejeon, 305-350, Republic of Korea 1 {hjkim78, dhlee, ljm80, sgchoi}@cbnu.ac.kr, 2{leekhe, wlyu}@etri.re.kr

Abstract
Currently, the satisfaction of user is becoming one of the most important topics concerned by the service providers. So we propose the approach method for the objective QoE measurement through the QoS parameters. The measurements and provisioning of the quality of service (QoS) are generally defined in terms of network delivery capacity and resource availability, not in terms of satisfaction to the end-user. The fundamental assumption behind such traditional provisioning is that the measured quality of service is closely related to the quality of experience (QoE) for the end-user. In the paper, we describe the QoS and QoE correlation model, and the QoE evaluation method using QoS parameter in the converged network environment is studied.

1. Introduction
The increasing deployment of broadband IP networks has enabled integration and convergence of voice service (e.g. VoIP), video service (e.g. VoD) and data service, which is referred as a Triply Play. Like this, as various services were provided in the integrated network environment, QoS and QoE concept was introduced in the IP network. Currently, the satisfaction of users is becoming one of the most import topics concerned by the service and network providers. However, existing methods used for evaluation the quality of experience (QoE) mostly rely on user survey and scores from the user, which are too subjective and need mush processing time and cost. So, our investigation is trying to relate the objective network service conditions with the human perception

of the quality of the service. Moreover, because existing QoE measurement schemes measure QoE by using a part QoS measurement parameters, they are difficult to reflect of the various services. The quality measurement scheme that uses the preexistence QoS measurement parameter can provide the objective quality information in a network-level. However, this quality measurement scheme is difficult to expect the quality which the service end-user experiences. So, in this paper, by using the quality of service information measured in a network-level, we explain the proposed QoS-QoE correlation model for the objective QoE, and describe the example of the service QoE evaluation. The paper proposed a method to evaluate the QoE of user in convergence network. The remainder of the paper is organized as follows. Section II outlines the current status of QoE evaluation method and related works. Section III introduces the QoS parameters related with QoE evaluation and the service classes. Section IV describes the QoS-QoE correlation model for the QoE evaluation and section V gives the example of the QoE evaluation by using this correlation model. Finally, section VI presents the conclusion.

2. Related works
2.1. The voice and video service quality measurement method
Existing quality assessment technologies can be classified into two categories: subjective quality assessment schemes and objective ones. Current

978-0-7695-3322-3/08 $25.00 2008 IEEE DOI 10.1109/NCM.2008.202

719

research status of these two kinds of evaluation methods are illustrated as followed paragraphs: Currently, the evaluation methods for the speech service are mature. For subjective evaluation methods, opinion rating (MOS) based on customers satisfaction has been studied to assess the perceptual QoS. It is specified in ITU-T recommendations E.800 initially [1]. On another hand, several objective quality assessed methods has been proposed in ITU-T, such as P.861[2] PSQM (Perceptual Speech Quality Measure), P.862[3] PESQ (Perceptual Evaluation of Speech Quality) and G.107 E-Model[4]. For the video service evaluation, subjective video quality evaluation method is the most reliable video quality measurement method. A group of viewers is selected and gathered in a room, the measurement environment is specified in the ITU-T Recommendation P.910[5]. For the research of objective video quality method, some estimation software has been developed which can analyze the video signals and produce the quality evaluation results. One traditional objective video quality measurement, Peak Signal to Noise Ratio (PSNR), has been widely used in many applications to assess video quality. PSNR does not take the visual masking phenomenon into consideration. In other words, every single pixel error contributes to the decrease of the PSNR, even if this error is not perceived. So, MPQM (Moving Pictures Quality Metric) was proposed for the objective the video quality measurement[6][10]. MPQM is an objective quality metric for moving picture which incorporates human vision characteristics. MPQM represents the typical image quality assessment models based on the error sensitivity. The widely adopted assumption of these models is that the loss of perceptual quality is directly related to the visibility of the error signal. From current research status of evaluation method, we can see that the subjective method based on user survey can reflect the experience of user more directly and match well to the feeling of user. However, this kind of method has several problems, such as, it required special environment and equipments, needs a mass of people to participate the test. In conclusion, subjective video quality measurement cannot provide real-time and in-service quality monitoring for realtime video applications. So the application of the method is limited.

2.2. The study of relationship between QoS and QoE

Currently, the investigation of QoS and QoE correlation is continued. Khirman and Henriksen were trying to relate the objective network service conditions with the human perception of the quality of the service. Their subject has been widely investigated for voice delivery and it is widely acknowledged that the relationship between voice transmission conditions and the human perception of quality is far from linear[7]. They discuss in detail how the human satisfaction of HTTP service is affected by the two main network QoS parameters, namely network delivery speed and latency. However, it is difficult to represent the feature of the provided and various services from only the bandwidth and latency time in the integrated network environment. In [8] the authors thought that pervasive computing environment brings the method of evidence context related to QoE. They studied the QoE evaluation method in pervasive computing environment, and proposed the enhanced QoE evaluation parameter model. In [8] rough-set based algorithm is proposed to reduce context attributes and determine the weight of each attribute, the algorithm has been validated on video streaming service, and the architecture of QoE evaluation system is described. As a mass of evidence information related to the experience of users can be gathered through the context-awareness computing, the calculation results of QoE evaluation method can highly match the real feeling of users. However, the method needs to be enhanced along with the development of pervasive computing.

3. The QoS parameters related with enduser

This clause addresses the QoS quality parameters which can be considered for the QoE evaluation. Transfer Capacity is a fundamental QoS parameter having primary influence on the performance perceived by end-users. Many user applications have minimum capacity requirements; these requirements should be considered when entering into service agreements. And lost bits or octets can be subtracted from the total sent in order to provisionally determine network capacity. An independent definition of capacity is for further study. It is assumed that the user and network provider have agreed on the maximum access capacity that will

720

be available to one or more packet flows in a specific QoS class (except the Unspecified class in Table ). A packet flow is the traffic associated with a given connection or connectionless stream having the same source host (SRC), destination host (DST), class of service, and session identification. Other documents may use the terms microflow or subflow when referring to traffic streams with this degree of classification. Initially, the agreeing parties may use whatever capacity specifications they consider appropriate, so long as they allow both network provider enforcement and user verification. For example, specifying the peak bit rate on an access link (including lower layer overhead) may be sufficient. The network provider agrees to transfer packets at the specified capacity in accordance with the agreed QoS class. The network performance objectives may no longer be applicable when there is packets submitted n excess of the capacity agreement or the negotiated traffic contract. If excess packets are observed, the network is allowed to discard a number of packets equal to the number of excess packets. Such discarded packets must not be included in the population of interest, which is the set of packets evaluated using the network performance parameters. In particular, discarded packets must not be counted as lost packets in assessing the network's IPLR performance. A discarded packet might be retransmitted, but then it must be considered as a new packet in assessing network performance. Each network QoS class creates a specific combination of bounds on the performance values. This clause includes guidance as to when each network QoS class might be used, but it does not mandate the use of any particular network QoS class in any particular context. Table 1. IP network QoS class definitions and network performance objectives [12]
Network performance Parameter IPTD IPDV IPLR IPER QoS Classes Class Class 2 3 100 400 ms ms U U 1x 1x 10-3 10-3 -4 1x 10

3.1. Delay
Delay manifests itself in a number of ways, including the time taken to establish a particular service from the initial user request and the time to receive specific information once the service is established. Delay has a very direct impact on user satisfaction depending on the application, and includes delays in the terminal, network, and any servers. Note that from a user point of view, delay also takes into account the effect of other network parameters such as throughput.

3.2. Delay variation

Delay variation is generally included as a performance parameter since it is very important at the transport layer in packetized data systems due to the inherent variability in arrival times of individual packets. However, services that are highly intolerant of delay variation will usually take steps to remove (or at least significantly reduce) the delay variation by means of buffering, effectively eliminating delay variation as perceived at the user level (although at the expense of adding additional fixed delay).

3.3. Information loss

Information loss has a very direct effect on the quality of the information finally presented to the user, whether it be voice, image, video or data. In this context, information loss is not limited to the effects of bit errors or packet loss during transmission, but also includes the effects of any degradation introduced by media coding for more efficient transmission (e.g. the use of low bit-rate speech codecs for voice). Table 2. Guidance for IP QoS classes [12]
QoS class 0 Applications (ex) Real-time, jitter sensitive, high interaction (VoIP, VTC) Real-time, jitter sensitive, interactive (VoIP, VTC) Transaction data, Highly interactive (Signalling) Transaction data, interactive Low loss only (short transactions, bulk data, video streaming) Traditional applications of default IP networks Network techniques Constrained routing and distance Less constrained routing and distances Constrained routing and distance Less constrained routing and distances Any route/path Any route/path

Class 0 100 ms 50ms 1x 10-3

Class 1 400 ms 50ms 1x 10-3

Class 4 1s U 1x 10-3

Class 5 U U U U

1 2 3 4 5

721

4. The proposed QoE measurement scheme using the QoS parameters

The provided QoS-QoE correlation of the service can variously show up in the integrated network environment by QoS quality parameters considered for a kind of service and QoE. We propose the correlation model approach method that can reflect all elements influencing on QoE in order to solve these problem. The relation between the QoE and QoS cannot be easily mapped to the service which can be provided in the converged network. According to the traffic characteristic of each service, the QoS level which is required in order to satisfy the QoE class is different. Moreover, the QoE influence element at the terminal layer which is required in order to evaluate QoE need to reflect. The normalized QoS score of the service which can be provided can be obtained through the next equation from the integrated network. (1) By using quality indexes of a network-level, the QoS score can obtain. It becomes with application so that there are delay, jitter, loss rate, error rate, bandwidth, the signal success rate, and etc, that is the objective quality parameter, and the weighted value of these quality parameters be different from the used service. For example, the service sensitive to delay time assigns the weighted value 10 to the delay time 100ms like VoIP and video conference service and the service which is a bit less sensitive to the delay time like VoD assigns the weighted value 5. In the same condition, the bandwidth proffer of over required is meaningless if VoIP and video conference service are satisfied by the minimum guaranteed bandwidth. In another hand, the case where the user experience quality increases as the bandwidth provided like VoD is big will be able to assign the large-scale weighted value.

calculated by the equation (1). The QoE class measured by the QoS quality parameter of a network-level is mapped like the existing MOS grade with 5 classes. The represents the QoS quality class of the networklevel which has to be provided as the QoS class constant in order to satisfy the required QoE class. The minimum range of the QoE quality standard of the provided service is determined by this value. Next, is determined according to the class of service as the grade of service constant. Finally, K is the Scale constant for showing the satisfaction about the use service. This generalizes the subjective quality about the using service. Although the same QoS quality is provided in the wire-network environment, this variable considers elements influencing on the user experience quality according to the radio channel and terminal. According to the class of service, the next paragraph shows the QoE measurement scheme that uses QoE and QoS correlation model.

4.1. The Guaranteed service

The services provided in the integrated network environment can classify as the Guaranteed service, the Premium service, and the Best Effort service, and the Guaranteed service has the top priority. The data priority processing and bandwidth guarantee, and etc. are required according to the service priority in order to provide the real-time multimedia service like VoIP, IPTV and Video Conference, and etc.
QoE
5 4 3 2 1

Guaranteed Service

10

20

30

40

50

60

70

QoS

(2) Here, the used variable and constant are determined by elements influencing on QoE including an environment and the using service kind, terminal position, using codec, and etc. The QoS of this equation is the normalized QoS score which is

Figure 1. The QoS-QoE correlation model of the Guaranteed service The most of guaranteed services is the real-time voice / video service like the VoIP, Video Phone and Video Conference service. The stable bandwidth has to be provided for the seamless usage of this service and the minimum connection delay time has to be guaranteed. The quality parameter needed for the

722

quality measure of this service is delay, jitter and packet Loss rate, and etc. And the call connection yield, and the use of service duration and audio / image MOS value are needed when the quality measure of this service considers QoE. The QoS-QoE correlation model of the Guaranteed service can represent the on / off model. If the QoS score which is required in order to guarantee the QoE of this service is unable to be satisfied, the Guaranteed service cannot be provided. As shown in Figure 1, the QoS score has to be provided over the minimum 50 in order to guarantee the QoE of the Guaranteed service. As to this model, the -value is small compared with the other model and this means that the minimum guaranteed class of the QoS which has to be supported is very sensitive. Here, the shown numerical value is one example and actually evaluates the quality customer satisfaction measurement about the service user and need to calculate the average distribution of QoE in order to establish the exact relation.

Guaranteed service and this can be controlled by the grade of service constant (B). The premium service can divide the quality of service according to this linearity. This accommodates the various class of the AF PHB of DiffServ. According to select the appropriate QoE class constant, a network and service providers can organize the various premium service QoS-QoE correlation models.

4.3. The Best Effort service

The best-effort service can include preexistence internet services and the quality level is similar to it. This says to be the more sensitive QoS-QoE correlation model the QoS parameters than QoE. This means that although the QoS parameter performance is improved, there is a limit in the QoE grade improvement which the end user experiences.
QoE
5 4

4.2. The Premium service

In the case of the premium service, it is more flexible than the Guaranteed service about the real-timeness. The target service kinds are the Movies streaming service, IPTV, VoD and Interactive Gaming, and etc. As to the traffic characteristic of these services, the minimum bandwidth has to be guaranteed as the realtime streaming service and the multicasting service. The major quality parameters are similar to the case of the Guaranteed service and are however a feature to be less sensitive to a delay than the Guaranteed service.
QoE
5 4

3 2 1

Best Effort Service

10 -1

20

30

40

50

60

70

QoS

Figure 3. The QoS-QoE correlation model of the Best Effort service In this way, we propose the QoS-QoE correlation model according to three service classes. The network performance (QoS class) which has to be guaranteed at least can be expected through this model according to the grade of service. Moreover, QoS and QoE relation of the provided service can be analyzed through this model in the integrated network environment. In the future, the research for setting up the quantified QoS class toward QoS parameter of the various network-levels in the integrated network environment is needed. Moreover, the research about which how will set up the value of , and K in consideration of the real time multimedia service kind and service usage environment is required.

Premium Service A
3 2 1

Premium Service B Premium Service C

10

20

30

40

50

60

70

QoS

Figure 2. The QoS-QoE correlation model of the Premium service The QoS-QoE correlation model of the premium service has the feature that is a linear, more than the

723

5. The example of the QoE measurement using the QoS-QoE correlation model
The process of measuring the QoE of the Premium service by using the QoS-QoE correlation model is as follows. The Table 1 shows the weighted value of the each QoS parameters needed to the QoE evaluation of the Premium service. If quality data measured about the video conference service of the Premium service are as follows, we can e xpect the QoE of a user through the QoS-QoE correlati on model. Delay (D) = 125ms Jitter (J) = 45ms Packet loss rate (L) = 2 x 10-5 Packet error rate (E) = 1 x 10-6 Bandwidth (B) = 2.5Mbps Call success rate(S) = 99.0% QoS = F(D, J, L, E, B, S) = 8+10+7+10+10+8=53 Table 3. QoS parameter and range related with QoE
QoS parameter and range Parameter Delay (D) Range 70ms ~ 100ms 100ms ~ 150ms 150ms ~ 200ms 30ms ~ 50ms Jitter (J) 50ms ~ 60ms 60ms ~ 70ms Packet loss rate (L) Packet error rate (E) ~ 10-5 10-5 ~ 10-4 10-4 ~ 10-3 ~ 10-6 10-6 ~ 10-5 10-5 ~ 10-4 256 ~ 128Kbps Voice Bandwidth (B) Video 128 ~ 80Kbps 80 ~ 64Kbps 3 ~ 2Mbps 2 ~ 1Mbps 1Mbps~512Kbps Call success rate (S) 100 ~ 99.9% 99.9% ~ 99.0% 99.0% ~ 98.0% HO within the cell Handover (H) HO between the cells HO between the frequencies Channel RF Power, RSSI, Score 10 8 6 10 5 3 10 7 3 10 7 4 10 8 6 10 8 5 10 9 8 N/A N/A N/A N/A Undefined Undefined Undefined G.1010 DSLForum WiMAX Y.1541 DSLForum Y.1541 DSLForum Y.1541 Note Y.1541, G.1010

Quality Indicator (CQI)

VSWR(Voltage Standing Wave Ratio), CINR (Carrier to Interference plus Noise Ratio)

The QoS normalized by the equation (1) is substituted for the equation (2), and then the QoE class can be evaluated. As shown in Figure 4, we can predict the class of Premium service B for the experience quality of an end-user in the above environment. We can confirm whether it is satisfied the class of service which the first user joins from this measurement result or not. Because of being the result that the QoE is evaluated by using the objective network quality parameters, it can be regarded as the more objective QoE than the other measurement schemes. Here, the used in order to digitize the QoE class network quality index and the standards aren't perfect, but are one example. The investigation of the weighted value about the network performance index has to be separately executed. The proper quality parameter score need to be selected against service users through the question survey.
QoE
5 4 3

Premium Service B
2 1

10

20

30

40

50

60

70

QoS

Figure 4. The example of the QoE evaluation using QoS-QoE correlation model

6. Conclusion
Because existing QoE measurement schemes measure QoE by using a part QoS measurement parameters, they are difficult to reflect of the various services. The quality measurement scheme that uses the preexistence QoS measurement parameter can provide the objective quality information in a network-level. However, this quality measurement scheme is difficult

724

to expect the quality which the service end-user experiences. So, in this paper, by using the quality of service information measured in a network-level, we explain the proposed QoS-QoE correlation model for the objective QoE, and describe the example of the service QoE evaluation. In the future, the research for setting up the quantified QoS class toward QoS parameter of the various network-levels in the integrated network environment is needed. Moreover, the research about which how will set up the value of , and K in consideration of the real time multimedia service kind and service usage environment is required.

[3] ITU-T P.862, Perceptual evaluation of speech quality (PESQ), Feb 2001. [4] ITU-T G.107, The E-model, a computational model for use in transmission planning, Mar 2005. [5] ITU-T P.910, Subjective video quality assessment methods for multimedia applications, Sep 1999. [6] Wang, Y. Survey of Objective Video Quality Measurements, Tech report, Worcester Polytechnic Institute, June 2006. [7] Stas Khirman, Peter Henriksen, Relationship between Quality-of-Service and Quality-of-Experience for Public Internet Service, PAM 2002, March 2002. [8] Liu Li-yuan, Zhou Wen-an, Song Jun-de, The Research of Quality of Experience Evaluation Method in Pervasive Computing Environment, PCA 2006, Aug 2006. [9] Hideaki YAMADA, Norihiro FUKUMOTO, Manabu ISOMURA, A QoE based service control scheme for RACF in IP based FMC networks, CEC/EEE 2007, July 2007. [10] NetPredic, Inc. White Paper, Performance Analysis for Video Streams across Networks, December 2003. [11] Mukundan Venkataraman, Shamik Sengupta, Mainak Chatterjee, Towards a Video Measuring Video Quality, IEEE Transactions on Broadcasting, Vol 50, No. 3, Sep 2004. [12] ITU-T Recommendation Y.1541, Network performance objectives for IP-based service, Feb 2006.

Acknowledgment
This work was supported by the IT R&D program of MKE/IITA. [2008-S-006-01, Development of OpenIPTV (IPTV2.0) Technologies for Wired and Wireless Networks]

*Corresponding Author: Seong Gon Choi (sgchoi@cbnu.ac.kr)

References
[1] ITU-T P.800, Mean Opinion Score(MOS) terminology, Mar 2003. [2] ITU-T P.862, Objective quality measurement of telephone-band (300-3400Hz) speech codec, Feb 1998.

725