Towards Improved Model for User Satisfaction Assessment of Multimedia Cloud Services

Aleksandar Karadimce1 and Danco P. Davcev2

1University of Information Science and Technology St. Paul the Apostle, Ohrid

2University Ss Cyril and Methodius, Skopje

E-mail: akaradimce@ieee.org, aleksandar.karadimce@uist.edu.mk; danco.davcev@finki.ukim.mk

Received 30 January 2018; Accepted 30 January 2018;
Publication 24 April 2018

Abstract

Service providers are in constant search for appropriate tools that will help them to determine and measure the satisfaction of end-users. In terms of efficient utilization of network resources, they are constantly striving to guaranty the quality, availability, and responsibilities of the directly measurable parameters with the service-level agreements (SLA). The introduction of cloud computing technology aims to provide stable, reliable and encapsulated environment for users who use different types of mobile and desktop devices to simultaneously access shared resources that are available anywhere and at any time. However, what is lacking in this direction is an assessment of how much the end-users themselves are satisfied with the cloud-based services offered. This research has considered different cloud computing service categories, some of them have been extensively used and others are still under development. These services use a variety of organizational and infrastructure access models, making them easily accessible and practical for work. The very nature of cloud computing-based services allows dynamic allocation of resources based on end-user needs, which makes the process of evaluating the offered services complex. This research will deliver the most appropriate model for assessing the impacts of objective and subjective factors by using the Bayesian networks. The dynamic nature of these networks allows flexibility and adaptation to measure the impact of various influencing factors. The main contribution of this research is the introduction of a metrics for assessing the user satisfaction of the particular type of offered service. In that direction, this research has provided improved Quality of Experience (QoE) model for measuring and assessing the perception of the multimedia cloud-based services quality among end-users.

Keywords

1 Introduction

Cloud computing consists of hardware and software resources, available on the Internet as a set of services for users. The users are likely to be offered a number of applications, which are based on various computing services. Here the most common problem that occurs for users, especially in mobile applications, is the dissatisfaction with the speed of access to necessary information. This dissatisfaction often results in a reduced perception of quality of the offered services. In this direction, service providers, from a technical point of view, have defined metric parameters, known as the notion of Quality of Services (QoS), which is carried out by provision and regulation of network parameters that are directly measurable. When the problem of the unsatisfactory level of quality of the offered services is considered from the user’s point, it requires a subjective quality assessment of services. Most often, to make this kind of analysis, service providers use empirical research, using field research in their opinion. Furthermore, these methods of analysis are usually bulky, long and costly surveys for the quality assessment of the offered services from a user perspective.

One way to overcome this problem is to propose models for quality assessment, which will consider the most significant factors affecting the quality of perception of the offered services. The proposed models within this research will provide a way of measuring and assessing the perception of quality of end-users, also known as Quality of Experience (QoE). The multimedia content delivery is even more difficult and challenging using the cloud-based services because the content should be presented as on-demand multimedia service that will meet user expectation of perceived audio-visual quality.

The major advantage of this approach is that allows the influencing factors from subjective and objective background to be applicable for quality estimation. In this direction, the most significant impact has the use of services that are based on the cloud computing technology, in the process of improving customer perception for quality. The empirical study made within this research aimed to differentiate the most significant factors that have the greatest impact on the quality of the offered services within an academic environment. Our previous research paper [1] has highlighted the scientific contribution of the importance of modelling the Quality of Experience for multimedia cloud-based services, in order to propose a new way of determining the end user quality perception.

However, there has not been a research study that provides complete evaluation methodology for user satisfaction assessment of multimedia cloud-based services. This research goes beyond existing state-of-the-art knowledge on perceived quality for multimedia cloud services delivery systems. Therefore, in this research, we are going to use the benefit of modelling the quality using the Bayesian networks in order to propose improved QoE-based multimedia cloud services quality evaluation methodology.

In this paper, we show how an iterative Bayesian model is used for end-user satisfaction assessment of multimedia cloud services. The research begins, in Section 2, by discussing state of the art approaches for quality estimation. Some of the most relevant research studies on estimating the quality of cloud-based services have been elaborated. We also explore the related work of the most popular cloud-based services for multimedia content delivery. In Section 3, we have discussed the multidimensional aspect of Quality of Experience (QoE).

In Section 3, the results of an empirical research survey that study the influencing factors that affect multimedia cloud-based services are analysed. In Section 4 we then describe the proposed improved model for perception of quality using the Bayesian network. Then we explore the collected empirical data to develop and build the Bayesian network for end used quality assessment. In Section 5 we have provided discussion and validation of our proposed Bayes-based model for end-user satisfaction assessment of multimedia cloud services. Finally, in Section 6 the conclusion and benefits of the proposed model are given.

2 Current Practice and Related Work

Lately, we have observed that university education institutions are largely using the cloud computing technology to improve the process of education. The delivery of user-centric multimedia content is very important because students tend to focus and pay more attention to the learning material. In this manner, education based on the mobile cloud [2] has been introduced as a novel state of the art education in the area of intelligent learning and researchers have developed a theoretical model framework for cloud-based intelligent learning to guide the design and development of an intelligent mobile cloud education system [3]. The proposed theoretical model is using an intelligent engine to create personalized learning plan for the users that can be adjusted in time manner based on learners’ feedback and outcomes [3].

In this direction cloud-based education needs a model to measure the level of quality for the multimedia services that are delivered to users. The more clear understanding of the term quality has been revealed by its use of the Quality of Service (QoS), which defines the ability of the network to classify its traffic and let them traverse the network based on their priority [4]. In order to categorize the traffic, we consider the widely known influencing factors: network availability, bandwidth, delay, jitter, and loss, which have an important part in measuring QoS in a network. An important role for the quality estimation methodology has the perception of quality for delivery of multimedia services. In this manner, a service is considered as an activity rather than a physical object, and so it has four unique features, i.e., intangibility, heterogeneity, inseparability, and perishability [5]. Otherwise, goods or products, which are tangible and have physical dimensions with, predefined quantifiable metrics that could be measured. Seeing this difference, we consider the services as intangibles whose output is viewed as an experience from user’s point of view.

A literature review and analysis on the current state of cloud computing has identified six key performance indicators (PKIs) that are essential for the proposed QoE framework [6]. The statistical analysis results have confirmed that the use of cloud QoE adoption has helped students to be more motivated for learning [6]. The typical representatives of real-time and on-demand multimedia content delivery systems require extensive research on Quality of Experience (QoE) estimation, which is based on selecting the appropriate services that will maximize the user’s satisfaction with the delivered content. They usually are considering only the influence of QoS factors, such as jitter, delay, and packet loss [7].

Today there are various multimedia services for video communication, online video/audio sharing, voice messaging, multimedia broadcasting, etc. The perceived quality evaluation for multimedia services that is exhausting only the network measurement metric is not enough. Furthermore, this process requires taking in consideration the user’s perspective, represented by the subjective parameters for the overall quality estimation process. The subjective methods for assessing the quality of telephony and television systems have been around for a long time and accordingly, and they are using the established perceptual quality models [8].

In order to capture student’s expectation, it is important to understand their cognitive learning styles and depending on the available network conditions to deliver the adapted multimedia content. Using this research approach requires to consider a broad collection of factors that influence the user’s perception of quality for end-to-end delivery of multimedia content. The proposed theoretical framework for QoE measurement has provided only quantitative mappings between QoE and QoS using the correlation analysis [9]. Authors Shin and Huh, in [10], have proposed service quality assessment model named mobile cloud service quality assessment model (mCSQAM). This model has used the ISO/IEC 9126, which is an international standard for software quality assessment, to apply assessment on mobile cloud services with different quality attributes with their own policy [10].

The concept of the Quality of Experience has been identified as a key differentiator of end users’ requirements for multimedia services, in terms of service quality and expectations. Authors Zepernick and Engelke, in [11], have provided a survey on the evolution of QoE concepts for multimedia systems. Their research has presented many accepted standardized QoE issues for speech, image, and video services. There have been many approaches to measure the user-perceived video quality [12]. The correlation between QoS and QoE for video streaming services has been studied by using machine learning classifiers to classify the collected datasets [13]. Another research has associated QoE in the context of cloud-based services for the purpose of accurately mapping the QoE with the performance indicators [14]. According to Suznjevic et al. in [14], statistical user behaviour detection has been used in order to confirm that document editing and web browsing are activities where users are much more tolerant towards network impairments than in the case of audio and video streaming.

Because of the attractiveness of cloud-based services in the academic environment this has brought a challenge for the OTT (Over-The-Top) service providers how end-users perceive the quality of the provided service [15]. Existing research has been focused on determining how the network traffic patterns generated by these services have impact on the network QoS parameters [16]. The bandwidth is one of the major factors for all cloud-based services, and it has very significant influence for less interactive services [16]. These include the data storage and synchronization services and collaborative editing services, which depend strictly on the user perceived interaction. On the other hand, the cloud gaming, Remote Desktop and live telepresence services have been considered as highly interactive multimedia entertainment service [17]. For these services, low round-trip time (RTT) of 150 milliseconds, is within the boundaries of acceptability [16]. Also these services are very sensitive to high packed loss and long delays, which deteriorates the performance of the interactive services. In summary, it has been confirmed that depending on the type of used service, there are parameters that have a significant impact on the end-users perceived quality. Unfortunately, only the measurement of the QoS parameters is not enough to reflect the true satisfaction of end-users for a particular service.

The Quality of Experience for remote virtual desktop services has been studied in a subjective experiment using 52 participants and recorded the user feedback using Mean Opinion Score (MOS) [18]. Authors Casas et al. in [18], the main finding is the correlation between QoE and application QoS requirement. Furthermore, they concluded that interactive remote virtual desktop services are more delay sensitive, high round trip time (RTT) degrades user quality of experience [18]. The mobile cloud streaming service has been assessed by a group of 33 mobile broadband users [19]. The evaluation period lasted for 31 days [19], during that period users regularly reported their perceived QoE assessment on surfing their preferred YouTube and Facebook contents.

Another research has considered the user’s location, satisfaction, technology acceptance and ease-of-use as context-aware influencing factors that represent different context states in the systems. This has allowed them to propose an approach for measurement and prediction on context-aware quality of experience (QoE), which is based on the Bayesian networks to be able to predict the users’ QoE perception under uncertainty [20]. Furthermore, the benefit of Bayesian hierarchical models has been used to understand the influence of the network bandwidth for visual perception, which has led to propose a model of QoE assessment in audio-visual communications [21].

These related studies have confirmed that the Bayesian Network (BN) provides a commonly accepted tool for modelling the overall quality perceived features. Initially, we have proposed QoE model, which takes into consideration the influencing factors that are affecting the user perception, in order to express the importance of the QoE metrics [22]. However, there is not a research that uses BN that analyses subjective survey results for the use of different types of multimedia cloud services.

3 Multidimensional Aspect of Quality of Experience Evaluation

In today’s technology driven society, the users’ expectations, satisfaction, and perceived quality have been recognized as essential contributing factors for the success of the technology. The user-centred content delivery, user expectancies and usability are subjectively estimated, while the more technically oriented research domains have rich traditions in measurements and monitoring on the network (related to QoS). When we observe the quality of experience (QoE) from a technical perspective it -comprises the end-to-end system effects (client, terminal, network, services infrastructure, etc). This gives the QoE broader concept that considers estimation of human preferred quality and context-aware delivery of multimedia content. In order to meet these user requirements, it is necessary to understand the relations and interaction between the influencing factors. Therefore, the main challenge is to establish a connection between objective technically measurable parameters and the subjective measures like usability, user expectations, and user experiences. They together define the Quality of Experience, as a measure of the degree of delight or annoyance of the user of an application or service [23]. Furthermore, the QoE assessment process results from the person’s evaluation of the fulfilment of his or her expectations and needs with respect to the utility (pragmatic and hedonic) in the light of the person’s context, personality and current state [23]. In this way, the QoE influencing factors are roughly grouped into three categories: human-related, system-related and context-related characteristics.

The main idea in this research is to provide an assessment of the perception of the quality of the multimedia content delivery with cloud-based services, which brings us closer to the definition of QoE. Therefore, simply we are going beyond and we are ensuring that not only the technical performance requirements are met. Also, at the same time, we are assuring that QoE is based on adopting a user perspective in judging that the actual needs and expectations of the end-user are encountered. It is important to note that QoE is no longer expressed as a verbal descriptor of satisfaction, but it has a multi-dimensional value that represents the dynamic and transient appeal of quality. Based on the mentioned classification of the three groups of characteristics we have developed the diagram for multimedia quality evaluation methodology, in Figure 1. The provided methodology for the perception of quality is proposed as a layered approach to investigate the influence of the human cognitive style, context-aware factors and the technical system capabilities of the end-user device in the multimedia environment. The overall process of QoE modelling consists of measurement of the system characteristics, gathering the context-aware influence parameters and conducting an assessment of human influence factors.

images

Figure 1 Multimedia quality evaluation methodology.

In order to understand the perception of quality, we need to consider many extended influencing factors that affect the end user’s quality of experience. The methodology has identified that the system characteristics, also known as QoS factors, are directly measurable. The QoS measurement process collects information on network parameters, such as jitter, delay, latency, packet loss, and application parameters, such as resolution, frame rate, video/audio encoding type, sampling rate and a number of channels. The system influencing factors mainly refer to properties and characteristics that determine application-related, network-related and device-related characteristics. The application refers to the software product running on the device and having the interface for the user to provide services. The network is the infrastructure that provides communication between remote devices and applications. The device is the hardware that runs the application and connected to the network in the given context. These characteristics have a high influence on multimedia service distribution at access, content, and transmission points. Therefore, it is important for multimedia OTT (Over-The-Top) providers to understand the impact of these system characteristics over the perception of quality so that they can ensure a higher quality of experience during cloud service delivery.

The context-dependent influencing factors represent any circumstances, situations, and environment at the time of interaction between human and system. These context-aware factors can have static and dynamic nature. The static context does not change often and includes user’s application preferences. In contrast, the dynamic context changes over a period of time and it can exhibit a range of temporal characteristics; it can have several alternative representations, which makes it more difficult to be predicted. The importance of modelling the context factors might provide a selection of appropriate quality levels for the given experience, improving efficiency and reliability of the multimedia system, or provide adaptation to the system characteristics. These conditions are affected by the network bandwidths, because of heterogeneous connections between user’s devices and applications. Another context factor that should be considered in the process of adaptation of the multimedia content is the technical capabilities of the mobile device to support the multimedia format. In order to considerate the technical capabilities, it requires using a methodology for gathering the end-users mobile device technical capabilities.

The human influence factors have the tendency to affect the personalization process based on the context-aware aspects, which ensures a significant impact on the overall perception of quality. These characteristics provide a description of the demographic and socio-economic background, the physical and mental constitution, or the user’s emotional state. They are highly complex because of their subjectivity, which makes them relatively intangible and therefore much more difficult to understanding. Therefore, the human domain consists of various demographic attributes (e.g., age, gender), have different roles (e.g., customer or user), and when interacting with technology, has a strong influence on quality. The subjective assessment of the perception plays an important part of the user system for content delivery, which not only includes user enjoyment and pleasure of a multimedia presentation but also their ability to the perception of content [24].

Together, all of these factors have multidimensional influence in the process of evaluating the perceived quality for delivery of multimedia cloud services. However, the awareness of different nature of these factors and an appropriate categorization might provide patterns and tools that allow measuring or improving the QoE level. Considering the proposed categories of factors that influence the perception of quality can result in very different modelling approaches. Given the complexity and multi-dimensional aspect of QoE metrics, it is a real challenge to provide adequate models for the end user satisfaction assessment of multimedia cloud services.

4 Empirical Research Survey on Using Multimedia Cloud-Based Services

Existing research has confirmed that the most common way to gather user’s subjective quality estimations is typically performed using individual survey questionnaires. Although the service oriented systems have rich traditions in measurements and monitoring on the network related performance indicators. In this research, we have faced with the challenge how to estimate the user’s subjective measures, which usually are expressed as usability, desires, user needs and user experiences. A survey directed to investigate habits and identify the factors that influence the choices of using various multimedia services is proposed. Initially, it has been intended for students of University of Information Science and Technology St. Paul the Apostle in Ohrid to study the interest for multimedia applications and services in the “cloud”. In general, the service delivery model for cloud computing services consists of three areas: IaaS (Infrastructure–as–a–service), PaaS (Platform–as–a–service) and SaaS (Software–as–a–service). This distinction is made according to the type of the infrastructure that has been offered to the end-users.

Today, the most widely used cloud service delivery model, when we relate to end-users, is the Software as a Service model. This model allows the most common software applications to be offered as services to end-users. In other words, the common everyday examples of SaaS model services include e-mail, web-based text processors (Google Docs, Microsoft Office 365 and etc.), video streaming services (YouTube, iTunes and etc.), social networks (Facebook, Twitter and etc.), cloud gaming services and many others. According to the abovementioned distinction of cloud-based services we have structured the survey questionnaire used in this research, given in Appendix 1. The survey was anonymous, and respondents were asked to carefully read and honestly answer to all of the questions. This research has provided relevant information to domain problem, which will investigate the acceptance, experience, and satisfaction of using the multimedia cloud-based services in the higher educational institution.

The continuous evaluation has been used in order to improve the validity of the quality assessment of the end-user perception. Therefore, we have continually conducted the same evaluation with the survey questionnaire to students in the IV year, within the spring semester during 2014–2017 academic year. The yearly survey was conducted at the end of the spring semester because students finish their team projects. This period has been chosen because it reflects student’s experience of collaborative activities done throughout the team project. In this manner, the questionnaire has been developed in order to collect detailed information for their needs and experiences of cloud-based services. Therefore, the survey was focused on detecting the interaction level and the student’s desire for using the cloud-based services during their everyday activities.

4.1 Description of the Survey Questionnaire

The classification of multimedia services in the cloud where end-users have direct interaction determined 9 (nine) types of cloud services:

The survey questionnaire has been organized into three general sections. The first section collected general data about the respondent (gender, faculty, year of study) and general data on the habits of using electronic devices for sharing and exchanging multimedia content with colleagues. This section, collected data for the different number of computing devices (laptop, tablet, mobile and similar devices) and their usage for communication and online entertainment, which included information the frequency of accessing the Internet.

The second section of the survey questionnaire was followed by eight questions, marked with the prefix 7. This section has asked the participants to estimate the students need of cloud services. The general question is: How often have you been in a situation:

The third section of the survey questionnaire contained nine questions, marked with the prefix 8. The questions asked in this section estimate the actual frequency of using various cloud-based services. In this section of the questionnaire, we have listed cloud services and the names of the some of the most popular services, from these categories. The general question that has been stated in the third section is: How would you evaluate the interaction with:

The evaluation of the answers in the second and third sections has been done using a 4-point interval scale. Participants have been asked to give an assessment on a quality rating with regard to the impact of using the interval rating scale. Where the value of 1, on the 4-point scale, represents that the user had rare or less interaction with the service. On the other hand, the value of 4, on the 4-point scale, states that the user had very frequent or high interaction with the service. The values of 2 and 3, on the 4-point scale, represents the occurrence from time to time and often interaction with the service, respectively. This proposed way of grouping and categorization of questions can assist in the identification of student’s interaction situations for different types of cloud-based services. The well-structured questionnaire can provide guidance for better understanding the relationships between these services. In this way, the questionnaire has been devoted to provide a clear description of the objective of the survey and to collect subjective rating for student’s habits of using different cloud-based services.

4.2 Statistical Analysis of the Results of the Survey

The analysis of the questionnaire data begins with the assumption of independent observations of the survey data results. This is because they are not related; in the survey questionnaire, we have used different students every year, which confirms that the survey data from each year have no effect on the one another. During the survey, every score rated by students in the experiment have been independently collected. Each questionnaire has been completed by only one student and the rating process on the cloud-based services is also keep independent from each other. User opinions were collected using the data with interval scales and the categories are ordered numerical labels, with values from 1 to 4. The Pearson correlation on the second section of the survey questionnaire has been given in Table 1.

Table 1 Summarized Pearson’s correlation on the survey questionnaire part two

Question Pearson Correlations (r) and Significance (2-tailed) (p)
*. Correlation is significant at the 0.05 level (2-tailed).
Q_7A 2015 2016 2017 Q_7A 2016 2017 Q_7A 2017
2014 r=-.072 r=-.192 r=-.263 2015 r=.211 r=0.000 2016 r=-.036
n=58 p=.649 p=.164 p=.055 n=42 p=.181 p=1.000 n=54 p=.796
Q_7B 2015 2016 2017 Q_7B 2016 2017 Q_7B 2017
2014 r=.226 r=.106 r=-.236 2015 r=-.183 r=.099 2016 r=-.093
n=58 p=.150 p=.445 p=.085 n=42 p=.245 p=.533 n=54 p=.504
Q_7C 2015 2016 2017 Q_7C 2016 2017 Q_7C 2017
2014 r=.005 r=.088 r=-.028 2015 r=-.049 r=.203 2016 r=-.033
n=58 p=.976 p=.525 p=.843 n=42 p=.759 p=.196 n=54 p=.812
Q_7D 2015 2016 2017 Q_7D 2016 2017 Q_7D 2017
2014 r=-.262 r=-.139 r=.021 2015 r=.324* r=-.160 2016 r=-.238
n=58 p=.094 p=.316 p=.880 n=42 p=.036 p=.312 n=54 p=.084
Q_7E 2015 2016 2017 Q_7E 2016 2017 Q_7E 2017
2014 r=-.073 r=-.045 r=.129 2015 r=.008 r=.218 2016 r=-.189
n=58 p=.647 p=.749 p=.354 n=42 p=.959 p=.164 n=54 p=.170
Q_7F 2015 2016 2017 Q_7F 2016 2017 Q_7F 2017
2014 r=.013 r=.066 r=.070 2015 r=.046 r=.091 2016 r=.058
n=58 p=.937 p=.635 p=.613 n=42 p=.775 p=.566 n=54 p=.676
Q_7G 2015 2016 2017 Q_7G 2016 2017 Q_7G 2017
2014 r=.147 r=-.265 r=-.272* 2015 r=.033 r=.044 2016 r=.042
n=58 p=.354 p=.053 p=.047 n=42 p=.838 p=.783 n=54 p=.761
Q_7H 2015 2016 2017 Q_7H 2016 2017 Q_7H 2017
2014 r= .113 r=-.118 r=.105 2015 r=-.324* r=.048 2016 r=-.164
n=58 p=.476 p=.396 p=.450 n=42 p=.036 p=.765 n=54 p=.237

The Pearson correlation on the third section of the survey questionnaire has been given in Table 2.

Table 2 Summarized Pearson’s correlation on the survey questionnaire part three

Question Pearson correlations (r) and Significance (2-tailed) (p)
*. Correlation is significant at the 0.05 level (2-tailed).
Q_8A 2015 2016 2017 Q_8A 2016 2017 Q_8A 2017
2014 r=-.051 r=-.147 r=-.006 2015 r=-.053 r=-.097 2016 r=.117
n=58 p=.747 p=.290 p=.963 n=42 p=.737 p=.541 n=54 p=.400
Q_8B 2015 2016 2017 Q_8B 2016 2017 Q_8B 2017
2014 r=-.015 r=.057 r=-.051 2015 r=.181 r=-.093 2016 r=-.095
n=58 p=.924 p=.682 p=.713 n=42 p=.252 p=.560 n=54 p=.494
Q_8C 2015 2016 2017 Q_8C 2016 2017 Q_8C 2017
2014 r=.045 r=-.095 r=.107 2015 r=.050 r=-.074 2016 r=.101
n=58 p=.775 p=.492 p=.442 n=42 p=.752 p=.640 n=54 p=.466
Q_8D 2015 2016 2017 Q_8D 2016 2017 Q_8D 2017
2014 r=-.005 r=-.316* r=.241 2015 r=-.127 r=-.023 2016 r=.040
n=58 p=.975 p=.020 p=.079 n=42 p=.425 p=.886 n=54 p=.772
Q_8E 2015 2016 2017 Q_8E 2016 2017 Q_8E 2017
2014 r=-.060 r=.235 r=-.079 2015 r=-.137 r=.167 2016 r=-.171
n=58 p=.705 p=.087 p=.571 n=42 p=.386 p=.290 n=54 p=.290
Q_8F 2015 2016 2017 Q_8F 2016 2017 Q_8F 2017
2014 r= .078 r=-.243 r=.161 2015 r=.033 r=-.076 2016 r=-.166
n=58 p=.622 p=.076 p=.246 n=42 p=.834 p=.632 n=54 p=.229
Q_8G 2015 2016 2017 Q_8G 2016 2017 Q_8G 2017
2014 r= -.164 r=.179 r=.072 2015 r=.132 r=.194 2016 r=.126
n=58 p=.299 p=.195 p=.607 n=42 p=.405 p=.218 n=54 p=.365
Q_8H 2015 2016 2017 Q_8H 2016 2017 Q_8H 2017
2014 r= .054 r=-.177 r=0.000 2015 r=-.061 r=.084 2016 r=-.047
n=58 p=.736 p=.200 p=1.000 n=42 p=.703 p=.596 n=54 p=.734
Q_8I 2015 2016 2017 Q_8I 2016 2017 Q_8I 2017
2014 r=.026 r=-.097 r=.130 2015 r=.157 r=-.120 2016 r=-.013
n=58 p=.871 p=.484 p=.349 n=42 p=.321 p=.450 n=54 p=.928

Correlation is a bivariate analysis that measures the strength of association between two variables and the direction of the relationship. The correlation coefficient, r, is a summary measure that describes the extent of the statistical relationship between two interval or ratio level variables. The correlation coefficient is scaled so that it is always between -1 and +1. When r is close to 0 this means that there is little relationship between the variables and the farther away from 0 r is, in either the positive or negative direction, the greater the relationship between the two variables. The results of p and r for each question have been compared from all consecutive 4 years surveys. First, the study of 2014 with 58 participants has been correlated with the 3 other surveys. Next, the study of 2015 with 42 participants has been correlated with the 2 other surveys. Last, the study of 2016 with 54 participants has been correlated with the survey of 2017 with 54 participants. Based on the results of the correlation analysis it has been developed scatter diagram for questions that have significant correlations between the yearly surveys, marked with stars (*) in Table 1 and Table 2.

The question Q_7D, confirms the importance of sharing with others and commenting on photos or videos in a fast and easy way, between surveys from 2015 and 2016 has shown moderate positive correlation (r=.324*), given in Figure 2. Once again, the necessity for sharing thoughts, ideas and comment on them with a group of friends, observed in question Q_7G between surveys from 2014 and 2017 has shown weak negative correlation (r=-.272*), given in Figure 3. The effective and quick communication with friends and colleagues, asked with question Q_7H between surveys from 2015 and 2016 has shown moderate negative correlation (r=-.324*), given in Figure 4. Furthermore, the service for live video streaming, observed with the question Q_8D, between surveys from 2014 and 2016 has shown moderate negative correlation (r=-.316*), given in Figure 5.

images

Figure 2 Question 7D correlations between surveys from 2015 and 2016.

images

Figure 3 Question 7G correlations between surveys from 2014 and 2017.

images

Figure 4 Question 7H correlations between surveys from 2015 and 2016.

images

Figure 5 Question 8D correlations between surveys from 2014 and 2016.

The conducted correlation analysis does not provide qualitative assessment features, in terms of interaction degree, service complexity, multimedia communication, collaborative working activities, relevant to the specific cloud-based services. This subsection has proven that the statistical tools that examine the four-scale user opinion scores against the conditions of the parametric tests are insufficient for measuring satisfaction of the provided multimedia cloud-based services. However, these survey data results will be used as input datasets for the proposed Bayesian network model in order to provide significantly important feedback for the multimedia cloud-based services.

5 Improved Model for Assessment of Quality Using the Bayesian Network

The process of measuring a degree of belief is commonly referred to as a probability assessment. In the same manner, the stochastic nature of human perception and the interpretation of audio-visual information influences to delight the overall quality and its attributes as probability distributions [25]. Human perception has an important role in the subjective quality experience, which affects the probabilistic estimation of the service. In order to create probabilistic statements for a broad class of distributions one of the powerful aspects of models is the use of graphs. These graphs are the most appropriate way to quantify the relationships between connected nodes and they allow specifying a conditional probability distribution for each node. Therefore, in this paper, we chose to focus on a special class of models called Bayesian Networks (BNs). These models have been mostly used to replace the subjective assessments and to represent knowledge about a specific uncertain domain [26].

5.1 Bayesian Theory

The Bayesian approach of reasoning is parameter-free and the user input is not required, in its place, the prior distributions of the model offer a theoretically justifiable method for affecting the model construction [27]. This allows to simply working with probabilities and hereafter it can be expected to produce smooth and robust visualizations with discrete data values containing nominal and ordinal attributes. Bayesian modelling allows the researcher to analyse both linear and non-linear relationships between variables [27]. The main benefits of using BNs are the clear and intuitive graphical representation of nodes and edges. Each node in the graph presents a random variable while the edges between the nodes represent probabilistic dependencies among the corresponding random variables. The Bayesian approach has no limit for minimum sample size or a maximum number of nodes limitations.

Important research case is segmentation and structuring of low-level features during meetings into more complex group behaviors using a multi-stream modeling framework based on multi-stream dynamic Bayesian networks (DBNs) [28]. This process of reasoning under uncertainty, using the Bayesian Networks, allows them to be widely used for knowledge representation. Also, they are very effective and handle with subjectivity using probabilities. By using the probability theory, the information gets propagated between nodes in the graph, simply as defined by the Bayes’ theorem (5) [26]. It describes how prior knowledge about cause C given the observed evidence E. The theorem relates the conditional and marginal probabilities of course C and evidence E as follows:

P(C|E)=P(E|C)·P(C)P(E)                                       (5)

where P(C) is the prior probability of the cause (the likelihood that C will be in a particular state, prior to consideration of any evidence); P(E|C) is the conditional probability distribution of observed evidence given the cause; and P(C|E) is the posterior probability of the cause (the likelihood that C is in a particular state, conditional on the evidence provided).

The presentation in the form of probabilities gives an explicit representation of uncertainty [28]. Therefore, the BN can be very useful for considering for relationships among a large number of variables. This feature has been used in the proposed model since in the existing survey questionnaire we have identified 17 questions for this research. This stage provides the basis to determine the degree of decomposition and it has been used for the initial construction of the model. The conducted detailed analysis of the questionnaire contributes to the first step of identifying the key questions and relationships that will be used in the BN graphical structure in the first stage of model development. This stage provides a method for representing relationships between variables, called ‘nodes’ in the BN, even if the relationships involve uncertainty. The directed acyclic graph of a BN allows entity representation as a set of vertices (nodes). On the other side identification of the links between different systems, entities have been displayed with links, as a set of arcs (edges). The Bayesian networks represent the causal probabilistic relationship among a set of random variables (nodes).

5.2 Proposed Model for Assessment of Quality Using the Bayesian Network

The initial model for assessment of quality using the Bayesian network is consists of four stages: development, quantification, interrogation and validation stage [1]. The proposed improved model considers learning stage, after quantifying the BN and before interrogation of BN. The structure of the improved model for assessment of quality using the Bayesian network is given in Figure 6. The development stage provides the basis to determine the degree of decomposition and it has been used for the initial construction of the model. It requires establishing a plan how to organize the nodes within the levels of the BN. In this way, the proposed BN model will have complete network and hierarchy of all the nodes. The quantification stage determines the conditional relationship between parent nodes and the child node is defined by a conditional probability table (CPT). Parameter learning requires estimating of the CPT for each node, by considering the link structures and the data. This stage is followed by selecting the sampling algorithm and applying the direct and indirect quantification for quantification of the nodes.

images

Figure 6 Improved model for assessment of quality using the Bayesian network.

Next, follows the learning stage when training dataset is used for training the proposed BN model. The training dataset is a usually smaller set of data acquired from larger dataset. After the learning stage, BN is ready to be analysed and validated. Then follows the interrogation stage consisted of the sensitivity and influence analysis. The main purpose of this analysis is to investigate how sensitive are the conditional probabilities to small changes (i.e., probabilities) in the parameters and/or evidence values [29]. The algorithm proposed by Kjaerulff and van der Gaag [30] that performs simple sensitivity analysis in BN has been implemented and used in GeNIe software tool. Overall, the interrogation stage provides an important role in the fine-tuning and visualizing the strengths of relationships between the nodes in the Bayesian network.

Last is the validation stage that provides information on the accuracy of the proposed BN model. This process starts by loading the results of the survey dataset into Bayesian Network. In order to run the validation step, you have to predetermine the validation method. After selecting the validation method, the next important element in the dialog is a selection of the class nodes. These nodes will be affected by the validation process and appropriate results will be given to them. The accuracy results of the validation contain information are presented with confusion matrix for each of the class nodes, which gives more detailed inside analysis. This matrix confirms how many of the records are correctly and incorrectly classified and values are represented on the Receiver Operating Characteristic (ROC) curves for each of the states of each of the class nodes. This curve plots the true positive rate as a function of the false positive rate. Once that the validation results are compared we gain relevant conclusions for the validity of the proposed improved model for assessment of quality using the Bayesian network.

Having in consideration that the questionnaire was dedicated to estimating user’s perception of cloud-based services, we have considered these questions as evidence for the BN at level 4 (L4). In order to differentiate them in BN, we have decided to use different prefix names and colours for the nodes, as described in Section 3.1. In relation to the proposed QoE Bayes model in Figure 6, the questions of the second section have the prefix 7 in the names and they estimate the user’s desired behaviour. While the questions of the third section have the prefix 8 in the names and they estimate the user’s perceived interaction of using various cloud-based services. Next, the identified and determined relational dependencies of nodes in the proposed BN model are given in Table 3. Any node in a BN is always conditionally independent of its all no descendants given that node’s parents.

Table 3 Relational dependencies of nodes in the proposed BN model

Level in BN Node Name It’s a Parent of
L4 Q_7A – Q_7H, Q_8A – Q_8I /
L3 Sync doc version (SD) Q_7A, Q_7B, Q_7E
L3 Edit doc service (ED) Q_8A, Q_8E, Q_8F
L3 Multimedia sharing (MS) Q_7C, Q_7D
L3 Voice/Video communication (VV) Q_8B, Q_8C
L3 Joint communication (JC) Q_7F, Q_7G, Q_7H
L3 Live presence (LP) Q_8D, Q_8H
L3 Remote Access (RA) Q_8G, Q_8I
L2 Synchronization of files (SF) SD, ED
L2 Collaborative working (CW) ED, MS
L2 Multimedia communication (MC) MS, VV, JC
L2 Online telepresence (OT) LP, RA
L1 Interaction degree (ID) SF, CW, MC, OT
L1 Service complexity (SC) SF, CW, MC, OT
L1 Multimedia Intensity (MI) SF, CW, MC, OT
L0 Overall perception of quality (OPQ) ID, SC, MI

Following the proposed Bayesian-based QoE model, in the third layer (L3) of the BN, we have made classification of nodes from the questionnaire based on the types of cloud-based services. In this layer, we have identified 7 nodes and each of them has two probability states High/Low. This layer is comprised of services for synchronization of document versions, collaborative document editing, multimedia sharing, video-voice communication, joint activity communication, live streaming presence and remote interactive access.

The middle layer (L2) of the proposed BN is used for assessment of the cloud-based services, which forms another more general grouping of the services based on the user’s activities. In this layer, we have identified 4 latent nodes and each of them has two probability states “Normal/Abnormal”. This second layer is comprised of activities for synchronizing cloud files, collaborative working, multimedia communication and online telepresence. The next layer (L1) of the proposed BN consisted of three nodes, which identifies the acceptability of cloud services based on the user preferences. This has been done based on the users’ interaction degree, the complexity of the service and intensity of multimedia content. Finally, the top layer (L0) completes the BN with a node for estimating the overall user-perceived quality of cloud-based services. This categorical order of the questions in the survey questionnaire has been used to arrange the structured organization of the nodes of Bayesian Network. This stage of developing the structure of BN completes with formalized graphical layered structure of the BN for estimating cloud-based services, as given in Figure 7.

images

Figure 7 The graphical layered structure of the Bayesian Network model.

images

Figure 8 BN Learning, final outcome after third stage.

5.3 Development of Bayesian Network for End Used Quality Assessment

The proposed improved Bayesian model development approach for assessment of multimedia cloud-based services was quantified based on the survey questionnaire results. The second stage of quantification starts with the selection of the Likelihood Sampling algorithm and direct quantification of the child nodes (terminal nodes). The marginal probability tables for terminal nodes will have four states, which corresponds to the 4-point scale answer provided in the survey questionnaire. The CPT table for the parent nodes will have 2 × 2 × 4 (in total 16) different states, where 2 is the number of states of the service presence node High/Low, the next 2 is the number of input nodes and 4 is the number of states of terminal nodes, as given in Figure 8. For example, the Voice/Video (VV) communication service is in High state, with very high probability under the condition when nodes Q_8C (On-demand video service) and Q_8B (voice conference service) have high values, or specifically when statistically answers to these questions are closer to 4 in this case.

Following the formalized graphical layered structure of the BN the different types of cloud services, from layer 3, have propagated their evidence to four identified cloud activity services. The main difference in this quantification approach is that highly correlated cloud services typically provide normal cloud service activities. This means, when the service presence nodes that belong to the layer 3 of the BN, have identical states in a majority, i.e. both nodes have a High state. Then the influence of that factor propagates Normal state, with higher probability, for service activity nodes. In the opposed case, when the service presence nodes have opposite states in a majority, i.e. one of the nodes has High state and the other one has a Low state. In this case, the influence of that factor propagates “Abnormal” state, with higher probability, for service activity nodes. This quantification approach has been used to propagate the influence from service presence nodes towards the BN layer with service activity nodes. The CPT table for the service activity nodes will have 2 × 2 × 2 (in total 8) different states, where 2 are states of the service presence nodes, the next 2 is the number of input nodes and 2 is the different states of service activity node, “Normal/Abnormal” respectively.

The last step of the second stage of quantification is using classification of the perceived quality of multimedia cloud services, based on the established subjective features that are important to users. These quality features emphasize the importance of the user’s perceived classification for multimedia cloud services, which have been elaborated in the previous research paper [22]. They include the degree of Multimedia Intensity (MI), which describes to what extent the service based on cloud computing is expressing the different intensity of interaction of various multimedia content. The degree of Service Complexity (SC) that considers the physical and logical service organization in terms of technological features to manage the required services. The Degree of Interactivity (DI) that determines the intensity of the interaction between the end user and the service as well as between different end-users. The indirect quantification with three different states (“High”, “Medium” and “Low”) of the nodes on the first level (L1) of the BN finalizes the quantification stage with this we have prepared BN for interrogation stage. Finally, the top layer (L0) completes the BN with a node for estimating the overall user-perceived quality of multimedia cloud services.

The learning stage is the third step of the improved model for assessment of quality using the Bayesian network. During this stage, we select a training dataset from the survey questionnaire results. In this case, we have made the assessment that the survey data from the first year 2014 would be most suitable for training the proposed BN model. After learning the developed BN with GeNIe software tool we have a completely quantified model that can be interrogated and validated, see Figure 8.

6 Discussion and Validation of Our Proposed Bayes-based Model

For complete interrogation process of BN, the sensitivity analysis can provide very detailed insight into the level of accuracy that is required for the various parameters and provides further knowledge elicitation efforts [31]. The interrogation stage, based on the analysis of the sensitivity parameters and strength of influence, allows applying an appropriate adjustment of the probabilities of the BN network. This is a very important process because it provides valuable insight to understand the responsiveness of BN nodes. The strength of influence is always calculated from the CPT of the child node and essentially it expresses some form of distance between the probability distributions of the child node conditional on the state of the parent node.

The interrogation stage has to be done in a systematic iterative manner, in order to precisely assess the beliefs and preferences underlying the assumptions of a BN model. The adjustment analysis will systematically vary one of the parameters of the Bayesian network while keeping all other parameters fixed. In this way, the adjustment process investigates the properties of a Bayesian network by studying its output probability variations arising from changes in the input probability values. The last step of this stage completes the adjustment analysis process by setting suitable probability values for nodes of BN model. The targeted node for the sensitivity analysis is chosen to be the root node for overall perception of quality. In this direction, the sensitivity analysis has confirmed that the overall experience for the used multimedia cloud services is more sensitive to the nodes in the first level (L1) of the BN. These relevant data for the conducted sensitivity and influence analysis have been used in the adjustment analysis iteration loop. The repeated adjustment of the probabilities has provided robustness assurances of the established BN, as given in Figure 9.

images

Figure 9 Sensitivity analysis of BN.

The proposed model for estimating user’s satisfaction of the multimedia cloud services in the last stage of the development process has been validated. The validation stage has provided evaluation for the proposed BN model using the data from the three surveys. This stage has been focused on the detailed analysis of the multimedia cloud service activity nodes. The general analysis of the ROC curves, presented by the comparison Table 4, has proven that the Synchronization of files (SF) and Online Telepresence (OT) have biggest average AUC value (0.93), which provides excellent validation of the model. Accordingly, the ROC curves for Multimedia Communication (MC) have smaller average AUC value (0.91) that is considered an excellent validation of the model. The ROC curves for Collaborative working (CW) have average AUC value (0.88) which represents a good validation of the model.

Table 4 Comparison of the AUC values of the three surveys, validation stage V

Sync Collaborative Multimedia Online
Survey Year Cloud Files Working Communication Telepresence
2015 0.941106 0.917772 0.811765 0.970252
2016 0.897377 0.855159 0.975735 0.907129
2017 0.981066 0.887784 0.947321 0.918382

The proposed model has achieved improvement with the introduction of the third learning stage. This can be observed by the comparison of the ROC curves. After training the developed BN with the survey data from the first year 2014 the process of validation has been conducted. In this way, the newly proposed Bayesian network confirms the improvement of the model in the case for Sync_cloud_files nodes. The analysis confirmed that survey answers from 2017 have provided an excellent classification. This implies that the BN model is able to make a prediction very correctly using the improved BN model, see Figure 10.

images

Figure 10 Comparison of the ROC curves for sync cloud files nodes (2015, 2016 and 2017 years).

images

Figure 11 Comparison of the ROC curves for collaborative working nodes (2015, 2016 and 2017 years).

The ROC curve analysis of the collaborative working part of the model has shown interesting predictions based on all of the survey datasets, see Figure 11. The ROC curves at the beginning of the prediction, for small FPRate values, provide low TPRates, which makes the model better in identifying the positive classifications only with strong evidence.

The multimedia communication service activities have been validated with the 3 surveys as very good estimators of student’s observations. In this case, the proposed BN model has not shown improvement in the prediction, see Figure 12. The ROC curves at the beginning of the prediction, for large FPRate values (>0.3), provide high TPRates, which makes the model better in identifying the negative classifications. Furthermore, the cloud services for online telepresence can make a prediction very correctly using the proposed BN model. This has been confirmed by the high AUC values from the three surveys and the very good classification visible from the ROC curves, see Figure 13.

images

Figure 12 Comparison of the ROC curves for multimedia communication nodes (2015, 2016 and 2017 years).

images

Figure 13 Comparison of the ROC curves for online telepresence nodes (2015, 2016 and 2017 years).

Overall, Bayesian networks are used in this research as an effective tool for grouping and classifying factors that influence the quality of multimedia cloud services. The QoE modelling of customer perception of quality in cloud services is a very challenging task. We chose to make an assessment of subjective responses using the established Bayes-based network model. The whole process of building the Bayes model was made by identifying the stages of the development approach. Each phase is divided into specific sequential steps that have been implemented to build it. The proposed Bayes-based model was developed by analysing the structure of the questionnaire for subjective assessment of cloud services. In this way, using the available set of measurement indicators, the proposed QoE model can provide relevant information about end-user perception of quality for the specific set of factors we are considering.

Detailed sensitivity analysis for sync cloud files has shown very high sensitivity to questions 8E and 8F, but very low sensitivity for questions 7A and 7B. These findings confirm that the proposed BN model is sensitive to the actual user’s need for multimedia sharing (7C and 7D), and usage of services for live video streaming services (8D). The strength of influence has confirmed that the actual use of the cloud sync services is more important than the student’s aspiration for sharing the last version of the document. The activity for collaborative working analysis shows that multimedia sharing is more sensitive to user’s interests for sharing comments with others.

Furthermore, the services for live presence have shown stronger influence compared to the need for document editing services in the cloud. In general, based on the strength of influence analysis we can observe that students show the highest impression towards the service for on-demand video usage. For them, it is very important to efficiently and quickly communicate with colleagues using a different kind of devices. When observing the online telepresence activity, it is evident equally stronger influence from the live presence services.

7 Discussion

The advantage of this model is that it enables the collection of very important information from the data from empirical research. Additionally, the Bayesian model’s analytical capacity creates a very effective diagnostic tool and a relevant assessment of the desired user quality features. The main benefit of using this model is that it can be used for a detailed assessment of the results of the survey. Furthermore, it can be used as a planning tool by the providers of OTT services to create hypothetical scenarios and simulate results before the action plan is finalized. This approach will provide service providers with quantitative and visual comparisons when they need to make meaningful decisions. This means that on the basis of the values given by the model, they will be able to identify the most sensitive factors affecting the quality of perception.

The nature of the existence of different cloud computing services requires the identification of different impact factors that need to be modelled accordingly. Based on our research, the most appropriate approach for assessing the impacts of objective and subjective factors is the use of Bayes networks. By accurately assessing the quality of service and application perceptions in the “cloud”, greater quality control of the delivered multimedia content is ensured and it achieves good management of cloud computing resources. This research led to the development of QoE indicators that could be used to evaluate the quality of efficient cloud computing-based services by looking at user influencing factors such as the degree of interactivity, the complexity of the service, the domain of application, such as and multimedia intensity that most affect user perception.

Since we come across numerous cloud computing services every day, it is very important to perceive the quality of the service by the end user. Therefore, all these findings have the leading quality management in the development and design of cloud-based computing services. The main contribution of this research is to propose a model for determining the quality of various cloud computing-based services. In doing so, it is important to note that influencing parameters should be selected from the user aspect based on the complexity of the service, the degree of interaction, the intensity of multimedia content and depending on the role of the user. The proposed Bayesian model for measuring the perceived quality was tested for the nine most commonly used cloud computing services, and of course, in the future, it can be expanded for multiple services and for multiple parameters of influence.

References

[1] Karadimce, A., and Davcev, D. (2017). Bayesian Network Model for Estimating User Satisfaction of Multimedia Cloud Services. In Proceedings of the 15th International Conference on Advances in Mobile Computing & Multimedia 3–12. Available at: https://doi.org/10.1145/3151848.315 1850

[2] Abunaser, A., and Alshattnawi, S. (2012). Mobile cloud computing and other mobile technologies: Survey. Journal of Mobile Multimedia, 8(4), 241–252.

[3] Wang, M., and Ng, J. W. (2012). Intelligent mobile cloud education: smart anytime-anywhere learning for the next generation campus environment. In 8th International Conference on Intelligent Environments (IE), 149–156. Available at: https://doi.org/10.1109/IE.2012.8

[4] Noor, R. M., and Khorsandroo, S. (2011). Quality of experience key metrics framework for network mobility user. International Journal of the Physical Sciences, 6(28), 6521–6528. DOI: 10.5897/IJPS11.318

[5] Parry G., Newnes L., and Huang X., ( 2011). Goods, products and services, in Service Design and Delivery, Service Science: Research and Innovations in the Service Economy, M. Macintyre et al., Eds. New York, NY, USA: Springer, 19–29. DOI: 10.1007/978-1-4419-8321-3_2

[6] Safdari, F., and Chang, V. (2014). “Review and Analysis of Cloud Computing Quality of Experience”, SCITEPRESS – Science and Technology Publications, 2014, 83–88. DOI: 10.5220/0004982800830088

[7] El-Khatib, K., Bochmann, G. v., and El-Saddik, A. (2017). A QoS-based Service Composition for Content Adaptation, 331–338. Available at: https://doi.org/10.1109/ICDEW.2007.4401013

[8] ITU-T G.1011 Reference guide to quality of experience assessment methodologies, 2013.

[9] Wu, W., Arefin, A., Rivas, R., Nahrstedt, K., Sheppard, R., and Yang, Z. (2009). Quality of experience in distributed interactive multimedia environments: toward a theoretical framework. In Proceedings of the 17th ACM International Conference on Multimedia, 481–490. DOI: 10.1145/1631272.1631338

[10] Shin, Y. R., and Huh, E. N. (2016). mCSQAM: Service Quality Assessment Model in Mobile Cloud Services Environment. Mobile Information Systems, 1–9. Available at: https://doi.org/10.1155/2016/2517052

[11] Zepernick, H.-J., and Engelke, U. (2014). Quality of Mobile Multimedia Experience: Past, Present and Future. In IEEE International Conference on Communications and Electronics, .

[12] Nam, H., Kim, K.-H., Calin, D., and Schulzrinne, H. (2014). YouSlow: a performance analysis tool for adaptive bitrate video streaming. ACM SIGCOMM Computer Communication Review, 44(4), 111–112. Available at: https://doi.org/10.1145/2740070.2631433

[13] Mushtaq, M. S., Augustin, B., and Mellouk, A. ( 2012). Empirical study based on machine learning approach to assess the QoS/QoE correlation, 1–7. Available at: https://doi.org/10.1109/NOC.2012.6249939

[14] Suznjevic, M., Skorin-Kapov, L., and Humar, I. (2015). Statistical user behavior detection and QoE evaluation for thin client services. Computer Science and Information Systems, 12(2), 587–605. Available at: . https://doi.org/10.2298/CSIS140810018S

[15] Gholami, M. F., Daneshgar, F., Low, G., and Beydoun, G. (2016). Cloud migration process—A survey, evaluation framework, and open challenges. Journal of Systems and Software, 120, 31–69. Available at: https://doi.org/10.1016/j.jss.2016.06.068

[16] Casas, P., and Schatz, R. (2014). Quality of experience in cloud services: survey and measurements. Computer Networks, 68, 149–165. DOI:10.1016/j.comnet.2014.01.008

[17] Hobfeld, T., Schatz, R., Varela, M., and Timmerer, C. (2012). Challenges of QoE management for cloud applications. IEEE Communications Magazine, 50(4). 28–36. Available at: https://doi.org/10.1109/MCOM.2012. 6178831

[18] Casas, P., Seufert, M., Egger, S., and Schatz, R. (2013). Quality of experience in remote virtual desktop services. In IFIP/IEEE International Symposium on Integrated Network Management (IM 2013), 1352–1357.

[19] Casas, P., Sackl, A., Egger, S., and Schatz, R. (2012). YouTube & facebook quality of experience in mobile broadband networks. In Globecom Workshops (GC Wkshps), 1269–1274.

[20] Mitra, K., Zaslavsky, A., and Åhlund, C. (2015). Context-aware QoE modelling, measurement, and prediction in mobile computing systems. IEEE Transactions on Mobile Computing, 14(5), 920–936. Available at: https://doi.org/10.1109/TMC.2013.155

[21] Tasaka, S. (2015). A Bayesian hierarchical model of QoE in interactive audiovisual communications. In IEEE International Conference on Communications (ICC), 6983–6989. Available at: https://doi.org/10.1109/ICC.2015.7249439

[22] Karadimce, A., and Davcev, D. (2016). Perception of quality in cloud computing based services. In Eighth International Conference on Quality of Multimedia Experience (QoMEX), 1–6. Available at: https://doi.org/10.1109/QoMEX.2016.7498925

[23] Le Callet P., Möller S. and Perkis A. (eds), (2012). Qualinet white paper on definitions of quality of experience–output version of the dagstuhl seminar 12181, in European network on quality of experience in multimedia systems and services (COST Action IC 1003).

[24] Sun, Y.-C., and Chen, C.-M. (2010). Assessing learning emotion for both the cognitive styles of visualizer and verbalizer distributed to different types of multimedia learning materials. In 2010 International Symposium on Computer Communication Control and Automation (3CA), 148–151. Available at: https://doi.org/10.1109/3CA.2010.5533628

[25] Eerola, T., Lensu, L., Kamarainen, J. K., Leisti, T., Ritala, R., Nyman, G., and Kälviäinen, H. (2011). Bayesian network model of overall print quality: construction and structural optimisation. Pattern Recognition Letters, 32(11), 1558–1566. Available at: https://doi.org/10.1016/j.patrec.2011.04.006

[26] Kevin Korb, B., and Ann Nicholson, E. (2010). Bayesian Artificial Intelligence, in Second Edition (2nd ed.). CRC Press, Inc., Boca Raton, FL, USA,. ISBN:1439815917.

[27] Nokelainen, P., Silander, T., Ruohotie, P., and Tirri, H., (2003). Investigating Non-linearities with Bayesian Networks. In 11th Annual Convention of the American Psychology Association at Toronto, Division of Evaluation, Measurement and Statistics. Toronto, Canada.

[28] Dielmann, A., and Renals, S. (2007). Automatic meeting segmentation using dynamic Bayesian networks. IEEE Transactions on Multimedia, 9(1), 25–36. DOI: 10.1109/TMM.2006.886337

[29] Castillo, E., Gutiérrez, J. M., and Hadi, A. S. (1997). Sensitivity analysis in discrete Bayesian networks. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 27(4), 412–423. Available at: https://doi.org/10.1109/3468.594909

[30] Kjærulff, U., and van der Gaag, L. C. (2000). Making Sensitivity Analysis Computationally Efficient. In Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, 317–325.

[31] Coupe, V. M. H., van der Gaag, L. C., and Habbema, J. D. F. (2000). Sensitivity analysis: an aid for probability elicitation. Knowledge Engineering Review, (15), 1–18.

Appendix 1: Questionnaire for Using Services in the Cloud

The survey questionnaire is conducted to investigate which aims at identifying the factors that influence the choice and use of services in the “cloud”. The survey is intended for students of University of information science and technology St. Paul the Apostle in Ohrid. It is anonymous, and respondents are asked to carefully read and honest answers to all questions. Since the issues related to your behaviour, experiences, attitudes, and perceptions, there are no wrong answers. The survey consists of a general part and the three groups (sites) issues. To fill out the survey is required 10–15 minutes.

  1. Gender: images
  2. Nationality group: images
  3. Year of study: images
  4. Faculty: images

    images

  5. How often are you online:

    (1 answer)

    images

  6. Which all devices you are accessing the Internet, applications, and your data?

    (multiple answers)

  7. How often have you been in a situation that (mark the frequency of these situations on a scale:

    1 = never; 2 = rarely; 3 = sometimes; 4 = often or very often):

    1. You need to manually copy and synchronize versions of your documents stored on different computers or devices?

      images

    2. You did not in have the document or the last version of document that you need at a given time?

      images

    3. Want to create a collection of your favorite music or movies and listen to / watch them on different computers and devices without the use of removable media memory?

      images

    4. Do you want to share with others and comment on photos or videos in a fast and easy way?

      images

    5. Would together edit the same documents, presentations, spreadsheets with collaborators?

      images

    6. You need to work with colleagues to develop a joint project or task?

      images

    7. Want to share your thoughts, ideas and comment on them with a group of friends?

      images

    8. You need to effectively and quickly communicate with friends and colleagues, no matter where you are and what your computer or mobile device you are using.

      images

  8. How often (mark the frequency of these situations on a scale:

    1 = never; 2 = rarely; 3 = sometimes; 4 = often or very often):

    1. Communicate via a web-mail services such as Gmail, Hotmail or YahooMail

      images

    2. Communicate via social networks like Facebook, Twitter, MySpace or Google+

      images

    3. Use the service to store, sync and share your data, such as Dropbox, SkyDrive or Google Drive

      images

    4. Use the service to share files such as Flickr, Instagram or Picasa

      images

    5. Use services for the exchange of audio/video content,

      Example: You Tube, iTunes, RealPlayer Cloud or Deezer

      images

    6. Use online word processing application, such as Google Docs, Office 365 Word or Zoho Writer

      images

    7. Use other online office applications (spreadsheets, presentations, forms) from GoogleDocs, ZohoDoc or Office 365

      images

    8. Use the system for e-learning, such as Moodle, Coursera, EdX…

      images

    9. Use the application to create the intellectual/mental maps such as Mindomo, Google MindMap or MindMeister

      images

    10. Use the application to find location such as Google Earth, Google Maps, MyMaps, GPS Location

      images

*The results from this survey will be used for scientific research purposes in benefit of the UIST-Ohrid.

Biographies

images

Aleksandar Karadimce is a research and teaching assistant at the University of Information Science and Technology “St. Paul the Apostle” – Ohrid, Macedonia. Also, he is a PhD candidate at the Faculty of Computer Science and Engineering, University Ss Cyril and Methodius in Skopje on Computer Science and Engineering department. His research interests lie in the area of computer science and engineering, mobile network communications, cloud computing, mobile computing, quality of experience (QoE) in delivery of multimedia content. As a researcher, he is focused on multimedia content media services and towards expanding the quality of experience expertise into smart mobile domains, in particular, those related to mobile cloud services. He has published more than 30 research papers in International Conferences proceedings or International Journals in Computer Science and Engineering. Actively participates in COST Actions, IEEE and ACM community.

images

Danco P. Davcev obtained the degree of Engineer in Electronics and Computer Science from the Faculty of Electrical Engineering at the University of Belgrade (YU) in 1972. He received his “Doctor – Engineer” degree in Computer Science from the University of Paris 11 (Paris, France) in 1975 and Ph.D. degree in Informatics from the University of Belgrade (YU) in 1981. He is currently a professor in Computer Science and Engineering at the University “Ss. Cyril & Methodius”, Skopje. He has more than 350 research papers presented on International Conferences or published in International Journals in Computer Science and Informatics. His research interests include Distributed Systems, Cloud computing, mobile computing etc. He is a Senior member of ACM and a Senior Member of IEEE.

Journal of Mobile Multimedia, Vol. 14_2, 157–196.
doi: 10.13052/jmm1550-4646.1422
© 2018 River Publishers

Abstract

Keywords

1 Introduction

2 Current Practice and Related Work

3 Multidimensional Aspect of Quality of Experience Evaluation

images

4 Empirical Research Survey on Using Multimedia Cloud-Based Services

4.1 Description of the Survey Questionnaire

4.2 Statistical Analysis of the Results of the Survey

images

images

images

images

5 Improved Model for Assessment of Quality Using the Bayesian Network

5.1 Bayesian Theory

5.2 Proposed Model for Assessment of Quality Using the Bayesian Network

images

images

images

5.3 Development of Bayesian Network for End Used Quality Assessment

6 Discussion and Validation of Our Proposed Bayes-based Model

images

images

images

images

images

7 Discussion

References

Appendix 1: Questionnaire for Using Services in the Cloud

Biographies