Strategies and Metrics for Evaluating the Quality of Experience in Virtual Reality Applications

  • Xiangjie KongEmail author
  • Yuqing Liu
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 876)


The development of virtual reality makes it possible to apply the technology in many fields, but the experience of users received limited attention. Quality of experience (QoE) is an evaluation metrics can be used to evaluate applications and services from the perspective of users. By an intensive analysis of the literature about user experience models and QoE related projects, three strategies for QoE evaluation were put forward: emphasizing user needs, emphasizing system characteristics and emphasizing emotional experience. As an example, a set of metrics for the evaluation of astronaut virtual training system (AVTS) are designed under the guidance of the strategies, which consists of five first-level indicators and ten second-level indicators and can be used to quantify the quality of experience of the astronaut virtual training system.


Quality of experience Evaluation strategy Evaluation metrics Virtual reality 



This work was supported by the Foundation of National Key Laboratory of Human Factors Engineering, Grant NO. SYFD160051802.


  1. 1.
    Qinping, Z.: Review virtual reality. Sci. Chin. Ser. F-Inf. Sci. 39(1), 2–46 (2009). (in Chinese)zbMATHGoogle Scholar
  2. 2.
    Qualinet White Paper on Definitions of Quality of Experience (2012). European Net-work on Quality of Experience in Multimedia Systems and Services (COST Action IC 1003), Patrick Le Callet, Sebastian Möller and Andrew Perkis, eds., Lausanne, Switzer-land, Version 1.2, (2013)Google Scholar
  3. 3.
    International Organization for Standardization. Ergonomics of human system interaction - Part 210: Human-centered design for interactive systems (formerly known as 13407). ISO F ± DIS 9241-210 (2009)Google Scholar
  4. 4.
    Garrett, J.J.: Elements of User Experience: User-Centered Design for the Web and Beyond. Pearson Education, London (2010)Google Scholar
  5. 5.
  6. 6.
    Quesenbery, W.: The five dimensions of usability. In: Content and Complexity, pp. 93–114. Routledge (2014)Google Scholar
  7. 7.
    Anderson, S.P.: Seductive Interaction Design: Creating Playful, Fun, and Effective User Experiences. Pearson Education, London (2011)Google Scholar
  8. 8.
    Krug, S.: Don’t Make Me Think! A Common Sense Approach to Web Usability. Pearson Education India, Delhi (2000)Google Scholar
  9. 9.
    Sheikh, H., Bovik, A.: Image information and visual quality. IEEE Trans. Image Process. 15(2), 430–444 (2006)CrossRefGoogle Scholar
  10. 10.
    Wang, Z., Bovik, A.C., Sheikh, H.R., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  11. 11.
    Zujovic, J., Pappas, T.N., Neuhoff, D.L.: Structural texture similarity metrics for image analysis and retrieval. IEEE Trans. Image Process. 22(7), 2545–2558 (2013)CrossRefGoogle Scholar
  12. 12.
    Chen, W., Fournier, J., Barkowsky, M., Le Callet, P.: Quality of experience model for 3DTV. In: Woods, A.J., Holliman, N.S., Favalora, G.E. (eds.) SPIE Stereoscopic Displays and Applications XXIII, 82881P (2012)Google Scholar
  13. 13.
    Keighrey, C., Flynn, R., Murray, S., Murray, N.A.: QoE evaluation of immersive augmented and virtual reality speech and language assessment applications. In: Ninth International Conference on Quality of Multimedia Experience. IEEE (2017)Google Scholar
  14. 14.
    Hamam, A., Saddik, A.E.: Evaluating the quality of experience of haptic-based applications through mathematical modeling. In: IEEE International Workshop on Haptic Audio Visual Environments and Games, pp. 56–61. IEEE (2012)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.National Key Laboratory of Human Factors EngineeringChina Astronaut Research and Training CenterBeijingChina

Personalised recommendations