Skip to main content

Advertisement

SpringerLink
Log in

What did you expect?

Modelling quality of experience for virtual reality using the repertory grid technique

  • Research Article
  • Published:
Quality and User Experience Aims and scope Submit manuscript

Abstract

The development of immersive technologies has brought with it the need to redefine the concept of quality of experience (QoE). Studies have explored QoE in virtual reality (VR) by adopting a top-down approach—these are solely based on existing frameworks and theory, and complemented with novel technical considerations. It can be argued that any QoE framework derived in this manner is limited, as its scope is fixed even prior to any data gathering process. To this end, the current study proposes a bottom-up approach, involving the user in the formulation of a broader QoE model. The repertory grid technique (RGT) was used to analyse and group 360 attributes, listed by participants as criteria they used in judging the quality of a VR experience. The advantage of RGT is that it has a holistic approach towards the interpretation of the user’s experience combined with the precision of quantitative analysis. The study resulted in a QoE model that consists of three main groups of attributes (i.e., user, content, and system). Furthermore, the analysis showed that participants listed attributes related to their experience and appraisal of VR, and to the content that they viewed. In contrast, very few system-related attributes were mentioned. Finally, the current study discussed the RGT methodology—and user-driven approaches in general—as a complementary research approach to create a comprehensive and practical QoE model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. 360 Labs: As it is—a grand Canyon VR documentary—YouTube

  2. Aldridge R, Davidoff J, Ghanbari M, Hands D, Pearson D (1995) Recency effect in the subjective assessment of digitally-coded television pictures. In: IEE conference publication, vol 410. IEE, pp 336–339. https://doi.org/10.1049/cp:19950676

  3. Baobab Studios: Crow: the Legend

  4. BBC Home: BBC Home—a VR spacewalk on oculus rift—oculus

  5. Binstock A (2015) Powering the Rift. https://www.oculus.com/en-us/blog/powering-the-rift/

  6. Bowman DA, Hodges LF (1997) An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments. In: Proceedings of the 1997 symposium on interactive 3D graphics, pp 35–ff

  7. Chessa M, Maiello G, Borsari A, Bex PJ (2019) The perceptual quality of the oculus rift for immersive virtual reality. Hum Comput Interact. https://doi.org/10.1080/07370024.2016.1243478

    Article  Google Scholar 

  8. Dangxiao W, Yuan G, Shiyi L, Yuru Z, Weiliang X, Jing X (2019) Haptic display for virtual reality: progress and challenges. Virtual Real Intell Hardw 1(2):136. https://doi.org/10.3724/sp.j.2096-5796.2019.0008

    Article  Google Scholar 

  9. De Marez L, De Moor K (2007) The challenge of user-and QoE-centric research and product development in today’s ICT-environment. Observatorio (OBS*) 1(3). https://doi.org/10.15847/obsobs132007141

  10. Egan D, Brennan S, Barrett J, Qiao Y, Timmerer C, Murray N (2016) An evaluation of Heart Rate and ElectroDermal Activity as an objective QoE evaluation method for immersive virtual reality environments. In: 2016 8th International conference on quality of multimedia experience, QoMEX 2016. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/QoMEX.2016.7498964

  11. Fallman D, Waterworth J (2005) Dealing with user experience and affective evaluation in HCI design: a repertory grid approach. Focus 1:2–7 March

    Google Scholar 

  12. Fallman D, Waterworth J (2010) Capturing user experiences of mobile information technology with the repertory grid technique. Hum Technol Interdiscip J Hum ICT Environ. https://doi.org/10.17011/HT/URN.201011173094

    Article  Google Scholar 

  13. Felix and Paul Studios: Jurassic world: apatosaurus on oculus go—oculus

  14. Gaines BR, Shaw ML (1993) Eliciting knowledge and transferring it effectively to a knowledge-based system. IEEE Trans Knowl Data Eng 5(1):4–14. https://doi.org/10.1109/69.204087

    Article  Google Scholar 

  15. Hamam A, El Saddik A, Aljáam J (2014) A quality of experience model for haptic virtual environments. ACM Trans Multimed Comput Commun Appl 10(3):1–23. https://doi.org/10.1145/2540991

    Article  Google Scholar 

  16. Huynh-Thu Q, Garcia MN, Speranza F, Corriveau P, Raake A (2011) Study of rating scales for subjective quality assessment of high-definition video. IEEE Trans Broadcast 57(1):1–14. https://doi.org/10.1109/tbc.2010.2086750

    Article  Google Scholar 

  17. Israr A, Schwemler Z, Mars J, Krainer B (2016) VR360HD: a VR360\(^\circ\) player with enhanced haptic feedback. In: Proceedings of the ACM symposium on virtual reality software and technology, VRST, vol 02-04-Nove, pp 183–186. https://doi.org/10.1145/2993369.2993404

  18. Itu-t (2009) Subjective video quality assessment methods for multimedia applications. Tech. rep., International Telecommunication Union Telecomunication Standardization Sector

  19. Itu-t (2016) G.1011: reference guide to quality of experience assessment methodologies. Tech. rep., International Telecommunication Union Telecomunication Standardization Sector

  20. ITU-T (2016) P.916 Information and guidelines for assessing and minimizing visual discomfort and visual fatigue from 3D video. Tech. rep., International Telecommunication Union

  21. Jain R (2004) Quality of experience. IEEE Multimed. https://doi.org/10.1109/mmul.2004.1261114

    Article  Google Scholar 

  22. Jang Y, Park E (2019) An adoption model for virtual reality games: the roles of presence and enjoyment. Telematics Inform 42:101239. https://doi.org/10.1016/j.tele.2019.101239

    Article  Google Scholar 

  23. Kawaf F, Tagg S (2017) The construction of online shopping experience: a repertory grid approach. Comput Hum Behav 72:222–232. https://doi.org/10.1016/j.chb.2017.02.055

    Article  Google Scholar 

  24. Kelly G (1955) Personal construct psychology. Norton, Nueva York

    Google Scholar 

  25. Marsden D, Littler D (2000) Repertory grid technique—an interpretive research framework. Eur J Mark 34(7):816–834. https://doi.org/10.1108/03090560010331261

    Article  Google Scholar 

  26. Muhanna MA (2015) Virtual reality and the CAVE: taxonomy, interaction challenges and research directions. J King Saud Univ Comput Inf Sci 27(3):344–361. https://doi.org/10.1016/j.jksuci.2014.03.023

    Article  Google Scholar 

  27. Papadopoulos L (2020) This AR/VR full body haptic suit is a breakthrough in human performance training. https://interestingengineering.com/video/this-ar-vr-full-body-haptic-suit-is-a-breakthrough -in-human-performance-training

  28. Paroz A, Potter LE (2017) Cybersickness and migraine triggers: exploring common ground. In: ACM international conference proceeding series. Association for Computing Machinery, pp 417–421. https://doi.org/10.1145/3152771.3156148

  29. Perkis A, Timmerer C, Barakovic S, Husic J.B, Bech S, Bosse S, Botev J, Brunnström K, da Silva Cruz LA, Moor KD, de Polo Saibanti A, Durnez W, Egger-Lampl S, Engelke U, Falk TH, Hameed A, Hines A, Kojic T, Kukolj D, Liotou E, Milovanovic D, Möller S, Murray N, Naderi B, Pereira M, Perry SW, Pinheiro AMG, Palacios AP, Raake A, Agrawal SR, Reiter U, Rodrigues R, Schatz R, Schelkens P, Schmidt S, Sabet SS, Singla A, Skorin-Kapov L, Suznjevic M, Uhrig S, Vlahovic S, Voigt-Antons J, Zadtootaghaj S (2020) QUALINET white paper on definitions of immersive media experience (IMEx). CoRR. arXiv:abs/2007.07032

  30. Pinson MH, Janowski L, Pepion R, Huynh-Thu Q, Schmidmer C, Corriveau P, Younkin A, Le Callet P, Barkowsky M, Ingram W (2012) The influence of subjects and environment on audiovisual subjective tests: an international study. IEEE J Sel Top Sign Proces 6(6):640–651. https://doi.org/10.1109/jstsp.2012.2215306

    Article  Google Scholar 

  31. Pinson MH, Wolf S (2003) Comparing subjective video quality testing methodologies. In: Ebrahimi T, Sikora T (eds) Visual Communications and image processing 2003, vol 5150. International Society for Optics and Photonics, SPIE, pp 573–582. https://doi.org/10.1117/12.509908

  32. Raake A, Egger S (2014) Quality and quality of experience. Springer, Cham, pp 11–33. https://doi.org/10.1007/978-3-319-02681-7_2

    Book  Google Scholar 

  33. Rambli DRA, Muhaiyuddin NDM (2014) Story line in image-based virtual Reality for enhancing spatial presence experience. In: 2014 International conference on computer and information sciences, ICCOINS 2014—a conference of world engineering, science and technology congress, ESTCON 2014—proceedings. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ICCOINS.2014.6868404

  34. Recommendation I (2008) E. 800: definitions of terms related to quality of service. International Telecommunication Union’s Telecommunication Standardization Sector (ITU-T) Std

  35. Reiter U, Brunnström K, De Moor K, Larabi MC, Pereira M, Pinheiro A, You J, Zgank A (2014) Factors influencing quality of experience. Springer, Cham, pp 55–72. https://doi.org/10.1007/978-3-319-02681-7_4

    Book  Google Scholar 

  36. Rodríguez D, Rosa R, Costa E, Abrahão J, Bressan G (2014) Video quality assessment in video streaming services considering user preference for video content. IEEE Trans Consum Electron 60(3):436–444. https://doi.org/10.1109/TCE.2014.6937328

    Article  Google Scholar 

  37. Singla A, FremeSinrey S, Robitza W, Raake A (2017) Measuring and comparing QoE and simulator sickness of omnidirectional videos in different head mounted displays. In: 2017 9th International conference on quality of multimedia experience, QoMEX 2017. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/QoMEX.2017.7965658

  38. Skorin-Kapov L, Varela M, Hoßfeld T, Chen KT (2018) A survey of emerging concepts and challenges for QoE management of multimedia services. ACM Trans Multimed Comput Commun Appl 14(2s):1–29. https://doi.org/10.1145/3176648

    Article  Google Scholar 

  39. Sloan L, Rowland W, Altman A (1952) Comparison of three types of test target for the measurement of visual acuity. Q Rev Ophthalmol 8:4–15

    Google Scholar 

  40. Steed A, McDonnell J (2003) Experiences with repertory grid analysis for investigating effectiveness of virtual environments. Tech. rep., Institute for Telecommunication Sciences (ITS), National Telecommunications and Information Administration (NTIA), U.S. Department of Commerce

  41. Strohmeier D (2011) Open profiling of quality : a mixed methods research approach for audiovisual quality evaluations. PhD thesis, Ilmenau University of Technology

  42. Strohmeier D, Jumisko-Pyykkö S, Kunze K (2010) Open profiling of quality: a mixed method approach to understanding multimodal quality perception. Adv Multimed 2010:1–28. https://doi.org/10.1155/2010/658980

    Article  Google Scholar 

  43. STX Entertainment: THE LIMIT: A Virtual Reality Film

  44. Tan FB, Gordon Hunter M (2002) The repertory grid technique: a method for the study of cognition in information systems. MIS Q Manag Inf Syst 26(1):39–57. https://doi.org/10.2307/4132340

    Article  Google Scholar 

  45. Toast: Richie’s Plank Experience—Steam

  46. Varela M, Skorin-Kapov L, Ebrahimi T (2014) Quality of service versus quality of experience. In: quality of service versus quality of experience. Springer, Cham, pp 85–96. https://doi.org/10.1007/978-3-319-02681-7_6

  47. Varga, S.: HaptX j Haptic gloves for VR training, simulation,and design (2019). https://haptx.com/https://haptx.com/%0Ahttps://haptx.com/%0Ahttp://files/1821/haptx.com.html

  48. Veryzer RW, De Mozota BB (2005) The impact of user-oriented design on new product development: an examination of fundamental relationships. J Prod Innov Manag 22(2):128–143. https://doi.org/10.1111/j.0737-6782.2005.00110.x

    Article  Google Scholar 

  49. Wechsung I, De Moor K (2014) Quality of experience versus user experience. In: Quality of experience versus user experience. Springer, Cham, pp 35–54. https://doi.org/10.1007/978-3-319-02681-7_3

  50. Zielinski S, Rumsey F, Bech S (2008) On some biases encountered in modern listening tests. J. Audio Eng Soc. https://doi.org/10.17743/jaes.2015.0094

    Article  Google Scholar 

Download references

Author information

Author notes

  1. Jolien De Letter and Aleksandra Zheleva authors contributed equally to the manuscript.

Authors and Affiliations

  1. imec-mict-UGent, Department of Communication Sciences, Ghent University, Ghent, Belgium

    Jolien De Letter, Aleksandra Zheleva, Mathias Maes, Anissa All, Lieven De Marez & Wouter Durnez

Authors
  1. Jolien De Letter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aleksandra Zheleva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mathias Maes
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anissa All
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lieven De Marez
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wouter Durnez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aleksandra Zheleva.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research work is funded by the research project imec ICON ILLUMINATE HBC.2018.0201.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

De Letter, J., Zheleva, A., Maes, M. et al. What did you expect?. Qual User Exp 6, 5 (2021). https://doi.org/10.1007/s41233-021-00045-6

Download citation

  • Received:

  • Published:

  • DOI: https://doi.org/10.1007/s41233-021-00045-6

Keywords

Search

Navigation