Skip to main content
SpringerLink
Log in

Spherical image QoE approximations for vision augmentation scenarios

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Augmented Reality (AR) devices are commonly head-worn to overlay context-dependent information into the field of view of the device operators. One particular scenario is the overlay of still images, for which we evaluate the interplay of user ratings as Quality of Experience (QoE) with (i) the non-referential BRISQUE objective image quality metric as Quality of Service (QoS) and (ii) human subject dry electrode EEG signals gathered with a commercial off-the-shelf device. We employ basic machine learning approaches to perform QoE and QoS predictions based on this data. We find strong correlations for QoS inputs with aggregated user ratings as Mean Opinion Scores with spherical images. For subject-specific EEG portfolios, overall predictability of the QoE for both media types can be attained. Our overall results can be employed in practical scenarios by content and network service providers to optimize the user experience in augmented reality scenarios with a passive human in-the-loop in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Acqualagna L, Bosse S, Porbadnigk AK, Curio G, Müller KR, Wiegand T, Blankertz B (2015) EEG-based classification of video quality perception using steady state visual evoked potentials (SSVEPs). J Neural Eng 12(2):026012

    Article  Google Scholar 

  2. Antons JN, Arndt S, Schleicher R, Möller S (2014) Brain activity correlates of quality of experience. In: Möller S, Raake M (eds) Quality of experience advanced concepts, applications and methods. Springer International Publishing. https://doi.org/10.1007/978-3-319-02681-7_8, pp 109–119

  3. Arampatzis A, Kamps J (2009) A signal-to-noise approach to score normalization. In: Proceedings of the 18th ACM conference on information and knowledge management, CIKM ’09. https://doi.org/10.1145/1645953.1646055. ACM, New York

  4. Arnau-Gonzalez P, Althobaiti T, Katsigiannis S, Ramzan N (2017) Perceptual video quality evaluation by means of physiological signals. In: 2017 9th international conference on quality of multimedia experience (qoMEX), pp 1–6. https://doi.org/10.1109/QoMEX.2017.7965651

  5. Bauman B, Seeling P (2017) Towards still image experience predictions in augmented vision settings. In: Proceedings of the IEEE consumer communications and networking conference (CCNC). Las Vegas, NV, USA, pp 1–6

  6. Bauman B, Seeling P (2017) Visual interface evaluation for wearables datasets: predicting the subjective augmented vision image qoe and qos. Future Internet 9(3):40

    Article  Google Scholar 

  7. Blankertz B, Acqualagna L, Dähne S, Haufe S, Schultze-Kraft M, Sturm I, Ušćumlic M, Wenzel MA, Curio G, Müller KR (2016) The Berlin brain-computer interface: progress beyond communication and control. Frontiers in Neuroscience, p 10

  8. Bosse S, Müller KR, Wiegand T, Samek W (2016) Brain-computer interfacing for multimedia quality assessment. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC). https://doi.org/10.1109/SMC.2016.7844669, pp 002834–002839

  9. Brunnström K, Beker SA, De Moor K, Dooms A, Egger S, Garcia MN, Hossfeld T, Jumisko-Pyykkö S, Keimel C, Larabi C et al (2013) Qualinet white paper on definitions of quality of experience

  10. Chen Y, Wu K, Zhang Q (2015) From qos to qoe: a tutorial on video quality assessment. IEEE Commun Surv Tutorials 17(2):1126–1165

    Article  Google Scholar 

  11. Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: a classification, review, and performance comparison. IEEE Trans Broadcast 57(2):165–182. https://doi.org/10.1109/TBC.2011.2104671

    Article  Google Scholar 

  12. Davis P, Creusere CD, Kroger J (2016) The effect of perceptual video quality on EEG power distribution. In: 2016 IEEE international conference on image processing (ICIP). Phoenix, AZ, USA, pp 2420–2424

  13. Eiris Pereira R, Moore HF, Gheisari M, Esmaeili B (2019) Development and usability testing of a panoramic augmented reality environment for fall hazard safety training. In: Mutis I, Hartmann T (eds) Advances in informatics and computing in civil and construction engineering. Springer International Publishing, Cham, pp 271–279

  14. Engelke U, Nguyen H, Ketchell S (2017) Quality of augmented reality experience: a correlation analysis. In: 2017 9th international conference on quality of multimedia experience (qoMEX), pp 1–3, DOI https://doi.org/10.1109/QoMEX.2017.7965638, (to appear in print)

  15. Fiedler M, Hossfeld T, Tran-Gia P (2010) A generic quantitative relationship between quality of experience and quality of service. IEEE Netw 24(2):36–41

    Article  Google Scholar 

  16. Han Y, Yuan Z, Muntean GM (2016) An innovative no-reference metric for real-time 3d stereoscopic video quality assessment. IEEE Trans Broadcast 62(3):654–663. https://doi.org/10.1109/TBC.2016.2529294

    Article  Google Scholar 

  17. He L, Gao F, Hou W, Hao L (2014) Objective image quality assessment: a survey. Int J Comput Math 91(11):2374–2388. https://doi.org/10.1080/00207160.2013.816415

    Article  MathSciNet  MATH  Google Scholar 

  18. Holm A, Lukander K, Korpela J, Sallinen M, Müller K (2009) Estimating brain load from the eeg. Sci World J 9:639–651

    Article  Google Scholar 

  19. Irshad S, Awang Rambli DR, Muhamad Nazri NIA, Mohd Shukri SRb, Omar Y (2018) Measuring user experience of mobile augmented reality systems through non-instrumental quality attributes. In: Abdullah N, Wan Adnan WA, Foth M (eds) User science and engineering. Springer Singapore, Singapore, pp 349–357

  20. ITU-T (2008) Recommendation ITU-T P.910: subjective video quality assessment methods for multimedia applications

  21. ITU-T (2012) Recommendation ITU-R BT.500-13: methodology for the subjective assessment of the quality of television picture

  22. Jain R (1991) The art of computer systems performance analysis: techniques for experimental design, measurement, simulation, and modeling. Wiley

  23. Kroupi E, Hanhart P, Lee JS, Rerabek M, Ebrahimi T (2014) Eeg correlates during video quality perception. In: 2014 22nd European signal processing conference (EUSIPCO), pp 2135–2139

  24. Kumar N, Kumar J (2016) Measurement of cognitive load in hci systems using eeg power spectrum: an experimental study. Procedia Comput Sci 84:70–78. https://doi.org/10.1016/j.procs.2016.04.068

    Article  Google Scholar 

  25. Lin W, Kuo CCJ (2011) Perceptual visual quality metrics: a survey. J Vis Commun Image Represent 22(4):297–312. https://doi.org/10.1016/j.jvcir.2011.01.005

    Article  Google Scholar 

  26. Lindemann L, Magnor MA (2011) Assessing the quality of compressed images using EEG. In: Proceedings of the IEEE international conference on image processing (ICIP), pp 3109–3112. https://doi.org/10.1109/ICIP.2011.6116324

  27. Mazher M, Aziz AA, Malik AS, Amin HU (2017) An eeg-based cognitive load assessment in multimedia learning using feature extraction and partial directed coherence. IEEE Access 5:14819–14829

    Article  Google Scholar 

  28. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708. https://doi.org/10.1109/TIP.2012.2214050

    Article  MathSciNet  MATH  Google Scholar 

  29. Moldovan AN, Ghergulescu I, Weibelzahl S, Muntean CH (2013) User-centered eeg-based multimedia quality assessment. In: 2013 IEEE international symposium on broadband multimedia systems and broadcasting (BMSB). https://doi.org/10.1109/BMSB.2013.6621743, pp 1–8

  30. Moldovan AN, Ghergulescu I, Muntean CH (2016) Vqamap: a novel mechanism for mapping objective video quality metrics to subjective mos scale. IEEE Trans Broadcast 62(3):610–627. https://doi.org/10.1109/TBC.2016.2570002

    Article  Google Scholar 

  31. Moon SE, Lee JS (2017) Implicit analysis of perceptual multimedia experience based on physiological response: a review. IEEE Trans Multimed 19(2):340–353. https://doi.org/10.1109/TMM.2016.2614880

    Article  Google Scholar 

  32. Nagelkerke N (1991) A note on a general definition of the coefficient of determination. Biometrika 78(3):691

    Article  MathSciNet  MATH  Google Scholar 

  33. Pan C, Xu Y, Yan Y, Gu K, Yang X (2016) Exploiting neural models for no-reference image quality assessment. In: 2016 visual communications and image processing (VCIP), pp 1–4. IEEE

  34. Perrin AFNM, Xu H, Kroupi E, Řeřábek M, Ebrahimi T (2015) Multimodal dataset for assessment of quality of experience in immersive multimedia. In: Proceedings of the 23rd ACM international conference on multimedia, MM ’15. https://doi.org/10.1145/2733373.2806387. ACM, New York, pp 1007–1010

  35. Plechawska-Wójcik M, Wawrzyk M, Wesołowska K, Kaczorowska M, Tokovarov M, Dmytruk R, Borys M (2017) Eeg spectral analysis of human cognitive workload study. Studia Informatica 38(2):17–30

    Google Scholar 

  36. Reichl P, Tuffin B, Schatz R (2013) Logarithmic laws in service quality perception: where microeconomics meets psychophysics and quality of experience. Telecommun Syst 52(2):587–600

    Google Scholar 

  37. Rein S, Fitzek FHP, Reisslein M (2005) Voice quality evaluation in wireless packet communication systems: a tutorial and performance results for rhc. IEEE Wirel Commun 12(1):60–67. https://doi.org/10.1109/MWC.2005.1404574

    Article  Google Scholar 

  38. Scholler S, Bosse S, Treder MS, Blankertz B, Curio G, Mueller KR, Wiegand T (2012) Toward a direct measure of video quality perception using EEG. IEEE Trans Image Process 21(5):2619–2629

    Article  MathSciNet  MATH  Google Scholar 

  39. Seeling P (2015) Augmented vision and quality of experience assessment: Towards a unified evaluation framework. In: Proceedings of IEEE ICC workshop on quality of experience-based management for future internet applications and services (qoe-FI). London, United Kingdom

  40. Seeling P (2015) Towards quality of experience determination for video in augmented binocular vision scenarios. Signal Process Image Commun 33:41–50

    Article  Google Scholar 

  41. Seeling P (2016) Visual user experience difference: image compression impacts on the quality of experience in augmented binocular vision. In: Proceedings of IEEE consumer communications and networking conference (CCNC). Las Vegas, NV, USA, pp 931–936

  42. Shahid M, Rossholm A, Lövström B, Zepernick HJ (2014) No-reference image and video quality assessment: a classification and review of recent approaches. EURASIP J Image Video Process 2014(1):40. https://doi.org/10.1186/1687-5281-2014-40

    Article  Google Scholar 

  43. Staelens N, Wallendael GV, Crombecq K, Vercammen N, Cock JD, Vermeulen B, de Walle RV, Dhaene T, Demeester P (2012) No-reference bitstream-based visual quality impairment detection for high definition h.264/avc encoded video sequences. IEEE Trans Broadcast 58(2):187–199. https://doi.org/10.1109/TBC.2012.2189334

    Article  Google Scholar 

  44. Xiao X, Ni LM (1999) Internet qos: a big picture. IEEE Netw 13(2):8–18. https://doi.org/10.1109/65.768484

    Article  MathSciNet  Google Scholar 

  45. You J, Reiter U, Hannuksela MM, Gabbouj M, Perkis A (2010) Perceptual-based quality assessment for audio-visual services: a survey. Signal Process Image Commun 25:482–501

    Article  Google Scholar 

Download references

Acknowledgements

This material is based upon work supported by the Faculty Research and Creative Endeavors (FRCE) program at Central Michigan University under grant #48146.

Author information

Authors and Affiliations

  1. Department of Computer Science, Central Michigan University, Mount Pleasant, MI, 48859, USA

    B. Bauman & P. Seeling

Authors
  1. B. Bauman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. Seeling
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to P. Seeling.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bauman, B., Seeling, P. Spherical image QoE approximations for vision augmentation scenarios. Multimed Tools Appl 78, 18113–18135 (2019). https://doi.org/10.1007/s11042-019-7171-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7171-x

Keywords

Search

Navigation