Subjective Visual Quality Assessment of Immersive 3D Media Compressed by Open-Source Static 3D Mesh Codecs

  • Kyriaki ChristakiEmail author
  • Emmanouil Christakis
  • Petros Drakoulis
  • Alexandros Doumanoglou
  • Nikolaos Zioulis
  • Dimitrios Zarpalas
  • Petros Daras
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11295)


While studies for objective and subjective evaluation of the visual quality of compressed 3D meshes has been discussed in the literature, those studies were covering the evaluation of 3D-meshes either created by 3D artists or generated by a computationally expensive reconstruction process applied on high quality 3D scans. With the advent of RGB-D sensors operating at high frame-rates and the utilization of fast 3D reconstruction algorithms, humans can be captured and reconstructed into a 3D representation in real-time, enabling new (tele-)immersive experiences. The produced 3D mesh content is structurally different in the two cases. The first type of content is nearly perfect and clean while the second type is much more irregular and noisy. Evaluating compression artifacts on this new type of immersive 3D media, constitutes a yet unexplored scientific area. In this paper, we conduct a survey to subjectively assess the perceived fidelity of 3D meshes subjected to compression using three open-source static 3D mesh codecs compared to the original uncompressed models. The subjective evaluation of the content is conducted in a Virtual Reality setting, using the forced-choice pairwise comparison methodology with existing reference. The results of this study are two-fold; first, the design of an experimental setup that can be used for the subjective evaluation of 3D media, and second, a mapping of the compared conditions to a continuous ranking scale. The latter can be used when selecting codecs and optimizing their compression parameters to achieve optimum balance between bandwidth and perceived quality in tele-immersive platforms.


Subjective visual quality study 3D Compression Tele-immersion Forced pairwise comparison Virtual reality (VR) 



This work was supported and received funding from the EU H2020 Programme under Grant Agreement no 762111 VRTogether.


  1. 1.
    Corto. Accessed 07 June 2018
  2. 2.
    Google Draco. Accessed 07 June 2018
  3. 3.
    Open 3D Graphics Compression (O3DGC). Accessed 07 June 2018
  4. 4.
    OpenCTM. Accessed 07 June 2018
  5. 5.
    The Stanford 3D Scanning Repository. Accessed 07 June 2018
  6. 6.
    Alexiadis, D.S., et al.: An integrated platform for live 3D human reconstruction and motion capturing. IEEE Trans. Circ. Syst. Video Technol. 27(4), 798–813 (2017)CrossRefGoogle Scholar
  7. 7.
    Alexiou, E., Ebrahimi, T.: On subjective and objective quality evaluation of point cloud geometry. In: 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX), pp. 1–3, May 2017.
  8. 8.
    Alexiou, E., Upenik, E., Ebrahimi, T.: Towards subjective quality assessment of point cloud imaging in augmented reality. In: 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP), pp. 1–6, October 2017.
  9. 9.
    Aspert, N., Santa-Cruz, D., Ebrahimi, T.: Mesh: measuring errors between surfaces using the hausdorff distance. In: Proceedings of IEEE International Conference on Multimedia and Expo, vol. 1, pp. 705–708 (2002).
  10. 10.
    Berjón, D., Morán, F., Manjunatha, S.: Objective and subjective evaluation of static 3D mesh compression. Sig. Process.: Image Commun. 28(2), 181–195 (2013). mPEG-VCrossRefGoogle Scholar
  11. 11.
    Bulbul, A., Capin, T., Lavoué, G., Preda, M.: Assessing visual quality of 3-D polygonal models. IEEE Sig. Process. Mag. 28(6), 80–90 (2011). Scholar
  12. 12.
    Silverstein, D.A., Farrell, J.E.: Efficient method for paired comparison. J. Electron. Imaging 10, 10–10-5 (2001). Scholar
  13. 13.
    International Telecommunication Union: Recommendation ITU-T P.910: subjective video quality assessment methods for multimedia applications (2008)Google Scholar
  14. 14.
    International Telecommunication Union: Recommendation ITU-R BT.500: methodology for the subjective assessment of the quality of television pictures (2012)Google Scholar
  15. 15.
    Javaheri, A., Brites, C., Pereira, F., Ascenso, J.: Subjective and objective quality evaluation of 3D point cloud denoising algorithms. In: 2017 IEEE International Conference on Multimedia Expo Workshops (ICMEW), pp. 1–6, July 2017.
  16. 16.
    Javaheri, A., Brites, C., Pereira, F., Ascenso, J.: Subjective and objective quality evaluation of compressed point clouds. In: 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP), pp. 1–6, October 2017.
  17. 17.
    Karakottas, A., Papachristou, A., Doumanoglou, A., Zioulis, N., Zarpalas, D., Daras, P.: Augmented VR, IEEE Virtual Reality, 18–22 March 2018.
  18. 18.
    Mamou, K., Zaharia, T., Prêteux, F.: TFAN: a low complexity 3D mesh compression algorithm. Comput. Animat. Virtual Worlds 20, 343–354 (2009). Scholar
  19. 19.
    Mantiuk, R., Tomaszewska, A., Mantiuk, R.: Comparison of four subjective methods for image quality assessment, vol. 31, November 2012Google Scholar
  20. 20.
    Mekuria, R., Cesar, P., Doumanis, I., Frisiello, A.: Objective and subjective quality assessment of geometry compression of reconstructed 3d humans in a 3d virtual room. In: Proceedings of the SPIE Applications of Digital Image Processing XXXVIII, vol. 9599, p. 95991M, September 2015.
  21. 21.
    Perez-Ortiz, M., Mantiuk, R.K.: A practical guide and software for analysing pairwise comparison experiments. ArXiv e-prints, December 2017Google Scholar
  22. 22.
    Ponchio, F., Dellepiane, M.: Fast decompression for web-based view-dependent 3d rendering. In: Proceedings of the 20th International Conference on 3D Web Technology, Web3D 2015, pp. 199–207. ACM, New York (2015).
  23. 23.
    Ponchio, F., Dellepiane, M.: Multiresolution and fast decompression for optimal web-based rendering. Graph. Models 88, 1–11 (2016). Scholar
  24. 24.
    Rossignac, J.: Edgebreaker: connectivity compression for triangle meshes. IEEE Trans. Vis. Comput. Graph. 5, 47–61 (1999)CrossRefGoogle Scholar
  25. 25.
    Thorn, J., Pizarro, R., Spanlang, B., Bermell-Garcia, P., González-Franco, M.: Assessing 3d scan quality through paired-comparisons psychophysics test. CoRR abs/1602.00238 (2016).
  26. 26.
    Zerman, E., Hulusic, V., Valenzise, G., Mantiuk, R., Dufaux, F.: The relation between MOS and pairwise comparisons and the importance of cross-content comparisons. In: Human Vision and Electronic Imaging Conference, IS&T International Symposium on Electronic Imaging (EI 2018), Burlingame, United States, January 2018.
  27. 27.
    Zhang, J., Huang, W., Zhu, X., Hwang, J.N.: A subjective quality evaluation for 3d point cloud models, pp. 827–831, January 2015Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Visual Computing Lab, Information Technologies Institute, Centre for Research and Technology HellasThessalonikiGreece

Personalised recommendations