Skip to main content
Log in

A machine learning framework for full-reference 3D shape quality assessment

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

To decide whether the perceived quality of a mesh is influenced by a certain modification such as compression or simplification, a metric for estimating the visual quality of 3D meshes is required. Today, machine learning and deep learning techniques are getting increasingly popular since they present efficient solutions to many complex problems. However, these techniques are not much utilized in the field of 3D shape perception. We propose a novel machine learning-based approach for evaluating the visual quality of 3D static meshes. The novelty of our study lies in incorporating crowdsourcing in a machine learning framework for visual quality evaluation. We deliberate that this is an elegant way since modeling human visual system processes is a tedious task and requires tuning many parameters. We employ crowdsourcing methodology for collecting data of quality evaluations and metric learning for drawing the best parameters that well correlate with the human perception. Experimental validation of the proposed metric reveals a promising correlation between the metric output and human perception. Results of our crowdsourcing experiments are publicly available for the community.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://www.mturk.com/mturk/welcome.

  2. https://requester.mturk.com/developer/tools/clt.

  3. http://get.webgl.org/.

  4. http://threejs.org/.

References

  1. Abouelaziz, I., El Hassouni, M., Cherifi, H.: No-reference 3D mesh quality assessment based on dihedral angles model and support vector regression. In: International Conference on Image and Signal Processing, pp. 369–377. Springer (2016)

  2. Abouelaziz, I., El Hassouni, M., Cherifi, H.: Blind 3D mesh visual quality assessment using support vector regression. Multimedia Tools and Applications pp. 1–22 (2018)

  3. Alliez, P., Cohen-Steiner, D., Devillers, O., Lévy, B., Desbrun, M.: Anisotropic polygonal remeshing. ACM Trans. Graph. 22, 485–493 (2003)

    Article  Google Scholar 

  4. Bulbul, A., Capin, T., Lavoué, G., Preda, M.: Assessing visual quality of 3-d polygonal models. IEEE Signal Processing Mag. 28(6), 80–90 (2011)

    Article  Google Scholar 

  5. Chetouani, A.: A 3D mesh quality metric based on features fusion. Electron. Imaging 2017(20), 4–8 (2017)

    Article  Google Scholar 

  6. Cignoni, P., Corsini, M., Ranzuglia, G.: Meshlab: an open-source 3D mesh processing system. ERCIM News 73, 45–46 (2008)

    Google Scholar 

  7. Cignoni, P., Rocchini, C., Scopigno, R.: Metro: measuring error on simplified surfaces. In: Computer Graphics Forum, vol. 17, pp. 167–174. Wiley Online Library (1998)

  8. Corsini, M., Gelasca, E., Ebrahimi, T., Barni, M.: Watermarked 3-D mesh quality assessment. IEEE Trans. Multimedia 9(2), 247–256 (2007)

    Article  Google Scholar 

  9. Corsini, M., Larabi, M.C., Lavoué, G., Petřík, O., Váša, L., Wang, K.: Perceptual metrics for static and dynamic triangle meshes. Comput. Graph. Forum 32, 101–125 (2013)

    Article  Google Scholar 

  10. Daly, S.J.: Visible differences predictor: an algorithm for the assessment of image fidelity. In: SPIE/IS&T 1992 Symposium on Electronic Imaging: Science and Technology, pp. 2–15. International Society for Optics and Photonics (1992)

  11. Dong, L., Fang, Y., Lin, W., Seah, H.S.: Perceptual quality assessment for 3D triangle mesh based on curvature. IEEE Trans. Multimedia 17(12), 2174–2184 (2015)

    Article  Google Scholar 

  12. Garces, E., Agarwala, A., Gutierrez, D., Hertzmann, A.: A similarity measure for illustration style. ACM Trans. Graph. 33(4), 93 (2014)

    Article  Google Scholar 

  13. Gingold, Y., Shamir, A., Cohen-Or, D.: Micro perceptual human computation for visual tasks. ACM Trans. Graph. 31(5), 119 (2012)

    Article  Google Scholar 

  14. Heer, J., Bostock, M.: Crowdsourcing graphical perception: using mechanical turk to assess visualization design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 203–212. ACM (2010)

  15. Karni, Z., Gotsman, C.: Spectral compression of mesh geometry. In: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pp. 279–286. ACM Press/Addison-Wesley Publishing Co. (2000)

  16. Kleiman, Y., Goldberg, G., Amsterdamer, Y., Cohen-Or, D.: Toward semantic image similarity from crowdsourced clustering. Vis. Comput. 32(6–8), 1045–1055 (2016)

    Article  Google Scholar 

  17. Koenderink, J.J., van Doorn, A.J.: Surface shape and curvature scales. Image Vis. Comput. 10(8), 557–564 (1992)

    Article  Google Scholar 

  18. Koyama, Y., Sakamoto, D., Igarashi, T.: Crowd-powered parameter analysis for visual design exploration. In: Proceedings of the 27th annual ACM symposium on User interface software and technology, pp. 65–74. ACM (2014)

  19. Kulis, B.: Metric learning: a survey. Found. Trends Mach. Learn. 5(4), 287–364 (2012)

    Article  Google Scholar 

  20. Kundu, D., Ghadiyaram, D., Bovik, A.C., Evans, B.L.: No-reference quality assessment of tone-mapped hdr pictures. IEEE Trans. Image Process. 26(6), 2957–2971 (2017)

    Article  MathSciNet  Google Scholar 

  21. Lavoué, G.: A local roughness measure for 3D meshes and its application to visual masking. ACM Trans. Appl. Percept. 5(4), 21 (2009)

    Article  Google Scholar 

  22. Lavoué, G.: A multiscale metric for 3D mesh visual quality assessment. Comput. Graph. Forum 30, 1427–1437 (2011)

    Article  Google Scholar 

  23. Lavoué, G., Cheng, I., Basu, A.: Perceptual quality metrics for 3D meshes: towards an optimal multi-attribute computational model. In: Systems, Man, and Cybernetics (SMC), 2013 IEEE International Conference on, pp. 3271–3276. IEEE (2013)

  24. Lavoué, G., Gelasca, E.D., Dupont, F., Baskurt, A., Ebrahimi, T.: Perceptually driven 3D distance metrics with application to watermarking. In: Optics & Photonics, pp. 63,120L–63,120L. International Society for Optics and Photonics (2006)

  25. Lavoué, G., Mantiuk, R.: Quality assessment in computer graphics. In: Visual Signal Quality Assessment, pp. 243–286. Springer (2015)

  26. Lee, C., Varshney, A., Jacobs, D.: Mesh saliency. In: ACM SIGGRAPH 2005 Papers, pp. 659–666. ACM (2005)

  27. Lin, W., Kuo, C.C.J.: Perceptual visual quality metrics: a survey. J. Vis. Commun. Image Represent. 22(4), 297–312 (2011)

    Article  Google Scholar 

  28. Liu, T., Hertzmann, A., Li, W., Funkhouser, T.: Style compatibility for 3D furniture models. ACM Trans. Graph. 34(4), 85 (2015)

    Article  Google Scholar 

  29. Lun, Z., Kalogerakis, E., Sheffer, A.: Elements of style: learning perceptual shape style similarity. ACM Trans. Graph. 34(4), 84 (2015)

    Article  Google Scholar 

  30. Maglo, A., Lavoué, G., Dupont, F., Hudelot, C.: 3D mesh compression: survey, comparisons, and emerging trends. ACM Comput. Surv. 47(3), 44 (2015)

    Article  Google Scholar 

  31. Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)

    Article  MathSciNet  Google Scholar 

  32. Nocedal, J., Wright, S.: Numerical optimization. Springer Science & Business Media (2006)

  33. Nouri, A., Charrier, C., Lézoray, O.: 3D blind mesh quality assessment index. Electron. Imaging 2017(20), 9–26 (2017)

    Article  Google Scholar 

  34. Ramanarayanan, G., Ferwerda, J., Walter, B., Bala, K.: Visual equivalence: towards a new standard for image fidelity. In: ACM SIGGRAPH 2007 papers, SIGGRAPH ’07. ACM, New York (2007)

  35. Saad, M.A., Bovik, A.C., Charrier, C.: Blind image quality assessment: a natural scene statistics approach in the dct domain. IEEE Trans. Image Process. 21(8), 3339–3352 (2012)

    Article  MathSciNet  Google Scholar 

  36. Saleh, B., Dontcheva, M., Hertzmann, A., Liu, Z.: Learning style similarity for searching infographics. In: Proceedings of the 41st Graphics Interface Conference, pp. 59–64. Canadian Information Processing Society (2015)

  37. Secord, A., Lu, J., Finkelstein, A., Singh, M., Nealen, A.: Perceptual models of viewpoint preference. ACM Trans. Graph. 30(5), 109 (2011)

    Article  Google Scholar 

  38. Sorkine, O., Cohen-Or, D., Toledo, S.: High-pass quantization for mesh encoding. In: Symposium on Geometry Processing, vol. 42 (2003)

  39. Torkhani, F., Wang, K., Chassery, J.M.: A curvature-tensor-based perceptual quality metric for 3D triangular meshes. Mach. Graph. Vis. 23(1–2), 59–82 (2014)

    Google Scholar 

  40. Torkhani, F., Wang, K., Chassery, J.M.: Perceptual quality assessment of 3D dynamic meshes: subjective and objective studies. Signal Process. Image Commun. 31, 185–204 (2015). https://doi.org/10.1016/j.image.2014.12.008

    Article  Google Scholar 

  41. Váša, L., Rus, J.: Dihedral angle mesh error: a fast perception correlated distortion measure for fixed connectivity triangle meshes. Comput. Graph. Forum 31, 1715–1724 (2012)

    Article  Google Scholar 

  42. Wang, K., Lavoué, G., Denis, F., Baskurt, A., He, X.: A benchmark for 3D mesh watermarking. In: Proc. of the IEEE International Conference on Shape Modeling and Applications, pp. 231–235 (2010)

  43. Wang, K., Torkhani, F., Montanvert, A.: A fast roughness-based approach to the assessment of 3D mesh visual quality. Comput. Graph. 36(7), 808–818 (2012)

    Article  Google Scholar 

  44. Yildiz, Z.C., Capin, T.: A perceptual quality metric for dynamic triangle meshes. EURASIP J. Image Video Process. 2017(1), 12 (2017). https://doi.org/10.1186/s13640-016-0157-y

    Article  Google Scholar 

  45. Yumer, M.E., Chaudhuri, S., Hodgins, J.K., Kara, L.B.: Semantic shape editing using deformation handles. ACM Trans. Graph. 34(4), 86 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the Scientific and Technical Research Council of Turkey (TUBITAK). Also, we would like to thank Yeojin Yun for her kind help in setting up the crowdsourcing platform.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zeynep Cipiloglu Yildiz.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yildiz, Z.C., Oztireli, A.C. & Capin, T. A machine learning framework for full-reference 3D shape quality assessment. Vis Comput 36, 127–139 (2020). https://doi.org/10.1007/s00371-018-1592-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-018-1592-9

Keywords

Navigation