Advertisement

3D visual saliency and convolutional neural network for blind mesh quality assessment

  • Ilyass AbouelazizEmail author
  • Aladine Chetouani
  • Mohammed El Hassouni
  • Longin Jan Latecki
  • Hocine Cherifi
IAPR-MedPRAI
First Online:
  • 29 Downloads

Abstract

A number of full reference and reduced reference methods have been proposed in order to estimate the perceived visual quality of 3D meshes. However, in most practical situations, there is a limited access to the information related to the reference and the distortion type. For these reasons, the development of a no-reference mesh visual quality (MVQ) approach is a critical issue, and more emphasis needs to be devoted to blind methods. In this work, we propose a no-reference convolutional neural network (CNN) framework to estimate the perceived visual quality of 3D meshes. The method is called SCNN-BMQA (3D visual saliency and CNN for blind mesh quality assessment). The main contribution is the usage of a CNN and 3D visual saliency to estimate the perceived visual quality of distorted meshes. To do so, the CNN architecture is fed by small patches selected carefully according to their level of saliency. First, the visual saliency of the 3D mesh is computed. Afterward, we render 2D projections from the 3D mesh and its corresponding 3D saliency map. Then the obtained views are split into 2D small patches that pass through a saliency filter in order to select the most relevant patches. Finally, a CNN is used for the feature learning and the quality score estimation. Extensive experiments are conducted on four prominent MVQ assessment databases, including several tests to study the effect of the CNN parameters, the effect of visual saliency and comparison with existing methods. Results show that the trained CNN achieves good rates in terms of correlation with human judgment and outperforms the most effective state-of-the-art methods.

Keywords

Mesh visual quality assessment Mean opinion score Mesh visual saliency Convolutional neural network 
This is a preview of subscription content, log in to check access.

Notes

References

  1. 1.
    Botsch M, Kobbelt L, Pauly M, Alliez P, Lévy B (2010) Polygon mesh processing. CRC Press, Boca RatonCrossRefGoogle Scholar
  2. 2.
    Alliez P, Gotsman C (2005) Recent advances in compression of 3d meshes. In: Dodgson N, Floater M, Sabin M (eds) Advances in multiresolution for geometric modelling. Springer, Berlin, pp 3–26CrossRefGoogle Scholar
  3. 3.
    Lee H, Dikici Ç, Lavoué G, Dupont F (2011) Joint reversible watermarking and progressive compression of 3d meshes. Vis Comput 27(6):781–792CrossRefGoogle Scholar
  4. 4.
    Wang K, Lavoué G, Denis F, Baskurt A (2008) A comprehensive survey on three-dimensional mesh watermarking. IEEE Trans Multimed 10(8):1513–1527CrossRefGoogle Scholar
  5. 5.
    Wang Y-P, Shi-Min H (2009) A new watermarking method for 3d models based on integral invariants. IEEE Trans Vis Comput Graph 15(2):285–294CrossRefGoogle Scholar
  6. 6.
    Garland M, Heckbert PS (1997) Surface simplification using quadric error metrics. In: Proceedings of the 24th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., pp 209–216Google Scholar
  7. 7.
    Luebke DP (2001) A developer’s survey of polygonal simplification algorithms. IEEE Comput Graph Appl 21(3):24–35CrossRefGoogle Scholar
  8. 8.
    Corsini M, Larabi M-C, Lavoué G, Petřík O, Váša L, Wang K (2013) Perceptual metrics for static and dynamic triangle meshes. In: Computer graphics forum, vol 32, issue 1. Blackwell, Oxford, UK, pp 101–125Google Scholar
  9. 9.
    Cignoni P, Rocchini C, Scopigno R (1998) Metro: measuring error on simplified surfaces. In: Computer graphics forum, vol 17, issue 2. Blackwell, Oxford, UK, pp 167–174Google Scholar
  10. 10.
    Aspert N, Santa-Cruz D, Ebrahimi T (2002) Mesh: measuring errors between surfaces using the Hausdorff distance. In: Proceedings. 2002 IEEE international conference on multimedia and expo, 2002. ICME’02, vol 1. IEEE, pp 705–708Google Scholar
  11. 11.
    Lavoué G, Corsini M (2010) A comparison of perceptually-based metrics for objective evaluation of geometry processing. IEEE Trans Multimed 12(7):636–649CrossRefGoogle Scholar
  12. 12.
    Bulbul A, Capin T, Lavoué G, Preda M (2011) Assessing visual quality of 3-d polygonal models. IEEE Signal Process Mag 28(6):80–90CrossRefGoogle Scholar
  13. 13.
    Lin W, Jay Kuo C-C (2011) Perceptual visual quality metrics: a survey. J Vis Commun Image Represent 22(4):297–312CrossRefGoogle Scholar
  14. 14.
    Lin W, Ebrahimi T, Loizou PC, Moller S, Reibman AR (2012) Introduction to the special issue on new subjective and objective methodologies for audio and visual signal processing. IEEE J Sel Top Signal Process 6(6):614–615CrossRefGoogle Scholar
  15. 15.
    Karni Z, Gotsman C (2000) Spectral compression of mesh geometry. In: Proceedings of the 27th annual conference on computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., pp 279–286Google Scholar
  16. 16.
    Sorkine O, Cohen-Or D, Toledo S (2003) High-pass quantization for mesh encoding. In: Symposium on geometry processing, vol 42Google Scholar
  17. 17.
    Pan Y, Cheng I, Basu A (2005) Quality metric for approximating subjective evaluation of 3-d objects. IEEE Trans Multimed 7(2):269–279CrossRefGoogle Scholar
  18. 18.
    Bian Z, Shi-Min H, Martin RR (2009) Evaluation for small visual difference between conforming meshes on strain field. J Comput Sci Technol 24(1):65–75CrossRefGoogle Scholar
  19. 19.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612CrossRefGoogle Scholar
  20. 20.
    Lavoué G, Gelasca ED, Dupont F, Baskurt A, Ebrahimi T (2006) Perceptually driven 3d distance metrics with application to watermarking. In: SPIE optics + photonics. International Society for Optics and Photonics, pp 63120L–63120LGoogle Scholar
  21. 21.
    Lavoué G (2011) A multiscale metric for 3d mesh visual quality assessment. In: Computer graphics forum, vol 30, issue 5. Blackwell, Oxford, UK, pp 1427–1437Google Scholar
  22. 22.
    Torkhani F, Wang K, Chassery J-M (2012) A curvature tensor distance for mesh visual quality assessment. In: Computer vision and graphics, pp 253–263Google Scholar
  23. 23.
    Corsini M, Gelasca ED, Ebrahimi T, Barni M (2007) Watermarked 3-d mesh quality assessment. IEEE Trans Multimed 9(2):247–256CrossRefGoogle Scholar
  24. 24.
    Váša L, Rus J (2012) Dihedral angle mesh error: a fast perception correlated distortion measure for fixed connectivity triangle meshes. In: Computer graphics forum, vol 31. Wiley Online Library, pp 1715–1724Google Scholar
  25. 25.
    Wang K, Torkhani F, Montanvert A (2012) A fast roughness-based approach to the assessment of 3d mesh visual quality. Comput Graph 36(7):808–818CrossRefGoogle Scholar
  26. 26.
    Abouelaziz I, El Hassouni M, Cherifi H (2016) No-reference 3d mesh quality assessment based on dihedral angles model and support vector regression. In: International conference on image and signal processing. Springer, pp 369–377Google Scholar
  27. 27.
    Abouelaziz I, El Hassouni M, Cherifi H (2016) A curvature based method for blind mesh visual quality assessment using a general regression neural network. In: 2016 12th international conference on signal-image technology & internet-based systems (SITIS). IEEE, pp 793–797Google Scholar
  28. 28.
    Nouri A, Charrier C, Lézoray O (2017) 3d blind mesh quality assessment index. Electron Imaging 2017(20):9–26CrossRefGoogle Scholar
  29. 29.
    Moorthy AK, Bovik AC (2010) A two-step framework for constructing blind image quality indices. IEEE Signal Process Lett 17(5):513–516CrossRefGoogle Scholar
  30. 30.
    Saad MA, Bovik AC, Charrier C (2010) A dct statistics-based blind image quality index. IEEE Signal Process Lett 17(6):583–586CrossRefGoogle Scholar
  31. 31.
    Li C, Bovik AC, Wu X (2011) Blind image quality assessment using a general regression neural network. IEEE Transa Neural Netw 22(5):793–799CrossRefGoogle Scholar
  32. 32.
    LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436CrossRefGoogle Scholar
  33. 33.
    Zhang W, Chenfei Q, Ma L, Guan J, Huang R (2016) Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network. Pattern Recognit 59:176–187CrossRefGoogle Scholar
  34. 34.
    Nouri A, Charrier C, Lézoray O (2016) Full-reference saliency-based 3d mesh quality assessment index. In: 2016 IEEE international conference on image processing (ICIP). IEEE, pp 1007–1011Google Scholar
  35. 35.
    Engelke U, Pepion R, Le Callet P, Zepernick H-J (2010) Linking distortion perception and visual saliency in h. 264/AVC coded video containing packet loss. In: Visual communications and image processing 2010. International Society for optics and photonics, vol 7744, p 774406Google Scholar
  36. 36.
    Lee CH, Varshney A, Jacobs DW (2005) Mesh saliency. ACM Trans Graph: TOG 24:659–666CrossRefGoogle Scholar
  37. 37.
    Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259CrossRefGoogle Scholar
  38. 38.
    Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708MathSciNetCrossRefGoogle Scholar
  39. 39.
    Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 807–814Google Scholar
  40. 40.
    Kang L, Ye P, Li Y, Doermann D (2014) Convolutional neural networks for no-reference image quality assessment. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1733–1740Google Scholar
  41. 41.
    Lavoué G, Larabi MC, Váša L (2016) On the efficiency of image metrics for evaluating the visual quality of 3d models. IEEE Trans Vis Comput Graph 22(8):1987–1999CrossRefGoogle Scholar
  42. 42.
    Silva S, Santos BS, Ferreira C, Madeira J (2009) A perceptual data repository for polygonal meshes. In: Second international conference in visualisation, 2009. VIZ’09. IEEE, pp 207–212Google Scholar
  43. 43.
    Wang Z, Bovik AC (2006) Modern image quality assessment. Synth Lect Image Video Multimed Process 2(1):1–156CrossRefGoogle Scholar
  44. 44.
    Engeldrum PG (2001) Psychometric scaling: avoiding the pitfalls and hazards. In: PICS, pp 101–107Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  • Ilyass Abouelaziz
    • 1
    Email author
  • Aladine Chetouani
    • 2
  • Mohammed El Hassouni
    • 1
    • 3
  • Longin Jan Latecki
    • 4
  • Hocine Cherifi
    • 5
  1. 1.LRIT, URAC No 29, Faculty of SciencesMohammed V University in RabatRabatMorocco
  2. 2.University of Orleans PRISME LaboratoryOrléansFrance
  3. 3.FLSHRMohammed V University in RabatRabatMorocco
  4. 4.Department of Computer and Information SciencesTemple UniversityPhiladelphiaUSA
  5. 5.LE2I, UMR 6306 CNRSUniversity of BurgundyDijonFrance

Personalised recommendations

Cite article

Buy options