Skip to main content

Learning Self-prior for Mesh Denoising Using Dual Graph Convolutional Networks

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13663))

Included in the following conference series:

Abstract

This study proposes a deep-learning framework for mesh denoising from a single noisy input, where two graph convolutional networks are trained jointly to filter vertex positions and facet normals apart. The prior obtained only from a single input is particularly referred to as a self-prior. The proposed method leverages the framework of the deep image prior (DIP), which obtains the self-prior for image restoration using a convolutional neural network (CNN). Thus, we obtain a denoised mesh without any ground-truth noise-free meshes. Compared to the original DIP that transforms a fixed random code into a noise-free image by the neural network, we reproduce vertex displacement from a fixed random code and reproduce facet normals from feature vectors that summarize local triangle arrangements. After tuning several hyperparameters with a few validation samples, our method achieved significantly higher performance than traditional approaches working with a single noisy input mesh. Moreover, its performance is better than the other methods using deep neural networks trained with a large-scale shape dataset. The independence of our method of either large-scale datasets or ground-truth noise-free mesh will allow us to easily denoise meshes whose shapes are rarely included in the shape datasets. Our code is available at: https://github.com/astaka-pe/Dual-DMP.git.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    All the meshes used in this study are shown in the supplementary document.

References

  1. Arvanitis, G., Lalos, A.S., Moustakas, K., Fakotakis, N.: Feature preserving mesh denoising based on graph spectral processing. IEEE Trans. Vis. Comput. Graphics 25(3), 1513–1527 (2019). https://doi.org/10.1109/tvcg.2018.2802926

    Article  Google Scholar 

  2. Calvarons, A.F.: Improved Noise2Noise denoising with limited data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 796–805 (2021). https://doi.org/10.1109/cvprw53098.2021.00089

  3. Clarenz, U., Diewald, U., Rumpf, M.: Anisotropic geometric diffusion in surface processing (2000). https://doi.org/10.1109/VISUAL.2000.885721

  4. Desbrun, M., Meyer, M., Schröder, P., Barr, A.H.: Implicit fairing of irregular meshes using diffusion and curvature flow. In: Proceedings of ACM International Conference on Computer Graphics and InteractDive Techniques (SIGGRAPH). SIGGRAPH 1999, pp. 317–324. ACM Press/Addison-Wesley Publishing Co. (1999). https://doi.org/10.1145/311535.311576

  5. Fleishman, S., Drori, I., Cohen-Or, D.: Bilateral mesh denoising. ACM Trans. Graphics 22(3), 950–953 (2003). https://doi.org/10.1145/882262.882368

    Article  Google Scholar 

  6. Hanocka, R., Hertz, A., Fish, N., Giryes, R., Fleishman, S., Cohen-Or, D.: MeshCNN: a network with an edge. ACM Trans. Graphics 38(4) (2019). https://doi.org/10.1145/3306346.3322959

  7. Hanocka, R., Metzer, G., Giryes, R., Cohen-Or, D.: Point2Mesh: a self-prior for deformable meshes. ACM Trans. Graphics 39(4) (2020). https://doi.org/10.1145/3386569.3392415

  8. He, L., Schaefer, S.: Mesh denoising via L0 minimization. ACM Trans. Graphics 32(4), 1–8 (2013). https://doi.org/10.1145/2461912.2461965

    Article  Google Scholar 

  9. Jones, T.R., Durand, F., Desbrun, M.: Non-iterative, feature-preserving mesh smoothing. ACM Trans. Graphics 22(3), 943–949 (2003). https://doi.org/10.1145/882262.882367

    Article  Google Scholar 

  10. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: Proceedings of International Conference on Learning Representations (ICLR) (2017)

    Google Scholar 

  11. Krull, A., Buchholz, T.O., Jug, F.: Noise2Void - learning denoising from single noisy images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2019 (2019). https://doi.org/10.1109/cvpr.2019.00223

  12. Lee, K.W., Wang, W.P.: Feature-preserving mesh denoising via bilateral normal filtering (2005). https://doi.org/10.1109/CAD-CG.2005.40

  13. Lehtinen, J., et al.: Noise2Noise: learning image restoration without clean data. In: Proceedings of International Conference on Machine Learning (ICML), pp. 2965–2974. PMLR (2018)

    Google Scholar 

  14. Li, T., Wang, J., Liu, H., Liu, L.: Efficient mesh denoising via robust normal filtering and alternate vertex updating. Front. Inf. Technol. Electron. Eng. 18(11), 1828–1842 (2017). https://doi.org/10.1631/FITEE.1601229

    Article  Google Scholar 

  15. Li, X., Li, R., Zhu, L., Fu, C.W., Heng, P.A.: DNF-Net: a deep normal filtering network for mesh denoising. IEEE Trans. Visual Comput. Graphics (2020). https://doi.org/10.1109/TVCG.2020.3001681

    Article  Google Scholar 

  16. Li, X., Zhu, L., Fu, C.W., Heng, P.A.: Non-local low-rank normal filtering for mesh denoising. Comput. Graphics Forum 37(7), 155–166 (2018). https://doi.org/10.1111/cgf.13556

    Article  Google Scholar 

  17. Li, Z., et al.: NormalF-Net: normal filtering neural network for feature-preserving mesh denoising. Comput. Aided Des. 127, 102861 (2020). https://doi.org/10.1016/j.cad.2020.102861

    Article  MathSciNet  Google Scholar 

  18. Liaw, R., Liang, E., Nishihara, R., Moritz, P., Gonzalez, J.E., Stoica, I.: Tune: a research platform for distributed model selection and training. arXiv preprint arXiv:1807.05118 (2018)

  19. Lu, X., Deng, Z., Chen, W.: A robust scheme for feature-preserving mesh denoising. IEEE Trans. Visual Comput. Graphics 22(3), 1181–1194 (2016). https://doi.org/10.1109/TVCG.2015.2500222

    Article  Google Scholar 

  20. Newcombe, R.A., et al.: KinectFusion: real-time dense surface mapping and tracking. In: Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 127–136. IEEE (2011). https://doi.org/10.1109/ismar.2011.6092378

  21. Ohtake, Y., Belyaev, A., Bogaevski, I.: Mesh regularization and adaptive smoothing. Comput.-Aided Des. 33(11), 789–800 (2001). https://doi.org/10.1016/s0010-4485(01)00095-1

  22. Ohtake, Y., Belyaev, A.G., Seidel, H.P.: Mesh smoothing by adaptive and anisotropic Gaussian filter applied to mesh normals. In: Proceeding of Vision, Modeling, and Visualization (VMV), vol. 2, pp. 203–210. Citeseer (2002)

    Google Scholar 

  23. Pinkall, U., Polthier, K.: Computing discrete minimal surfaces and their conjugates. Exp. Math. 2(1), 15–36 (1993). https://doi.org/10.1080/10586458.1993.10504266

    Article  MathSciNet  MATH  Google Scholar 

  24. Shen, Y., et al.: GCN-denoiser: mesh denoising with graph convolutional networks. ACM Trans. Graphics 41(1), 1–14 (2022). https://doi.org/10.1145/3480168

    Article  Google Scholar 

  25. Sun, X., Rosin, P.L., Martin, R., Langbein, F.: Fast and effective feature-preserving mesh denoising. IEEE Trans. Visual Comput. Graphics 13(5), 925–938 (2007). https://doi.org/10.1109/TVCG.2007.1065

    Article  Google Scholar 

  26. Taubin, G.: Curve and surface smoothing without shrinkage. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) 1995, pp. 852–857. IEEE (1995)

    Google Scholar 

  27. Taubin, G.: Linear anisotropic mesh filtering. Research Report RC2213 (2001)

    Google Scholar 

  28. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) 1998, pp. 839–846. IEEE (1998)

    Google Scholar 

  29. Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep image prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018, pp. 9446–9454 (2018). https://doi.org/10.1109/cvpr.2018.00984

  30. Wang, J., Huang, J., Wang, F.L., Wei, M., Xie, H., Qin, J.: Data-driven geometry-recovering mesh denoising. Comput. Aided Des. 114, 133–142 (2019). https://doi.org/10.1016/j.cad.2019.05.027

    Article  Google Scholar 

  31. Wang, P.S., Liu, Y., Tong, X.: Mesh denoising via cascaded normal regression. ACM Trans. Graphics 35(6), 1–12 (2016). https://doi.org/10.1145/2980179.2980232

    Article  Google Scholar 

  32. Wang, R., Yang, Z., Liu, L., Deng, J., Chen, F.: Decoupling noise and features via weighted \(\ell \)1-analysis compressed sensing. ACM Trans. Graphics 33(2), 1–12 (2014). https://doi.org/10.1145/2557449

    Article  Google Scholar 

  33. Wei, M., Huang, J., Xie, X., Liu, L., Wang, J., Qin, J.: Mesh denoising guided by patch normal co-filtering via kernel low-rank recovery. IEEE Trans. Visual Comput. Graphics 25(10), 2910–2926 (2019). https://doi.org/10.1109/TVCG.2018.2865363

    Article  Google Scholar 

  34. Wei, M., Liang, L., Pang, W.M., Wang, J., Li, W., Wu, H.: Tensor voting guided mesh denoising. IEEE Trans. Autom. Sci. Eng. 14(2), 931–945 (2017). https://doi.org/10.1109/TASE.2016.2553449

    Article  Google Scholar 

  35. Wei, M., et al.: Bi-normal filtering for mesh denoising. IEEE Trans. Visual Comput. Graphics 21(1), 43–55 (2015). https://doi.org/10.1109/TVCG.2014.2326872

    Article  Google Scholar 

  36. Wei, X., Chen, Z., Fu, Y., Cui, Z., Zhang, Y.: Deep hybrid self-prior for full 3D mesh generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 2021 (2021). https://doi.org/10.1109/iccv48922.2021.00575

  37. Williams, F., Schneider, T., Silva, C., Zorin, D., Bruna, J., Panozzo, D.: Deep geometric prior for surface reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2019, pp. 10122–10131 (2019). https://doi.org/10.1109/CVPR.2019.01037

  38. Xu, J., et al.: Noisy-as-Clean: learning self-supervised denoising from corrupted image. IEEE Trans. Image Process. 29, 9316–9329 (2020). https://doi.org/10.1109/tip.2020.3026622

    Article  MATH  Google Scholar 

  39. Yagou, H., Ohtake, Y., Belyaev, A.: Mesh smoothing via mean and median filtering applied to face normals. In: Proceedings of Geometric Modeling and Processing. Theory and Applications. GMP. (2002). https://doi.org/10.1109/GMAP.2002.1027503

  40. Zhang, W., Deng, B., Zhang, J., Bouaziz, S., Liu, L.: Guided mesh normal filtering. Comput. Graphics Forum 34, 23–34 (2015). https://doi.org/10.1111/cgf.12742

    Article  Google Scholar 

  41. Zhang, Y., Shen, G., Wang, Q., Qian, Y., Wei, M., Qin, J.: GeoBi-GNN: geometry-aware bi-domain mesh denoising via graph neural networks. Comput. Aided Des. 144, 103154 (2022). https://doi.org/10.1016/j.cad.2021.103154

    Article  MathSciNet  Google Scholar 

  42. Zhao, W., Liu, X., Wang, S., Fan, X., Zhao, D.: Graph-based feature-preserving mesh normal filtering. IEEE Trans. Visual Comput. Graphics 27(3), 1937–1952 (2021). https://doi.org/10.1109/TVCG.2019.2944357

    Article  Google Scholar 

  43. Zhao, W., Liu, X., Zhao, Y., Fan, X., Zhao, D.: NormalNet: learning-based mesh normal denoising via local partition normalization. IEEE Trans. Circuits Syst. Video Technol. 31(12), 4697–4710 (2021). https://doi.org/10.1109/TCSVT.2021.3099939

    Article  Google Scholar 

  44. Zhao, Y., Qin, H., Zeng, X., Xu, J., Dong, J.: Robust and effective mesh denoising using L0 sparse regularization. Comput. Aided Des. 101, 82–97 (2018). https://doi.org/10.1016/j.cad.2018.04.001

    Article  Google Scholar 

  45. Zheng, Y., Fu, H., Au, O.K.C., Tai, C.L.: Bilateral normal filtering for mesh denoising. IEEE Trans. Visual Comput. Graphics 17(10), 1521–1530 (2011). https://doi.org/10.1109/TVCG.2010.264

    Article  Google Scholar 

Download references

Acknowledgment

This study is financially supported by a JSPS Grant-in-Aid for Early-career Scientists (22K17907).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shota Hattori .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 2566 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hattori, S., Yatagawa, T., Ohtake, Y., Suzuki, H. (2022). Learning Self-prior for Mesh Denoising Using Dual Graph Convolutional Networks. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13663. Springer, Cham. https://doi.org/10.1007/978-3-031-20062-5_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20062-5_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20061-8

  • Online ISBN: 978-3-031-20062-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics