Skip to main content
SpringerLink
Log in

RePCD-Net: Feature-Aware Recurrent Point Cloud Denoising Network

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The captured 3D point clouds by depth cameras and 3D scanners are often corrupted by noise, so point cloud denoising is typically required for downstream applications. We observe that: (i) the scale of the local neighborhood has a significant effect on the denoising performance against different noise levels, point intensities, as well as various kinds of local details; (ii) non-iteratively evolving a noisy input to its noise-free version is non-trivial; (iii) both traditional geometric methods and learning-based methods often lose geometric features with denoising iterations, and (iv) most objects can be regarded as piece-wise smooth surfaces with a small number of features. Motivated by these observations, we propose a novel and task-specific point cloud denoising network, named RePCD-Net, which consists of four key modules: (i) a recurrent network architecture to effectively remove noise; (ii) an RNN-based multi-scale feature aggregation module to extract adaptive features in different denoising stage; (iii) a recurrent propagation layer to enhance the geometric feature perception across stages; and (iv) a feature-aware CD loss to regularize the predictions towards multi-scale geometric details. Extensive qualitative and quantitative evaluations demonstrate the effectiveness and superiority of our method over state-of-the-arts, in terms of noise removal and feature preservation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Alexa, M., Behr, J., Cohen-Or, D., Fleishman, S., Levin, D., & Silva, C. T. (2001). Point set surfaces. In Proceedings visualization, 2001. VIS’01 (pp. 21–29). IEEE.

  • Alexa, M., Behr, J., Cohen-Or, D., Fleishman, S., Levin, D., & Silva, C. T. (2003). Computing and rendering point set surfaces. IEEE Transactions on Visualization and Computer Graphics, 9(1), 3–15.

    Article  Google Scholar 

  • Avron, H., Sharf, A., Greif, C., & Cohen-Or, D. (2010). L1-sparse reconstruction of sharp point set surfaces. ACM Transactions on Graphics (TOG), 29(5), 1–12.

    Article  Google Scholar 

  • Chen, H., Wei, M., Sun, Y., Xie, X., & Wang, J. (2019). Multi-patch collaborative point cloud denoising via low-rank recovery with graph constraint. IEEE Transactions on Visualization and Computer Graphics, 26(11), 3255–3270.

    Article  Google Scholar 

  • Digne, J., Valette, S., & Chaine, R. (2017). Sparse geometric representation through local shape probing. IEEE Transactions on Visualization and Computer Graphics, 24(7), 2238–2250.

    Article  Google Scholar 

  • Fan, H., Su, H., & Guibas, L. J. (2017). A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 605–613.

  • Fleishman, S., Cohen-Or, D., & Silva, C. T. (2005). Robust moving least-squares fitting with sharp features. ACM Transactions on Graphics (TOG), 24(3), 544–552.

    Article  Google Scholar 

  • Gersho, A., & Gray, R. M. (2012). Vector quantization and signal compression (Vol. 159). Berlin.

  • Guo, Y., Bennamoun, M., Sohel, F., Lu, M., & Wan, J. (2014). An integrated framework for 3-d modeling, object detection, and pose estimation from point-clouds. IEEE Transactions on Instrumentation and Measurement, 64(3), 683–693.

    Google Scholar 

  • Hermosilla, P., Ritschel, T., & Ropinski, T. (2019). Total denoising: Unsupervised learning of 3d point cloud cleaning. In Proceedings of the IEEE international conference on computer vision, pp. 52–60.

  • Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.

    Article  Google Scholar 

  • Hu, W., Gao, X., Cheung, G., & Guo, Z. (2020). Feature graph learning for 3d point cloud denoising. IEEE Transactions on Signal Processing, 68, 2841–2856.

    Article  MathSciNet  Google Scholar 

  • Huang, C., Li, R., Li, X., & Fu, C. W. (2020). Non-local part-aware point cloud denoising. arXiv:200306631

  • Huang, H., Li, D., Zhang, H., Ascher, U., & Cohen-Or, D. (2009). Consolidation of unorganized point clouds for surface reconstruction. ACM Transactions on Graphics (TOG), 28(5), 1–7.

    Article  Google Scholar 

  • Huang, H., Wu, S., Gong, M., Cohen-Or, D., Ascher, U., & Zhang, H. (2013). Edge-aware point set resampling. ACM Transactions on Graphics (TOG), 32(1), 1–12.

    Article  Google Scholar 

  • Kong, D., Xu, L., Li, X., & Li, S. (2013). K-plane-based classification of airborne lidar data for accurate building roof measurement. IEEE Transactions on Instrumentation and Measurement, 63(5), 1200–1214.

    Article  Google Scholar 

  • Li, R., Li, X., Fu, C. W., Cohen-Or, D., & Heng, P. A. (2019). Pu-gan: a point cloud upsampling adversarial network. In Proceedings of the IEEE international conference on computer vision, pp. 7203–7212.

  • Li, X., Li, R., Zhu, L., Fu, C. W., & Heng, P. A. (2020). Dnf-net: a deep normal filtering network for mesh denoising. IEEE Transactions on Visualization and Computer Graphics, 27(10), 4060–4072.

  • Lipman, Y., Cohen-Or, D., Levin, D., & Tal-Ezer, H. (2007). Parameterization-free projection for geometry reconstruction. ACM Transactions on Graphics (TOG), 26(3), 22-es.

    Article  Google Scholar 

  • Liu, X., Han, Z., Liu, Y. S., & Zwicker, M. (2019). Point2sequence: Learning the shape representation of 3d point clouds with an attention-based sequence to sequence network. In Proceedings of the AAAI conference on artificial intelligence, (Vol. 33, 8778–8785).

  • Lu, D., Lu, X., Sun, Y., & Wang, J. (2020). Deep feature-preserving normal estimation for point cloud filtering. Computer-Aided Design, 125, 102860.

    Article  MathSciNet  Google Scholar 

  • Lu, X., Schaefer, S., Luo, J., Ma, L., & He, Y. (2018). Low rank matrix approximation for geometry filtering. CoRR arXiv:abs/1803.06783.

  • Lu, X., Wu, S., Chen, H., Yeung, S. K., Chen, W., & Zwicker, M. (2017). Gpf: Gmm-inspired feature-preserving point set filtering. IEEE Transactions on Visualization and Computer Graphics, 24(8), 2315–2326.

    Google Scholar 

  • Luo, S., & Hu, W. (2020). Differentiable manifold reconstruction for point cloud denoising. In Proceedings of the 28th ACM international conference on multimedia, pp. 1330–1338.

  • Nguyen, C. V., Izadi, S., & Lovell, D. (2012). Modeling kinect sensor noise for improved 3d reconstruction and tracking. In 2012 second international conference on 3D imaging, modeling, processing, visualization & transmission (pp. 524–530). IEEE.

  • Öztireli, A. C., Guennebaud, G., & Gross, M. (2009). Feature preserving point set surfaces based on non-linear kernel regression. Computer Graphics Forum, 28(2), 493–501.

    Article  Google Scholar 

  • Pistilli, F., Fracastoro, G., Valsesia, D., & Magli, E. (2020). Learning graph-convolutional representations for point cloud denoising. In European conference on computer vision (pp. 103–118). Springer.

  • Preiner, R., Mattausch, O., Arikan, M., Pajarola, R., & Wimmer, M. (2014). Continuous projection for fast l1 reconstruction. ACM Transactions on Graphics (TOG), 33(4), 47-1.

  • Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2017a). Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652–660.

  • Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017b). Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, & R. Garnett (Eds.), Advances in neural information processing systems (pp. 5099–5108). Red Hook, NY: Curran Associates Inc.

  • Rakotosaona, M., Barbera, V. L., Guerrero, P., Mitra, N. J., & Ovsjanikov, M. (2020). Pointcleannet: Learning to denoise and remove outliers from dense point clouds. Computer Graphics Forum, 39(1), 185–203.

    Article  Google Scholar 

  • Remil, O., Xie, Q., Xie, X., Xu, K., & Wang, J. (2017). Surface reconstruction with data-driven exemplar priors. Computer-Aided Design, 88, 31–41.

    Article  Google Scholar 

  • Rosman, G., Dubrovina, A., & Kimmel, R. (2013). Patch-collaborative spectral point-cloud denoising. Computer Graphics Forum, 32(8), 1–12.

    Article  Google Scholar 

  • Roveri, R., Öztireli, A. C., Pandele, I., & Gross, M. (2018). Pointpronets: Consolidation of point clouds with convolutional neural networks. Computer Graphics Forum, 37, 87–99.

    Article  Google Scholar 

  • Serna, A., Marcotegui, B., Goulette. F., & Deschaud, J. E. (2014). Paris-rue-madame database: A 3d mobile laser scanner dataset for benchmarking urban detection, segmentation and classification methods. In 4th International Conference on Pattern Recognition, Applications and Methods ICPRAM 2014.

  • Sun, Y., Schaefer, S., & Wang, W. (2015). Denoising point sets via l0 minimization. Computer Aided Geometric Design, 35, 2–15.

    Article  MathSciNet  Google Scholar 

  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł, & Polosukhin, I. (2017). Attention is all you need. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, & R. Garnett (Eds.), Advances in neural information processing systems (pp. 5998–6008). Red Hook, NY: Curran Associates Inc.

    Google Scholar 

  • Wang, P. S., Liu, Y., & Tong, X. (2016). Mesh denoising via cascaded normal regression. ACM Transactions on Graphics, 35(6), 232–1.

    Google Scholar 

  • Wang, Y., Liu, Y., Xie, Q., Wu, Q., Guo, X., Yu, Z., & Wang, J. (2020). Density-invariant registration of multiple scans for aircraft measurement. IEEE Transactions on Instrumentation and Measurement, 70, 1–15.

    Article  Google Scholar 

  • Wang, Y., Serena, F., Wu, S., Öztireli, C., & Sorkine-Hornung, O. (2019). Differentiable surface splatting for point-based geometry processing. ACM Transactions on Graphics (TOG), 38(6), 1–14.

    Google Scholar 

  • Wei, M., Feng, Y., & Chen, H. (2020). Selective guidance normal filter for geometric texture removal. IEEE Transactions on Visualization and Computer Graphics, 27(12), 4469–4482.

  • Xie, Q., Lu, D., Huang, A., Yang, J., Li, D., Zhang, Y., & Wang, J. (2020). Rrcnet: Rivet region classification network for rivet flush measurement based on 3-d point cloud. IEEE Transactions on Instrumentation and Measurement, 70, 1–12.

    Google Scholar 

  • Yu, L., Li, X., Fu, C. W., Cohen-Or, D., & Heng, P. A. (2018a). Ec-net: An edge-aware point set consolidation network. In Proceedings of the European conference on computer vision (ECCV), pp. 386–402.

  • Yu, L., Li, X., Fu, C. W., Cohen-Or, D., & Heng, P. A. (2018b). Pu-net: Point cloud upsampling network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2790–2799.

  • Zhang, D., Lu, X., Qin, H., & He, Y. (2020). Pointfilter: Point cloud filtering via encoder-decoder modeling. IEEE Transactions on Visualization and Computer Graphics.

  • Zhou, H., Chen, K., Zhang, W., Fang, H., Zhou, W., & Yu, N. (2019). Dup-net: Denoiser and upsampler network for 3d adversarial point clouds defense. In Proceedings of the IEEE international conference on computer vision, pp. 1961–1970.

Download references

Acknowledgements

The work was supported by the Hong Kong Centre for Logistics Robotics.

Author information

Authors and Affiliations

  1. Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Honghua Chen, Zeyong Wei, Yabin Xu, Mingqiang Wei & Jun Wang

  2. Shenzhen Research Institute, Shenzhen, China

    Mingqiang Wei

  3. Huazhong University of Science and Technology, Wuhan, China

    Xianzhi Li

Authors
  1. Honghua Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zeyong Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xianzhi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yabin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mingqiang Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Wang.

Additional information

Communicated by Zuzana Kukelova.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the National Key Research and Development Program of China (No. 2020YFB2010702, No. 2019YFB1707504), National Natural Science Foundation of China (No. 61772267, No. 62172218), Natural Science Foundation of Jiangsu Province (No. BK20190016), and the Free Exploration of Basic Research Project, Local Science and Technology Development Fund Guided by the Central Government of China (No. 2021Szvup060). The corresponding author for this paper is Jun Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, H., Wei, Z., Li, X. et al. RePCD-Net: Feature-Aware Recurrent Point Cloud Denoising Network. Int J Comput Vis 130, 615–629 (2022). https://doi.org/10.1007/s11263-021-01564-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-021-01564-7

Keywords

Search

Navigation