DeepFit: 3D Surface Fitting via Neural Network Weighted Least Squares

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12346)


We propose a surface fitting method for unstructured 3D point clouds. This method, called DeepFit, incorporates a neural network to learn point-wise weights for weighted least squares polynomial surface fitting. The learned weights act as a soft selection for the neighborhood of surface points thus avoiding the scale selection required of previous methods. To train the network we propose a novel surface consistency loss that improves point weight estimation. The method enables extracting normal vectors and other geometrical properties, such as principal curvatures, the latter were not presented as ground truth during training. We achieve state-of-the-art results on a benchmark normal and curvature estimation dataset, demonstrate robustness to noise, outliers and density variations, and show its application on noise removal.


Normal estimation Surface fitting Least squares Unstructured 3D point clouds 3D point cloud deep learning 



This research was conducted by the Australian Research Council Centre of Excellence for Robotic Vision (CE140100016).

Supplementary material (31.8 mb)
Supplementary material 1 (zip 32581 KB)


  1. 1.
    Ben-Shabat, Y., Avraham, T., Lindenbaum, M., Fischer, A.: Graph based over-segmentation methods for 3D point clouds. Comput. Vis. Image Underst. 174, 12–23 (2018)CrossRefGoogle Scholar
  2. 2.
    Ben-Shabat, Y., Lindenbaum, M., Fischer, A.: 3DmFV: three-dimensional point cloud classification in real-time using convolutional neural networks. IEEE Robot. Autom. Lett. 3(4), 3145–3152 (2018)CrossRefGoogle Scholar
  3. 3.
    Ben-Shabat, Y., Lindenbaum, M., Fischer, A.: Nesti-Net: normal estimation for unstructured 3D point clouds using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10112–10120 (2019)Google Scholar
  4. 4.
    Boulch, A., Marlet, R.: Fast and robust normal estimation for point clouds with sharp features. In: Computer Graphics Forum, vol. 31, pp. 1765–1774. Wiley Online Library (2012)Google Scholar
  5. 5.
    Boulch, A., Marlet, R.: Deep learning for robust normal estimation in unstructured point clouds. Comput. Graph. Forum 35(5), 281–290 (2016)CrossRefGoogle Scholar
  6. 6.
    Cazals, F., Pouget, M.: Estimating differential quantities using polynomial fitting of osculating jets. Comput. Aided Geom. Des. 22(2), 121–146 (2005)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Dang, Z., Yi, K.M., Hu, Y., Wang, F., Fua, P., Salzmann, M.: Eigendecomposition-free training of deep networks with zero eigenvalue-based losses. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11209, pp. 792–807. Springer, Cham (2018). Scholar
  8. 8.
    Guennebaud, G., Gross, M.: Algebraic point set surfaces. ACM Trans. Graph. (TOG) 26(3), 23 (2007)CrossRefGoogle Scholar
  9. 9.
    Guerrero, P., Kleiman, Y., Ovsjanikov, M., Mitra, N.J.: PCPNet learning local shape properties from raw point clouds. Comput. Graph. Forum 37(2), 75–85 (2018)CrossRefGoogle Scholar
  10. 10.
    Hoppe, H., DeRose, T., Duchampt, T., McDonald, J., Stuetzle, W.: Surface reconstruction from unorganized points. Comput. Graph. 26, 2 (1992)CrossRefGoogle Scholar
  11. 11.
    Kamberov, G., Kamberova, G.: Geometric integrability and consistency of 3D point clouds. In 2007 IEEE 11th International Conference on Computer Vision, pp. 1–6. IEEE (2007)Google Scholar
  12. 12.
    Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Proceedings of the 4th Eurographics Symposium on Geometry processing, vol. 7 (2006)Google Scholar
  13. 13.
    Klokov, R., Lempitsky, V.: Escape from cells: deep Kd-networks for the recognition of 3D point cloud models. In The IEEE International Conference on Computer Vision (ICCV), pp. 863–872 (October 2017)Google Scholar
  14. 14.
    Lenssen, J.E., Osendorfer, C., Masci, J.: Differentiable iterative surface normal estimation. arXiv preprint arXiv:1904.07172 (2019)
  15. 15.
    Maturana, D., Scherer, S.: VoxNet: a 3D convolutional neural network for real-time object recognition. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 922–928. IEEE (2015)Google Scholar
  16. 16.
    Meyer, M., Desbrun, M., Schröder, P., Barr, A.H.: Discrete differential-geometry operators for triangulated 2-manifolds. In: Hege, H.C., Polthier, K. (eds.) Visualization and Mathematics III. Mathematics and Visualization. Springer, Heidelberg (2003). Scholar
  17. 17.
    Mitra, N.J., Nguyen, A.: Estimating surface normals in noisy point cloud data. In: Proceedings of the 19th Annual Symposium on Computational Geometry, pp. 322–328. ACM (2003)Google Scholar
  18. 18.
    Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (July 2017)Google Scholar
  19. 19.
    Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)Google Scholar
  20. 20.
    Rusinkiewicz, S.: Estimating curvatures and their derivatives on triangle meshes. In: 2004 Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 3DPVT 2004, pp. 486–493. IEEE (2004)Google Scholar
  21. 21.
    Spivak, M.D.: A Comprehensive Introduction to Differential Geometry. Publish or Perish, Inc., Houston (1970)zbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Australian Centre for Robotic VisionAustralian National UniversityCanberraAustralia

Personalised recommendations