3D Object Recognition with Ensemble Learning—A Study of Point Cloud-Based Deep Learning Models

  • Daniel KoguciukEmail author
  • Łukasz Chechliński
  • Tarek El-Gaaly
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11845)


In this study, we present an analysis of model-based ensemble learning for 3D point-cloud object classification. An ensemble of multiple model instances is known to outperform a single model instance, but there is little study of the topic of ensemble learning for 3D point clouds. First, an ensemble of multiple model instances trained on the same part of the ModelNet40 dataset was tested for seven deep learning, point cloud-based classification algorithms: PointNet, PointNet++, SO-Net, KCNet, DeepSets, DGCNN, and PointCNN. Second, the ensemble of different architectures was tested. Results of our experiments show that the tested ensemble learning methods improve over state-of-the-art on the ModelNet40 dataset, from 92.65% to 93.64% for the ensemble of single architecture instances, 94.03% for two different architectures, and 94.15% for five different architectures. We show that the ensemble of two models with different architectures can be as effective as the ensemble of 10 models with the same architecture. Third, a study on classic bagging (i.e. with different subsets used for training multiple model instances) was tested and sources of ensemble accuracy growth were investigated for best-performing architecture, i.e. SO-Net. We measure the inference time of all 3D classification architectures on a Nvidia Jetson TX2, a common embedded computer for mobile robots, to allude to the use of these models in real-life applications.


Point cloud Point set Classification Ensemble learning 3D Deep Learning 



This research was partially supported by the Dean of Faculty of Mechatronics (Grant No. 504/03731 and Grant No. 504/03272). We want to thank authors of all architectures for providing a public repository. We would also like to gratefully acknowledge the helpful comments and suggestions of Tomasz Trzciński and Robert Sitnik.


  1. 1.
    Arvind, V., Costa, A., Badgeley, M., Cho, S., Oermann, E.: Wide and deep volumetric residual networks for volumetric image classification. arXiv preprint arXiv:1710.01217 (2017)
  2. 2.
    Chang, A.X., et al.: ShapeNet: an information-rich 3D model repository. Technical report. arXiv:1512.03012 [cs.GR], Stanford University – Princeton University – Toyota Technological Institute at Chicago (2015)
  3. 3.
    Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T., Nießner, M.: ScanNet: richly-annotated 3D reconstructions of indoor scenes (2017). arxiv:1702.04405
  4. 4.
    Gan, Y., Tang, Y., Zhang, Q.: 3D model retrieval method based on mesh segmentation. In: Proceedings of SPIE - The International Society for Optical Engineering, vol. 8334, p. 120 (2012).
  5. 5.
    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Rob. Res. (IJRR) (2013)Google Scholar
  6. 6.
    Himmelsbach, M., Luettel, T., Wuensche, H.J.: Real-time object classification in 3D point clouds using point feature histograms. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2009, pp. 994–1000. IEEE (2009)Google Scholar
  7. 7.
    Li, J., Chen, B.M., Lee, G.H.: SO-Net: self-organizing network for point cloud analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9397–9406 (2018)Google Scholar
  8. 8.
    Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B.: PointCNN: convolution on X-transformed points. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 31, pp. 828–838. Curran Associates, Inc. (2018)Google Scholar
  9. 9.
    Opitz, D., Maclin, R.: Popular ensemble methods: an empirical study. J. Artif. Intell. Res. 11, 169–198 (1999)CrossRefGoogle Scholar
  10. 10.
    Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings Computer Vision and Pattern Recognition (CVPR), vol. 1, no. 2, p. 4. IEEE (2017)Google Scholar
  11. 11.
    Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)Google Scholar
  12. 12.
    Rokach, L.: Ensemble-based classifiers. Artif. Intell. Rev. 33(1–2), 1–39 (2010)CrossRefGoogle Scholar
  13. 13.
    Rutzinger, M., Höfle, B., Hollaus, M., Pfeifer, N.: Object-based point cloud analysis of full-waveform airborne laser scanning data for urban vegetation classification. Sensors 8(8), 4505–4528 (2008)CrossRefGoogle Scholar
  14. 14.
    Sfikas, K., Pratikakis, I., Theoharis, T.: Ensemble of PANORAMA-based convolutional neural networks for 3D model classification and retrieval. Comput. Graph. 71, 208–218 (2018)CrossRefGoogle Scholar
  15. 15.
    Shen, Y., Feng, C., Yang, Y., Tian, D.: Mining point cloud local structures by kernel correlation and graph pooling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 4 (2018)Google Scholar
  16. 16.
    Song, S., Lichtenberg, S.P., Xiao, J.: SUN RGB-D: A RGB-D scene understanding benchmark suite. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 567–576. IEEE (2015)Google Scholar
  17. 17.
    Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E.: Multi-view convolutional neural networks for 3D shape recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 945–953 (2015)Google Scholar
  18. 18.
    Su, J., Gadelha, M., Wang, R., Maji, S.: A deeper look at 3D shape classifiers. CoRR abs/1809.02560 (2018)Google Scholar
  19. 19.
    Tang, J., Ren, Y., Liu, S.: Real-time robot localization, vision, and speech recognition on Nvidia Jetson TX1. CoRR abs/1705.10945 (2017)Google Scholar
  20. 20.
    Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. arXiv preprint arXiv:1801.07829 (2018)
  21. 21.
    Wu, Z., et al.: 3D ShapeNets: a deep representation for volumetric shapes. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1912–1920, June 2015.
  22. 22.
    Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R.R., Smola, A.J.: Deep sets. In: Advances in Neural Information Processing Systems, pp. 3391–3401 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Faculty of MechatronicsWarsaw University of TechnologyWarsawPoland
  2. 2.VoyagePalo AltoUSA

Personalised recommendations