Skip to main content
SpringerLink
Log in

A Hierarchical Optimization Algorithm Based on GPU for Real-Time 3D Reconstruction

  • 3DR Express
  • Published:
3D Research

Abstract

In machine vision sensing system, it is important to realize high-quality real-time 3D reconstruction in large-scale scene. The recent online approach performed well, but scaling up the reconstruction, it causes pose estimation drift, resulting in the cumulative error, usually requiring a large number of off-line operation to completely correct the error, reducing the reconstruction performance. In order to optimize the traditional volume fusion method and improve the old frame-to-frame pose estimation strategy, this paper presents a real-time CPU to Graphic Processing Unit reconstruction system. Based on a robust camera pose estimation strategy, the algorithm fuses all the RGB-D input values into an effective hierarchical optimization framework, and optimizes each frame according to the global camera attitude, eliminating the serious dependence on the tracking timeliness and continuously tracking globally optimized frames. The system estimates the global optimization of gestures (bundling) in real-time, supports for robust tracking recovery (re-positioning), and re-estimation of large-scale 3D scenes to ensure global consistency. It uses a set of sparse corresponding features, geometric and ray matching functions in one of the parallel optimization systems. The experimental results show that the average reconstruction time is 415 ms per frame, the ICP pose is estimated 20 times in 100.0 ms. For large scale 3D reconstruction scene, the system performs well in online reconstruction area, keeping the reconstruction accuracy at the same time.

Graphical Abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Weise, T., Wismer, T., & Leibe, B., et al. (2009). In-hand scanning with online loop closure. In IEEE international conference on computer vision workshops (pp. 1630–1637).

  2. Henry, P., Krainin, M., Herbst, E., et al. (2012). RGB-D mapping: Using depth cameras for dense 3d modeling of indoor environments. International Journal of Robotics Research, 31(5), 647–663.

    Article  Google Scholar 

  3. Keller, M., Lefloch, D., & Lambers, M., et al. (2013). Real-time 3D reconstruction in dynamic scenes using point-based fusion. In International conference on 3D Vision-3DV (Vol. 8768, Issue 2, pp. 1–8).

  4. Whelan, T., Leutenegger, S., & Salas-Moreno, R. F., et al. (2015). ElasticFusion: Dense SLAM without a pose graph. Robotics: Science and Systems (RSS).

  5. Merrell, P., Akbarzadeh, A., & Wang, L., et al. (2007). Real-time visibilitybased fusion of depth maps. In IEEE international conference on computer vision (Vol. 8, pp. 1–8).

  6. Meilland, M., & Comport, A. (2013). On unifying key-frame and voxel-based dense visual slam at large scales. IEEE/RSJ International Conference on Intelligent Robots & Systems, 8215(2), 3677–3683.

    Google Scholar 

  7. Gallup, D., Pollefeys, M., & Frahm, J. M. (2010). 3D reconstruction using an n-layer heightmap. Pattern Recognition, 6376, 1–10.

    Google Scholar 

  8. Wurm, K. M., Hornung, A., & Bennewitz, M., et al. (2010). OctoMap: A probabilistic, flexible, and compact 3D map representation for robotic systems. In IEEE international conference on robotics and automation.

  9. Curless, B., & Levoy, M. (1996). A volumetric method for building complex models from range images. In In Proceedings of SIGGRAPH. ACM (pp. 303–312).

  10. Newcombe, R. A., Izadi, S., & Hilliges, O., et al. (1996). KinectFusion: Real-time dense surface mapping and tracking. In Conference on computer graphics and interactive techniques (Vol. 3, pp. 303–312).

  11. Steinbruecker, F., Sturm, J., & Cremers, D. (2014). Volumetric 3D mapping in real-time on a CPU. In IEEE international conference on robotics and automation (pp. 2021–2028).

  12. Zollhöfer, M., Thies, J., Colaianni, M., et al. (2014). Interactive model-based reconstruction of the human head using an RGB-D sensor. Computer Animation and Virtual Worlds, 25(25), 213–222.

    Article  Google Scholar 

  13. Zhou, Q. Y., & Koltun, V. (2014). Color map optimization for 3d reconstruction with consumer depth cameras. ACM Transactions on Graphics, 33(4), 1–10.

    Google Scholar 

  14. Choi, S., Zhou, Q. Y., & Koltun, V. (2015). Robust reconstruction of indoor scenes. In: Computer vision and pattern recognition (pp. 5556–5565).

  15. Wikowski, A., Kornuta, T., Stefańczyk, M., et al. (2016). Efficient generation of 3D surfel maps using RGB-D sensors. International Journal of Applied Mathematics and Computer Science, 1, 99–122.

    MathSciNet  MATH  Google Scholar 

  16. Kornuta, T., & Laszkowski, M. (2016). Perception subsystem for object recognition and pose estimation in RGB-D images. Automation, Springer International Publishing, 44(10), 995–1003.

    Google Scholar 

  17. Whelan, T., Johannsson, H., & Kaess, M., et al. (2013). Robust real-time visual odometry for dense RGB-D mapping. In IEEE international conference on robotics and automation.

  18. Lin, J. H., Wang, Y. J., & Sun, H. H. (2017). A feature-adaptive subdivision method for real-time 3D reconstruction of repeated topology surfaces. 3D Research, 8, 6. doi:10.1007/s13319-017-0117-z.

    Article  Google Scholar 

  19. Qu, Y., Liu, Z., Jiang, Y., et al. (2017). Self-adaptative variable-metric feature point extraction method. Editorial Office of Optics and Precision Engineering, 25(1), 188–197. (In Chinese).

    Article  Google Scholar 

  20. Liu, Y., Wang, C., Gao, N., et al. (2017). Point cloud adaptive simplification of feature extraction. Editorial Office of Optics and Precision Engineering, 25(1), 245–254. (In Chinese).

    Article  Google Scholar 

  21. Maier, R., Sturm, J., & Cremers, D. (2014). Submap-based bundle adjustment for 3D reconstruction from RGB-D Data. In Pattern Recognition (pp. 54–65).

  22. Engel, J., Schöps, T., & Cremers, D. (2014). LSD-SLAM: Large-scale direct monocular SLAM (pp. 834–849). Zurich: Springer.

    Google Scholar 

  23. Stückler, J., & Behnke, S. (2014). Multi-resolution surfel maps for efficient dense 3D modeling and tracking. Journal of Visual Communication and Image Representation, 25(1), 137–147.

    Article  Google Scholar 

  24. Nießner, M., Dai, A., & Fisher, M. (2014). Combining inertial navigation and ICP for real-time 3D Surface Reconstruction.

  25. Kerl, C., Sturm, J., & Cremers, D. (2013). Dense visual SLAM for RGB-D cameras. In IEEE international conference on intelligent robots and systems (pp. 2100–2106).

  26. Zhang, L., Wang, Y., Sun, H., et al. (2016). Adaptive scale object tracking with kernelized correlation filters. Editorial Office of Optics and Precision Engineering, 24(2), 448–459. (In Chinese).

    Article  Google Scholar 

  27. Wang, Y., Zhang, Q., & Zhou, Y. (2015). Dense 3D mapping for indoor environment based on Kinect-style depth cameras (Vol. 345, pp. 317–330). Cham: Springer.

    Google Scholar 

  28. Sturm, J., Engelhard, N., Endres, F., et al. (2012). A benchmark for the evaluation of RGB-D SLAM systems. In IEEE/RSJ international conference on intelligent robots and systems (pp. 573–580). IEEE.

Download references

Acknowledgements

This work is funded by National High-tech R&D Program (863 Program) (No. 2014AA7031010B), Science and Technology Project of the thirteenth Five-Year Plan (JJZ[2016]345).

Author information

Authors and Affiliations

  1. Machinery and Electronics Engineering, Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, 130033, China

    Jin-hua Lin & Yan-jie Wang

  2. Machinery and Electronics Engineering, Chinese Academy of Sciences University, Changchun, 130033, China

    Jin-hua Lin

  3. Computer Application Technology, Changchun University of Technology, Changchun, 130033, China

    Jin-hua Lin & Lu Wang

Authors
  1. Jin-hua Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan-jie Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin-hua Lin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, Jh., Wang, L. & Wang, Yj. A Hierarchical Optimization Algorithm Based on GPU for Real-Time 3D Reconstruction. 3D Res 8, 16 (2017). https://doi.org/10.1007/s13319-017-0127-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13319-017-0127-x

Keywords

Search

Navigation