Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Robust dense reconstruction by range merging based on confidence estimation

  • 165 Accesses

  • 58 Citations

Abstract

Although the stereo matching problem has been extensively studied during the past decades, automatically computing a dense 3D reconstruction from several multiple views is still a difficult task owing to the problems of textureless regions, outliers, detail loss, and various other factors. In this paper, these difficult problems are handled effectively by a robust model that outputs an accurate and dense reconstruction as the final result from an input of multiple images captured by a normal camera. First, the positions of the camera and sparse 3D points are estimated by a structure-from-motion algorithm and we compute the range map with a confidence estimation for each image in our approach. Then all the range maps are integrated into a fine point cloud data set. In the final step we use a Poisson reconstruction algorithm to finish the reconstruction. The major contributions of the work lie in the following points: effective range-computation and confidence-estimation methods are proposed to handle the problems of textureless regions, outliers and detail loss. Then, the range maps are merged into the point cloud data in terms of a confidence-estimation. Finally, Poisson reconstruction algorithm completes the dense mesh. In addition, texture mapping is also implemented as a post-processing work for obtaining good visual effects. Experimental results are presented to demonstrate the effectiveness of the proposed approach.

This is a preview of subscription content, log in to check access.

References

  1. 1

    Chen J W, Bautembach D, Izadi S. Scalable real-time volumetric surface reconstruction. ACM Trans Graphic, 2013, 32: 1–16

  2. 2

    Chen Y D, Hao C Y, Wu W, et al. Recursive video segmentation (in Chinese). Sci Sin Inform, 2014, 44: 1361–1369

  3. 3

    Hao C Y, Chen Y D, Wu W, et al. Image completion with perspective constraint based on a single image. Sci China Inf Sci, 2015, 58: 092109

  4. 4

    Collins R T. A space-sweep approach to true multi-image matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, 1996. 358–363

  5. 5

    Seitz S M, Dyer C R. Photorealistic scene reconstruction by voxel coloring. Int J Comput Vision, 1999, 35: 151–173

  6. 6

    Newcombe R A, Davison A J. Live dense reconstruction with a single moving camera. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, 2010. 1498–1505

  7. 7

    Chen Y D, Hao C Y, Wu W, et al. Live accurate and dense reconstruction from a handheld camera. Comput Animat Virtual Worlds, 2013, 24: 387–397

  8. 8

    Furukawa Y, Ponce J. Accurate, dense, and robust multiview stereopsis. IEEE Trans Pattern Anal Mach Intell, 2010, 32: 1362–1376

  9. 9

    Furukawa Y, Curless B, Seitz S M, et al. Towards internet-scale multi-view stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, 2010. 1434–1441

  10. 10

    Wang Y, Ji X Y, Dai Q H. Key technologies of light field capture for 3d reconstruction in microscopic scene. Sci China Inf Sci, 2010, 53: 1917–1930

  11. 11

    Agarwal S, Furukawa Y, Snavely N, et al. Building rome in a day. Commun ACM, 2011, 54: 105–112

  12. 12

    Sun J, Li Y, Kang S B, et al. Symmetric stereo matching for occlusion handling. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, 2005. 399–406

  13. 13

    Strecha C, Fransens R, van Gool L. Combined depth and outlier estimation in multi-view stereo. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, 2006. 2394–2401

  14. 14

    Ryan K, Camillo J T. Optical flow with geometric occlusion estimation and fusion of multiple frames. In: Proceedings of the 10th International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition, Hongkong, 2015. 364–377

  15. 15

    Zhang G F, Jia J Y, Hua W, et al. Robust bilayer segmentation and motion depth estimation with a handheld camera. IEEE Trans Pattern Anal Mach Intell, 2011, 33: 603–617

  16. 16

    Felzenszwalb P F, Huttenlocher D P. Efficient belief propagation for early vision. Int J Comput Vision, 2006, 70: 41–54

  17. 17

    Woodford O, Torr P, Reid I, et al. Global stereo reconstruction under second-order smoothness priors. IEEE Trans Pattern Anal Mach Intell, 2009, 31: 2115–2128

  18. 18

    Fuhrmann S, Goesele M. Fusion of depth maps with multiple scales. In: Proceedings of the SIGGRAPH Asia Conference. New York: ACM, 2011. 1–8

  19. 19

    Zach C, Pock T, Bischof H. A globally optimal algorithm for robust TV-L1 range image integration. In: Proceedings of the IEEE International Conference on Computer Vision, Rio de Janeiro, 2007. 1–8

  20. 20

    Stühmer J, Gumhold S, Cremers D. Real-time dense geometry from a handheld camera. In: Proceedings of the Conference on Pattern Recognition, Lecture Notes in Computer Science, Berlin, 2010. 11–20

  21. 21

    Graber G, Pock T, Bischof H. Online 3d reconstruction using convex optimization. In: Proceedings of IEEE International Conference on Computer Vision Workshops, Barcelona, 2011. 708–711

  22. 22

    Bischof H. Dense reconstruction on-the-fly. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Washington, 2012. 1450–1457

  23. 23

    Shan Q, Curless B, Furukawa Y, et al. Occluding contours for multi-view stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Washington, 2014. 4002–4009

  24. 24

    Snavely N, Seitz S M, Szeliski R. Photo tourism: exploring photo collections in 3D. In: Proceedings of the ACM SIGGRAPH, New York, 2006. 835–846

  25. 25

    Wu C C, Agarwal S, Curless B, et al. Multicore bundle adjustment. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Providence, 2011. 3057–3064

  26. 26

    Dong Z L, Zhang G F, Jia J Y, et al. Efficient keyframe-based real-time camera tracking. Comput Vision Image Und, 2014, 118: 97–110

  27. 27

    Kazhdan M, Hoppe H. Screened Poisson surface reconstruction. ACM Trans Graphic, 2013, 32: 1–13

  28. 28

    Kang S B, Szeliski R. Extracting view-dependent depth maps from a collection of images. Int J Comput Vision, 2004, 58: 139–163

Download references

Author information

Correspondence to Chuanyan Hao.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Hao, C., Wu, W. et al. Robust dense reconstruction by range merging based on confidence estimation. Sci. China Inf. Sci. 59, 092103 (2016). https://doi.org/10.1007/s11432-015-0957-4

Download citation

Keywords

  • stereo matching
  • 3D reconstruction
  • textureless regions
  • outliers
  • details loss
  • range map