3D Reconstruction of Indoor Scenes via Image Registration

Article
  • 38 Downloads

Abstract

With the development of computer vision technologies, 3D reconstruction has become a hotspot. At present, 3D reconstruction relies heavily on expensive equipment and has poor real-time performance. In this paper, we aim at solving the problem of 3D reconstruction of an indoor scene with large vertical span. In this paper, we propose a novel approach for 3D reconstruction of indoor scenes with only a Kinect. Firstly, this method uses a Kinect sensor to get color images and depth images of an indoor scene. Secondly, the combination of scale-invariant feature transform and random sample consensus algorithm is used to determine the transformation matrix of adjacent frames, which can be seen as the initial value of iterative closest point (ICP). Thirdly, we establish the relative coordinate relation between pair-wise frames which are the initial point cloud data by using ICP. Finally, we achieve the 3D visual reconstruction model of indoor scene by the top-down image registration of point cloud data. This approach not only mitigates the sensor perspective restriction and achieves the indoor scene reconstruction of large vertical span, but also develops the fast algorithm of indoor scene reconstruction with large amount of cloud data. The experimental results show that the proposed algorithm has better accuracy, better reconstruction effect, and less running time for point cloud registration. In addition, the proposed method has great potential applied to 3D simultaneous location and mapping.

Keywords

Indoor scene 3D reconstruction Relative coordinate ICP Top and bottom registration 3D SLAM 

Notes

Acknowledgements

The paper was supported in part by the National Natural Science Foundation (NSFC) of China under Grant Nos. 61373077, 61365003 and Gansu Province Basic Research Innovation Group Project No. 1506RJIA031.

References

  1. 1.
    Barnard ST, Fischler MA (1982) Computational stereo. ACM Comput Surv 14(4):553–572CrossRefGoogle Scholar
  2. 2.
    Thrun S, Burgard W, Fox D (2000) A real-time algorithm for mobile robot mapping with applications to multi-robot and 3D mapping. In: IEEE international conference on robotics and automation, pp 321–328Google Scholar
  3. 3.
    Huang MQ, Wang ZY, Hao LQ, Zhou LZ (2006) Laser aerosol time-of-flight mass spectrometry analysis of individual aerosol particles from photooxidation of toluene. Opt Appl 36(1):1–5Google Scholar
  4. 4.
    Filipik A, Jan J, Peterlik I (2012) Time-of-Flight based calibration of an ultrasonic computed tomography system. Radio Eng 23(8):346–355Google Scholar
  5. 5.
    Zhang Z (2012) Microsoft kinect sensor and its effect. IEEE Multimed 19(2):4–10CrossRefGoogle Scholar
  6. 6.
  7. 7.
    Mair E, Strobl KH, Bodenmüller T, Suppa M, Burschka D (2010) Real-time image-based localization for hand-held 3D-modeling. KI-Künstl Intell 24(3):207–214CrossRefGoogle Scholar
  8. 8.
    Besl PJ, McKay ND (2002) A method for registration of 3-D shapes. In: IEEE transactions on pattern analysis & machine intelligence, vol 14, no 2, pp 239–256Google Scholar
  9. 9.
    Rusinkiewicz S, Levoy M (2001) Efficient variants of the ICP algorithm. In: IEEE conference on 3D digital imaging and modeling, pp 145–152Google Scholar
  10. 10.
    Henry P, Krainin M, Herbst E, Ren X, Fox D (2012) RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments. Int J Robot Res 31(5):647–663CrossRefGoogle Scholar
  11. 11.
    Newcombe RA, Izadi S, Hilliges O, Molyneaux D, Kim D, Davison AJ, Kohi P, Shotton J, Hodges S, Fitzgibbon A (2011)KinectFusion: real-time dense surface mapping and tracking. In: IEEE international symposium on mixed and augmented reality, pp 127–136Google Scholar
  12. 12.
    Izadi S, Kim D, Hilliges O, Molyneaux D, Newcombe R,Kohli P, Shotton J, Hodges S, FreemanD, Davison A (2011) KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th annual ACM symposium on user interface software and technology, pp 559–568Google Scholar
  13. 13.
    Liu X, Xu HR, Hu ZY (2012) GPU based fast 3D-object modeling with kinect. Acta Autom Sin 38(8):1288–1297CrossRefGoogle Scholar
  14. 14.
    Zou Y, Chen W, Wu X, Liu Z (2012) Indoor localization and 3D scene reconstruction for mobile robots using the Microsoft Kinect sensor. In: IEEE international conference on industrial informatics, pp 1182–1187Google Scholar
  15. 15.
    Yue H, Chen W, Wu X, Liu J (2014) Fast 3D modeling in complex environments using a single Kinect sensor. Opt Lasers Eng 53(1):104–111CrossRefGoogle Scholar
  16. 16.
    Wang K, Zhang G, Bao H (2014) Robust 3D reconstruction with an RGB-D camera. IEEE Trans Image Process 23(11):4893–4906MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Choi S, Zhou QY, Koltun V (2015) Robust reconstruction of indoor scenes. In: IEEE conference on computer vision and pattern recognition, pp 5556–5565Google Scholar
  18. 18.
    Mei F, Liu J, Li CP, Wang ZQ (2015) Improved RGB-D camera based indoor scene reconstruction. J Image Graph 20(10):1366–1373Google Scholar
  19. 19.
    Thomas D, Sugimoto A (2013) A flexible scene representation for 3D reconstruction using an RGB-D camera. In: IEEE international conference on computer vision, pp 2800–2807Google Scholar
  20. 20.
    Yang Y, Gao M, Yin K, Wu Z (2015) High-quality depth map reconstruction combining stereo image pair. J Image Graph 20(1):1–10Google Scholar
  21. 21.
    Xiao JX, Owens A, Torralba A (2013) SUN3D: a database of big spaces reconstructed using sfm and object labels. In: IEEE international conference on computer vision, pp 1625–1632Google Scholar
  22. 22.
    Zhang CH, Zhu RJ, Zhuang Y (2013) Indoor 3D scene reconstruction based on Kinect depth camera. Netw J Grad Sch Dalian Univ Technol 5:1–8Google Scholar
  23. 23.
    Bay H, Ess A, Tuytelaars T, Gool LV (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359CrossRefGoogle Scholar
  24. 24.
    Chen XM, Jiang LT, Ying RD (2013) Research of 3D reconstruction and filtering algorithm based on depth information of Kinect. Appl Res Comput 30(4):1216–1218Google Scholar
  25. 25.
    Izadi S, Stamminger M (2013) Real-time 3D reconstruction at scale using voxel hashing. ACM Trans Graph 32(6):169–187Google Scholar
  26. 26.
    Liu Z, Zhang Y, Wu W, Liu K, Sun Z (2015) Model-driven indoor scenes modeling from a single image. In: Graphics interface conference canadian information processing society, pp 25–32Google Scholar
  27. 27.
    Li YF, Zhang GL, Xu J, Yao EL (2016) Improved ICP in frame-to-frame registration based on Kinect. Electron Opt Control 23(2):56–60Google Scholar
  28. 28.
    Du SY, Liu J, Zhang CJ, Zhu JH, Li K (2015) Probability iterative closest point algorithm for m-D point set registration with noise. Neurocomputing 157:187–198CrossRefGoogle Scholar
  29. 29.
    Du SY, Liu J, Bi B, Zhu JH, Xue JR (2016) New iterative closest point algorithm for isotropic scaling registration of point sets with noise. J Vis Commun Image Represent 38:207–216CrossRefGoogle Scholar
  30. 30.
    Ying S, Wu G, Wang Q, Shen D (2014) Hierarchical unbiased graph shrinkage (HUGS): a novel groupwise registration for large data set. Neuroimage 84(1):626–638CrossRefGoogle Scholar
  31. 31.
    Ying S, Wang Y, Wen Z, Lin Y (2016) Nonlinear 2D shape registration via thin-plate spline and Lie group representation. Neurocomputing 195:129–136CrossRefGoogle Scholar
  32. 32.
    Santos DRD, Basso MA, Khoshelham K, Oliveira ED, Pavan NL, Vosselman G (2016) Mapping indoor spaces by adaptive coarse-to-fine registration of RGB-D data. IEEE Geosci Remote Sens Lett 13(2):262–266Google Scholar
  33. 33.
    Xi L (2017) A real time implementation of 3D symmetric object reconstruction. Faculty of the Graduate School of the University of Maryland, College ParkGoogle Scholar
  34. 34.
    Lv Q, Lin H, Wang G, Wei H, Wang Y (2017) ORB-SLAM-based tracing and 3D reconstruction for robot using Kinect 2.0. In: IEEE control and decision conference, pp 3319–3324Google Scholar
  35. 35.
    Nuchter A, Lingemann K, Hertzberg J (2007) Cached k-d tree search for ICP algorithms. In: IEEE international conference on 3D digital imaging and modeling, pp 419–426Google Scholar
  36. 36.
    Chen HM, Lin TH (2006) An algorithm to build convex hulls for 3D objects. J Chin Inst Eng 29(6):945–952MathSciNetCrossRefGoogle Scholar
  37. 37.
    Lowe DG (1999) Object recognition from local scale-invariant features. Int Conf Comput Vis 2:1150–1157Google Scholar
  38. 38.
    Helmer S, Lowe DG (2004) Object class recognition with many local features. In: IEEE conference on computer vision and pattern recognition workshop, pp 182–187Google Scholar
  39. 39.
    Lindeberg T (2013) Image matching using generalized scale-space interest points. In: International conference on scale space and variational methods in computer vision, pp 355–367Google Scholar
  40. 40.
    Xu M, Lu J (2012) Distributed RANSAC for the robust estimation of three-dimensional reconstruction. In: IEEE international conference on computer vision, pp 324–333Google Scholar
  41. 41.
    Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395MathSciNetCrossRefGoogle Scholar
  42. 42.
    Li C, Xue JR, Du SY, Zheng NN (2010) A fast multi-resolution iterative closest point algorithm. In: Chinese conference on pattern recognition, pp 1–5Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Ce Li
    • 1
  • Bing Lu
    • 1
  • Yachao Zhang
    • 1
  • Hao Liu
    • 1
  • Yanyun Qu
    • 2
  1. 1.Lanzhou University of TechnologyLanzhouPeople’s Republic of China
  2. 2.Xiamen UniversityXiamenPeople’s Republic of China

Personalised recommendations