Skip to main content
SpringerLink
Log in

Video Frames Selection Method for 3D Reconstruction Depending on ROS-Based Monocular SLAM

  • Chapter
  • First Online:
Robot Operating System (ROS)

Part of the book series: Studies in Computational Intelligence ((SCI,volume 895))

  • 2955 Accesses

Abstract

In this research chapter, a new method for building a point cloud for an object in a static scene is presented. The method uses images taken by an RGB camera mounted on a controllable robot while moving around that object. During the estimation of the pose of every video frame, a selection method is applied to extract the best frames from the video. Based on these selected images and their estimated rotation and transition vectors, a sparse 3D reconstruction process is conducted. The estimation of these vectors is done by applying Extended Kalman Filter to solve the Simultaneous Localization and Mapping (SLAM) problem with ROS (Robotics Operating System) as a framework. Covariance information provided by Kalman filter is utilized as additional selection criterion. Then, a ROS-based sparse bundle adjustment (SBA) process is performed on both of the new point cloud and the estimated pose vectors. Finally, a dense 3D reconstruction is performed on the optimized values of the rotations and transitions vectors to get a denser point cloud. This method is tested using simulation in Gazebo framework and the results are discussed. All the experiments are explained in details in this chapter. The source code of this project is available online and divided to two public repositories, one for the filtering phase (https://github.com/engyasin/EKF-MonoSLAM_for_3D-reconstruction) and the other for the 3D reconstruction phase (https://github.com/engyasin/3D-reconstruction_with_known_poses).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/engyasin/EKF-MonoSLAM_for_3D-reconstruction.

  2. 2.

    https://www.sympy.org/.

  3. 3.

    https://github.com/rrg-polito/mono-slam.

  4. 4.

    https://www.meshlab.net.

  5. 5.

    https://code.ros.org/svn/ros-pkg/stacks/vslam/trunk.

  6. 6.

    https://github.com/ros-teleop/teleop_twist_keyboard.

  7. 7.

    https://bitbucket.org/osrf/gazebo_models.

  8. 8.

    https://youtu.be/gxz2JRf8LvQ.

  9. 9.

    http://wiki.ros.org/turtlebot_teleop.

  10. 10.

    https://youtu.be/XabY1OyT6_4.

  11. 11.

    https://github.com/engyasin/3D-reconstruction_with_known_poses.

  12. 12.

    https://cython.org.

  13. 13.

    https://youtu.be/BRDCWAJDuM0.

  14. 14.

    https://youtu.be/ucTHoqFaFig.

  15. 15.

    https://youtu.be/XabY1OyT6_4.

  16. 16.

    https://youtu.be/jrnJ9LEAygE.

References

  1. S. Agarwal, Y. Furukawa, N. Snavely, I. Simon, B. Curless, S.M. Seitz, R. Szeliski, Building Rome in a day. Commun. ACM 54(10), 105–112 (2011)

    Google Scholar 

  2. B. Williams, M. Cummins, J. Neira, P. Newman, I. Reid, J. Tardós, A comparison of loop closing techniques in monocular SLAM. Robot. Auton. Syst. 57(12), 1188–1197 (2009)

    Article  Google Scholar 

  3. O.G. Grasa, J. Civera, A. Guemes, V. Munoz, J.M.M. Montiel, EKF monocular SLAM 3D modeling, measuring and augmented reality from endoscope image sequences, in Medical Image Computing and Computer-Assisted Intervention (MICCAI), vol. 2 (2009)

    Google Scholar 

  4. N. Mahmoud, I. Cirauqui, A. Hostettler, C. Doignon, L. Soler, J. Marescaux, J.M.M. Montiel, ORBSLAM-based endoscope tracking and 3d reconstruction, in International Workshop on Computer-Assisted and Robotic Endoscopy (Springer, 2016), pp. 72–83

    Google Scholar 

  5. G. Pavoni, M. Dellepiane, M. Callieri, R. Scopigno, Automatic selection of video frames for path regularization and 3D reconstruction, in Proceedings of the 14th Eurographics Workshop on Graphics and Cultural Heritage, GCH’16, Goslar Germany, Germany (Eurographics Association, 2016), pp. 1–10

    Google Scholar 

  6. A. Rachmielowski, N. Birkbeck, M. Jägersand, D. Cobzas, Realtime visualization of monocular data for 3d reconstruction, in 2008 Canadian Conference on Computer and Robot Vision (IEEE, 2008), pp. 196–202

    Google Scholar 

  7. A.J. Davison, I.D. Reid, N.D. Molton, O. Stasse, MonoSLAM: real-time single camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007)

    Google Scholar 

  8. J. Civera, A.J. Davison, J.M.M. Montiel, Inverse depth parametrization for monocular SLAM. IEEE Trans. Robot. 24(5), 932–945 (2008)

    Google Scholar 

  9. L. Russo, S. Rosa, B. Bona, M. Matteucci, A ros implementation of the mono-slam algorithm. Int. J. Comput. Sci. Inf. Technol. 6(1), 339–351 (2014)

    Google Scholar 

  10. M. Li, A.I. Mourikis, High-precision, consistent EKF-based visual-inertial odometry. Int. J. Robot. Res. 32(6), 690–711 (2013)

    Google Scholar 

  11. B. Williams, I. Reid, On combining visual SLAM and visual odometry, in 2010 IEEE International Conference on Robotics and Automation (IEEE, 2010), pp. 3494–3500

    Google Scholar 

  12. H. Strasdat, J.M.M. Montiel, A.J. Davison, Visual SLAM: why filter? Image Vis. Comput. 30(2), 65–77 (2012)

    Google Scholar 

  13. J. Engel, T. Schöps, D. Cremers, LSD-SLAM: large-scale direct monocular SLAM, in European Conference on Computer Vision (Springer, 2014), pp. 834–849

    Google Scholar 

  14. J. Engel, V. Koltun, D. Cremers, Direct sparse odometry. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 611–625 (2018)

    Article  Google Scholar 

  15. R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge University Press, Cambridge, 2003)

    Google Scholar 

  16. M.I.A. Lourakis, A.A. Argyros, SBA: a software package for generic sparse bundle adjustment. ACM Trans. Math. Softw. (TOMS) 36(1), 2 (2009)

    Google Scholar 

  17. Rudolph Emil Kalman, A new approach to linear filtering and prediction problems. J. Basic Eng. 82(1), 35–45 (1960)

    Article  MathSciNet  Google Scholar 

  18. J. Civera, A.J. Davison, J.M.M. Montiel, Structure from Motion Using the Extended Kalman Filter, vol. 75 (Springer Science & Business Media, Berlin, 2011)

    Google Scholar 

  19. B. Triggs, P.F. McLauchlan, R.I. Hartley, A.W. Fitzgibbon, Bundle adjustment–a modern synthesis, in International Workshop on Vision Algorithms (Springer, 1999), pp. 298–372

    Google Scholar 

  20. R.C. Bolles, M.A. Fischler, A RANSAC-based approach to model fitting and its application to finding cylinders in range data, in IJCAI, vol. 1981 (1981), pp. 637–643

    Google Scholar 

  21. J. Civera, O.G. Grasa, A.J. Davison, J.M.M. Montiel, 1-Point RANSAC for extended Kalman filtering: application to real-time structure from motion and visual odometry. J. Field Robot. 27(5), 609–631 (2010)

    Google Scholar 

  22. J. Civera, A.J. Davison, J.M.M. Montiel, Inverse depth to depth conversion for monocular SLAM, in Proceedings 2007 IEEE International Conference on Robotics and Automation, April 2007, pp. 2778–2783

    Google Scholar 

  23. E. Rublee, V. Rabaud, K. Konolige, G.R. Bradski, ORB: an efficient alternative to SIFT or SURF, in ICCV, vol. 11 (Citeseer, 2011), p. 2

    Google Scholar 

  24. D.G. Lowe et al., Object recognition from local scale-invariant features, in ICCV, vol. 99 (1999), pp. 1150–1157

    Google Scholar 

  25. B. Ochoa, S. Belongie, Covariance propagation for guided matching, in Proceedings of the Workshop on Statistical Methods in Multi-Image and Video Processing (SMVP), vol. 83 (2006)

    Google Scholar 

  26. N. Koenig, A. Howard, Design and use paradigms for gazebo, an open-source multi-robot simulator, in 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), vol. 3 (IEEE, 2004), pp. 2149–2154

    Google Scholar 

  27. M.J. Brooks, W. Chojnacki, D. Gawley, A. Van Den Hengel, What value covariance information in estimating vision parameters? in Proceedings 8th IEEE International Conference on Computer Vision. ICCV 2001, vol. 1 (IEEE, 2001), pp. 302–308

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Al-Baath University, Homs, Syria

    Yasin Maan Yousif

  2. University of Tishreen, Lattakia, Syria

    Iyad Hatem

Authors
  1. Yasin Maan Yousif
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Iyad Hatem
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yasin Maan Yousif .

Editor information

Editors and Affiliations

  1. College of Computer Science and Information Systems, Prince Sultan University, Riyadh, Saudi Arabia

    Anis Koubaa

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Yousif, Y.M., Hatem, I. (2021). Video Frames Selection Method for 3D Reconstruction Depending on ROS-Based Monocular SLAM. In: Koubaa, A. (eds) Robot Operating System (ROS). Studies in Computational Intelligence, vol 895. Springer, Cham. https://doi.org/10.1007/978-3-030-45956-7_11

Download citation

Publish with us

Policies and ethics

Search

Navigation