Skip to main content
SpringerLink
Log in

Monocular-Based SLAM for Mobile Robots: Filtering-Optimization Hybrid Approach

  • Regular paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

The SLAM or Simultaneous Localization and Mapping still remains one of the most important problems to be fully addressed in the path to building fully autonomous mobile robots. This work aims to contribute to the above objective by presenting a novel hybrid architecture for implementing monocular-based SLAM systems for mobile robots. The key idea is to take advantage in a complementary manner of the inherent properties of the two main methodologies available today for implementing visual-based SLAM systems: the filter-based methods, and the optimization-based methods. The proposed method differs from previous approaches among other aspects because filter-based and optimization-based functionalities run concurrently in separate processes, improving the modularity and robustness of the system due to a degree of redundancy. Also, due to its modularity, the proposed approach can be easily applied to different mobile robot platforms using a monocular camera as its main sensor and different setups of additional sensors. In this sense, experiments with real data obtained from a differential robot and a quadrotor are presented to show that the proposed SLAM system can perform well in out-of-the-lab environments by using low-cost sensors. The experiments also show that the SLAM system exhibits real-time and stable computational performance. Moreover, a ROS-2 C++ open-source implementation of the proposed system is provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Jiang, D., et al.: Manipulator grabbing position detection with information fusion of color image and depth image using deep learning. J. Ambient. Intell. Humaniz. Comput. 12, 10809–10822 (2021)

    Article  Google Scholar 

  2. Parikh, P., Trivedi, R., Dave, J., Joshi, K., Adhyaru, D.: Design and development of a low-cost vision-based 6 dof assistive feeding robot for the aged and specially-abled people. IETE J. Res. 0, 1–29 (2023). https://doi.org/10.1080/03772063.2023.2173665

    Article  Google Scholar 

  3. An, X., Wang, Y.: Smart wearable medical devices for isometric contraction of muscles and joint tracking with gyro sensors for elderly people. J. Ambient. Intell. Humaniz. Comput. 1–12 (2021)

  4. Kumar, A.: Real-time performance comparison of vision-based autonomous landing of quadcopter on a ground moving target. IETE J. Res. 0, 1–18 (2021). https://doi.org/10.1080/03772063.2021.1963332

    Article  Google Scholar 

  5. Rao, D., Gupta, M.: Neuro-fuzzy controller for control and robotics applications. Engineering Applications of Articial Intelligence 7,479–491 (1994). https://www.sciencedirect.com/science/article/pii/0952197694900272

  6. Ding, H.: Motion path planning of soccer training auxiliary robot based on genetic algorithm in fixed-point rotation environment. J. Ambient. Intell. Humaniz. Comput. 11, 6261–6270 (2020)

    Article  Google Scholar 

  7. Tong, C.: Three-dimensional reconstruction of the dribble track of soccer robot based on heterogeneous binocular vision. J. Ambient. Intell. Humaniz. Comput. 11, 6361–6372 (2020)

    Article  Google Scholar 

  8. Hamidi, K.E., Mjahed, M., Kari, A.E., Ayad, H., Gmili, N.E.: Design of hybrid neural controller for nonlinear mimo system based on narma-l2 model. IETE J. Res. 69, 3038–3051 (2023). https://doi.org/10.1080/03772063.2021.1909507

    Article  Google Scholar 

  9. Singh, S., Khosla, A., Kapoor, R.: Visual-thermal fusion-based object tracking via a granular computing backed particle filtering. IETE J. Res. 0, 1–16 (2022). https://doi.org/10.1080/03772063.2022.2030251

    Article  Google Scholar 

  10. Khan, M. F., ul Islam, R., Iqbal, J.: Control strategies for robotic manipulators, 26–33 (2012)

  11. Asha, C.S., Narasimhadhan, A.V.: Visual tracking using kernelized correlation filter with conditional switching to median ow tracker. IETE J. Res. 66, 427–438 (2020). https://doi.org/10.1080/03772063.2018.1492356

    Article  Google Scholar 

  12. Li, X.: Robot target localization and interactive multi-mode motion trajectory tracking based on adaptive iterative learning. J. Ambient. Intell. Humaniz. Comput. 11, 6271–6282 (2020)

    Article  Google Scholar 

  13. Yağ, İ, Altan, A.: Artificial intelligence-based robust hybrid algorithm design and implementation for real-time detection of plant diseases in agricultural environments. Biology 11, 1732 (2022)

    Article  Google Scholar 

  14. Belge, E., Altan, A., Hacıoğlu, R.: Metaheuristic optimization-based path planning and tracking of quadcopter for payload hold-release mission. Electronics 11, 1208 (2022)

    Article  Google Scholar 

  15. Altan, A.: Performance of metaheuristic optimization algorithms based on swarm intelligence in attitude and altitude control of unmanned aerial vehicle for path following. IEEE, 1–6 (2020)

  16. Gupta, A., Fernando, X.: Simultaneous localization and mapping (slam) and data fusion in unmanned aerial vehicles: recent advances and challenges. Drones 6, 85 (2022)

    Article  Google Scholar 

  17. Ullah, I., Su, X., Zhang, X., Choi, D.: Simultaneous localization and mapping based on kalman filter and extended kalman filter. Wirel. Commun. Mob. Comput. 2020, 12 (2020)

    Google Scholar 

  18. Kim, P., Coltin, B., Kim, H.J.: Linear rgb-d slam for planar environments (2018)

  19. Chen, M., Yang, S., Yi, X., Wu, D.: Real-time 3d mapping using a 2d laser scanner and imu-aided visual slam, 297–302 (2017)

  20. Motlagh, H.D.K., Lot, F., Taghirad, H.D., Germi, S.B.: Position estimation for drones based on visual slam and imu in gps-denied environment, 120–124 (2019)

  21. Zhou, H., et al.: Structslam: visual slam with building structure lines. IEEE Trans. Veh. Technol. 64, 1364–1375 (2015)

    Article  Google Scholar 

  22. Bloesch, M., Burri, M., Omari, S., Hutter, M., Siegwart, R.: Iterated extended kalman filter based visual-inertial odometry using direct photometric feedback. The International Journal of Robotics Research 36, 1053–1072 (2017). https://doi.org/10.1177/0278364917728574

    Article  Google Scholar 

  23. Quan, M., Piao, S., Tan, M., Huang, S.-S.: Accurate monocular visual-inertial slam using a map-assisted ekf approach. IEEE Access 7, 34289–34300 (2019)

    Article  Google Scholar 

  24. Holmes, S., Klein, G., Murray, D.W.: A square root unscented kalman filter for visual monoslam, 3710–3716 (2008)

  25. Lee, S.-H.: Real-time camera tracking using a particle filter combined with unscented Kalman filters. J. Electron. Imaging 23, 1–19 (2014). https://doi.org/10.1117/1.JEI.23.1.013029

    Article  MathSciNet  Google Scholar 

  26. Tang, M., Chen, Z., Yin, F.: Robot tracking in slam with masreliez-martin unscented kalman filter. Int. J. Control Autom. Syst. 18, 2315–2325 (2020)

    Article  Google Scholar 

  27. Celik, K., Chung, S.-J., Clausman, M., Somani, A.K.: Monocular vision slam for indoor aerial vehicles, 1566–1573 (2009)

  28. Wen, S., Chen, J., Lv, X., Tong, Y.: Cooperative simultaneous localization and mapping algorithm based on distributed particle filter. Int. J. Adv. Rob. Syst. 16, 1729881418819950 (2019). https://doi.org/10.1177/1729881418819950

    Article  Google Scholar 

  29. Liu, W.: Slam algorithm for multi-robot communication in unknown environment based on particle filter. J. Ambient. Intell. Human Comput. (2021)

  30. Eustice, R.M., Singh, H., Leonard, J.J., Walter, M.R.: Visually mapping the rms titanic: conservative covariance estimates for slam information filters. The International Journal of Robotics Research 25, 1223–1242 (2006). https://doi.org/10.1177/0278364906072512

    Article  Google Scholar 

  31. Zhou, W., Valls MirÓ, J., Dissanayake, G.: Information-efficient 3-d visual slam for unstructured domains. IEEE Trans. Rob. 24, 1078–1087 (2008)

    Article  Google Scholar 

  32. Klein, G., Murray, D.: Parallel tracking and mapping for small ar workspaces, 225–234 (2007)

  33. Mur-Artal, R., Montiel, J.M.M., Tardós, J.D.: Orb-slam: a versatile and accurate monocular slam system. IEEE Trans. Rob. 31, 1147–1163 (2015)

    Article  Google Scholar 

  34. Concha, A., Civera, J.: Dpptam: dense piecewise planar tracking and mapping from a monocular sequence, 5686–5693 (2015)

  35. Lin, X., Wang, F., Guo, L., Zhang, W.: An automatic key-frame selection method for monocular visual odometry of ground vehicle. IEEE Access 7, 70742–70754 (2019)

    Article  Google Scholar 

  36. Zhang, Y., Xu, X., Zhang, N., Lv, Y.: A semantic slam system for catadioptric panoramic cameras in dynamic environments. Sensors 21 (2021). https://www.mdpi.com/1424-8220/21/17/5889

  37. Liao, Z., Wang, W., Qi, X., Zhang, X.: Rgb-d object slam using quadrics for indoor environments. Sensors 20 (2020). https://www.mdpi.com/1424-8220/20/18/5150

  38. Zhao, Y., Vela, P.A.: Good feature selection for least squares pose optimization in vo/vslam, 1183–1189 (2018)

  39. Reif, K., Gunther, S., Yaz, E., Unbehauen, R.: Stochastic stability of the discrete-time extended kalman filter. IEEE Trans. Autom. Control 44, 714–728 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  40. Kluge, S., Reif, K., Brokate, M.: Stochastic stability of the extended kalman filter with intermittent observations. IEEE Trans. Autom. Control 55, 514–518 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  41. Huang, G.P., Mourikis, A.I., Roumeliotis, S.I.: Observability-based rules for designing consistent ekf slam estimators. The International Journal of Robotics Research 29, 502–528 (2010). https://doi.org/10.1177/0278364909353640

    Article  Google Scholar 

  42. Huang, G.P., Mourikis, A.I., Roumeliotis, S.I.: A quadratic-complexity observability-constrained unscented kalman filter for slam. IEEE Trans. Rob. 29, 1226–1243 (2013)

    Article  Google Scholar 

  43. Lee, S.-M., Jung, J., Kim, S., Kim, I.-J., Myung, H.: Dv-slam (dual-sensor-based vector- field slam) and observability analysis. IEEE Trans. Industr. Electron. 62, 1101–1112 (2015)

    Article  Google Scholar 

  44. Poulose, A., Han, D.S.: Hybrid indoor localization using imu sensors and smart-phone camera. Sensors 19 (2019). https://www.mdpi.com/1424-8220/19/23/5084

  45. Strasdat, H., Montiel, J., Davison, A.J. Visual slam: why filter? Image Vis. Comput. 30,65–77 (2012). https://www.sciencedirect.com/science/article/pii/S0262885612000248

  46. Munguia, R., Trujillo, J.-C., Guerra, E., Grau, A.: A hybrid visual-based slam architecture: Local filter-based slam with keyframe-based global mapping. Sensors 22 (2022). https://www.mdpi.com/1424-8220/22/1/210

  47. Macenski, S., Foote, T., Gerkey, B., Lalancette, C., Woodall, W.: Robot operating system 2: design, architecture, and uses in the wild. Science Robotics 7, eabm6074 (2022). https://doi.org/10.1126/scirobotics.abm6074

    Article  Google Scholar 

  48. Agarwal, S., Mierle, K., Others.: Ceres solver (2023). http://ceres-solver.org

  49. Dellaert, F., Kaess, M.: Factor graphs for robot perception. Foundations and Trends in Robotics 6, 1–139 (2017). https://doi.org/10.1561/2300000043

    Article  Google Scholar 

  50. Conte, F., Martinelli, A.: A hybrid filter-based and graph-based approach to slam, 999 (2010)

  51. Ding, Y., Xiong, Z., Xiong, J., Cui, Y., Cao, Z.: Ogi-slam2: a hybrid map slam framework grounded in inertial-based slam. IEEE Trans. Instrum. Meas. 71, 1–14 (2022)

    Google Scholar 

  52. Bouguet, J.: Camera calibration toolbox for matlab (2008). http://www.vision.caltech.edu/bouguetj/calib_doc

  53. Durrant-Whyte, H., Bailey, T.: Simultaneous localization and mapping: part i. IEEE Robotics Automation Magazine 13, 99–110 (2006)

    Article  Google Scholar 

  54. Bailey, T., Durrant-Whyte, H.: Simultaneous localization and mapping (slam): part ii. IEEE Robotics Automation Magazine 13, 108–117 (2006)

    Article  Google Scholar 

  55. Munguia, R., Grau, A.: Monocular slam for visual odometry: a full approach to the delayed inverse-depth feature initialization method. Mathematical Problems in Engineering 2012 (2012)

  56. Urzua, S., Munguía, R., Grau, A.: Vision-based slam system for mavs in gps-denied environments. International Journal of Micro Air Vehicles 9, 283–296 (2017). https://doi.org/10.1177/1756829317705325

    Article  Google Scholar 

  57. Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W. Triggs, B., Zisserman, A., Szeliski, R. (eds.) Bundle adjustment — a modern synthesis. Triggs, B., Zisserman, A., Szeliski, R. (eds.) Vision Algorithms: Theory and Practice, 298–372. Springer Berlin Heidelberg, Berlin, Heidelberg (2000)

  58. Hartley, R., Zisserman, A.: Multiple view geometry in computer vision, 2 edn. Cambridge University Press (2004)

  59. Wu, H.Z., Y. Pnp problem revisited. J. Math Imaging Vis. 24, 131–141 (2006)

  60. Itseez.: Open source computer vision library (2015). https://github.com/itseez/opencv

  61. Source code (2023). https://github.com/rodrigo-munguia/Hybrid_VSLAM

  62. Parrot bebop drone for developers (2023). https://github.com/Parrot-Developers/arsdk_manifests

Download references

Acknowledgements

The first author wants to thank to Roderic Munguia for his contribution to this work as a research assistant.

Funding

This research was conducted with no funding.

Author information

Author notes

  1. Juan-Carlos Trujillo, Guillermo Obregón-Pulido and Carlos I. Aldana contributed equally to this work.

Authors and Affiliations

  1. Department of Computer Science, CUCEI University of Guadalajara, Guadalajara, 44430, Jalisco, Mexico

    Rodrigo Munguia, Juan-Carlos Trujillo & Carlos I. Aldana

  2. Department of Electro-photonics, CUCEI University of Guadalajara, Guadalajara, 44430, Jalisco, Mexico

    Guillermo Obregón-Pulido

Authors
  1. Rodrigo Munguia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan-Carlos Trujillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guillermo Obregón-Pulido
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Carlos I. Aldana
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The contribution of each author is: Conceptualization, R.M. and J.-C.T; methodology, G.O.-P and C.I.A.; software, R.M; validation, J.-C.T, G.O.-P. and C.I.A.; investigation, R.M. and G.O.-P; resources, C.I.A.; writing—original draft preparation, R.M.; writing—review and editing, J.-C.T. and C.I.A.; visualization, G.O.-P; supervision, R.M. and J.-C.T; lab resources, R.M. All authors have commented on previous versions of the manuscripts. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Rodrigo Munguia.

Ethics declarations

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent to publish

Not applicable.

Competing Interests

All the authors declare they have no financial interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Munguia, R., Trujillo, JC., Obregón-Pulido, G. et al. Monocular-Based SLAM for Mobile Robots: Filtering-Optimization Hybrid Approach. J Intell Robot Syst 109, 53 (2023). https://doi.org/10.1007/s10846-023-01981-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-023-01981-5

Keywords

Search

Navigation