Skip to main content
SpringerLink
Log in

New hybrid vision-based control approach for automated guided vehicles

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Automated guided vehicles (AGVs) are a common choice made by many companies for material handling (MH) in manufacturing systems. AGV-based internal transport of raw materials, goods, and parts is becoming improved with advances in technology. Demands for fast, efficient, and reliable transport imply the usage of the flexible AGVs with onboard sensing and special kinds of algorithms needed for daily operations. So far, the majority of these transport solutions have not considered the modern techniques for visual servoing, monocular SLAM, and consequently, the usage of camera as onboard sensor for AGVs. In this research, a new hybrid control of AGV is proposed. The main control algorithm consists of two independent control loops: position-based control (PBC) for global navigation and image based visual seroving (IBVS) for fine motions needed for accurate steering towards loading/unloading point. By separating the initial transportation task into two parts (global navigation towards the goal pose near the loading/unloading point and fine motion from the goal pose to the loading/unloading point), the proposed hybrid control bypasses the need for artificial landmarks or accurate map of the environment. The state estimation of the robot pose is determined in terms of monocular SLAM, via extended Kalman filter coupled with feedforward neural network—the neural extended Kalman filter (NEKF). NEKF is used to model unknown disturbances and to improve the robot state transition model. The integration of the new hybrid control and NEKF has been tested in laboratory with the mobile robot and simple camera. Experimental results present the effectiveness of the proposed hybrid control approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alenya G, Escoda J, Martinez AB, Torras C (2005) Using laser and vision to locate a robot in an industrial environment: a practical experience. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, Barcelona, Spain, pp 3528–3533

  2. Baglivo L, Biasi N, Biral F, Bellomo N, Bertolazzi E, Lio MD, Cecco MD (2011) Autonomous pallet localization and picking for industrial forklifts: a robust range and look method. Meas Sci Technol 22:085502. doi:10.1088/0957-0233/22/8/085502

    Article  Google Scholar 

  3. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) SURF: Speeded Up Robust Features. Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  4. Bonin-Font F, Ortiz A, Oliver G (2008) Visual navigation for mobile robots: a survey. J Intell Robot Syst 53(3):263–296

    Article  Google Scholar 

  5. Borenstein J, Everett HR, Feng L (1996) Where am I? Sensors and methods for mobile robot positioning. University of Michigan, Ann Arbor. http://www-personal.engin.unimch.edu/~johannb/position.htm

  6. Cadena C, Neira J (2010) SLAM in O(log n) with the combined Kalman—information filter. Robot Auton Syst 58(11):1207–1219

    Article  Google Scholar 

  7. Castellanos JA, Martinez-Cantin R, Tardos JD, Neira J (2007) Robocentric map joining: Improving the consistency of EKF-SLAM. Robot Auton Syst 55(1):21–29

    Article  Google Scholar 

  8. Chen J, Dawson DM, Dixon WE, Behal A (2005) Adaptive homography-based visual servo tracking for a fixed camera configuration with a camera-in-hand extension. IEEE Trans Control Syst Technol 13(5):814–825

    Article  Google Scholar 

  9. Choi M, Sakthivel R, Chung WK (2007) Neural network-aided extended Kalman filter for SLAM problem. In: Proceedings of the IEEE International Conference on Robotics and Automation, Roma, Italy, paper 1-4244-0602-1/07, pp. 1686–1690.

  10. Civera J, Davison AJ, Montiel JMM (2008) Inverse depth parametrization for monocular SLAM. IEEE Trans Robot 24(5):932–945

    Article  Google Scholar 

  11. Civera J, Grasa OG, Davison AJ, Montiel JMM (2010) 1-point RANSAC for EKF filtering. Application to real-time structure from motion and visual odometry. J Field Robot 27(5):609–631

    Article  Google Scholar 

  12. Davidson A, Reid I, Molton N, Stasse O (2007) MonoSLAM: real-time single camera SLAM. IEEE Trans Pattern Anal 29(6):1052–1067

    Article  Google Scholar 

  13. DeSouza GN, Kak AC (2002) Vision for mobile robot navigation: a survey. IEEE Trans Pattern Anal 24(2):237–267

    Article  Google Scholar 

  14. Durrant-Whyte HF (1988) Uncertain geometry in robotics. IEEE Trans Robot Autom Mag 4(1):23–31

    Article  Google Scholar 

  15. Dissanayake MWMG, Newman P, Clark S, Durrant-Whyte HF, Csorba M (2001) A solution to the simultaneous localization and map building (SLAM) problem. IEEE Trans Robot Autom Mag 17(3):229–241

    Article  Google Scholar 

  16. Fang Y, Dixon WE, Dawson DM, Chawda P (2005) Homography-based visual servo regulation of mobile robots. IEEE Trans Syst Man Cybern Part B: Cybern 35(5):1041–1050

    Article  Google Scholar 

  17. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    Article  MathSciNet  Google Scholar 

  18. Garibotto G, Masciangelo S, Ilic M, Basino P (1996) Robolift: a vision guided autonomous fork-lift for pallet handling. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Osaka, Japan. pp. 656–663

  19. Garibotto G, Masciangelo S, Bassino P, Coelho C, Pavan A, Marson M, Elsag BG (1998) Industrial exploitation of computer vision in logistic automation: autonomous control of an intelligent forklift truck. In: Proceedings of the IEEE International Conference on Robotics and Automation, Leuven, Belgium, 2:1459–1464

  20. Gaspar J, Winters N, Santos-Victor J (2000) Vision-based navigation and environmental representations with an omnidirectional camera. IEEE Trans Robot Autom Mag 16(6):890–898

    Article  Google Scholar 

  21. Hartley RI, Zisserman A (2004) Multiple view geometry in computer vision (2nd edn). Cambridge University Press, Cambridge

  22. Kang JG, An SY, Oh SY (2010) Modified neural network aided EKF based SLAM for improving an accuracy of the feature map. In: International Joint Conference on Neural Networks (IJCNN), Barcelona, Spain, paper 978-1-4244-6916-1, pp 1–7.

  23. Kelly A, Nagy B, Stager D, Unnikrishnan R (2007) An infrastructure-free automated guided vehicle based on computer vision. IEEE Trans Robot Autom Mag 14(3):24–34

    Article  Google Scholar 

  24. Kim YH, Lee SW, Yang H, Shell D (2012) Toward autonomous robotic containment booms: visual servoing for robust inter-vehicle docking of surface vehicles. Intell Serv Robot 5:1–18. doi:10.1007/s11370-011-0100-0

    Article  Google Scholar 

  25. Konolige K, Agrawal M, Solà J (2007) Large scale visual odometry for rough terrain. In: Proceedings of the International Symposium on Research in Robotics (ISRR), Hiroshima, Japan

  26. Kramer KA, Stubberud SC (2006) Analysis and implementation of a neural extended Kalman filter for target tracking. Int J Neural Syst 16(1):1–13

    Article  Google Scholar 

  27. Kramer KA, Stubberud SC (2010) Tracking of multiple target types with a single neural extended kalman filter. Int J Intell Syst 25:440–459

    MATH  Google Scholar 

  28. Leonard JJ, Durrant-Whyte HF, Cox IJ (1990) Dynamic map building for an autonomous mobile robot. In: IEEE International Workshop on Intelligent Robots and Systems IROS ‘90, pp. 89–96

  29. López-Nicolás G, Sagüés C, Guerrero JJ, Kragic D, Jensfelt P (2008) Switching visual control based on epipoles for mobile robots. Robot Auton Syst 56(7):592–603

    Article  Google Scholar 

  30. López-Nicolás G, Guerrero JJ, Sagüés C (2010) Visual control of vehicles using two-view geometry. Mechatronics 20(2):315–325

    Article  Google Scholar 

  31. Lozoya C, Marti P, Velasco M, Fuertes JM, Martin EX (2011) Simulation study of a remote wireless path tracking control with delay estimation for an autonomous guided vehicle. Int J Adv Manuf Technol 52(5–8):751–761

    Article  Google Scholar 

  32. Ma Y, Kosecka J, Sastry S (1999) Vision guided navigation for a nonholonomic mobile robot. IEEE Trans Robot Autom 15(3):521–536

    Article  Google Scholar 

  33. Mariottini GL, Oriolo G, Prattichizzo D (2007) Image-based visual servoing for nonholonomic mobile robots using epipolar geometry. IEEE Trans Robot 23(1):87–100

    Article  Google Scholar 

  34. Martinez-Barbera H, Herrero-Perez D (2010) Autonomous navigation of an automated guided vehicle in industrial environments. Robot Comput Integr Manuf 26(4):296–311

    Article  Google Scholar 

  35. Martinez-Barbera H, Herrero-Perez D (2010) Development of a flexible AGV for flexible manufacturing systems. Ind Robot 37(5):459–468

    Article  Google Scholar 

  36. Menegatti E, Maeda T, Ishiguro H (2004) Image-based memory for robot navigation using properties of the omnidirectional images. Robot Auton Syst 47(4):251–267

    Article  Google Scholar 

  37. Miljković Z, Aleksendrić D (2009) Artificial neural networks—solved examples with theoretical background (In Serbian). University of Belgrade-Faculty of Mechanical Engineering, Belgrade

    Google Scholar 

  38. Ming X (1995) Trinocular vision for AGV guidance: path localization and obstacle detection. Comput Electr Eng 21(6):441–452

    Article  Google Scholar 

  39. Miyazaki F, Masutani Y (1990) Robustness of sensory feedback control based on imperfect Jacobian. In: Proceedings of the 5th international symposium on Robotics research, pp. 201–208

  40. Mouragnon E, Lhuillier M, Dhome M, Dekeyser F, Sayd P (2009) Generic and real-time structure from motion using local bundle adjustment. Image Vis Comput 27(8):1178–1193

    Article  Google Scholar 

  41. Nemra A, Aouf N (2010) Experimental airborne NH∞ vision-based simultaneous localization and mapping in unknown environments. Proc Inst Mech Eng Part G: J Aerosp Eng 223(8):1253–1270

    Google Scholar 

  42. Nister D, Naroditsky O, Bergen J (2004) Visual odometry. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Washington DC, USA, pp. 652–659

  43. Nygards J, Hogstrom T, Wernersson A (2000) Docking to pallets with feedback from a sheet-of-light range camera. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Takamatsu, Japan, vol 3, pp. 1853–1859

  44. Paz LM, Pinies P, Tardos JD, Neira J (2008) Large scale 6DOF SLAM with stereo-in-hand. IEEE Trans Robot 24(5):946–957

    Article  Google Scholar 

  45. Pradalier C, Tews A, Roberts J (2008) Vision-based operations of a large industrial vehicle—autonomous hot metal carrier. J Field Robot 25(4–5):243–267

    Article  Google Scholar 

  46. Rodriguez FJ, Mazo M, Sotelo MA (1998) Automation of an industrial fork lift truck, guided by artificial vision in open environments. Auton Robot 5:215–231

    Article  Google Scholar 

  47. Ronzoni D, Olmi R, Secchi C, Fantuzzi C (2011) AGV global localization using indistinguishable artificial landmarks. In: Proceedings of IEEE International Conference on Robotics and Automation, Shanghai, China, pp. 287–292

  48. Rosten E, Porter R, Drummond T (2010) FASTER and better: a machine learning approach to corner detection. IEEE Trans Pattern Anal 32(1):105–119

    Article  Google Scholar 

  49. Scaramuzza D, Fraundorfer F (2011) Visual odometry: part I—the first 30 years and fundamentals. IEEE Trans Robot Autom Mag 18(4):80–92

    Article  Google Scholar 

  50. Scaramuzza D, Fraundorfer F, Pollefeys M (2010) Closing the loop in appearance-guided omnidirectional visual odometry by using vocabulary trees. Robot Auton Syst 58(6):820–827

    Article  Google Scholar 

  51. Seelinger M, Yoder JD (2006) Automatic visual guidance of a forklift engaging a pallet. Robot Auton Syst 54:1026–1038

    Article  Google Scholar 

  52. Slotine JJE, Li W (1991) Applied nonlinear control. Prentice Hall, Englewood Cliffs

    MATH  Google Scholar 

  53. Smith R, Cheeseman P (1986) On the representation and estimation of spatial uncertainty. Int J Robot Res 5(4):56–68

    Article  Google Scholar 

  54. Stubberud AR (2006) A validation of the neural extended Kalman filter. In: Proceedings of the Eighteenth International Conference on Systems Engineering, Coventry University, London, pp. 3–8

  55. Stubberud SC, Kramer KA (2008) Analysis of system identification using the neural extended Kalman filter. In: Proceedings of 19th International Conference on System Engineering, Las Vegas, paper 978-0-7695-3331-5/08:153-158.

  56. Sunderhauf N, Lange S, Protzel P (2007) Using the unscented Kalman filter in mono-SLAM with inverse depth parametrization for autonomous airship control. In: IEEE International Workshop on Safety Security and Rescue Robotics, Rome, Italy, paper 978-1-4244-1569-4, pp. 1–6

  57. Tamba TA, Hong H, Hong KS (2009) A path following control of an unmanned autonomous forklift. Int J Control Autom 7(1):113–122

    Article  Google Scholar 

  58. Tardós JD, Neira J, Newman P, Leonard J (2002) Robust mapping and localization in indoor environments using sonar data. Int J Robot Res 21(4):311–330

    Article  Google Scholar 

  59. Tsakiris D, Rives P, Samson C (1998) Extending visual servoing techniques to nonholonomic mobile robots. In: Lecture notes in control and information sciences: the confluence of vision and control. Springer, Berlin, pp 106–117

  60. Thrun S, Montemerlo M (2005) The GraphSLAM algorithm with applications to large-scale mapping of urban structure. Int J Robot Res 25(5/6):403–430

    Google Scholar 

  61. Thrun S, Koller D, Ghahramani Z, Durrant-Whyte H, Ng A (2004) Simultaneous localization and mapping with sparse extended information filters. Int J Robot Res 23(7/8):693–716

    Article  Google Scholar 

  62. Vuković N, Miljković Z (2009) New hybrid control architecture for intelligent mobile robot navigation in a manufacturing environment. FME Trans 37(1):9–18

    Google Scholar 

  63. Walter MR, Eustice RM, Leonard JJ (2007) Exactly sparse extended information filters for feature-based SLAM. Int J Robot Res 26(4):335–359

    Article  Google Scholar 

  64. Wang Y, Lang H, de Silva CW (2010) A hybrid visual servo controller for robust grasping by wheeled mobile robots. IEEE-ASME Trans Mechatron 15(5):757–769

    Article  Google Scholar 

  65. Yahyaei M, Jam JE, Hosnavi R (2010) Controlling the navigation of automatic guided vehicle (AGV) using integrated fuzzy logic controller with programmable logic controller (IFLPLC)—stage 1. Int J Adv Manuf Technol 47(5–8):795–807

    Article  Google Scholar 

  66. Zhan R, Wan J (2006) Neural network-aided adaptive unscented Kalman filter for nonlinear state estimation. IEEE Signal Proc Lett 13(7):445–448

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Production Engineering Department, Faculty of Mechanical Engineering, University of Belgrade, Kraljice Marije 16, 11120, Belgrade 35, Serbia

    Zoran Miljković, Marko Mitić & Bojan Babić

  2. Innovation Center, Faculty of Mechanical Engineering, University of Belgrade, Kraljice Marije 16, 11120, Belgrade 35, Serbia

    Najdan Vuković

Authors
  1. Zoran Miljković
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Najdan Vuković
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marko Mitić
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bojan Babić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zoran Miljković.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Miljković, Z., Vuković, N., Mitić, M. et al. New hybrid vision-based control approach for automated guided vehicles. Int J Adv Manuf Technol 66, 231–249 (2013). https://doi.org/10.1007/s00170-012-4321-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-012-4321-y

Keywords

Search

Navigation