Vision-Guided Autonomous Forklift

  • Mohammad M. Aref
  • Reza Ghabcheloo
  • Antti Kolu
  • Jouni Mattila
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 540)


This paper tackles the problem of integrating Visual Servoing Control (VSC) into the functionalities of an Articulated-Frame-Steering (AFS) hydraulic forklift. The controller is capable of breaking down high-level messages into piecewise commands for the different software modules of the vehicle. It also preserves seamless cooperation of the modules for a successful pallet-picking mission. The proposed architecture has been verified on a real machine. Videos of the test runs are available on YouTube.


Mobile manipulation Visual servoing State machine 



This work is supported by the Academy of Finland under the project “Integrated Multimodal Sensing of 3D Environment for Intelligent Manipulators,” grant no. 286260.


  1. 1.
    Wurman, P.R., D’andrea, R., Barbehenn, M.T., Hoffman, A.E., Mountz, M.C.: Kiva transporting inventory items#3, uS Patent Ap. 14/095,751.
  2. 2.
    Aref, M.M., Ghabcheloo, R., Mattila, J.: A macro-micro controller for pallet picking by an articulated-frame-steering hydraulic mobile machine. In: IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, pp. 6816–6822 (2014)Google Scholar
  3. 3.
    Aref, M.M., Ghabcheloo, R., Mattila, J.: Real-time vision-based navigation for nonholonomic mobile robots. In: Accepted for IEEE International Conference on Automation Science and Engineering (CASE) (2016)Google Scholar
  4. 4.
    Aref, M.M., Ghabcheloo, R., Kolu, A., Mattila, J.: A multistage controller with smooth switching for autonomous pallet picking. In: IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden (2016)Google Scholar
  5. 5.
    Yoder, J.D., West, J., Baumgartner, E., Perrollaz, M., Seelinger, M., Robinson, M.: Experiments comparing precision of stereo-vision approaches for control of an industrial manipulator. In: Desai, J.P., Dudek, G., Khatib, O., Kumar, V. (eds.) Experimental Robotics, pp. 245–256. Springer International Publishing, Switzerland (2013)CrossRefGoogle Scholar
  6. 6.
    Holeva, L.F., Elston, E.R., Seelinger, M.J., Yoder, J.D.S.: Identifying and selecting objects that may correspond to pallets in an image scene, uS Patent 9,025,886, 5 May 2015Google Scholar
  7. 7.
    Ban, K., Warashina, F., Yamada, M., Namiki, Y.: Robot system comprising visual sensor, uS Patent 8,326,460, 4 December 2012Google Scholar
  8. 8.
    Byun, S., Kim, M.: Real-time positioning and orienting of pallets based on monocular vision. In: 20th IEEE International Conference on Tools with Artificial Intelligence, 2008, ICTAI 2008, vol. 2, pp. 505–508. IEEE (2008)Google Scholar
  9. 9.
    Baglivo, L., Biasi, N., Biral, F., Bellomo, N., Bertolazzi, E., Da Lio, M., De Cecco, M.: Autonomous pallet localization and picking for industrial forklifts: a robust range and look method. Meas. Sci. Technol. 22, 085502 (2011)CrossRefGoogle Scholar
  10. 10.
    Ghabcheloo, R., Hyvönen, M.: Modeling and motion control of an articulated-frame-steering hydraulic mobile machine. In: Proceedings of the 17th Mediterranean Conference on Control and Automation, Greece, June 2009Google Scholar
  11. 11.
    Kolu, A., Lauri, M., Hyvonen, M., Ghabcheloo, R., Huhtala, K.: A mapping method tolerant to calibration and localization errors based on tilting 2D laser scanner. In: European Control Conference (ECC), pp. 2348–2353. IEEE (2015)Google Scholar
  12. 12.
    Choi, J.W., Huhtala, K.: Constrained global path optimization for articulated steering vehicles. IEEE Trans. Veh. Technol. 65(4), 1868–1879 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Mohammad M. Aref
    • 1
  • Reza Ghabcheloo
    • 1
  • Antti Kolu
    • 1
  • Jouni Mattila
    • 1
  1. 1.Department of Intelligent Hydraulics and AutomationTampere University of TechnologyTampereFinland

Personalised recommendations