Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

A computer vision assisted system for autonomous forklift vehicles in real factory environment

  • 639 Accesses

  • 2 Citations


Industry 4.0 is an important trend in factory automation nowadays. Among the Automated-Storage-and-Retrieval-System (ASRS) is one of the most important issues for industry. It is widely used in a variety of industries for a variety of storage applications in factories and warehouses. However, the cost of constructing an ASRS is so high that most small/medium enterprises cannot afford it. A forklift system is a cheaper alternative to a complicated ASRS. In this work, a new pallet detection method that uses an Adaptive Structure Feature (ASF) and Direction Weighted Overlapping (DWO) ratio to allow forklifts to pick up a pallet is proposed, using a monocular vision system on the forklift. Combining the ASF and DWO ratio for pallet detection, the proposed method removes most of the non-stationary (dynamic) background and significantly increases the processing efficiency. A Haar like-based Adaboost scheme uses an AS for pallets algorithm to detect pallets. It detects the pallet in a dark environment. Finally, by calculating the DWO ratio between the detected pallets and tracking records, it avoids erroneous candidates during object tracking. Therefore, this work improves the pallet detection to solve the problem with an effective design. As results show that the hybrid algorithms that are proposed in this work increase the average pallet detection rate by 95 %.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15


  1. 1.

    Byun S, Kim M (2008) Real-time positioning and orienting of pallets based on monocular vision. IEEE International Conference on Tools with Artificial Intelligence, pp. 505–508

  2. 2.

    Chen C-Y, Hsia C-H, Yang C-Y (2016) Evolutionary design of multiplierless 2D discrete wavelet transform using symmetric mask-based for low-resolution image applications. Multimed Tools and Appl 75(16):9949–9972

  3. 3.

    Chen G, Peng R, Wang Z, Zhao W (2012) Pallet recognition and localization method for vision guided forklift. International Conference on Wireless Communications, Networking and Mobile Computing, pp. 1–4

  4. 4.

    Cucchiara R, Piccardi M, Prati A (2000) Focus based feature extraction for pallets recognition. British Machine Vision Conf., pp. 1–10

  5. 5.

    Cui G, Lu L, He Z, Yao L, Yang C, Huang B, Hu Z (2010) A robust autonomous mobile forklift pallet recognition. International Asia Conference on Informatics in Control, Automation and Robotics, pp. 286–290

  6. 6.

    Fogel M, Burkhart N, Ren H, Schiff J, Meng M, Goldberg K (2007) Automated tracking of pallets in warehouses: beacon layout and asymmetric ultrasound observation models. IEEE International Conference on Automation Science and Engineering, pp. 678–685

  7. 7.

    Garibotto G, Masciangelo S, Ilic M, Bassino P (1996) ROBOLIFT: a vision guided autonomous fork-lift for pallet handling. IEEE/ RSJ Int. Conf. on Robots and Systems, pp. 656–663

  8. 8.

    He Z, Wang X, Liu J, Sun J, Cui G (2010) Feature-to-feature based laser scan matching for pallet recognition. International Conference on Measuring Technology and Mechatronics Automation, pp. 260–263

  9. 9.

    Horng YR, Tseng YC, Chang TS (2010) Stereoscopic images generation with directional Gaussian filter. IEEE International Symposium on Circuits and Systems, pp. 2650–2653

  10. 10.

    Hsia C-H, Chiang J-S, Li H-T, Lin C-S, Chou K-Y (2016) A 3D endoscopic imaging system with content-adaptive filtering and hierarchical similarity analysis. IEEE Sensors J 16(11):4521–4530

  11. 11.

    Hsia C-H, Guo J-M (2014) Efficient modified directional lifting-based discrete wavelet transform for moving object detection. Signal Processing 96, part B, pp. 138–152

  12. 12.

    Hsia C-H, Guo J-M, Chiang J-S (2009) Improved low-complexity algorithm for 2-D integer lifting-based discrete wavelet transform using symmetric mask-based scheme. IEEE Trans Circ Syst Video Technol 19(8):1202–1208

  13. 13.

    Jeon S, Choi M, Kim G, Hong B (2010) Localization of pallets based on passive RFID tags. International Conference on Information Technology: New Generations, pp. 834–839

  14. 14.

    Kalal Z, Mikolajczyk K, Matas J (2012) Tracking-learning-detection. IEEE Trans Pattern Anal Mach Intell 34(7):1409–1422

  15. 15.

    Lecking D, Wulf O, Wagner B (2006) Variable pallet pick-up for automatic guided vehicles in industrial environments. IEEE Conference on Emerging Technologies and Factory Automation, pp. 1169–1174

  16. 16.

    Li J, Li X, Yang B, Sun X (2015) Segmentation-based image copy-move forgery detection scheme. IEEE Trans Inf Forensics Secur 10(3):507–518

  17. 17.

    Lienhart R, Maydt J (2002) An extended set of Haar-like features for rapid object detection. Int Conf Image Process 1:I-900–I-903

  18. 18.

    Liu X, Tao D, Song M, Zhang L, Bu J, Chen C (2015) Learning to track multiple targets. IEEE Trans Neural Netw Learn Syst 26(5):1060–1073

  19. 19.

    Pagès J, Armangué X, Salvi J, Freixenet J, Martí J (2001) A computer vision system for autonomous forklift vehicles in industrial environments. Mediterranean Conf. on Control and Automation, pp. 379–384

  20. 20.

    Pan Z, Lei J, Zhang Y, Sun X, Kwong S (2016) Fast motion estimation based on content property for low-complexity H.265/HEVC encoder. IEEE Trans Broadcast 62(3):675–684

  21. 21.

    Seelinger M, Yoder JD (2005) Automatic pallet engagement by a vision guided forklift. IEEE International Conference on Robotics and Automation, pp. 4068–4073

  22. 22.

    Shen J, Tan H, Wang J, Wang J, Lee S (2015) A novel routing protocol providing good transmission reliability in underwater sensor networks. J Internet Technol 16(1):171–178

  23. 23.

    Varga R, Nedevschi S (2014) Vision-based autonomous load handling for automated guided vehicles. In: Intelligent Computer Communication and Processing (ICCP), 2014 I.E. International Conference on, pp. 239–244

  24. 24.

    Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. EEE Comput Soc Conf Comput Vis Pattern Recognit 1:I-511–I-518

  25. 25.

    Xia Z, Wang X, Sun X, Liu Q, Xiong N (2016) Steganalysis of LSB matching using differences between nonadjacent pixels. Multimed Tools Appl 75(4):1947–1962

  26. 26.

    Zhang L, Tam WJ (2005) Stereoscopic image generation based on depth images for 3D TV. IEEE Trans Broadcast 51:191–199

  27. 27.

    Zhao G, Wang S, Wang T, Chen J (2008) HSV color space and face detection based objectionable image detecting. International Conference on Future Generation Communication and Networking Symposia, pp. 107–110

Download references


The authors would like to thank the anonymous reviewers of their paper for the many helpful suggestions. This work was supported by the Ministry of Science and Technology of Taiwan. under grant number MOST 104-2221-E-034-013-MY2.

Author information

Correspondence to Chih-Hsien Hsia.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Syu, J., Li, H., Chiang, J. et al. A computer vision assisted system for autonomous forklift vehicles in real factory environment. Multimed Tools Appl 76, 18387–18407 (2017).

Download citation


  • Industry 4.0
  • Automated storage and retrieval systems
  • Forklift
  • Adaboost
  • Pallet detection