Robust visual line-following navigation system for humanoid robots

  • Li-Hong JuangEmail author
  • Jian-Sen Zhang


This paper implements a novel line-following system for humanoid robots. Camera embedded on the robot’s head captures the image and then extracts the line using a high-speed and high-accuracy rectangular search method. This method divides the search location into three sides of rectangle and performs image convolution by edge detection matrix. The extracted line is used to calculate relative parameters, including forward velocity, lateral velocity and angular velocity that drive line-following walking. A proposed path curvature estimation method generates the forward velocity and guidance reference point of the robot. A classical PID controller and a PID controller with angle compensation are then used to set the lateral velocity and angular velocity of the robot, improving the performance in tracking a curved line. Line-following experiments for various shapes were conducted using humanoid robot NAO. Experimental results demonstrate the robot can follow different line shapes with the tracking error remaining at a low level. This is a significant improvement from existing biped robot visual navigation systems.


Line-following Humanoid robot Computer vision Rectangle search Angle compensation PID controller Visual navigation 



The authors deeply acknowledge the financial support from Xiamen University of Technology, Fujian, P.R. China under the Xiamen University of Technology Scientific Research Foundation for Talents plan.

Authors’ contribution

The contributions of this paper are: (1) a line-following method that uses the Gaussian filter and a simple convolution kernel to extract line information and improves the processing speed by using rectangle search; (2) A curvature approximation method to generate forward velocity; and (3) a PID controller with angle compensation to adapt to different curves. When applied collectively, the humanoid robot NAO’s navigation performance was comparatively robust and accurate.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Aldana-Murillo NG, Hayet JB, Becerra HM (2015) Evaluation of local descriptors for vision-based localization of humanoid robots. In: Mexican conference on pattern recognition. Springer International Publishing, pp 179–189Google Scholar
  2. Alexiadis DS, Zarpalas D (2013) Real-time, full 3-D reconstruction of moving foreground objects from multiple consumer depth cameras. IEEE Trans Multimed 15:339–358CrossRefGoogle Scholar
  3. Antonelli G, Chiaverini S, Fusco G (2007) A fuzzy-logic based approach for mobile robot path tracking. IEEE Trans Fuzzy Syst 15(2):211–221CrossRefGoogle Scholar
  4. Ariffin IM et al. (2015) Vision tracking application for mobile navigation using Humanoid robot Nao. In: International symposium on micro-nanomechatronics and human science IEEE, pp 1–7Google Scholar
  5. Armesto L, Tornero J (2009) Automation of industrial vehicles: a visionbased line tracking application. In: Proceedings of IEEE conference on emerging technologies & factory automation, pp 1–7Google Scholar
  6. Balaji V, Balaji M, Chandrasekaran M et al. (2015) Optimization of PID control for high speed line tracking robots. In: IEEE international symposium on robotics and intelligent sensors. IEEE, pp 147–154Google Scholar
  7. Bernabe A, De San Dios MD, Ollero A (2017) Efficient integration of RSSI for tracking using wireless camera networks. Inf Fusion 36:296–312CrossRefGoogle Scholar
  8. Brandão AS, Martins FN, Soneguetti HB (2015) A vision-based line following strategy for an autonomous UAV. In: International conference on informatics in control, automation and robotics IEEE, pp 314–319Google Scholar
  9. Tessier C et al (2010) Map aided localization and vehicle guidance using an active landmark search. Inf Fusion 11(3):283–296CrossRefGoogle Scholar
  10. Delfin JH, Becerra M, Arechavaleta G (2014) Visual path following using a sequence of target images and smooth robot velocities for humanoid navigation. In: IEEE-RAS international conference on humanoid robots IEEE, pp 354–359Google Scholar
  11. Delgado-Galvan J et al. (2015) Vision-based humanoid robot navigation in a featureless environment. In: Mexican Conference on pattern recognition. Springer, Cham, pp 169–178Google Scholar
  12. Du X, Tan KK, Htet KKK (2015) Vision-based lane line detection for autonomous vehicle navigation and guidance. In: Control conference IEEE, pp 1–5Google Scholar
  13. Faragasso A et al. (2016) Vision-based corridor navigation for humanoid robots. In: IEEE international conference on robotics and automation. IEEE, pp 3190–3195Google Scholar
  14. George L, Mazel A (2013) Humanoid robot indoor navigation based on 2D bar codes: application to the NAO robot. In: IEEE-RAS international conference on humanoid robots IEEE, pp 329–335Google Scholar
  15. Głowicki M, Butkiewicz BS (2013) Autonomous line-follower with fuzzy control. In: Signal processing symposium IEEE, pp 1–6Google Scholar
  16. Hornung A, Bennewitz M, Strasdat H (2010) Efficient vision-based navigation. Auton Robots 29(2):137–149CrossRefGoogle Scholar
  17. Bazylev D, Popchenko F, Ibraev D, et al. (2017) Humanoid robot walking on track using computer vision. In: Control and automation, IEEE, pp 1310–1315Google Scholar
  18. Ismail AH et al. (2009) Vision-based system for line following mobile robot. In: IEEE symposium on industrial electronics & applications. ISIEA 2009, pp 642–645Google Scholar
  19. Lobos-Tsunekawa K, Leiva F, Ruiz-Del-Solar J (2018) Visual navigation for biped humanoid robots using deep reinforcement learning. IEEE Robot Autom Lett 3:3247–3254CrossRefGoogle Scholar
  20. Luo B et al. (2015) Research on mobile robot path tracking based on color vision. In: Chinese automation congress IEEE, pp 371–375Google Scholar
  21. Martinez S, Cortes J, Bullo F (2007) Motion coordination with distributed information. IEEE Control Syst 27(4):75–88CrossRefGoogle Scholar
  22. Morrison JG, Gavez-Lopez D, Sibley G (2016) Scalable multirobot localization and mapping with relative maps: introducing MOARSLAM. IEEE Control Syst 36(2):75–85MathSciNetCrossRefGoogle Scholar
  23. Ng KH, Che FY, Su ELM et al (2012) Adaptive Phototransistor Sensor for Line Finding. Procedia Engineering 41(41):237–243CrossRefGoogle Scholar
  24. Okarma K, Lech P (2010) A fast image analysis technique for the line tracking robots. In: Artifical intelligence and soft computing. Springer Berlin Heidelberg, pp 329–336Google Scholar
  25. Oriolo G, Ulivi G, Vendittelli M (1995) On-line map building and navigation for autonomous mobile. In: Proceedings - IEEE international conference on robotics and automation, vol 3. pp 2900–2906Google Scholar
  26. Oriolo G et al. (2013) Vision-based trajectory control for humanoid navigation. In: IEEE-RAS international conference on humanoid robots IEEE, pp 118–123Google Scholar
  27. Pakdaman M, Mehdi M (2009) Design and implementation of line follower robot. In: Second international conference on computer and electrical engineering, pp 585–590Google Scholar
  28. Rahman M, Rahman MHR, Haque AL, Islam MT (2005) Architecture of the vision system of a line following mobile robot operating is static environment. In: 9th IEEE international multitopic conference, pp 1–8Google Scholar
  29. Reyes LAV, Tanner HG (2015) Flocking, formation control, and path following for a group of mobile robots. IEEE Trans Control Syst Technol 23(4):1268–1282CrossRefGoogle Scholar
  30. Roy A, Noel MM (2016) Design of a high-speed line following robot that smoothly follows tight curves. Comput Electr Eng 56:732–747CrossRefGoogle Scholar
  31. Saitoh T, Tada N, Konishi R (2009) Indoor mobile robot navigation by center following based on monocular vision. In: Computer Vision, In-the Publishers, pp 352–366Google Scholar
  32. Shiao YS, Yang JL, Su DT (2013) Path tracking laws and implementation of a vision-based wheeled mobile robot. Proc Inst Mech Eng Part I J Syst Control Eng 223(6):847–862Google Scholar
  33. Thuy PX, Cuong NT (2016) Vision Based autonomous path/line following of a mobile robot using a hybrid fuzzy pid controller. In: Hnkh Toàn Quốc Lần Thứ 3 Về Điều Khiển & Tự Động HoGoogle Scholar
  34. Wu JJ et al (2014) A real-time method for motion blur detection in visual navigation with a humanoid robot. Acta Autom Sin 40(2):267–276Google Scholar
  35. Xiafu L, Yong C (2009) A design of autonomous tracing in Intelligent vehicle based on infrared photoelectric sensor. In: International conference on information engineering & computer science, pp 1–4Google Scholar
  36. NAO-technical overview Accessed 5 May 2014
  37. Yan Lu, Song D (2017) Visual navigation using heterogeneous landmarks and unsupervised geometric constraints. IEEE Trans Robot 31(3):736–749Google Scholar
  38. Yussof H et al (2015) Sensor based mobile navigation using humanoid robot Nao. Procedia Comput Sci 76:474–479CrossRefGoogle Scholar
  39. Zhang J, Wang N, Wang S (2004) A developed method of tuning PID controllers with fuzzy rules for integrating processes. In: Proceedings of the IEEE American control conference 2004, pp 1109–1114Google Scholar
  40. Zhang L, Zhuan X, Gao X (2010) Design and implementation of a Vision based 4-wheeled line track robot. In: 2010 WASE international conference on information engineering, pp 3–6Google Scholar

Copyright information

© Springer Nature B.V. 2018

Authors and Affiliations

  1. 1.School of Electrical Engineering and AutomationXiamen University of TechnologyXiamenPeople’s Republic of China
  2. 2.Engineering CollegeHuaQiao UniversityFengze District, QuanzhouPeople’s Republic of China

Personalised recommendations