Skip to main content
SpringerLink
Log in

An Effective Proposal to Reliable Forward Velocity Variation of NMPC-Based Visual Path-Following Control

  • Published:
Journal of Control, Automation and Electrical Systems Aims and scope Submit manuscript

Abstract

Vision-based control has become an interesting alternative for increasing the autonomy of mobile robot navigation in many real scenarios. Classic path-following models did not originally predict a metric for reference forward velocity variation, and this becomes an even more pronounced problem when using visual techniques that are very sensitive to parameter calibration, such as curvature. This paper proposes a novel approach to reliable forward velocity variation in NMPC (nonlinear model predictive control)-based visual path-following controllers, directly from the image plane. The main contribution arises as improvements in the image processing stage for the acquisition of practicable reference velocities and a new state capable of capturing the characteristics of the path and calculating, at runtime, an optimal forward velocity capable of safely driving the robot around the visual path. The new set of internal control inputs defined for the NMPC framework allows the application of a computationally efficient technique to handle feasibility through the relaxation of input and state constraints. Simulations and experimental results with the Husky UGV platform navigating on an imperfect visual reference path and with an arbitrary curvature profile demonstrate the correctness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. The Robotics Laboratory (LaR) world of the Federal University of Bahia (UFBA) available at https://github.com/lar-deeufba/lar_gazebo

References

  • Arakeri, M.P., Vijaya Kumar, B.P., Barsaiya, S., Sairam, H.V. (2017). Computer vision based robotic weed control system for precision agriculture. In: 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 1201–1205, https://doi.org/10.1109/ICACCI.2017.8126005.

  • Burke, M. (2012). Path-following control of a velocity constrained tracked vehicle incorporating adaptive slip estimation. In: 2012 IEEE International Conference on Robotics and Automation, pp. 97–102, https://doi.org/10.1109/ICRA.2012.6224684.

  • Castano, M., & Tan, X. (2019). Model predictive control-based path-following for tail-actuated robotic fish. Journal of Dynamic Systems, Measurement, and Control. https://doi.org/10.1115/1.4043152.

    Article  Google Scholar 

  • Castelli, F., Michieletto, S., Ghidoni, S., & Pagello, E. (2017). A machine learning-based visual servoing approach for fast robot control in industrial setting. International Journal of Advanced Robotic Systems14(6).

  • Chang, W. C., Cheng, M. Y., & Tsai, H. J. (2017). Image feature command generation of contour following tasks for SCARA robots employing image-based visual servoing—a PH-spline approach. Robotics and Computer-Integrated Manufacturing, 44(C), 57–66.

    Article  Google Scholar 

  • Chen, Z., & Birchfield, S. T. (2009). Qualitative vision-based path following. IEEE Transactions on Robotics, 25(3), 749–754. https://doi.org/10.1109/TRO.2009.2017140.

    Article  Google Scholar 

  • Cherubini, A., Chaumette, F., Oriolo, G. (2008). An image-based visual servoing scheme for following paths with nonholonomic mobile robots. In: International Conference on Control, Automation, Robotics and Vision, ICARCV 2008, Hanoi, Vietnam, France, pp. 108–113.

  • Coulaud, J., Campion, G., Bastin, G., & De Wan, M. (2006). Stability analysis of a vision-based control design for an autonomous mobile robot. IEEE Transactions on Robotics, 22(5), 1062–1069. https://doi.org/10.1109/TRO.2006.878934.

    Article  Google Scholar 

  • Delfin, J., Becerra, H. M., Arechavaleta, G. (2014). Visual path following using a sequence of target images and smooth robot velocities for humanoid navigation. In: 2014 IEEE-RAS International Conference on Humanoid Robots, pp. 354–359, https://doi.org/10.1109/HUMANOIDS.2014.7041384

  • Dughman, S., & Rossiter, J. (2015). A survey of guaranteeing feasibility and stability in MPC during target changes. IFAC-PapersOnLine,48(8), 813–818. https://doi.org/10.1016/j.ifacol.2015.09.069, 9th IFAC Symposium on Advanced Control of Chemical Processes ADCHEM 2015.

  • Faulwasser, T., & Findeisen, R. (2009). Nonlinear Model Predictive Path-Following Control (pp. 335–343). Berlin Heidelberg, Berlin, Heidelberg: Springer. https://doi.org/10.1007/978-3-642-01094-1_28.

    Book  MATH  Google Scholar 

  • Franco, I. J. P. B., Ribeiro, T. T., & Conceição, A. G. S. (2021). A novel visual lane line detection system for a NMPC-based path following control scheme. Journal of Intelligent and Robotic Systems, 101(1), 12. https://doi.org/10.1007/s10846-020-01278-x.

    Article  Google Scholar 

  • Gorbunov, V., Bobkov, V., Win Htet, N., Ionov, E. (2018). Automated control system of fabrics parameters that uses computer vision. pp. 1728–1730, https://doi.org/10.1109/EIConRus.2018.8317438.

  • Ibarguren, A., Martínez-Otzeta, J. M., & Maurtua, I. (2014). Particle filtering for industrial 6dof visual servoing. Journal of Intelligent and Robotic Systems, 74(3), 689–696.

    Article  Google Scholar 

  • Kanjanawanishkul, K., Hofmeister, M., & Zell, A. (2010). Path following with an optimal forward velocity for a mobile robot. IFAC Proceedings Volumes,43(16), 19–24. https://doi.org/10.3182/20100906-3-IT-2019.00006, 7th IFAC Symposium on Intelligent Autonomous Vehicles.

  • Kim, Z. (2008). Robust lane detection and tracking in challenging scenarios. IEEE Transactions on Intelligent Transportation Systems, 9(1), 16–26. https://doi.org/10.1109/TITS.2007.908582.

    Article  Google Scholar 

  • Kumar, A., Gupta, S., Fouhey, D., Levine, S., & Malik, J. (2018). Visual memory for robust path following. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 31 (pp. 765–774). New York: Curran Associates Inc.

    Google Scholar 

  • Kuo, Y. C., Pai, N. S., & Li, Y. F. (2011). Vision-based vehicle detection for a driver assistance system. Computers and Mathematics with Applications,61(8), 2096–2100. https://doi.org/10.1016/j.camwa.2010.08.081, advances in Nonlinear Dynamics.

  • Mayne, D., Rawlings, J., Rao, C., & Scokaert, P. (2000). Constrained model predictive control: Stability and optimality. Automatica, 36(6), 789–814. https://doi.org/10.1016/S0005-1098(99)00214-9.

    Article  MathSciNet  MATH  Google Scholar 

  • Micaelli, A., & Samson, C. (1994). Trajectory tracking for two-steering-wheels mobile robots. IFAC Proceedings Volumes,27(14), 249–256. https://doi.org/10.1016/S1474-6670(17)47322-8, fourth IFAC Symposium on Robot Control, Capri, Italy, September 19–21, 1994

  • Muñoz-Benavent, P., Solanes, J. E., Gracia, L., & Tornero, J. (2019). Robust auto tool change for industrial robots using visual servoing. International Journal of Systems Science, 50(2), 432–449.

    Article  MathSciNet  Google Scholar 

  • Ribeiro, T. T., & Conceição, A. G. S. (2019). Nonlinear model predictive visual path following control to autonomous mobile robots. Journal of Intelligent and Robotic Systems, 95(2), 731–743. https://doi.org/10.1007/s10846-018-0896-3.

  • Ribeiro, T. T., Costa, A. L., & Conceição, A. G. S. (2015). Distributed constrained nmpc with infeasibility handling applied to formation control of nonholonomic vehicles. Journal of Control, Automation and Electrical Systems, 26(6), 599–608. https://doi.org/10.1007/s40313-015-0208-0.

    Article  Google Scholar 

  • Seon, J., Tamadazte, B., & Andreff, N. (2015). Decoupling path following and velocity profile in vision-guided laser steering. IEEE Transactions on Robotics, 31(2), 280–289. https://doi.org/10.1109/TRO.2015.2400660.

    Article  Google Scholar 

  • Spellucci, P. (1998). An SQP method for general nonlinear programs using only equality constrained subproblems. Mathematical Programming, 82, 413–448.

    MathSciNet  MATH  Google Scholar 

  • Tianqi, L. (2017) A review of lane perception and automobile control based on computer vision. pp. 6–11, https://doi.org/10.1109/ICMCCE.2017.10.

Download references

Acknowledgements

An earlier version of the paper was presented at XXIII Congresso Brasileiro de Automática (CBA 2020). We would like to thank the SEPIN/MCTI and the European Union’s Horizon 2020 Research and Innovation Programme through the Grant Agreement No. 777096, and the Brazilian funding agency (CNPq) Grant Number [311029/2020-5].

Author information

Authors and Affiliations

  1. LaR - Robotics Laboratory, Department of Electrical and Computer Engineering, Federal University of Bahia, Salvador, Bahia, Brazil

    Tiago T. Ribeiro, Iago José P. B. Franco & André Gustavo S. Conceição

Authors
  1. Tiago T. Ribeiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Iago José P. B. Franco
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. André Gustavo S. Conceição
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tiago T. Ribeiro.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (mp4 23569 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ribeiro, T.T., Franco, I.J.P.B. & Conceição, A.G.S. An Effective Proposal to Reliable Forward Velocity Variation of NMPC-Based Visual Path-Following Control. J Control Autom Electr Syst 33, 1376–1388 (2022). https://doi.org/10.1007/s40313-022-00898-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40313-022-00898-y

Keywords

Search

Navigation