Skip to main content
SpringerLink
Log in

Kinematics-based Fault-tolerant Techniques: Lane Prediction for an Autonomous Lane Keeping System

  • Regular Papers
  • Robot and Applications
  • Published:
International Journal of Control, Automation and Systems Aims and scope Submit manuscript

Abstract

In this paper, we propose the use of fault-tolerant techniques for an autonomous lane keeping system (LKS) under sensor failure of the camera vision sensor. When the output of the vision sensor is not available to the LKS due to malfunction and/or environmental conditions, it is necessary for the lateral control system to maintain its stability before the driver takes over control authority. We propose a method for fault-tolerant control using the lateral kinematic vehicle motion model. The kinematic motion model-based lane estimation scheme covers possible camera vision sensor failure that occurs in the presence of unreliable or unavailable data from the vision sensor due to complex shadowing, incomplete lane marks, and lighting changes. The proposed lane estimation method enables the LKS to maintain its performance in the presence of sensor failures. The developed algorithm was validated via computational simulation results with CarSim and MATLAB/Simulink. We also included experimental results with a test vehicle equipped with an AutoBox from dSPACE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. A. Eskandarian, Handbook of Intelligent Vehicles, Springer, 2014.

    Google Scholar 

  2. R. Rajamani, Vehicle Dynamics and Control, Springer Science & Business Media, 2011.

    MATH  Google Scholar 

  3. A. Broggi, A. Zelinsky, Ü. Özgüner, and C. Laugier, “Intelligent vehicles,” in Springer Handbook of Robotics, Springer, pp. 1627–1656, 2016.

    Chapter  Google Scholar 

  4. J. C. McCall and M. M. Trivedi, “Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation,” IEEE Transactions on Intelligent Transportation Systems, vol. 7, no. 1, pp. 20–37, 2006. [click]

    Article  Google Scholar 

  5. M. B. de Paula and C. R. Jung, “Automatic detection and classification of road lane markings using onboard vehicular cameras,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 6, pp. 3160–3169, 2015. [click]

    Article  Google Scholar 

  6. S. Jung, J. Youn, and S. Sull, “Efficient lane detection based on spatiotemporal images,” IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 1, pp. 289–295, 2016. [click]

    Article  Google Scholar 

  7. H. Zhao, O. Kim, J.-S. Won, and D.-J. Kang, “Lane detection and tracking based on annealed particle filter,” International Journal of Control, Automation and Systems, vol. 12, no. 6, pp. 1303–1312, 2014. [click]

    Article  Google Scholar 

  8. S. Sivaraman and M. M. Trivedi, “Integrated lane and vehicle detection, localization, and tracking: a synergistic approach,” IEEE Transactions on Intelligent Transportation Systems, vol. 14, no. 2, pp. 906–917, 2013. [click]

    Article  Google Scholar 

  9. Y. S. Son, W. Kim, S.-H. Lee, and C. C. Chung, “Predictive virtual lane method using relative motions between a vehicle and lanes,” International Journal of Control, Automation and Systems, vol. 13, no. 1, pp. 146–155, 2015. [click]

    Article  Google Scholar 

  10. Y. S. Son, W. Kim, S.-H. Lee, and C. C. Chung, “Robust multirate control scheme with predictive virtual lanes for lane-keeping system of autonomous highway driving,” IEEE Transactions on Vehicular Technology, vol. 64, no. 8, pp. 3378–3391, 2015. [click]

    Article  Google Scholar 

  11. C. Gold, D. Damböck, L. Lorenz, and K. Bengler, ““Take over!” how long does it take to get the driver back into the loop?” Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 57, no. 1, pp. 1938–1942, SAGE Publications Sage CA, Los Angeles, CA, 2013.

    Article  Google Scholar 

  12. Z. Tao and P. Bonnifait, “Tightly coupling GPS with lane markings for autonomous vehicle navigation,” Proc. of IEEE International Conference on Intelligent Transportation Systems, pp. 439–444, 2014. [click]

    Google Scholar 

  13. C. Kang, S.-H. Lee, and C. C. Chung, “On-road vehicle localization with gps under long term failure of a vision sensor,” Proc. of IEEE International Conference on Intelligent Transportation Systems, pp. 1638–1643, 2015. [click]

    Google Scholar 

  14. J. Kong, M. Pfeiffer, G. Schildbach, and F. Borrelli, “Kinematic and dynamic vehicle models for autonomous driving control design,” Proceedings of IEEE Intelligent Vehicles Symposium, pp. 1094–1099, 2015.

    Google Scholar 

  15. P. Polack, F. Altché, B. D’Andréa-Novel, and A. De La Fortelle, “The kinematic bicycle model: a consistent model for planning feasible trajectories for autonomous vehicles?” Proceedings of IEEE Intelligent Vehicles Symposium, 2017.

    Google Scholar 

  16. C. M. Kang, S.-H. Lee, and C. C. Chung, “Comparative evaluation of dynamic and kinematic vehicle models,” Proceedings of IEEE Conference on Decision and Control, pp. 648–653, 2014.

    Google Scholar 

  17. C. M. Kang, S.-H. Lee, and C. C. Chung, “Lane estimation using a vehicle kinematic lateral motion model under clothoidal road constraints,” Proc. of IEEE International Conference on Intelligent Transportation Systems, pp. 1066–1071, 2014.

    Google Scholar 

  18. C. M. Kang, Y. S. Gu, S. J. Jeon, Y. S. Son, W. Kim, S.-H. Lee, and C. C. Chung, “Lateral control system for autonomous lane change system on highways,” SAE International Journal of Passenger Cars-Mechanical Systems, vol. 9, no. 2016-01-1641, pp. 877–884, 2016.

    Google Scholar 

  19. C. M. Kang, J. Lee, S. G. Yi, S. J. Jeon, Y. S. Son, W. Kim, S.-H. Lee, and C. C. Chung, “Lateral control for autonomous lane keeping system on highways,” Proc. of IEEE International Conference on Control, Automation and Systems, pp. 1728–1733, 2015. [click]

    Google Scholar 

  20. S. Matzka and R. Altendorfer, “A comparison of trackto-track fusion algorithms for automotive sensor fusion,” Multisensor Fusion and Integration for Intelligent Systems, Springer, pp. 69–81, 2009.

    Chapter  Google Scholar 

  21. R. Mobus and U. Kolbe, “Multi-target multi-object tracking, sensor fusion of radar and infrared,” Proceedings of IEEE Intelligent Vehicles Symposium, pp. 732–737, 2004.

    Google Scholar 

  22. M. Aeberhard, S. Schlichtharle, N. Kaempchen, and T. Bertram, “Track-to-track fusion with asynchronous sensors using information matrix fusion for surround environment perception,” IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 4, 2012. [click]

    Google Scholar 

  23. S.-H. Lee, “Multirate digital control system design and its application to computer disk drives,” IEEE Transactions on Control Systems Technology, vol. 14, no. 1, pp. 124–133, 2006.

    Article  MathSciNet  Google Scholar 

  24. E. D. Dickmanns and A. Zapp, “A curvature-based scheme for improving road vehicle guidance by computer vision,” Cambridge Symposium_Intelligent Robotics Systems, International Society for Optics and Photonics, pp. 161–168, 1987.

    Google Scholar 

  25. M. Lützeler and E. D. Dickmanns, Road Recognition with Marveye, 1998.

    Google Scholar 

  26. M. Maurer, R. Behringer, S. Furst, F. Thomanek, and E. D. Dickmanns, “A compact vision system for road vehicle guidance,” Proceedings of IEEE International Conference on Pattern Recognition, vol. 3, pp. 313–317, 1996.

    Article  Google Scholar 

  27. F. Paetzold and U. Franke, “Road recognition in urban environment,” Image and Vision Computing, vol. 18, no. 5, pp. 377–387, 2000.

    Article  Google Scholar 

  28. A. B. Hillel, R. Lerner, D. Levi, and G. Raz, “Recent progress in road and lane detection: a survey,” Machine vision and applications, vol. 25, no. 3, pp. 727–745, 2014. [click]

    Article  Google Scholar 

  29. S. Sivaraman and M. M. Trivedi, “Looking at vehicles on the road: a survey of vision-based vehicle detection, tracking, and behavior analysis,” IEEE Transactions on Intelligent Transportation Systems, vol. 14, no. 4, pp. 1773–1795, 2013. [click]

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Hanyang University, Seoul, 04763, Korea

    Chang Mook Kang

  2. Agency for Defense Development, Daejeon, 305-152, Korea

    Chang Mook Kang

  3. Dept. of Electrical Engineering, Hanyang University, Seoul, 04763, Korea

    Seung-Hi Lee

  4. Smart Car Research Center, Chungbuk National University, Cheongju, 28644, Korea

    Seok-Cheol Kee

  5. Division of Electrical and Biomedical Engineering, Hanyang University, Seoul, 04763, Korea

    Chung Choo Chung

Authors
  1. Chang Mook Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seung-Hi Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seok-Cheol Kee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chung Choo Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chung Choo Chung.

Additional information

Recommended by Associate Editor Gon-Woo Kim under the direction of Editor Euntai Kim. This work was partly supported by a Korea Evaluation Institute of Industrial Technology (KEIT) grant funded by the Korea government(MOTIE) (No.10044620, Automatic lane change system for novice drivers) and an Institute for Information & communications Technology Promotion(IITP) grant funded by the government of Korea government(MSIP) (No.R7117-16-0164, Development of wide area driving environment awareness and cooperative driving technology which are based on V2X wireless communication).

Chang Mook Kang received his B.S. degree and Ph. D in electrical engineering in 2012 and 2018 from Hanyang University, Seoul, Korea. In 2018, he joined Agency for Defense Development, Daejeon, Korea, as a Senior Research Engineer. His main research interests include control theory, autonomous driving, machine learning, and system integration of intelligent vehicles. He is a member of the IEEE Intelligent Transportation Systems Society (ITSS), the IEEE Control Systems Society, the SOCIETY OF AUTOMOTIVE ENGINEERS (SAE), the Korean Society of Automotive Engineers (KSAE) and the Institute of Control, Robotics and Systems (ICROS).

Seung-Hi Lee received his B.S. degree in Mechanical Engineering from Korea University, Seoul, and an M.S. degree in Mechanical Engineering from Seoul National University, Seoul, Korea, in 1985 and 1987, respectively, He later received his Ph.D. in Mechanical Engineering and Applied Mechanics from the University of Michigan, Ann Arbor, in 1993. From 1988 to 1989, he was a Research Scientist with the Korea Institute of Science and Technology. After 1994, he was with the Samsung Advanced Institute of Technology, Korea, where he was a team leader responsible for advanced servomechanical systems. In 2009, he joined Hanyang University, Seoul, Korea, as a research professor, where he is also teaching advanced control systems. His research interests include robust sampled-data feedback control of uncertain systems, as well as its application to information storage, automotive, electromechanical, and manufacturing systems. Prof. Lee has served as a Member of the Editorial Board of the International Journal of Control, Automation, and Systems.

Seok-Cheol Kee received the B.S. and M.S. degrees in control and instrumentation engineering and the Ph.D. degree in electrical engineering, all from Seoul National University, Seoul, Korea, in 1987, 1989, and 2002, respectively. From 1989 to 2007, he worked as a Principal Research Staff in the Samsung Advanced Institute of Technology (SAIT). From 2010 to 2015, he worked as the head of Electronic R&D Center in Mando. He joined the faculty of Chungbuk National University in 2015, where he is currently an Associate Professor of smart car research center. His research interests are in computer vision, image processing, sensor fusion, embedded control system for automotive applications.

Chung Choo Chung received his B.S. and M.S. degrees in electrical engineering from Seoul National University, Seoul, South Korea, and his Ph.D. degree in electrical and computer engineering from the University of Southern California, Los Angeles, CA, USA, in 1993. From 1994 to 1997, he was with the Samsung Advanced Institute of Technology, Korea. In 1997, he joined the Faculty of Hanyang University, Seoul, South Korea. He was an Associate Editor for the Asian Journal of Control from 2000 to 2002 and an Editor for the International Journal of Control, Automation and Systems from 2003 to 2005. He served as associate editor for various international conferences, such as the IEEE Conference on Decision and Control (CDC), the American Control Conferences, the IEEE Intelligent Vehicles Symposium, and the Intelligent Transportation Systems Conference. He was also guest editor for a special issue on advanced servo control for emerging data storage systems published by the IEEE Transactions on Control System Technologies (TCST) and a guest editor for Special Section on 2015 IEEE Intelligent Vehicle Symposium in the IEEE Intelligent Transportation Systems Magazine. He is currently an associate editor for TCST, the IEEE Transactions on Intelligent Transportation Systems, and the IFAC Journal Mechatronics. Dr. Chung was an organizing chair for the International Conference on Control, Automation and Systems (ICCAS) 2011 and a program co-chair of ICCAS-SICE 2009, and the 2015 IEEE Intelligent Vehicles Symposium. He is currently a general co-chair of CDC 2020, to be held in Korea. He is President-elect of Institute of Control, Robotics and Systems (ICROS) in 2018.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, C.M., Lee, SH., Kee, SC. et al. Kinematics-based Fault-tolerant Techniques: Lane Prediction for an Autonomous Lane Keeping System. Int. J. Control Autom. Syst. 16, 1293–1302 (2018). https://doi.org/10.1007/s12555-017-0449-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12555-017-0449-8

Keywords

Search

Navigation