Skip to main content
SpringerLink
Log in

Energy-Efficient Autonomous Driving Using Cognitive Driver Behavioral Models and Reinforcement Learning

  • Chapter
  • First Online:
AI-enabled Technologies for Autonomous and Connected Vehicles

  • 985 Accesses

Abstract

Autonomous driving technologies are expected to not only improve mobility and road safety but also bring energy efficiency benefits. In the foreseeable future, autonomous vehicles (AVs) will operate on roads shared with human-driven vehicles. To maintain safety and liveness while simultaneously minimizing energy consumption, the AV planning and decision-making process should account for interactions between the autonomous ego vehicle and surrounding human-driven vehicles. In this chapter, we describe a framework for developing energy-efficient autonomous driving policies on shared roads by exploiting human-driver behavior modeling based on cognitive hierarchy theory and reinforcement learning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Amini MR, Kolmanovsky I, Sun J (2020) Hierarchical MPC for robust eco-cooling of connected and automated vehicles and its application to electric vehicle battery thermal management. IEEE Trans Control Syst Technol 29(1):316–328

    Article  Google Scholar 

  2. Aoki S, Jan LE, Zhao J, Bhat A, Chang CF et al (2021) Multicruise: eco-lane selection strategy with eco-cruise control for connected and automated vehicles. arXiv preprint arXiv:2104.11959

  3. Awal T, Murshed M, Ali M (2015) An efficient cooperative lane-changing algorithm for sensor-and communication-enabled automated vehicles. In: 2015 IEEE intelligent vehicles symposium (IV). IEEE, pp 1328–1333

    Google Scholar 

  4. Chen R, Cassandras CG, Tahmasbi-Sarvestani A, Saigusa S, Mahjoub HN, Al-Nadawi YK (2020) Cooperative time and energy-optimal lane change maneuvers for connected automated vehicles. IEEE Trans Intell Transp Syst

    Google Scholar 

  5. Claussmann L, Carvalho A, Schildbach G (2015) A path planner for autonomous driving on highways using a human mimicry approach with binary decision diagrams. In: 2015 European control conference (ECC). IEEE, pp 2976–2982

    Google Scholar 

  6. Delnevo G, Di Lena P, Mirri S, Prandi C, Salomoni P (2019) On combining big data and machine learning to support eco-driving behaviours. J Big Data 6(1):64

    Article  Google Scholar 

  7. Di Cairano S, Liang W, Kolmanovsky IV, Kuang ML, Phillips AM (2012) Power smoothing energy management and its application to a series hybrid powertrain. IEEE Trans Control Syst Technol 21(6):2091–2103

    Article  Google Scholar 

  8. Doshi N, Hanover D, Hemmati S, Morgan C, Shahbakhti M (2019) Modeling of thermal dynamics of a connected hybrid electric vehicle for integrated HVAC and powertrain optimal operation. In: Dynamic systems and control conference, vol 59155. American Society of Mechanical Engineers, p V002T23A005

    Google Scholar 

  9. Feldkamp L, Abou-Nasr M, Kolmanovsky IV (2009) Recurrent neural network training for energy management of a mild hybrid electric vehicle with an ultra-capacitor. In: 2009 IEEE workshop on computational intelligence in vehicles and vehicular systems. IEEE, pp 29–36

    Google Scholar 

  10. Gamage HD, Lee JB (2017) Reinforcement learning based driving speed control for two vehicle scenario. In: Australasian transport research forum (ATRF), 39th, 2017, Auckland, New Zealand

    Google Scholar 

  11. Grigorescu S, Trasnea B, Cocias T, Macesanu G (2020) A survey of deep learning techniques for autonomous driving. J Field Robot 37(3):362–386

    Article  Google Scholar 

  12. Guanetti J, Kim Y, Borrelli F (2018) Control of connected and automated vehicles: State of the art and future challenges. Annu Rev Control 45:18–40

    Article  MathSciNet  Google Scholar 

  13. Guzzella L, Sciarretta A et al (2007) Vehicle propulsion systems, vol 1. Springer, Berlin

    Google Scholar 

  14. Houshmand A, Cassandras CG, Zhou N, Hashemi N, Li B, Peng H (2020) Combined eco-routing and power-train control of plug-in hybrid electric vehicles in transportation networks. arXiv preprint arXiv:2004.05161

  15. Hu X, Liu T, Qi X, Barth M (2019) Reinforcement learning for hybrid and plug-in hybrid electric vehicle energy management: recent advances and prospects. IEEE Ind Electron Mag 13(3):16–25

    Article  Google Scholar 

  16. Jaakkola T, Singh SP, Jordan MI (1995) Reinforcement learning algorithm for partially observable markov decision problems. In: Advances in neural information processing systems, pp 345–352

    Google Scholar 

  17. Kamal MAS, Taguchi S, Yoshimura T (2015) Efficient vehicle driving on multi-lane roads using model predictive control under a connected vehicle environment. In: 2015 IEEE intelligent vehicles symposium (IV). IEEE, pp 736–741

    Google Scholar 

  18. Karimi S, Vahidi A (2020) Receding horizon motion planning for automated lane change and merge using Monte Carlo tree search and level-k game theory. In: 2020 American control conference (ACC). IEEE, pp 1223–1228

    Google Scholar 

  19. Kiran BR, Sobh I, Talpaert V, Mannion P, Al Sallab AA, Yogamani S, Pérez P (2021) Deep reinforcement learning for autonomous driving: a survey. IEEE Trans Intell Transp Syst

    Google Scholar 

  20. Kumaravel SD, Malikopoulos AA, Ayyagari R (2020) Decentralized cooperative merging of platoons of connected and automated vehicles at highway on-ramps. arXiv preprint arXiv:2002.04826

  21. Kuutti S, Bowden R, Jin Y, Barber P, Fallah S (2020) A survey of deep learning applications to autonomous vehicle control. IEEE Trans Intell Transp Syst 22(2):712–733

    Article  Google Scholar 

  22. Li H, Butts K, Zaseck K, Liao-McPherson D, Kolmanovsky I (2017) Emissions modeling of a light-duty diesel engine for model-based control design using multi-layer perceptron neural networks. Technical report, SAE technical paper

    Book  Google Scholar 

  23. Li H, Li N, Kolmanovsky I, Girard A (2020) Energy-efficient autonomous vehicle control using reinforcement learning and interactive traffic simulations. In: 2020 American control conference (ACC). IEEE, pp 3029–3034

    Google Scholar 

  24. Li N (2021) Game-theoretic and set-based methods for safe autonomous vehicles on shared roads. Ph.D. thesis, University of Michigan

    Google Scholar 

  25. Li N, Han K, Kolmanovsky I, Girard A (2021) Coordinated receding-horizon control of battery electric vehicle speed and gearshift using relaxed mixed-integer nonlinear programming. In: IEEE transactions on control systems technology. https://doi.org/10.1109/TCST.2021.3111538

  26. Li N, Oyler DW, Zhang M, Yildiz Y, Kolmanovsky I, Girard AR (2017) Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems. IEEE Trans Control Syst Technol 26(5):1782–1797

    Article  Google Scholar 

  27. Li N, Yao Y, Kolmanovsky I, Atkins E, Girard AR (2020) Game-theoretic modeling of multi-vehicle interactions at uncontrolled intersections. IEEE Trans Intell Transp Syst

    Google Scholar 

  28. Li N, Zhang M, Yildiz Y, Kolmanovsky I, Girard A (2019) Game theory-based traffic modeling for calibration of automated driving algorithms. In: Control strategies for advanced driver assistance systems and autonomous driving functions. Springer, pp 89–106

    Google Scholar 

  29. Liu C, Murphey YL (2014) Power management for plug-in hybrid electric vehicles using reinforcement learning with trip information. In: 2014 IEEE transportation electrification conference and expo (ITEC). IEEE, pp 1–6

    Google Scholar 

  30. Mahbub AI, Malikopoulos AA, Zhao L (2020) Decentralized optimal coordination of connected and automated vehicles for multiple traffic scenarios. Automatica 117:108958

    Article  MathSciNet  Google Scholar 

  31. McDonough K, Kolmanovsky I, Filev D, Szwabowski S, Yanakiev D, Michelini J (2014) Stochastic fuel efficient optimal control of vehicle speed. In: Optimization and optimal control in automotive systems. Springer, pp 147–162

    Google Scholar 

  32. Meng X, Cassandras CG (2020) Eco-driving of autonomous vehicles for nonstop crossing of signalized intersections. IEEE Trans Autom Sci Eng

    Google Scholar 

  33. Mohan G, Assadian, F., Longo, S.: Comparative analysis of forward-facing models vs backwardfacing models in powertrain component sizing. In: IET hybrid and electric vehicles conference 2013 (HEVC 2013). IET, pp 1–6

    Google Scholar 

  34. Murphey YL, Park J, Chen Z, Kuang ML, Masrur MA, Phillips AM (2012) Intelligent hybrid vehicle power control-Part I: machine learning of optimal vehicle power. IEEE Trans Veh Technol 61(8):3519–3530

    Article  Google Scholar 

  35. Qiu S, Qiu L, Qian L, Abdollahi Z, Pisu P (2018) Closed-loop hierarchical control strategies for connected and autonomous hybrid electric vehicles with random errors. IET Intel Transport Syst 12(10):1378–1385

    Article  Google Scholar 

  36. Rajendran AV, Hegde B, Ahmed Q, Rizzoni G (2017) Design and development of traffic-in-loop powertrain simulation. In: 2017 IEEE conference on control technology and applications (CCTA). IEEE, pp 261–266

    Google Scholar 

  37. Schildbach G, Borrelli F (2015) Scenario model predictive control for lane change assistance on highways. In: 2015 IEEE intelligent vehicles symposium (IV). IEEE, pp 611–616

    Google Scholar 

  38. Sciarretta A, Vahidi A et al (2020) Energy-efficient driving of road vehicles. Springer, Berlin

    Book  Google Scholar 

  39. Tian R, Li N, Kolmanovsky I, Yildiz Y, Girard AR (2020) Game-theoretic modeling of traffic in unsignalized intersection network for autonomous vehicle control verification and validation. IEEE Trans Intell Transp Syst

    Google Scholar 

  40. U.S. Department of Energy: next-generation energy technologies for connected and automated on-road vehicles. https://arpa-e.energy.gov/technologies/programs/nextcar. Accessed: 2021-07-15

  41. Vahidi A, Sciarretta A (2018) Energy saving potentials of connected and automated vehicles. Transp Res Part C Emerg Technol 95:822–843

    Article  Google Scholar 

  42. Wang H, Amini MR, Song Z, Sun J, Kolmanovsky I (2019) Combined energy and comfort optimization of air conditioning system in connected and automated vehicles. In: Dynamic systems and control conference, vol 59148. American Society of Mechanical Engineers, p V001T08A001

    Google Scholar 

  43. Waschl H, Kolmanovsky I, Willems F (2019) Control strategies for advanced driver assistance systems and autonomous driving functions. Springer, Berlin

    Book  Google Scholar 

  44. Wipke KB, Cuddy MR, Burch SD (1999) ADVISOR 2.1: a user-friendly advanced powertrain simulation using a combined backward/forward approach. IEEE Trans Veh Technol 48(6):1751–1761

    Google Scholar 

  45. Wu G, Ye F, Hao P, Esaid D, Boriboonsomsin K, Barth MJ et al (2019) Deep learning-based eco-driving system for battery electric vehicles. Technical report, Institute of Transportation Studies, UC Davis

    Google Scholar 

  46. Xu B, Hu X, Tang X, Lin X, Li H, Rathod D, Filipi Z (2020) Ensemble reinforcement learning-based supervisory control of hybrid electric vehicle for fuel economy improvement. IEEE Trans Transp Electrification 6(2):717–727

    Article  Google Scholar 

  47. Yildiz Y, Agogino A, Brat G (2014) Predicting pilot behavior in medium-scale scenarios using game theory and reinforcement learning. J Guid Control Dyn 37(4):1335–1343

    Article  Google Scholar 

  48. Yoo JH, Langari R (2013) Stackelberg game based model of highway driving. In: ASME 2012 5th annual dynamic systems and control conference joint with the JSME 2012 11th motion and vibration conference. American Society of Mechanical Engineers Digital Collection, pp 499–508

    Google Scholar 

  49. Yoo JH, Langari R (2014) A stackelberg game theoretic driver model for merging. In: ASME 2013 dynamic systems and control conference. American society of mechanical engineers digital collection

    Google Scholar 

  50. Zheng G, Peng Z (2021) Life cycle assessment (LCA) of BEV’s environmental benefits for meeting the challenge of ICExit (internal combustion engine exit). Energy Rep 7:1203–1216

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by Mcity, University of Michigan. This research was also supported in part through computational resources and services provided by Advanced Research Computing at the University of Michigan, Ann Arbor.

Author information

Authors and Affiliations

  1. Department of Aerospace Engineering, University of Michigan, Ann Arbor, MI, USA

    Huayi Li, Nan Li, Ilya Kolmanovsky & Anouck Girard

Authors
  1. Huayi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ilya Kolmanovsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anouck Girard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huayi Li .

Editor information

Editors and Affiliations

  1. College of Engineering and Computer Science, University of Michigan–Dearborn, Dearborn, MI, USA

    Yi Lu Murphey

  2. Aerospace Engineering, University of Michigan, Ann Arbor, MI, USA

    Ilya Kolmanovsky

  3. College of Engineering and Computer Science, University of Michigan–Dearborn, Dearborn, MI, USA

    Paul Watta

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Li, H., Li, N., Kolmanovsky, I., Girard, A. (2023). Energy-Efficient Autonomous Driving Using Cognitive Driver Behavioral Models and Reinforcement Learning. In: Murphey, Y.L., Kolmanovsky, I., Watta, P. (eds) AI-enabled Technologies for Autonomous and Connected Vehicles. Lecture Notes in Intelligent Transportation and Infrastructure. Springer, Cham. https://doi.org/10.1007/978-3-031-06780-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06780-8_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06779-2

  • Online ISBN: 978-3-031-06780-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation