Skip to main content
SpringerLink
Log in

Deep reinforcement learning-based drift parking control of automated vehicles

  • Article
  • Published:
Science China Technological Sciences Aims and scope Submit manuscript

Abstract

Drift parking usually requires precise control of a vehicle by a professional driver, which can reflect the performance of the vehicle under critical conditions. The obstacles to implementing this action include the high coupling between the longitudinal and lateral states, the high precision required for the vehicle initial state when the drift is triggered, and the difficulty in determining the reference state variables in the drift process. A two-segment drift parking control system is proposed in this paper. In the approaching control segment, the vehicle achieves the drift-triggered vehicle speed and pose, which relies on a path-tracking algorithm based on linear time-varying model predictive control. In the drifting control segment, the deep reinforcement learning algorithm twin-delayed deep deterministic policy gradient is creatively introduced to the controller design. It solves the precise vehicle motion control problem under the condition of the rear wheels having locked brakes. Through various simulations, the superiority and robustness to different initial conditions and abrupt changes in the parking space are verified. The effectiveness of the proposed control system is verified by a ground test.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Goh J Y, Gerdes J C. Simultaneous stabilization and tracking of basic automobile drifting trajectories. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV). Gothenburg, Sweden, 2016. 597–602

  2. Nakano H, Kinugawa J, Kosuge K. Control of a four-wheel independently driven electric vehicle with a large sideslip angle. In: Proceedings of IEEE International Conference on Robotics and Biomimetics (ROBIO). Bali, Indonesia, 2014. 265–270

  3. Hindiyeh R, Christian G. A controller framework for autonomous drifting: Design, stability, and experimental validation. In: Proceedings of ASME Dynamic Systems and Control Conference/Bath/ASME Symposium on Fluid Power and Motion Control. Arlington, VA, 2014. 901–908

  4. Zhang F, Gonzales J, Li S E, et al. Drift control for cornering maneuver of autonomous vehicles. Mechatronics, 2018, 54: 167–174

    Article  Google Scholar 

  5. Gonzales J M. Planning and control of drift maneuvers with the Berkeley autonomous race car. Dissertation for the Doctoral Degree. Berkeley: UC Berkeley, 2018

    Google Scholar 

  6. Silva M, Garrote L, Moita F, et al. Autonomous electric vehicle: Steering and path-following control systems. In: Proceedings of 16th IEEE Mediterranean Electrotechnical Conference (MELECON). Hammamet, Tunisia, 2012. 442–445

  7. Kolter J Z, Plagemann C, Jackson D T, et al. A probabilistic approach to mixed open-loop and closed-loop control with application to extreme autonomous driving. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA). Anchorage, AK, 2010. 839–845

  8. Peng B, Sun Q, Li S E, et al. End-to-end autonomous driving through dueling double deep Q-network. Automot Innov, 2021, 4: 328–337

    Article  Google Scholar 

  9. Liang Y, Li Y, Yu Y, et al. Path-following control of autonomous vehicles considering coupling effects and multi-source system uncertainties. Automot Innov, 2021, 4: 284–300

    Article  Google Scholar 

  10. Lin X, Zhou B, Xia Y. Online recursive power management strategy based on the reinforcement learning algorithm with cosine similarity and a forgetting factor. IEEE Trans Ind Electron, 2021, 68: 5013–5023

    Article  Google Scholar 

  11. Martinez-Marin T. A reinforcement learning algorithm for optimal motion of car-like vehicles. In: Proceedings of 7th IEEE International Conference on Intelligent Transportation Systems. Washington, DC, 2004. 47–51

  12. Gómez M, González R, Martínez-Marín T, et al. Optimal motion planning by reinforcement learning in autonomous mobile vehicles. Robotica, 2012, 30: 159–170

    Article  Google Scholar 

  13. Kang D H, Bong J H, Park J, et al. Reinforcement learning strategy for automatic control of real-time obstacle avoidance based on vehicle dynamics. J Korea Robot Soc, 2017, 12: 297–305

    Article  Google Scholar 

  14. Li G, Yang Y, Li S, et al. Decision making of autonomous vehicles in lane change scenarios: Deep reinforcement learning approaches with risk awareness. Transport Res Part C: Emerg Technol, 2022, 134: 113452

    Article  Google Scholar 

  15. An H I, Jung J. Decision-making system for lane change using deep reinforcement learning in connected and automated driving. Electronics, 2019, 8: 543–556

    Article  Google Scholar 

  16. Gheisarnejad M, Khooban M H. An intelligent non-integer PID controller-based deep reinforcement learning: Implementation and experimental results. IEEE Trans Ind Electron, 2021, 68: 3609–3618

    Article  Google Scholar 

  17. Ichikawa I, Ushio T. Application of reinforcement learning to adaptive control of connected vehicles. Nonlinear Theor Its Appl IEICE, 2019, 10: 443–454

    Article  Google Scholar 

  18. Fujimoto S, Hoof H V, Meger D. Addressing function approximation error in actor-critic methods. In: Proceedings of 35th International Conference on Machine Learning (ICML). Stockholm, Sweden, 2018. 2587–2601

  19. Chen L, Qin D, Xu X, et al. A path and velocity planning method for lane changing collision avoidance of intelligent vehicle based on cubic 3-D Bezier curve. Adv Eng Software, 2019, 132: 65–73

    Article  Google Scholar 

  20. Huang Y, Ding H, Zhang Y, et al. A motion planning and tracking framework for autonomous vehicles based on artificial potential field elaborated resistance network approach. IEEE Trans Ind Electron, 2020, 67: 1376–1386

    Article  Google Scholar 

  21. Yu Z, Zhang R, Xiong L, et al. Steering angle control of steer-by-wire systems considering nonlinear characteristic and uncertainty parameters (in Chinese). J Tongji Univ, 2017, 45: 79–86

    Google Scholar 

  22. Milliken W, Milliken D. Race Car Vehicle Dynamics. Warrendale, USA: SAE International, 1994. 57–58

    Google Scholar 

  23. Leng B, Jin D, Xiong L, et al. Estimation of tire-road peak adhesion coefficient for intelligent electric vehicles based on camera and tire dynamics information fusion. Mech Syst Signal Processing, 2021, 150: 107275

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Automotive Studies, Tongji University, Shanghai, 201804, China

    Bo Leng, YiZe Yu, Ming Liu, Lei Cao, Xing Yang & Lu Xiong

Authors
  1. Bo Leng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. YiZe Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ming Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lei Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xing Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lu Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lu Xiong.

Additional information

This work was supported by the National Key R&D Program of China (Grant No. 2021YFB2501201), the National Natural Science Foundation of China (Grant No. 52002284), and the Young Elite Scientists Sponsorship Program by China Association for Science and Technology (Grant No. 2021QNRC001).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Leng, B., Yu, Y., Liu, M. et al. Deep reinforcement learning-based drift parking control of automated vehicles. Sci. China Technol. Sci. 66, 1152–1165 (2023). https://doi.org/10.1007/s11431-022-2273-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11431-022-2273-5

Keywords

Search

Navigation