A Generalised Method for Adaptive Longitudinal Control Using Reinforcement Learning

  • Shashank PathakEmail author
  • Suvam Bag
  • Vijay Nadkarni
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 867)


Adaptive cruise control (ACC) seeks intelligent and adaptive methods for longitudinal control of the cars. Since more than a decade, high-end cars have been equipped with ACC typically through carefully designed model-based controllers. Unlike the traditional ACC, we propose a reinforcement learning based approach – RL-ACC. We present the RL-ACC and its experimental results from the automotive-grade car simulators. Thus, we obtain a controller which requires minimal domain knowledge, is intuitive in its design, can accommodate uncertainties, can mimic human-like behaviour and may enable human-trust in the automated system. All these aspects are crucial for a fully autonomous car and we believe reinforcement learning based ACC is a step towards that direction.


  1. 1.
  2. 2.
    Oktal - simulation in motion. Accessed Feb 2018
  3. 3.
    Vtd - virtual test drive. Accessed Feb 2018
  4. 4.
    Bando, M., Hasebe, K., Nakayama, A., Shibata, A., Sugiyama, Y.: Dynamical model of traffic congestion and numerical simulation. Phys. Rev. E 51(2), 1035 (1995)CrossRefGoogle Scholar
  5. 5.
    Boer, E.R.: Car following from the drivers perspective. Transp. Res. Part F: Traffic Psychol. Behav. 2(4), 201–206 (1999)CrossRefGoogle Scholar
  6. 6.
    Brackstone, M., McDonald, M.: Car-following: a historical review. Transp. Res. Part F: Traffic Psychol. Behav. 2(4), 181–196 (1999)CrossRefGoogle Scholar
  7. 7.
    Gazis, D.C., Herman, R., Rothery, R.W.: Nonlinear follow-the-leader models of traffic flow. Oper. Res. 9(4), 545–567 (1961)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Gipps, P.G.: A behavioural car-following model for computer simulation. Transp. Res. Part B: Methodol. 15(2), 105–111 (1981)CrossRefGoogle Scholar
  9. 9.
    Gray, R., Regan, D.: Accuracy of estimating time to collision using binocular and monocular information. Vision Res. 38(4), 499–512 (1998)CrossRefGoogle Scholar
  10. 10.
    Helly, W.: Simulation of bottlenecks in single lane traffic flow, presentation at the symposium on theory of traffic flow. Research laboratories, General Motors, New York, pp. 207–238 (1959)Google Scholar
  11. 11.
    Lagoudakis, M.G., Parr, R.: Least-squares policy iteration. J. Mach. Learn. Res. 4(Dec), 1107–1149 (2003)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Littman, M.L., Dean, T.L., Leslie, P.K.: On the complexity of solving markov decision problems, pp. 394–402. Morgan Kaufmann Publishers Inc. (1995)Google Scholar
  13. 13.
    Michaels, R.M.: Perceptual factors in car following. In: Proceedings of the 2nd International Symposium on the Theory of Road Traffic Flow, London, England, OECD (1963)Google Scholar
  14. 14.
    Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. arXiv preprintarXiv:1312.5602 (2013)
  15. 15.
    Moody, J., Saffell, M.: Learning to trade via direct reinforcement. IEEE Trans. Neural Netw. 12(4), 875–889 (2001)CrossRefGoogle Scholar
  16. 16.
    Nagel, K., Schreckenberg, M.: A cellular automaton model for freeway traffic. Journal de physique I 2(12), 2221–2229 (1992)CrossRefGoogle Scholar
  17. 17.
    Newell, G.F.: A simplified car-following theory: a lower order model. Transp. Res. Part B: Methodol. 36(3), 195–205 (2002)CrossRefGoogle Scholar
  18. 18.
    Peters, J., Vijayakumar, S., Schaal, S.: Reinforcement learning for humanoid robotics. In: Proceedings of the Third IEEE-RAS International Conference on Humanoid Robots, pp. 1–20 (2003)Google Scholar
  19. 19.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1. MIT Press Cambridge, Cambridge (1998)Google Scholar
  20. 20.
    Treiber, M., Hennecke, A., Helbing, D.: Congested traffic states in empirical observations and microscopic simulations. Phys. Rev. E 62(2), 1805 (2000)CrossRefGoogle Scholar
  21. 21.
    Wiedemann, R.: Simulation des straßenverkehrsflusses. schriftenreihe heft 8. Institute for Transportation Science, University of Karlsruhe, Germany (1994)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Visteon Electronics GmbHAn der RaumFabrik 33bKarlsruheGermany
  2. 2.Visteon CorporationSanta ClaraUSA

Personalised recommendations