Skip to main content
SpringerLink
Log in

Pedestrian Collision Avoidance Using Deep Reinforcement Learning

  • Published:
International Journal of Automotive Technology Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

The use of intelligent systems to prevent accidents and safety enhancement in vehicles is becoming a requirement. Besides, the development of autonomous cars is progressing every day. One of the main challenges in transportation is the high mortality rate of vehicles colliding with pedestrians. This issue becomes severe due to various and abnormal situations. This paper proposes a new intelligent algorithm for pedestrian collision avoidance based on deep reinforcement learning. A deep Q-network (DQN) is designed to discover an optimal driving policy for pedestrian collision avoidance in diverse environments and conditions. The algorithm interacts with the vehicle and the pedestrian agents and uses a specific reward function to train the model. We have used Car Learning to Act (CARLA), an open-source autonomous driving simulator, for training and verifying the model in various conditions. Applying the proposed algorithm to a simulated environment reduces vehicles and pedestrians’ collision by about 64 %, depending on the environment. Our findings offer an early-warning solution to mitigate the risk of a crash of vehicles and pedestrians in the real world.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

S:

state space

A:

action space

P:

probability function

R:

reward function

θ:

network weights

α:

learning rate, dimensionless

γ:

discount factor, dimensionless

LS :

safe distance, m

V:

velocity, m/s

X, Y:

position coordinate, m

θped :

pedestrian crossing angle, degree

O:

overall risk, dimensionless

K:

risk degree, dimensionless

References

  • Anowar, S., Alam, M. D. and Raihan, M. A. (2008). Analysis of accident patterns at selected intersections of an urban arterial. Proc. 21st ICTCT Workshop, Melbourne, Australia.

  • Badue, C., Guidolini, R., Carneiro, R. V., Azevedo, P., Cardoso, V. B., Forechi, A., Jesus, L., Berriel, R., Paixão, T. M., Mutz, F., Veronese, L. D. P., Oliveira-Santos, T. and De Souza, A. F. (2020). Self-driving cars: A survey. Expert Systems with Applications, 165, 113816.

    Article  Google Scholar 

  • Belmonte, F. J., Martin, S., Sancristobal, E., Ruipérez-Valiente, J. A. and Castro, M. (2020). Overview of embedded systems to build reliable and safe ADAS and AD systems. IEEE Intelligent Transportation Systems Magazine (in press), 1–10.

  • Busoniu, L., Babuska, R. and De Schutter, B. (2008). A comprehensive survey of multiagent reinforcement learning. IEEE Trans. Systems, Man, and Cybernetics, Part C (Applications and Reviews) 38, 2, 156–172.

    Article  Google Scholar 

  • Chae, H., Kang, C. M., Kim, B., Kim, J., Chung, C. C. and Choi, J. W. (2017). Autonomous braking system via deep reinforcement learning. IEEE 20th Int. Conf. Intelligent Transportation Systems (ITSC), Yokohama, Japan.

  • De, S., Mukherjee, A. and Ullah, E. (2018). Convergence guarantees for RMSProp and ADAM in non-convex optimization and an empirical comparison to Nesterov acceleration. arXiv: 1807. 06766.

  • Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A. and Koltun, V. (2017). CARLA: An open urban driving simulator. Proc. 1st Annual Conf. Robot Learning, in PMLR, 78, 1–16.

    Google Scholar 

  • Galvani, M. (2019). History and future of driver assistance. IEEE Instrumentation & Measurement Magazine 22, 1, 11–16.

    Article  MathSciNet  Google Scholar 

  • Gelbal, S. Y., Guvenc, B. A. and Guvenc, L. (2020). Collision avoidance of low speed autonomous shuttles with pedestrians. Int. J. Automotive Technology 21, 4, 903–917.

    Article  Google Scholar 

  • Gu, S., Holly, E., Lillicrap, T. and Levine, S. (2017). Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. IEEE Int. Conf. Robotics and Automation (ICRA), Singapore.

  • Hacohen, S., Shoval, S. and Shvalb, N. (2020). The paradox of pedestrian’s risk aversion. Accident Analysis & Prevention, 142, 105518.

    Article  Google Scholar 

  • Hamid, U. Z. A., Zakuan, F. R. A., Zulkepli, K. A., Azmi, M. Z., Zamzuri, H., Rahman, M. A. A. and Zakaria, M. A. (2017). Autonomous emergency braking system with potential field risk assessment for frontal collision mitigation. IEEE Conf. Systems, Process and Control (ICSPC), Karunya Nagar, Coimbatore, India.

  • Hojjati-Emami, K., Dhillon, B. and Jenab, K. (2012). Reliability prediction for the vehicles equipped with advanced driver assistance systems (ADAS) and passive safety systems (PSS). Int. J. Industrial Engineering Computations 3, 5, 731–742.

    Article  Google Scholar 

  • Hussain, Q., Feng, H., Grzebieta, R., Brijs, T. and Olivier, J. (2019). The relationship between impact speed and the probability of pedestrian fatality during a vehicle-pedestrian crash: A systematic review and meta-analysis. Accident Analysis & Prevention, 129, 241–249.

    Article  Google Scholar 

  • Huval, B., Wang, T., Tandon, S., Kiske, J., Song, W., Pazhayampallil, J., Andriluka, M., Rajpurkar, P., Migimatsu, T., Cheng-Yue, R., Mujica, F., Coates, A. and Ng, A. Y. (2015). An empirical evaluation of deep learning on highway driving. arXiv: 1504. 01716.

  • Kanarachos, S., Christopoulos, S. R. G. and Chroneos, A. (2018). Smartphones as an integrated platform for monitoring driver behaviour: The role of sensor fusion and connectivity. Transportation Research Part C: Emerging Technologies, 95, 867–882.

    Article  Google Scholar 

  • Koehler, F. and Risteski, A. (2018). Representational power of ReLU networks and polynomial kernels: Beyond worst-case analysis. arXiv: 1805.11405.

  • Lankarani, K. B., Heydari, S. T., Aghabeigi, M. R., Moafian, G., Hoseinzadeh, A. and Vossoughi, M. (2014). The impact of environmental factors on traffic accidents in Iran. J. Injury and Violence Research 6, 2, 64–71.

    Google Scholar 

  • Lee, D. and Yeo, H. (2016). Real-time rear-end collision-warning system using a multilayer perceptron neural network. IEEE Trans. Intelligent Transportation Systems 17, 11, 3087–3097.

    Article  Google Scholar 

  • Li, D., Ranjitkar, P., Zhao, Y., Yi, H. and Rashidi, S. (2017a). Analyzing pedestrian crash injury severity under different weather conditions. Traffic Injury Prevention 18, 4, 427–430.

    Article  Google Scholar 

  • Li, J., Yao, L., Xu, X., Cheng, B. and Ren, J. (2020). Deep reinforcement learning for pedestrian collision avoidance and human-machine cooperative driving. Information Sciences, 532, 110–124.

    Article  Google Scholar 

  • Li, Z., Yu, Q., Zhao, X., Yu, M., Shi, P. and Yan, C. (2017b). Crashworthiness and lightweight optimization to applied multiple materials and foam-filled front end structure of auto-body. Advances in Mechanical Engineering 9, 8, 1–21.

    Google Scholar 

  • Likmeta, A., Metelli, A. M., Tirinzoni, A., Giol, R., Restelli, M. and Romano, D. (2020). Combining reinforcement learning with rule-based controllers for transparent and general decision-making in autonomous driving. Robotics and Autonomous Systems, 131, 103568.

    Article  Google Scholar 

  • Lin, L., Zhang, D., Luo, P. and Zuo, W. (2020). Human Centric Visual Analysis with Deep Learning. Springer. Singapore.

    Book  Google Scholar 

  • Lucidi, F., Girelli, L., Chirico, A., Alivernini, F., Cozzolino, M., Violani, C. and Mallia, L. (2019). Personality traits and attitudes toward traffic safety predict risky behavior across young, adult, and older drivers. Frontiers in Psychology, 10, 536.

    Article  Google Scholar 

  • Malin, F., Norros, I. and Innamaa, S. (2019). Accident risk of road and weather conditions on different road types. Accident Analysis & Prevention, 122, 181–188.

    Article  Google Scholar 

  • McDonald, A. D., Lee, J. D., Schwarz, C. and Brown, T. L. (2018). A contextual and temporal algorithm for driver drowsiness detection. Accident Analysis & Prevention, 113, 25–37.

    Article  Google Scholar 

  • Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D. and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv: 1312. 5602.

  • Pinchon, N., Cassignol, O., Nicolas, A., Bernardin, F., Leduc, P., Tarel, J.-P., Brémond, R., Bercier, E. and Brunet, J. (2018). All-weather vision for automotive safety: Which spectral band?. Int. Forum on Advanced Microsystems for Automotive Applications. Springer. Cham, Switzerland.

    Google Scholar 

  • Pop, D. O. (2019). Detection of pedestrian actions based on deep learning approach. Studia Universitatis Babeş-Bolyai. Informatica, Babeş-Bolyai University.

  • Powell, W. B. (2021). From reinforcement learning to optimal control: A unified framework for sequential decisions. Handbook of Reinforcement Learning and Control. Springer. Cham, Switzerland.

    Google Scholar 

  • Ramachandran, P., Zoph, B. and Le, Q. V. (2017). Swish: A self-gated activation function. arXiv: 1710. 05941.

  • Roh, C. G., Kim, J. and Im, I. (2020). Analysis of impact of rain conditions on ADAS. Sensors 20, 23, 6720.

    Article  Google Scholar 

  • Rosén, E. and Sander, U. (2009). Pedestrian fatality risk as a function of car impact speed. Accident Analysis & Prevention 41, 3, 536–542.

    Article  Google Scholar 

  • Savchenko, V. V. and Litarovich, V. V. (2020). Classification of tablesemantically binary relevant information for drivers in highly automated vehicles. IOP Conf. Series: Materials Science and Engineering 819, 1, 012042.

    Article  Google Scholar 

  • Schaul, T., Quan, J., Antonoglou, I. and Silver, D. (2015). Prioritized experience replay. arXiv: 1511. 05952.

  • Schrum, K. D., De Albuquerque, F. D. B., Sicking, D. L., Falle, R. K. and Reid, J. D. (2014). Correlation between crash severity and embankment geometry. J. Transportation Safety & Security 6, 4, 321–334.

    Article  Google Scholar 

  • Sewalkar, P. and Seitz, J. (2019). Vehicle-to-pedestrian communication for vulnerable road users: Survey, design considerations, and challenges. Sensors 19, 2, 358.

    Article  Google Scholar 

  • Shaaban, K., Muley, D. and Mohammed, A. (2018). Analysis of illegal pedestrian crossing behavior on a major divided arterial road. Transportation Research Part F: Traffic Psychology and Behaviour, 54, 124–137.

    Article  Google Scholar 

  • Sombolestan, S. M., Rasooli, A. and Khodaygan, S. (2019). Optimal path-planning for mobile robots to find a hidden target in an unknown environment based on machine learning. J. Ambient Intelligence and Humanized Computing 10, 5, 1841–1850.

    Article  Google Scholar 

  • Tabata, T., Konet, H. and Kanuma, T. (2010). Development of Nissan approaching vehicle sound for pedestrians. EVS-25 Shenzhen, China 5, 9, 1–6.

    Google Scholar 

  • Wang, P., Motamedi, S., Canas Bajo, T., Zhou, X., Zhang, T., Whitney, D. and Chan, C. Y. (2019a). Safety Implications of Automated Vehicles Providing External Communication to Pedestrians. Research Report No. UC-ITS-2019-12.

  • Wang, Y., He, H. and Sun, C. (2018). Learning to navigate through complex dynamic environment with modular deep reinforcement learning. IEEE Trans. Games 10, 4, 400–112.

    Article  Google Scholar 

  • Wang, Z., Wan, Q., Qin, Y., Fan, S. and Xiao, Z. (2019b). Intelligent algorithm in a smart wearable device for predicting and alerting in the danger of vehicle collision. J. Ambient Intelligence and Humanized Computing, 11, 3841–3852.

    Article  Google Scholar 

  • Wang, Z., Wan, Q., Qin, Y., Fan, S. and Xiao, Z. (2020). Research on intelligent algorithm for alerting vehicle impact based on multi-agent deep reinforcement learning. J. Ambient Intelligence and Humanized Computing 12, 1, 1337–1347.

    Article  Google Scholar 

  • Yin, S., Chen, H., Wu, Y., Li, Y. and Xu, J. (2018). Introducing composite lattice core sandwich structure as an alternative proposal for engine hood. Composite Structures, 201, 131–140.

    Article  Google Scholar 

  • Zai, A. and Brown, B. (2020). Deep Reinforcement Learning in Action. Manning Publications. Shelter Island, NY, USA.

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Intelligent Mobile Robot Lab (IMRL), Department of Mechatronics Engineering, Faculty of New Sciences and Technologies, University of Tehran, Tehran, 1439957131, Iran

    Alireza Rafiei & Amirhossein Oliaei Fasakhodi

  2. College of Engineering and Science, Victoria University Sydney, Sydney, NSW, 2000, Australia

    Farshid Hajati

Authors
  1. Alireza Rafiei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amirhossein Oliaei Fasakhodi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Farshid Hajati
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Farshid Hajati.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rafiei, A., Fasakhodi, A.O. & Hajati, F. Pedestrian Collision Avoidance Using Deep Reinforcement Learning. Int.J Automot. Technol. 23, 613–622 (2022). https://doi.org/10.1007/s12239-022-0056-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12239-022-0056-4

Key Words

Search

Navigation