Pedestrian–Autonomous Vehicles Interaction Challenges: A Survey and a Solution to Pedestrian Intent Identification

  • Pranav PandeyEmail author
  • Jagannath V. Aghav
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 94)


Autonomous Vehicles are on rise around the globe, millions of them are already there on road with medium levels of automation but still there is a long way to go for full autonomy. One of the biggest roadblocks for autonomous vehicles to reach full autonomy is driving in urban environments. To make autonomous vehicles fully autonomous, they require the ability to communicate with other road users (pedestrian, vehicles, and other road users) and understand their intentions. Social interaction is a complex task, there are uncountable scenarios that happen on roads that require human interaction both verbal and nonverbal. Deciding whether a person standing on the sidewalk is about to cross the road, or they are just waiting near the sidewalk is a difficult task for an autonomous vehicle, and it could be a matter of life-and-death in case of a vehicle driving at very high speed. So, it is very important for self-driving cars to identify true intentions of on-road pedestrians and understand social interaction norms. In this paper, we go through some of the challenges in Pedestrian and Autonomous vehicles interaction that autonomous vehicles might face while driving in an urban environment; after that we propose a novel architecture for identifying pedestrian’s intention using pedestrian’s detection, pose estimation, and classification algorithms while discussing different methods of each.


Self-driving cars Machine learning LIDAR Automotive 


  1. 1.
    Mahadevan, K., Somanath, S., & Sharlin, E. (2018). Communicating awareness and intent in autonomous vehicle-pedestrian interaction. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems—CH’18.
  2. 2.
    Yan, J., Lei, Z., Wen, L., & Li, S.Z. (2014). The fastest deformable part model for object detection. In CVPR.Google Scholar
  3. 3.
    V ̈olz, B., Behrendt, K., Mielenz, H., Gilitschenski, I., Siegwart, R., & Nieto, J. (2016) A data-driven approach for pedestrian intention estimation. In Proceedings of the IEEE International Conference on Intelligent Transportation Systems (ITSC), Rio de Janeiro, Brazil, 1–4 November 2016.Google Scholar
  4. 4.
    Rasouli, A., Kotseruba, I., Tsotsos, J. (2017). Agreeing to cross: How drivers and pedestrians communicate. arXiv:1702.03555v1.
  5. 5.
    Xu, X., & Fan, C. (2018). Autonomous vehicles, risk percpetions and insurance demand: An individual survey in China. Transportation Research Part A: Policy and Practice.
  6. 6.
    Keller, C., & Gavrila, D. (2014). Will the pedestrian cross? A study on pedestrian path prediction. IEEE Transactions on Intelligent Transportation Systems, 15(494–506), 9.Google Scholar
  7. 7.
    WHO Survey on Road Safety-Mortality rate. Retrieved from
  8. 8.
    Pinheiro, P., & Collobert, R. (2014). Recurrent convolutional neural networks for scene labeling. In: JMLR.Google Scholar
  9. 9.
    Brunetti, A., Buongiorno, D., Trotta, G. F., & Bevilacqua, V. (2018). Computer vision and deep learning techniques for pedestrian detection and tracking: A survey. Neurocomputing, 300, 17–33. Scholar
  10. 10.
    Mogelmose, A., Trivedi, M. M., & Moeslund, T. B. (2015). Trajectory analysis and prediction for improved pedestrian safety: Integrated framework and evaluations. In 2015 IEEE Intelligent Vehicles Symposium (IV).
  11. 11.
    Cao, Z., et al. (2017). Realtime multi-person 2D pose estimation using part affinity fields. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  12. 12.
    Benenson, R., Omran, M., Hosang, J., & Schiele, B. (2014). Ten years of pedestrian detection, what have we learned? ECCV 2014: Computer Vision—ECCV 2014 Workshops (pp. 613–627).Google Scholar
  13. 13.
    Enzweiler, M., & Gavrila, D. (2009). Monocular pedestrian detection: Survey and experiments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 2179–2195.CrossRefGoogle Scholar
  14. 14.
    Dollar, P., Wojek, C., Schiele, B., & Perona, P. (2012). Pedestrian detection: An evaluation of the state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 743–761. Scholar
  15. 15.
    Cao, Z., Simon, T., Wei, S., & Sheikh, Y. (2017). Realtime multiperson 2D pose estimation using part affinity fields. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  16. 16.
    Zhang, S., Benenson, R., Omran, M., Hosang, J., & Schiele, B. (2017). Towards reaching human performance in pedestrian detection. IEEE Transactions on Pattern Analysis and Machine Intelligence. Scholar
  17. 17.
    Patel, H. A. Thakore, D. G. (2013). Moving object tracking using Kalman Filter. International Journal of Computer Science and Mobile Computing.Google Scholar
  18. 18.
    Rosenfeld, A., Zemel, R., & Tsotsos, J. K. The elephant in the room.Google Scholar
  19. 19.
    Cho, H., Rybski, P. E., Bar-Hillel, A., & Zhang, W. (2012). Real-time pedestrian detection with deformable part models. In 2012 IEEE Intelligent Vehicles Symposium.
  20. 20.
    Wojek, C., Walk, S., & Schiele, B. (2009). Multi-cue on-board pedestrian detection. In: CVPR.Google Scholar
  21. 21.
    Rasouli, A., Kotseruba, I., & Tsotsos, J. K. (2017). Are they going to cross? A benchmark dataset and baseline for pedestrian cross-walk behavior. In 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).
  22. 22.
    Fang, Z., Va ́zquez, D., & Lo ́pez, A. (2017). On-Board detection of pedestrian intentions. Sensors, 17(10), 2193.
  23. 23.
    Ratsamee, P., Mae, Y., Ohara, K., Takubo, T., & Arai, T. (2012). People tracking with body pose estimation for human path prediction. In 2012 IEEE International Conference on Mechatronics and Automation.
  24. 24.
    Schneider, N., Gavrila, D. M. (2013). Pedestrian path prediction with recursive Bayesian Filters: A comparative study. Lecture notes in computer science pattern recognition (pp. 174–183).
  25. 25.
    Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05).Google Scholar
  26. 26.
    Schulz, A. T., & Stiefelhagen, R. (2015). Pedestrian intention recognition using Latent-dynamic Conditional Random Fields. In 2015 IEEE Intelligent Vehicles Symposium (IV).

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.College of EngineeringWellesley Rd, ShivajinagarPuneIndia

Personalised recommendations