Computer Vision and Machine Learning for Enhancing Pedestrian Safety

  • Tarak Gandhi
  • Mohan Manubhai Trivedi
Part of the Studies in Computational Intelligence book series (SCI, volume 132)


Accidents involving pedestrians is one of the leading causes of death and injury around the world. Intelligent driver support systems hold a promise to minimize accidents and save many lives. Such a system would detect the pedestrian, predict the possibility of collision, and then warn the driver or engage automatic braking or other safety devices. This chapter describes the framework and issues involved in developing a pedestrian protection system. It is emphasized that the knowledge of the state of the environment, vehicle, and driver are important for enhancing safety. Classification, clustering, and machine learning techniques for effectively detecting pedestrians are discussed, including the application of algorithms such as SVM, Neural Networks, and AdaBoost for the purpose of distinguishing pedestrians from background. Pedestrians unlike vehicles are capable of sharp turns and speed changes, therefore their future paths are difficult to predict. In order to estimate the possibility of collision, a probabilistic framework for pedestrian path prediction is described along with related research. It is noted that sensors in vehicle are not always sufficient to detect all the pedestrians and other obstacles. Interaction with infrastructure based systems as well as systems from other vehicles can provide a wide area situational awareness of the scene. Furthermore, in infrastructure based systems, clustering and learning techniques can be applied to identify typical vehicle and pedestrian paths and to detect anomalies and potentially dangerous situations. In order to effectively integrate information from infrastructure and vehicle sources, the importance of developing and standardizing vehicle-vehicle and vehicle-infrastructure communication systems is also emphasized.


Support Vector Machine Ground Plane Intelligent Transportation System Pedestrian Detection Omnidirectional Camera 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Advanced Highway Systems Program, Japanese Ministry of Land, Infrastructure and Transport, Road Bureau.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
    Deliverable 3.3b report on initial algorithms 2. Technical Report IST-2001-34410, CAMELLIA: Core for Ambient and Mobile Intelligent Imaging Applications, December 2003.Google Scholar
  6. 6.
    IEEE Intelligent Vehicle Symposium, Istanbul, Turkey, June 2007.Google Scholar
  7. 7.
    IEEE International Transportation Systems Conference, Seattle, WA, September 2007.Google Scholar
  8. 8.
    Y. Abramson and B. Steux. Hardware-friendly pedestrian detection and impact prediction. In IEEE Intelligent Vehicle Symposium, pp. 590–595, June 2004.Google Scholar
  9. 9.
    G. Antonini, S. Venegas, J.P. Thiran, and M. Bierlaire. A discrete choice pedestrian behavior model for pedestrian detection in visual tracking systems. In Proceedings of Advanced Concepts for Intelligent Vision Systems, September 2004.Google Scholar
  10. 10.
    S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A tutorial on particle filters for on-line non-linear/non-gaussian bayesian tracking. IEEE Transactions on Signal Processing, 50(2):174–188, 2002.CrossRefGoogle Scholar
  11. 11.
    C.-C. Chang and C.-J. Lin. LIBSVM: A Library for Support Vector Machines, Last updated June 2007.Google Scholar
  12. 12.
    H. Cheng, N. Zheng, and J. Qin. Pedestrian detection using sparse gabor filters and support vector machine. In IEEE Intelligent Vehicle Symposium, pp. 583–587, June 2005.Google Scholar
  13. 13.
    N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, June 2005.Google Scholar
  14. 14.
    U. Franke. Real-time stereo vision for urban traffic scene understanding. In IEEE Intelligent Vehicle Symposium, pp. 273–278, 2000.Google Scholar
  15. 15.
    T. Gandhi and M.M. Trivedi. Motion analysis for event detection and tracking with a mobile omni-directional camera. Multimedia Systems Journal, Special Issue on Video Surveillance, 10(2):131–143, 2004.Google Scholar
  16. 16.
    T. Gandhi and M.M. Trivedi. Parametric ego-motion estimation for vehicle surround analysis using an omnidirectional camera. Machine Vision and Applications, 16(2):85–95, 2005.CrossRefGoogle Scholar
  17. 17.
    T. Gandhi and M.M. Trivedi. Vehicle mounted wide FOV stereo for traffic and pedestrian detection. In Proceedings of International Conference on Image Processing, pp. 2:121–124, 2005.Google Scholar
  18. 18.
    T. Gandhi and M.M. Trivedi. Vehicle surround capture: Survey of techniques and a novel omni video based approach for dynamic panoramic surround maps. IEEE Transactions on Intelligent Transportation Systems, 7(3):293–308, 2006.CrossRefGoogle Scholar
  19. 19.
    T. Gandhi and M.M. Trivedi. Pedestrian protection systems: Issues, survey, and challenges. IEEE Transactions on Intelligent Transportation Systems, 8(3), 2007.Google Scholar
  20. 20.
    D.M. Gavrila. Pedestrian detection from a moving vehicle. In Proceedings of European Conference on Computer Vision, pp. 37–49, 2000.Google Scholar
  21. 21.
    D.M. Gavrila and S. Munder. Multi-cue pedestrian detection and tracking from a moving vehicle. International Journal of Computer Vision, 73(1):41–59, 2007.CrossRefGoogle Scholar
  22. 22.
    R.C. Gonzalez and R.E. Woods. Digital Image Processing. Prentice Hall, Upper Saddle River, NJ, 3rd edition, 2008.Google Scholar
  23. 23.
    K. Kim, T.H. Chalidabhongse, D. Harwood, and L.S. Davis. Real-time foreground-background segmentation using codebook model. Real-Time Imaging, 11(3):172–185, 2005.CrossRefGoogle Scholar
  24. 24.
    K. Konolige. Small vision system: Hardware and implementation. In Eighth International Symposium on Robotics Research, pp. 111–116, 1997.
  25. 25.
    S.J. Krotosky and M.M. Trivedi. A comparison of color and infrared stereo approaches to pedestrian detection. In IEEE Intelligent Vehicles Symposium, June 2007.Google Scholar
  26. 26.
    R. Labayrade, D. Aubert, and J.-P. Tarel. Real time obstacle detection in stereovision on non flat road geometry through V-disparity representation. In IEEE Intelligent Vehicles Symposium, volume II, pp. 646–651, 2002.Google Scholar
  27. 27.
    S. Munder and D.M. Gavrila. An experimental study on pedestrian classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(11):1863–1868, 2006.CrossRefGoogle Scholar
  28. 28.
    C. Papageorgiou and T. Poggio. A trainable system for object detection. International Journal of Computer Vision, 38(1):15–33, 2000.zbMATHCrossRefGoogle Scholar
  29. 29.
    S. Park and M.M. Trivedi. Video Analysis of Vehicles and Persons for Surveillance. Intelligent and Security Informatics: Techniques and Applications, Springer, Berlin Heidelberg New York, 2007.Google Scholar
  30. 30.
    A. Prati, I. Mikic, M.M. Trivedi, and R. Cucchiara. Detecting moving shadows: Algorithms and evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 918–923, July 2003.Google Scholar
  31. 31.
    D.V. Prokhorov. Neural Networks in Automotive Applications. Computational Intelligence in Automotive Applications, Studies in Computational Intelligence, Springer, Berlin Heidelberg New York, 2008.CrossRefGoogle Scholar
  32. 32.
    M. Soga, T. Kato, M. Ohta, and Y. Ninomiya. Pedestrian detection with stereo vision. In International Conference on Data Engineering, April 2005.Google Scholar
  33. 33.
    C. Stauffer and W.E.L. Grimson. Adaptive background mixture model for real-time tracking. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 246–252, 1999.Google Scholar
  34. 34.
    M.M. Trivedi, T. Gandhi, and K.S. Huang. Distributed interactive video arrays for event capture and enhanced situational awareness. IEEE Intelligent Systems, Special Issue on AI in Homeland Security, 20(5):58–66, September–October 2005.Google Scholar
  35. 35.
    M.M. Trivedi, T. Gandhi, and J. McCall. Looking-in and looking-out of a vehicle: Computer vision based enhanced vehicle safety. IEEE Transactions on Intelligent Transportation Systems, 8(1):108–120, March 2007.CrossRefGoogle Scholar
  36. 36.
    P. Viola and M.J. Jones. Rapid object detection using a boosted cascade of simple features. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. I:511–518, June 2001.Google Scholar
  37. 37.
    P. Viola, M.J. Jones, and D. Snow. Detecting pedestrians using patterns of motion and appearance. International Journal of Computer Vision, 63(2):153–161, 2005.CrossRefGoogle Scholar
  38. 38.
    C. Wakim, S. Capperon, and J. Oksman. A markovian model of pedestrian behavior. In Proceedings of IEEE Intelligent Conference on Systems, Man, and Cybernetics, pp. 4028–4033, October 2004.Google Scholar
  39. 39.
    C. Wöhler and J. Anlauf. An adaptable time-delay neural-network algorithm for image sequence analysis. IEEE Transactions on Neural Networks, 10(6):1531–1536, 1999.CrossRefGoogle Scholar
  40. 40.
    Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, November 1998.CrossRefGoogle Scholar
  41. 41.
    L. Zhao and C. Thorpe. Stereo and neural network-based pedestrian detection. IEEE Transactions Intelligent Transportation, 1(3):148–154, September 2000.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Tarak Gandhi
    • 1
  • Mohan Manubhai Trivedi
    • 1
  1. 1.Laboratory for Safe and Intelligent Vehicles (LISA)University of California San DiegoLa JollaUSA

Personalised recommendations