Pedestrian and Vehicle Detection and Tracking with Object-Driven Vanishing Line Estimation

  • Yi-Ming ChanEmail author
  • Li-Chen Fu
  • Pei-Yung Hsiao
  • Shin-Shinh Huang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10116)


To robustly detect people and vehicle on the road in a video sequence is a challenging problem. Most researches focus on detecting or tracking of specific targets only. On the contrary, instead of detecting vehicle or pedestrian individually, an integration framework combining the geometric information is proposed. The camera’s pitch angle is estimated with a novel vanishing line estimator. Not only detecting the vanishing point using line intersection approach, but also the object information from tracker are considered. Specifically, the detected vehicle or pedestrian will cast votes for the hypothesized horizon line. The vanishing line can be estimated even when the scenes are cluttered or crowded, and thus the geometric information can be estimated under challenging circumstance. In turn, such information of scene can help the system refine our detection results through Bayes’ network. Finally, to verify the performance of the system, comprehensive experiments have been conducted with the KITTI dataset. It is quite promising that the state-of-the-art detector, in our case, Regionlet detector, can be improved.


Ground Plane Convolutional Neural Network Edge Pixel Vehicle Detection Pedestrian Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Hoiem, D., Efros, A., Hebert, M.: Putting objects in perspective. Int. J. Comput. Vision 80, 3–15 (2008)CrossRefGoogle Scholar
  2. 2.
    Nieto, M., Laborda, J.A., Salgado, L.: Road environment modeling using robust perspective analysis and recursive bayesian segmentation. Mach. Vis. Appli. 22, 927–945 (2011)CrossRefGoogle Scholar
  3. 3.
    Geiger, A., Lauer, M., Wojek, C., Stiller, C., Urtasun, R.: 3D traffic scene understanding from movable platforms. IEEE Trans. Pattern Anal. Mach. Intell. 36, 1012–1025 (2014)CrossRefGoogle Scholar
  4. 4.
    Richardson, E., Peleg, S., Werman, M.: Scene geometry from moving objects. In: IEEE International Conference on Advanced Video and Signal Based Surveillance, pp. 13–18 (2014)Google Scholar
  5. 5.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Computer Vision and Patter Recognition, vol. 1, pp. 886–893 (2005)Google Scholar
  6. 6.
    Dollar, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: an evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 34, 743–761 (2012)CrossRefGoogle Scholar
  7. 7.
    Ess, A., Leibe, B., Schindler, K., Van Gool, L.: A mobile vision system for robust multi-person tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  8. 8.
    Wojek, C., Roth, S., Schindler, K., Schiele, B.: Monocular 3D scene modeling and inference: understanding multi-object traffic scenes. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 467–481. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15561-1_34 CrossRefGoogle Scholar
  9. 9.
    Sudowe, P., Leibe, B.: Efficient use of geometric constraints for sliding-window object detection in video. In: Crowley, J.L., Draper, B.A., Thonnat, M. (eds.) ICVS 2011. LNCS, vol. 6962, pp. 11–20. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23968-7_2 CrossRefGoogle Scholar
  10. 10.
    Moghadam, P., Starzyk, J.A., Wijesoma, W.S.: Fast vanishing-point detection in unstructured environments. IEEE Trans. Image Process. 21, 425–430 (2012)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Geiger, A., Wojek, C., Urtasun, R.: Joint 3D estimation of objects and scene layout. In: Advances in Neural Information Processing Systems, pp. 1467–1475 (2011)Google Scholar
  12. 12.
    Lezama, J., Grompone von Gioi, R., Randall, G., Morel, J.M.: Finding vanishing points via point alignments in image primal and dual domains. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 509–515 (2014)Google Scholar
  13. 13.
    Yang, W., Luo, X., Fang, B., Zhang, D., Tang, Y.Y.: Fast and accurate vanishing point detection in complex scenes. In: IEEE Conference on Intelligent Transportation Systems, pp. 93–98 (2014)Google Scholar
  14. 14.
    Wildenauer, H., Hanbury, A.: Robust camera self-calibration from monocular images of manhattan worlds. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2831–2838 (2012)Google Scholar
  15. 15.
    Nieto, M., Salgado, L.: Real-time robust estimation of vanishing points through nonlinear optimization. In: SPIE Photonics Europe, pp. 772402–772402-14. International Society for Optics and Photonics (2010)Google Scholar
  16. 16.
    Quan, L., Mohr, R.: Determining perspective structures using hierarchical hough transform. Pattern Recogn. Lett. 9, 279–286 (1989)CrossRefzbMATHGoogle Scholar
  17. 17.
    Dubska, M., Herout, A., Havel, J.: Pclines - line detection using parallel coordinates. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1489–1494 (2011)Google Scholar
  18. 18.
    Vermaak, J., Doucet, A., Perez, P.: Maintaining multimodality through mixture tracking. In: IEEE International Conference on Computer Vision, vol. 2, pp. 1110–1116 (2003)Google Scholar
  19. 19.
    Lanz, O.: Approximate bayesian multibody tracking. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1436–1449 (2006)CrossRefGoogle Scholar
  20. 20.
    Kuo, C.H., Huang, C., Nevatia, R.: Multi-target tracking by on-line learned discriminative appearance models. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 685–692 (2010)Google Scholar
  21. 21.
    Breitenstein, M.D., Reichlin, F., Leibe, B., Koller-Meier, E., Van Gool, L.: Online multiperson tracking-by-detection from a single, uncalibrated camera. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1820–1833 (2011)CrossRefGoogle Scholar
  22. 22.
    Leibe, B., Schindler, K., Cornelis, N., Van Gool, L.: Coupled object detection and tracking from static cameras and moving vehicles. IEEE Trans. Pattern Anal. Mach. Intell. 30, 1683–1698 (2008)CrossRefGoogle Scholar
  23. 23.
    Milan, A., Roth, S., Schindler, K.: Continuous energy minimization for multitarget tracking. IEEE Trans. Pattern Anal. Mach. Intell. 36, 58–72 (2014)CrossRefGoogle Scholar
  24. 24.
    Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20, 226–239 (1998)CrossRefGoogle Scholar
  25. 25.
    Bota, S., Nedesvchi, S.: Multi-feature walking pedestrians detection for driving assistance systems. IET Intel. Trans. Syst. 2, 92–104 (2008)CrossRefGoogle Scholar
  26. 26.
    Isard, M., Blake, A.: Condensation-conditional density propagation for visual tracking. Int. J. Comput. Vision 29, 5–28 (1998)CrossRefGoogle Scholar
  27. 27.
    Matas, J., Galambos, C., Kittler, J.: Robust detection of lines using the progressive probabilistic hough transform. Comput. Vis. Image Underst. 78, 119–137 (2000)CrossRefGoogle Scholar
  28. 28.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)Google Scholar
  29. 29.
    Wang, X., Yang, M., Zhu, S., Lin, Y.: Regionlets for generic object detection. In: IEEE International Conference on Computer Vision, pp. 17–24 (2013)Google Scholar
  30. 30.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354–3361 (2012)Google Scholar
  31. 31.
    Xiang, Y., Alahi, A., Savarese, S.: Learning to track: Online multi-object tracking by decision making. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4705–4713 (2015)Google Scholar
  32. 32.
    Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the clear mot metrics. J. Image Video Process. 2008, 1–10 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Yi-Ming Chan
    • 1
    Email author
  • Li-Chen Fu
    • 1
  • Pei-Yung Hsiao
    • 2
  • Shin-Shinh Huang
    • 3
  1. 1.Computer Science and Information EngineeringNational Taiwan UniversityTaipei CityTaiwan
  2. 2.Electrical EngineeringNational University of KaohsiungKaohsiungTaiwan
  3. 3.Computer and Communication EngineeringNational Kaohsiung First University of Science and TechnologyKaohsiungTaiwan

Personalised recommendations