Local Feature Based Person Detection and Tracking Beyond the Visible Spectrum

Chapter
Part of the Augmented Vision and Reality book series (Augment Vis Real, volume 1)

Abstract

One challenging field in computer vision is the automatic detection and tracking of objects in image sequences. Promising performance of local features and local feature based object detection approaches in the visible spectrum encourage the application of the same principles to data beyond the visible spectrum. Since these dedicated object detectors neither make assumptions on a static background nor a stationary camera, it is reasonable to use these object detectors as a basis for tracking tasks as well. In this work, we address the two tasks of object detection and tracking and introduce an integrated approach to both challenges that combines bottom-up tracking-by-detection techniques with top-down model based strategies on the level of local features. By this combination of detection and tracking in a single framework, we achieve (i) automatic identity preservation in tracking, (ii) a stabilization of object detection, (iii) a reduction of false alarms by automatic verification of tracking results in every step and (iv) tracking through short term occlusions without additional treatment of these situations. Since our tracking approach is solely based on local features it works independently of underlying video-data specifics like color information—making it applicable to both, visible and infrared data. Since the object detector is trainable and the tracking methodology does not make any assumptions on object class specifics, the overall approach is general applicable for any object class. We apply our approach to the task of person detection and tracking in infrared image sequences. For this case we show that our local feature based approach inherently allows for object component classification, i.e., body part detection. To show the usability of our approach, we evaluate the performance of both, person detection and tracking in different real world scenarios, including urban scenarios where the camera is mounted on a moving vehicle.

Keywords

Person detection Person tracking Visual surveillance SURF 

References

  1. 1.
    Andriluka, M., Roth, S., Schiele, B.: People-tracking-by-detection and people-detection- by-tracking. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  2. 2.
    Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features. Comput. Vis. Image Underst. 110(3), 346–359 (2008)CrossRefGoogle Scholar
  3. 3.
    Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. Trans. Pattern Anal. Mach. Intell. 24(4), 509–522 (2002)CrossRefGoogle Scholar
  4. 4.
    Bernardi, K., Elbs, A., Stiefelhagen, R.: Multiple object tracking performance metrics and evaluation in a smart room environment. In: The IEEE International Workshop on Visual Surveillance, pp. 219–223 (2006)Google Scholar
  5. 5.
    Berrabah, S.A., De Cubber, G., Enescu, V., Sahli, H.: MRF-based foreground detection in image sequences from a moving camera. In: Proceedings of the IEEE International Conference on Image Processing, pp. 1125–1128 (2006)Google Scholar
  6. 6.
    Bertozzi, M., Broggi, A., Grisleri, P., Graf, T., Meinecke, M.: Pedestrian detection in infrared images. In: Proceedings of the IEEE Intelligent Vehicles Symposium, pp. 662–667 (2003)Google Scholar
  7. 7.
    Conaire, C.O., Cooke, E., O’Connor, N., Murphy, N., Smearson, A.: Background modeling in infrared and visible spectrum video for people tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 20–25 (2005)Google Scholar
  8. 8.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 886–893 (2005)Google Scholar
  9. 9.
    Davis, J., Sharma, V.: Background-subtraction using contour-based fusion of thermal and visible imagery. Comput. Vis. Image Underst. 106(2–3), 162–182 (2007)CrossRefGoogle Scholar
  10. 10.
    Davis, J.W., Sharma, V.: Robust background-subtraction for person detection in thermal imagery. In: Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop, pp. 128 (2004)Google Scholar
  11. 11.
    Enzweiler, M., Gavrila, D.M.: Monocular pedestrian detection: survey and experiments. IEEE Trans. Pattern Anal. Mach. Intell. 31(12):2179–2195 (2009)Google Scholar
  12. 12.
    Fang, Y., Yamada, K., Ninomiya, Y., Horn, B., Masaki, I.: Comparison between infrared-image-based and visible-image-based approaches for pedestrian detection. In: Proceedings of the IEEE Intelligent Vehicles Symposium, pp. 505–510 (2003)Google Scholar
  13. 13.
    Felzenszwalb, P., Mcallester, D., Ramanan, D.: A discriminatively trained, multiscale, deformable part model. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)Google Scholar
  14. 14.
    Gammeter, S., Ess, A., Jaeggli, T., Schindler, K., Leibe, B., Van Gool, L.: Articulated multi-body tracking under egomotion. In: Proceedings of the European Conference Computer Vision, pp. 816–830 (2008)Google Scholar
  15. 15.
    Gavrila, D., Munder, S.: Multi-cue pedestrian detection and tracking from a moving vehicle. Int. J. Comput. Vis. 73(1), 41–59 (2007)CrossRefGoogle Scholar
  16. 16.
    Haritaoglu, I., Harwood, D., Davis, L.: W4s: a real-time system for detecting and tracking people in 2.5 d. In: Proceedings of the European Conference on Computer Vision, pp. 877+ (1998)Google Scholar
  17. 17.
    Harris, C., Stephens, M.: A combined corner and edge detection. In: Proceedings of the Alvey Vision Conference, pp. 147–151 (1988)Google Scholar
  18. 18.
    Hu, W., Tan, T., Wang, L., Maybank, S.: A survey on visual surveillance of object motion and behaviors. IEEE Trans. Syst. Man Cybern. 34(3), 334–352 (2004)CrossRefGoogle Scholar
  19. 19.
    Jaccard, P.: Nouvelles recherches sur la distribution florale. Bull. Soc. Vaudoise Sci. Naturelles 4(3), 223–370 (1908)Google Scholar
  20. 20.
    Jüngling, K., Arens, M.: Detection and tracking of objects with direct integration of perception and expectation. In: Proceedings of the International Conference on Computer Vision, ICCV Workshops, pp. 1129–1136 (2009)Google Scholar
  21. 21.
    Jüngling, K., Arens, M.: Feature based person detection beyond the visible spectrum. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR Workshops, pp. 30–37 (2009)Google Scholar
  22. 22.
    Klinger, V., Arens, M.: Ragdolls in action–action recognition by 3d pose recovery from monocular video. In: Proceedings of the IADIS International Conference on Computer Graphics, Visualization, Computer Vision and Image Processing, pp. 219–223 (2009)Google Scholar
  23. 23.
    Leibe, B., Leonardis, A., Schiele, B.: Combined object categorization and segmentation with an implicit shape model. In: ECCV Workshop on Statistical Learning in Computer Vision, pp. 17–32 (2004)Google Scholar
  24. 24.
    Leibe, B., Leonardis, A., Schiele, B.: Robust object detection with interleaved categorization and segmentation. Int. J. Comput. Vis. 77(1–3), 259–289 (2008)CrossRefGoogle Scholar
  25. 25.
    Leibe, B., Schindler, K., Cornelis, N., Van Gool, L.: Coupled object detection and tracking from static cameras and moving vehicles. IEEE Trans. Pattern Anal. Mach. Intell. 30(10), 1683–1698 (2008)CrossRefGoogle Scholar
  26. 26.
    Leibe, B., Schindler, K., Van Gool, L.: Coupled detection and trajectory estimation for multi-object tracking. In: Proceedings of the International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  27. 27.
    Leykin, A., Hammoud, R.: Robust multi-pedestrian tracking in thermal-visible surveillance videos. In: Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop, pp. 136+ (2006)Google Scholar
  28. 28.
    David Lowe, G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  29. 29.
    Munkres, J.: Algorithms for the assignment and transportation problems. J. Soc. Industrial Appl. Math. 5, 32–38 (1957)MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    Nanda, H., Davis, L.: Probabilistic template based pedestrian detection in infrared videos. In: Proceedings of the IEEE Intelligent Vehicle Symposium, vol. 1, pp. 15–20 (2002)Google Scholar
  31. 31.
    Ren, Y., Chua, C., Ho, Y.: Statistical background modeling for non-stationary camera. Pattern Recognition Lett. 24(1–3), 183–196 (2003)MATHCrossRefGoogle Scholar
  32. 32.
    Seemann, E., Fritz, M., Schiele, B.: Towards robust pedestrian detection in crowded image sequences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)Google Scholar
  33. 33.
    Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 246–252 (1999)Google Scholar
  34. 34.
    Suard, F., Rakotomamonjy, A., Bensrhair, A., Broggi, A.: Pedestrian detection using infrared images and histograms of oriented gradients. In: Proceedings of the IEEE Intelligent Vehicles Symposium, pp. 206–212 (2006)Google Scholar
  35. 35.
    Tuytelaars, T., Mikolajczyk, K.: Local Invariant Feature Detectors: A Survey. Now Publishers Inc., Hanover (2008)Google Scholar
  36. 36.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 511–518 (2001)Google Scholar
  37. 37.
    Wren, C., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: real-time tracking of the human body. In: Proceedings of the International Conference on Automatic Face and Gesture Recognition, pp. 51–56 (1996)Google Scholar
  38. 38.
    Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet based part detectors. Int. J. Comput. Vis. 75(2), 247–266 (2007)CrossRefGoogle Scholar
  39. 39.
    Xu, F., Fujimura, K.: Pedestrian detection and tracking with night vision. In: Proceedings of the IEEE Intelligent Vehicle Symposium, pp. 21–30 (2002)Google Scholar
  40. 40.
    Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. 38(4), 13+ (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.Fraunhofer IOSBEttlingenGermany

Personalised recommendations