People Detection Using Color and Depth Images

  • Joaquín Salas
  • Carlo Tomasi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6718)


We present a strategy that combines color and depth images to detect people in indoor environments. Similarity of image appearance and closeness in 3D position over time yield weights on the edges of a directed graph that we partition greedily into tracklets, sequences of chronologically ordered observations with high edge weights. Each tracklet is assigned the highest score that a Histograms-of-Oriented Gradients (HOG) person detector yields for observations in the tracklet. High-score tracklets are deemed to correspond to people. Our experiments show a significant improvement in both precision and recall when compared to the HOG detector alone.


Depth Information Depth Image Scale Invariant Feature Transform Human Detection Pedestrian Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Dalal, N., Triggs, B.: Histograms of Oriented Gradients for Human Detection. In: IEEE Computer Vision and Pattern Recognition, vol. 1, pp. 886–893 (2005)Google Scholar
  2. 2.
    Dalal, N., Triggs, B., Schmid, C.: Human Detection using Oriented Histograms of Flow and Appearance. In: European Conference on Computer Vision, pp. 428–441 (2006)Google Scholar
  3. 3.
    Dalal, N.: INRIA Person Database (September 2010),
  4. 4.
    Elfes, A.: Using Occupancy Grids for Mobile Robot Perception and Navigation. Computer 22(6), 46–57 (2002)CrossRefGoogle Scholar
  5. 5.
    Gavrila, D.: Pedestrian Detection from a Moving Vehicle. In: European Conference on Computer Vision, pp. 37–49 (2000)Google Scholar
  6. 6.
    Gavrila, D., Giebel, J., Munder, S.: Vision-based Pedestrian Detection: The Protector System. In: Intelligent Vehicles Symposium, pp. 13–18 (2004)Google Scholar
  7. 7.
    Gavrila, D.: The Visual Analysis of Human Movement: A Survey. Computer Vision and Image Understanding 73(1), 82–98 (1999)CrossRefzbMATHGoogle Scholar
  8. 8.
    Giles, J.: Inside the Race to Hack the Kinect. The New Scientist 208 (2789) (2010)Google Scholar
  9. 9.
    Gordon, G., Darrell, T., Harville, M., Woodfill, J.: Background Estimation and Removal based on Range and Color. In: IEEE Computer Vision and Pattern Recognition, p. 2 (1999)Google Scholar
  10. 10.
    Javed, O., Shafique, K., Rasheed, Z., Shah, M.: Modeling Inter-Camera Space-Time and Appearance Relationships for Tracking Across Non-Overlapping Views. Computer Vision and Image Understanding 109(2), 146–162 (2008)CrossRefGoogle Scholar
  11. 11.
    Johansson, G.: Visual Perception of Biological Motion and a Model for its Analysis. Perceiving Events and Objects 3 (1973)Google Scholar
  12. 12.
    Kelly, M.: Visual Identification of People by Computer. Ph.D. thesis, Stanford University (1971)Google Scholar
  13. 13.
    Lowe, D.: Object Recognition from Local Scale-invariant Features. In: IEEE International Conference on Computer Vision, p. 1150 (1999)Google Scholar
  14. 14.
    Micilotta, A., Ong, E., Bowden, R.: Detection and Tracking of Humans by Probabilistic Body Part Assembly. In: British Machine Vision Conference, vol. 1, pp. 429–438 (2005)Google Scholar
  15. 15.
    Mikolajczyk, K., Schmid, C., Zisserman, A.: Human Detection based on a Probabilistic Assembly of Robust Part Detectors. In: European Conference on Computer Vision, pp. 69–82 (2004)Google Scholar
  16. 16.
    Mohan, A., Papageorgiou, C., Poggio, T.: Example-based Object Detection in Images by Components. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(4), 349 (2001)CrossRefGoogle Scholar
  17. 17.
    Muñoz, R., Aguirre, E., García, M.: People Detection and Tracking using Stereo Vision and Color. Image and Vision Computing 25(6), 995–1007 (2007)CrossRefGoogle Scholar
  18. 18.
    Nelder, J., Mead, R.: A Simplex Method for Function Minimization. The Computer Journal 7(4), 308 (1965)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Papageorgiou, C., Poggio, T.: A Trainable System for Object Detection. International Journal of Computer Vision 38(1), 15–33 (2000)CrossRefzbMATHGoogle Scholar
  20. 20.
    Phillips, P.: Human Identification Technical Challenges. In: IEEE International Conference on Image Processing (2002)Google Scholar
  21. 21.
    Ramanan, D., Forsyth, D., Zisserman, A.: Strike a Pose: Tracking People by Finding Stylized Poses. In: IEEE Computer Vision and Pattern Recognition, pp. 271–278 (2005)Google Scholar
  22. 22.
    Roberts, T., McKenna, S., Ricketts, I.: Human Pose Estimation using Learnt Probabilistic Region Similarities and Partial Configurations. In: European Conference on Computer Vision, pp. 291–303 (2004)Google Scholar
  23. 23.
    Ronfard, R., Schmid, C., Triggs, B.: Learning to Parse Pictures of People. In: European Conference on Computer Vision, pp. 700–714 (2006)Google Scholar
  24. 24.
    Rubner, Y., Tomasi, C., Guibas, L.: The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision 40(2), 99–121 (2000)CrossRefzbMATHGoogle Scholar
  25. 25.
    Schwartz, W., Kembhavi, A., Harwood, D., Davis, L.: Human Detection using Partial Least Squares Analysis. In: IEEE International Conference on Computer Vision, pp. 24–31 (2010)Google Scholar
  26. 26.
    Swets, J., Dawes, R., Monahan, J.: Better Decisions through Science. Scientific American, 83 (2000)Google Scholar
  27. 27.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition. Elsevier, Amsterdam (2009)zbMATHGoogle Scholar
  28. 28.
    Viola, P., Jones, M.: Rapid Object Detection using a Boosted Cascade of Simple Features. In: IEEE Computer Vision and Pattern Recognition, vol. 1 (2001)Google Scholar
  29. 29.
    Viola, P., Jones, M., Snow, D.: Detecting Pedestrians using Patterns of Motion and Appearance. International Journal of Computer Vision 63(2), 153–161 (2005)CrossRefGoogle Scholar
  30. 30.
    Vrubel, A., Bellon, O., Silva, L.: Planar Background Elimination in Range Images: A Practical Approach. In: IEEE International Conference on Image Processing, pp. 3197–3200 (2009)Google Scholar
  31. 31.
    Willow Garage: OpenCV (September 2010),
  32. 32.
    Xu, F., Fujimura, K.: Human Detection using Depth and Gray Images. In: IEEE Advanced Video and Signal Based Surveillance. pp. 115–121. IEEE, New York (2003)Google Scholar
  33. 33.
    Zhao, L., Davis, L.: Closely coupled object detection and segmentation. In: IEEE International Conference on Computer Vision, pp. 454–461 (2005)Google Scholar
  34. 34.
    Zhao, L., Thorpe, C.: Stereo and Neural Network-based Pedestrian Detection. IEEE Transactions on Intelligent Transportation Systems 1(3), 148–154 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Joaquín Salas
    • 1
  • Carlo Tomasi
    • 2
  1. 1.Instituto Politécnico NacionalMexico
  2. 2.Duke UniversityUSA

Personalised recommendations