International Journal of Computer Vision

, Volume 101, Issue 2, pp 350–366 | Cite as

Shadow Casting Out Of Plane (SCOOP) Candidates for Human and Vehicle Detection in Aerial Imagery

  • Vladimir ReillyEmail author
  • Berkan Solmaz
  • Mubarak Shah


In this paper, we propose a method for detecting humans and vehicles in imagery taken from a UAV. This is a challenging problem due to a limited number of pixels on target, which makes it more difficult to distinguish objects from background clutter, and results in much larger search space. We propose a method for constraining the search based on a number of geometric constraints obtained from the metadata. Specifically, we obtain the orientation of ground plane normal, the orientation of shadows cast by out of plane objects in the scene, and the relationship between object heights and the size of their corresponding shadows. We use the aforementioned information in a geometry-based shadow, and ground-plane normal blob detector, which provides an initial estimation for locations of shadow casting out of plane (SCOOP) objects in the scene. These SCOOP candidate locations are then classified as either human or clutter using a combination of wavelet features and a Support Vector Machine. To detect vehicles, we similarly find potential vehicle candidates by combining SCOOP and inverted-SCOOP candidates and then classify them using wavelet features and SVM. Our method works on a single frame, and unlike motion detection based methods, it bypasses the entire pipeline of registration, motion detection, and tracking. This method allows for detection of stationary and slowly moving humans and vehicles while avoiding the search across the entire image, allowing accurate and fast localization. We show impressive results on sequences from VIVID and CLIF datasets and provide comparative analysis.


Human detection Vehicle detection Aerial surveillance UAV Shadow Metadata 



This research was partially supported by the Harris corporation and Defense Advanced Research Projects Agency (DARPA) under Contract No. HR0011-10-C-0112. Any opinions, findings, and conclusions expressed in this material are those of the authors and do not necessarily reflect the views of the Harris corporation or DARPA.


  1. Bi, S., Liang, D., Shen, X., & Wang, Q. (2007). Human cast shadow elimination method bad on orientation information measures. In ICAL. Google Scholar
  2. Bissacco, A., & Yang, M. H. (2007). Detecting humans via their pose. In NIPS. Google Scholar
  3. Bose, B., & Grimson, E. (2004). Improving object classification in far-field video. In CVPR. Google Scholar
  4. Breckon, T., Barnes, S., Eichner, M., & Wahren, K. (2009). Autonomous real-time vehicle detection from a medium-level uav. In UAVS. Google Scholar
  5. Chang, C., & Lin, C. (2001). LIBSVM: a library for support vector machines. Software available at
  6. Chang, J. C., Hu, W. F., Hsieh, J. W., & Chen, Y. S. (2002). Shadow elimination for effective moving object detection with Gaussian models. In ICPR. Google Scholar
  7. Chen, Y. T., Chen, C. S., Hung, Y. P., & Chang, K. Y. (2009). Multi-class multi-instance boosting for part-based human detection. In ICCV. Google Scholar
  8. Cheng, H., Butler, D., & Basu, C. (2006). ViTex: video to tex and its application in aerial video surveillance. In CVPR. Google Scholar
  9. Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In CVPR (Vol. 1). Google Scholar
  10. Felzenszwalb, P., McAllester, D., & Ramanan, D. (2008). A discriminatively trained, multiscale, deformable part model. In CVPR. Google Scholar
  11. Finlayson, G., Hordley, S., Lu, C., & Drew, M. (2006). On the removal of shadows from images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1). Google Scholar
  12. Gaszczak, A., Breckon, T., & Han, J. (2011). Real-time people and vehicle detection from uav imagery. SPIE. Google Scholar
  13. Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd ed.). Cambridge: Cambridge University Press. ISBN: 0521540518. zbMATHCrossRefGoogle Scholar
  14. Hsieh, J. W., Yu, S. H., Chen, Y. S., & Hu, W. F. (2004). A shadow elimination method for vehicle analysis. In ICPR. Google Scholar
  15. Hu, H., Huang, Y. Q., & Li, L. M. (2010). Moving vehicle shadow elimination approach based on mark growing of multi-feature fusion. In ICACIA. Google Scholar
  16. Kembhavi, A., Harwood, D., & Davis, L. (2011). Vehicle detection using partial least squares. IEEE Transactions on Pattern Analysis and Machine Intelligence. Google Scholar
  17. Kluckner, S., Mauthner, T., Roth, P. M., & Bischof, H. (2009). Semantic classification in aerial imagery by integrating appearance and height information. In ACCV. Google Scholar
  18. Leibe, B., Seemann, E., & Schiele, B. (2005). Pedestrian detection in crowded scenes. In CVPR. Google Scholar
  19. Liu, Z., Zhao, F., & Yang, H. (2010). A new method of moving shadow elimination combining texture and chrominance of moving foreground region based on criterion. In WCICA. Google Scholar
  20. Martel-Brisson, N., & Zaccarin, A. (2005). Moving cast shadow detection from a Gaussian mixtrue shadow model. In CVPR. Google Scholar
  21. Mikolajczyk, K. C. S., & Zisserman, A. (2004). Human detection based on a probabilistic assembly of robust part detectors. In ECCV. Google Scholar
  22. Miller, A., Babenko, P., Hu, M., & Shah, M. (2007). Person tracking in UAV video. In CLEAR. Google Scholar
  23. Panagopoulos, A., Samaras, D., & Paragios, N. (2009). Robust shadow and illumination estimation using a mixture model. In CVPR. Google Scholar
  24. Porikli, F., & Thornton, J. (2005). Shadow flow: a recursive method to learn moving cast shadows. In ICCV. Google Scholar
  25. Prati, A., Mikic, I., Trivedi, M. M., & Cucchiara, R. (2003). Detecting moving shadows: algorithms and evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25. Google Scholar
  26. Quaritsch, M., Kruggl, K., Wischounig-Strucl, D., Bhattacharya, S., Shah, M., & Rinner, B. (2010). Networked uavs as aerial sensor network for disaster management applications. Elektrotechnik und Informationstechnik, 127. Google Scholar
  27. Reda, I., & Anreas, A. (2003). Solar position algorithm for solar radiation applications (NREL Report No. TP-560-34302). Google Scholar
  28. Rudol, P., & Doherty, P. (2008). Human body detection and geolocalization for uav search and rescue missions using color and thermal imagery. In IEEE aerospace. Google Scholar
  29. Sabzmeydani, P., & Mori, G. (2007). Detecting pedestrians by learning shapelet features. In CVPR. Google Scholar
  30. Sokalski, J., & Breckon, T. (2010). Automatic salient object detection in uav imagery. In UAVS. Google Scholar
  31. Tian, T. P., & Sclaroff, S. (2010). Fast multi-aspect 2d human detection. In ECCV. Google Scholar
  32. Tuzel, O., Porikli, F., & Meer, P. (2008). Pedestrian detection via classification on Riemannian manifolds. Pattern Analysis and Machine Intelligence, 30. Google Scholar
  33. Wang, X., Han, T., & Yan, S. (2009). An hog-lbp human detector with partial occlusion handling. In ICCV. Google Scholar
  34. Wu, B., & Nevatia, R. (2005). Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors. In ICCV. Google Scholar
  35. Wu, Q., Luo, X., Li, H., & Liu, P. (2010). An improved multi-scale retinex algorithm for vehicle shadow elimination based on variational kimmel. In SWUATC. Google Scholar
  36. Xiao, J., Cheng, H., Han, F., & Sawhney, H. (2008a). Geo-spatial aerial video processing for scene understanding and object tracking. In CVPR. Google Scholar
  37. Xiao, J., Yang, C., Han, F., & Cheng, H. (2008b). Vehicle and person tracking in aerial videos. In Multimodal Technologies for Perception of Humans. Google Scholar
  38. Xiao, J., Cheng, H., Sawhney, H., & Han, F. (2010). Vehicle detection and tracking in wide field-of-view aerial video. In CVPR. Google Scholar
  39. Xu, L., Qi, F., & Jiang, R. (2006). Shadow removal from a single image. Intelligent Systems Design and Applications, 2. Google Scholar
  40. Yahyanejad, S., Wischounig-Strucl, D., Quaritsch, M., & Rinner, B. (2010). Incremental mosaicking of images from autonomous, small-scale uavs. In AVSS. Google Scholar
  41. Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking a survey. ACM Computing Surveys, 38. Google Scholar
  42. Yoneyama, A., Yeh, C. H., & Jay Kuo, C. C. (2003). Moving cast shadow elimination for robust vehicle extraction based on 2d joint vehicle/shadow models. In AVSS. Google Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  1. 1.University of Central FloridaOrlandoUSA

Personalised recommendations