Abstract
In this paper, we propose a method for detecting humans and vehicles in imagery taken from a UAV. This is a challenging problem due to a limited number of pixels on target, which makes it more difficult to distinguish objects from background clutter, and results in much larger search space. We propose a method for constraining the search based on a number of geometric constraints obtained from the metadata. Specifically, we obtain the orientation of ground plane normal, the orientation of shadows cast by out of plane objects in the scene, and the relationship between object heights and the size of their corresponding shadows. We use the aforementioned information in a geometry-based shadow, and ground-plane normal blob detector, which provides an initial estimation for locations of shadow casting out of plane (SCOOP) objects in the scene. These SCOOP candidate locations are then classified as either human or clutter using a combination of wavelet features and a Support Vector Machine. To detect vehicles, we similarly find potential vehicle candidates by combining SCOOP and inverted-SCOOP candidates and then classify them using wavelet features and SVM. Our method works on a single frame, and unlike motion detection based methods, it bypasses the entire pipeline of registration, motion detection, and tracking. This method allows for detection of stationary and slowly moving humans and vehicles while avoiding the search across the entire image, allowing accurate and fast localization. We show impressive results on sequences from VIVID and CLIF datasets and provide comparative analysis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bi, S., Liang, D., Shen, X., & Wang, Q. (2007). Human cast shadow elimination method bad on orientation information measures. In ICAL.
Bissacco, A., & Yang, M. H. (2007). Detecting humans via their pose. In NIPS.
Bose, B., & Grimson, E. (2004). Improving object classification in far-field video. In CVPR.
Breckon, T., Barnes, S., Eichner, M., & Wahren, K. (2009). Autonomous real-time vehicle detection from a medium-level uav. In UAVS.
Chang, C., & Lin, C. (2001). LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
Chang, J. C., Hu, W. F., Hsieh, J. W., & Chen, Y. S. (2002). Shadow elimination for effective moving object detection with Gaussian models. In ICPR.
Chen, Y. T., Chen, C. S., Hung, Y. P., & Chang, K. Y. (2009). Multi-class multi-instance boosting for part-based human detection. In ICCV.
Cheng, H., Butler, D., & Basu, C. (2006). ViTex: video to tex and its application in aerial video surveillance. In CVPR.
Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In CVPR (Vol. 1).
Felzenszwalb, P., McAllester, D., & Ramanan, D. (2008). A discriminatively trained, multiscale, deformable part model. In CVPR.
Finlayson, G., Hordley, S., Lu, C., & Drew, M. (2006). On the removal of shadows from images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1).
Gaszczak, A., Breckon, T., & Han, J. (2011). Real-time people and vehicle detection from uav imagery. SPIE.
Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd ed.). Cambridge: Cambridge University Press. ISBN: 0521540518.
Hsieh, J. W., Yu, S. H., Chen, Y. S., & Hu, W. F. (2004). A shadow elimination method for vehicle analysis. In ICPR.
Hu, H., Huang, Y. Q., & Li, L. M. (2010). Moving vehicle shadow elimination approach based on mark growing of multi-feature fusion. In ICACIA.
Kembhavi, A., Harwood, D., & Davis, L. (2011). Vehicle detection using partial least squares. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Kluckner, S., Mauthner, T., Roth, P. M., & Bischof, H. (2009). Semantic classification in aerial imagery by integrating appearance and height information. In ACCV.
Leibe, B., Seemann, E., & Schiele, B. (2005). Pedestrian detection in crowded scenes. In CVPR.
Liu, Z., Zhao, F., & Yang, H. (2010). A new method of moving shadow elimination combining texture and chrominance of moving foreground region based on criterion. In WCICA.
Martel-Brisson, N., & Zaccarin, A. (2005). Moving cast shadow detection from a Gaussian mixtrue shadow model. In CVPR.
Mikolajczyk, K. C. S., & Zisserman, A. (2004). Human detection based on a probabilistic assembly of robust part detectors. In ECCV.
Miller, A., Babenko, P., Hu, M., & Shah, M. (2007). Person tracking in UAV video. In CLEAR.
Panagopoulos, A., Samaras, D., & Paragios, N. (2009). Robust shadow and illumination estimation using a mixture model. In CVPR.
Porikli, F., & Thornton, J. (2005). Shadow flow: a recursive method to learn moving cast shadows. In ICCV.
Prati, A., Mikic, I., Trivedi, M. M., & Cucchiara, R. (2003). Detecting moving shadows: algorithms and evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25.
Quaritsch, M., Kruggl, K., Wischounig-Strucl, D., Bhattacharya, S., Shah, M., & Rinner, B. (2010). Networked uavs as aerial sensor network for disaster management applications. Elektrotechnik und Informationstechnik, 127.
Reda, I., & Anreas, A. (2003). Solar position algorithm for solar radiation applications (NREL Report No. TP-560-34302).
Rudol, P., & Doherty, P. (2008). Human body detection and geolocalization for uav search and rescue missions using color and thermal imagery. In IEEE aerospace.
Sabzmeydani, P., & Mori, G. (2007). Detecting pedestrians by learning shapelet features. In CVPR.
Sokalski, J., & Breckon, T. (2010). Automatic salient object detection in uav imagery. In UAVS.
Tian, T. P., & Sclaroff, S. (2010). Fast multi-aspect 2d human detection. In ECCV.
Tuzel, O., Porikli, F., & Meer, P. (2008). Pedestrian detection via classification on Riemannian manifolds. Pattern Analysis and Machine Intelligence, 30.
Wang, X., Han, T., & Yan, S. (2009). An hog-lbp human detector with partial occlusion handling. In ICCV.
Wu, B., & Nevatia, R. (2005). Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors. In ICCV.
Wu, Q., Luo, X., Li, H., & Liu, P. (2010). An improved multi-scale retinex algorithm for vehicle shadow elimination based on variational kimmel. In SWUATC.
Xiao, J., Cheng, H., Han, F., & Sawhney, H. (2008a). Geo-spatial aerial video processing for scene understanding and object tracking. In CVPR.
Xiao, J., Yang, C., Han, F., & Cheng, H. (2008b). Vehicle and person tracking in aerial videos. In Multimodal Technologies for Perception of Humans.
Xiao, J., Cheng, H., Sawhney, H., & Han, F. (2010). Vehicle detection and tracking in wide field-of-view aerial video. In CVPR.
Xu, L., Qi, F., & Jiang, R. (2006). Shadow removal from a single image. Intelligent Systems Design and Applications, 2.
Yahyanejad, S., Wischounig-Strucl, D., Quaritsch, M., & Rinner, B. (2010). Incremental mosaicking of images from autonomous, small-scale uavs. In AVSS.
Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking a survey. ACM Computing Surveys, 38.
Yoneyama, A., Yeh, C. H., & Jay Kuo, C. C. (2003). Moving cast shadow elimination for robust vehicle extraction based on 2d joint vehicle/shadow models. In AVSS.
Acknowledgements
This research was partially supported by the Harris corporation and Defense Advanced Research Projects Agency (DARPA) under Contract No. HR0011-10-C-0112. Any opinions, findings, and conclusions expressed in this material are those of the authors and do not necessarily reflect the views of the Harris corporation or DARPA.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Reilly, V., Solmaz, B. & Shah, M. Shadow Casting Out Of Plane (SCOOP) Candidates for Human and Vehicle Detection in Aerial Imagery. Int J Comput Vis 101, 350–366 (2013). https://doi.org/10.1007/s11263-012-0580-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-012-0580-1