Abstract
Early delineation of the most salient portions of a temporal image stream (e.g., a video) could serve to guide subsequent processing to the most important portions of the data at hand. Toward such ends, the present paper documents an algorithm for spatiotemporal salience detection. The algorithm is based on a definition of salient regions as those that differ from their surrounding regions, with the individual regions characterized in terms of 3D, (x,y,t), measurements of visual spacetime orientation. The algorithm has been implemented in software and evaluated empirically on a publically available database for visual salience detection. The results show that the algorithm outperforms a variety of alternative algorithms and even approaches human performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kandel, E., Schwartz, J.: Principles of Neuroscience. Elsevier, NY (1996)
Derpanis, K., Wildes, R.: Spacetime texture representation and recognition based on a spatiotemporal orientation analysis. PAMI 34, 1193–1205 (2012)
Kullback, S.: Information Theory and Statistics. Dover, Mineola (1968)
Picard, M.: Background subtraction techniques: A review. In: SMC, pp. 3099–3104 (2004)
Stauffer, C., Grimson, E.: Adaptive background mixture models for real-time tracking. In: CVPR, pp. 2246–2252 (1999)
Elgammal, A., Duraiswami, R., Harwood, D., Davis, L.: Background and foreground modeling using nonparametric kernel density for visual surveillance. Proc. IEEE 90, 1151–1163 (2002)
Monnet, A., Mittal, A., Paragios, N., Ramesh, V.: Background modeling and subtraction for a moving observer. In: ICCV, pp. 1305–1312 (2003)
Sheikh, Y., Shah, M.: Bayesian modeling of dynamic scenes for object detection. PAMI 27, 1778–1792 (2005)
Hayman, E., Eklundh, J.: Statistical background subtraction for a moving observer. In: ICCV (2003)
Ren, Y., Chua, C., Ho, Y.: Motion detection with nonstationary background. MVA 13, 332–343 (2003)
Wixson, L.: Detecting salient motion by accumulating directionally-consistent flow. PAMI 22, 774–780 (2000)
Bugeau, A., Perez, P.: Detection and segmentation of moving objects in highly dynamic scenes. In: CVPR (2007)
Itti, L., Baldi, P.: Bayesian surprise attracts human attention. Vis. Res. 49, 1295–1306 (2009)
Bruce, N., Tsotsos, J.: Towards a hierarchical representation of visual saliency. In: WAPCV, pp. 98–111 (2009)
Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. PAMI 20, 1254–1259 (1998)
Doretto, K., Chiuso, A., Wu, Y., Soatto, S.: Dynamic textures. IJCV 51, 91–109 (2003)
Mahadevan, V., Vasconcelos, N.: Spatiotemporal saliency in dynamic scenes. PAMI 32, 171–177 (2010)
Bence, M., Olveczky, P., Baccus, S.: Segregation of object and background motion in the retina. Nature (2003)
Heeger, D.: Model for the extraction of image flow. JOSA-A 2, 1455–1471 (1987)
Granlund, G., Knuttson, H.: Signal Processing for Computer Vision. Kluwer, Norwell (1995)
Simoncelli, E., Heeger, D.: A model of neuronal responses in visual area MT. Vis. Res. 38 (1996)
Chomat, O., Crowley, J.: Probabilistic recognition of activity using local appearance. In: CVPR, pp. 104–109 (1999)
Derpanis, K., Sizintsev, M., Cannons, K., Wildes, R.: Efficient action spotting based on a spacetime orientation structure representation. In: CVPR (2010)
Zaharescu, A., Wildes, R.: Anomalous Behaviour Detection Using Spatiotemporal Oriented Energies, Subset Inclusion Histogram Comparison and Event-Driven Processing. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 563–576. Springer, Heidelberg (2010)
Sadanand, S., Corso, J.: Action bank: A high-level representation of activity in video. In: CVPR (2012)
Derpanis, K., Lecce, M., Daniilidis, K., Wildes, R.: Dynamic scene understanding: The role of orientation features in space and time in scene classification. In: CVPR (2012)
Klaser, A., Marszalek, M., Schmid, C.: A spatiotemporal descriptor based on 3d-gradients. In: BMVC (2008)
Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: CVPR (2008)
Derpanis, K., Sizintsev, M., Cannons, K., Wildes, R.: Action spotting and recognition based on a spatiotemporal orientation analysis. PAMI (to appear)
Freeman, W., Adelson, E.: The design and use of steerable filters. PAMI 13, 891–906 (1991)
Derpanis, K., Wildes, R.: Dynamic texture recognition based on distributions of spacetime oriented structure. In: CVPR (2010)
Lindeburg, T.: Scale-Space Theory in Computer Vision. Kluwer, Norwell (1993)
Bracewell, R.: The Fourier Transform and its Applications. McGraw-Hill, NY (2000)
Pearce, P., Pearce, S.: The Polyhedra Primer. Van Nostrand Reinhold, NY (1978)
Jahne, B.: Digital Image Processing. Springer, Berlin (2005)
Viola, P., Jones, M.J.: Robust real-time face detection. IJCV 57, 137–154 (2004)
Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. Wiley, NY (2000)
Koenderink, J.: The structure of images. Biological Cybernetics 50, 363–370 (1984)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zaharescu, A., Wildes, R. (2013). Spatiotemporal Salience via Centre-Surround Comparison of Visual Spacetime Orientations. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds) Computer Vision – ACCV 2012. ACCV 2012. Lecture Notes in Computer Science, vol 7726. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37431-9_41
Download citation
DOI: https://doi.org/10.1007/978-3-642-37431-9_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-37430-2
Online ISBN: 978-3-642-37431-9
eBook Packages: Computer ScienceComputer Science (R0)