Abstract
We address the problem of detecting irregularities in visual data, e.g., detecting suspicious behaviors in video sequences, or identifying salient patterns in images. The term “irregular” depends on the context in which the “regular” or “valid” are defined. Yet, it is not realistic to expect explicit definition of all possible valid configurations for a given context. We pose the problem of determining the validity of visual data as a process of constructing a puzzle: We try to compose a new observed image region or a new video segment (“the query”) using chunks of data (“pieces of puzzle”) extracted from previous visual examples (“the database”). Regions in the observed data which can be composed using large contiguous chunks of data from the database are considered very likely, whereas regions in the observed data which cannot be composed from the database (or can be composed, but only using small fragmented pieces) are regarded as unlikely/suspicious. The problem is posed as an inference process in a probabilistic graphical model. We show applications of this approach to identifying saliency in images and video, for detecting suspicious behaviors and for automatic visual inspection for quality assurance.
Similar content being viewed by others
References
Bart, E. and Ullman, S. 2004. Class-based matching of object parts. In Video Register04, p. 173.
Boiman, O. and Irani, M. 2005. Detecting irregularities in images and in video. In ICCV05, pp. I:462–469.
Efros, A.A. and Leung, T.K. 1999. Texture synthesis by non-parametric sampling. In ICCV, pp. 1033–1038.
Felzenszwalb, P. and Huttenlocher, D. 2005. Pictorial structures for object recognition. IJCV, 61(1):55–79.
Fergus, R., Perona, P., and Zisserman, A. 2003. Object class recognition by unsupervised scale-invariant learning. In CVPR03.
Freeman, W., Pasztor, E., and Carmichael, O. 2000. Learning low-level vision. IJCV, 40:25–47.
Honda, T. and Nayar, S. 2001. Finding ‘anomalies’ in an arbitrary image, pp. II:516–523.
Itti, L., Koch, C., and Niebur, E. 1998. A model of saliency-based visual attention for rapid scene analysis. PAMI.
Ivanov, Y. and Bobick, A. 1999. Recognition of multi-agent interaction in video surveillance. In ICCV.
Jurie, F. and Triggs, B. 2005. Creating efficient codebooks for visual recognition. In ICCV05, pp. I:604–610.
Laptev, I. and Lindeberg, T. 2003. Space-time interest points. In ICCV03, pp. 432–439.
Leibe, B., Leonardis, A., and Schiele, B. 2004. Combined object categorization and segmentation with an implicit shape model. In ECCV04 Workshop on Statistical Learning in CV.
Lowe, D. 2004. Distinctive image features from scale-invariant keypoints. IJCV, 60:91–110.
Shechtman, E. and Irani, M. 2005. Space-time behavior based correlation, pp. I:405–412.
Sivic, J. and Zisserman, A. 2003. Video google: A text retrieval approach to object matching in videos. In ICCV.
Stauffer, C. and Grimson, E. 2000. Learning patterns of activity using real-time tracking. PAMI.
Wexler, Y., Shechtman, E., and Irani, M. 2004. Space-time video completion. In CVPR04, pp. I:120–127.
Yedidia, J.S., Freeman, W.T., and Weiss, Y. 2003. Understanding belief propagation and its generalizations, pp. 239–269.
Zhong, H., Shi, J., and Visontai, M. 2004. Detecting unusual activity in video. In CVPR04, pp. II:819–826.
Author information
Authors and Affiliations
Additional information
Patent Pending
Rights and permissions
About this article
Cite this article
Boiman, O., Irani, M. Detecting Irregularities in Images and in Video. Int J Comput Vision 74, 17–31 (2007). https://doi.org/10.1007/s11263-006-0009-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-006-0009-9