Anomaly Detection and Knowledge Transfer in Automatic Sports Video Annotation

  • I. Almajai
  • F. Yan
  • T. de Campos
  • A. Khan
  • W. Christmas
  • D. Windridge
  • J. Kittler

Abstract

A key question in machine perception is how to adaptively build upon existing capabilities so as to permit novel functionalities. Implicit in this are the notions of anomaly detection and learning transfer. A perceptual system must firstly determine at what point the existing learned model ceases to apply, and secondly, what aspects of the existing model can be brought to bear on the newly-defined learning domain. Anomalies must thus be distinguished from mere outliers, i.e. cases in which the learned model has failed to produce a clear response; it is also necessary to distinguish novel (but meaningful) input from misclassification error within the existing models. We thus apply a methodology of anomaly detection based on comparing the outputs of strong and weak classifiers [10] to the problem of detecting the rule-incongruence involved in the transition from singles to doubles tennis videos. We then demonstrate how the detected anomalies can be used to transfer learning from one (initially known) rule-governed structure to another. Our ultimate aim, building on existing annotation technology, is to construct an adaptive system for court-based sport video annotation.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Almajai, I., Kittler, J., de Campos, T., Christmas, W., Yan, F., Windridge, D., Khan, A.: Ball event recognition using HMM for automatic tennis annotation. In: Proc. Int. Conf. on Image Processing, Hong Kong, September 26-29 (2010) (in Press)Google Scholar
  2. 2.
    Burget, L., Schwarz, P., Matejka, P., Hannemann, M., Rastrow, A., White, C., Khudanpur, S., Hermansky, H., Cernock, J.: Combination of strongly and weakly constrained recognizers for reliable detection of OOVs. In: Proc. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 4081–4084 (2008)Google Scholar
  3. 3.
    de Campos, T., Barnard, M., Mikolajczyk, K., Kittler, J., Yan, F., Christmas, W., Windridge, D.: An evaluation of bags-of-words and spatio-temporal shapes for action recognition. In: Proc. of the 10th IEEE Workshop on Applications of Computer Vision, Kona, Hawaii, January 5-6 (2011)Google Scholar
  4. 4.
    Khan, A., Windridge, D., de Campos, T., Kittler, J., Christmas, W.: Lattice-based anomaly rectification for sport video annotation. In: Proc. ICPR (2010)Google Scholar
  5. 5.
    Kittler, J., Christmas, W.J., Yan, F., Kolonias, I., Windridge, D.: A memory architecture and contextual reasoning for cognitive vision. In: Proc. SCIA, pp. 343–358 (2005)Google Scholar
  6. 6.
    Kläser, A., Marszałek, M., Schmid, C.: A spatio-temporal descriptor based on 3D-gradients. In: 19th British Machine Vision Conference, pp. 995–1004 (2008)Google Scholar
  7. 7.
    Kolonias, I.: Cognitive Vision Systems for Video Understanding and Retrieval. PhD thesis, University of Surrey (2007)Google Scholar
  8. 8.
    Tommasi, T., Caputo, B.: The more you know, the less you learn: from knowledge transfer to one-shot learning of object categories. In: British Machine Vision Conference (2009)Google Scholar
  9. 9.
    Wang, H., Ullah, M.M., Käser, A., Laptev, I., Schmid, C.: Evaluation of local spatio-temporal features for action recognition. In: 20th British Machine Vision Conference (2009)Google Scholar
  10. 10.
    Weinshall, D., Hermansky, H., Zweig, A., Luo, J., Jimison, H., Ohl, F., Pavel, M.: Beyond novelty detection: Incongruent events, when general and specific classifiers disagree. In: Advances in Neural Information Processing Systems (NIPS) (December 2009)Google Scholar
  11. 11.
    Yan, F., Christmas, W., Kittler, J.: Layered data association using graph-theoretic formulation with application to tennis ball tracking in monocular sequences. Transactions on Pattern Analysis and Machine Intelligence (2008)Google Scholar
  12. 12.
    Young, S., Kershaw, D., Odell, J., Ollason, D., Valtchev, V., Woodland, P.: The HTK Book Version 3.0. Cambridge University Press, Cambridge (2000)Google Scholar
  13. 13.
    Zimmermann, K., Svoboda, T., Matas, J.: Adaptive parameter optimization for real-time tracking. In: Proc. ICCV, Workshop on Non-rigid Registration and Tracking through Learning (2007)Google Scholar
  14. 14.
    Zimmermann, K., Svoboda, T., Matas, J.: Simultaneous learning of motion and appearance. In: The 1st International Workshop on Machine Learning for Vision-based Motion Analysis, Marseilles (2008), In Conjunction with ECCV Google Scholar
  15. 15.
    Zweig, A., Weinshall, D.: Exploiting object hierarchy: Combining models from different category levels. In: IEEE 11th International Conference on Computer Vision (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • I. Almajai
    • 1
  • F. Yan
    • 1
  • T. de Campos
    • 1
  • A. Khan
    • 1
  • W. Christmas
    • 1
  • D. Windridge
    • 1
  • J. Kittler
    • 1
  1. 1.CVSSP, University of SurreyGuildfordUK

Personalised recommendations