Skip to main content
SpringerLink
Log in

Filtering Surveillance Image Streams by Interactive Machine Learning

  • Chapter
Multimedia Analysis, Processing and Communications

Part of the book series: Studies in Computational Intelligence ((SCI,volume 346))

Abstract

As surveillance cameras become widespread, filters are needed to process large streams of image data and spot interesting events. Programmed image filters generally result in low to medium performing solutions. Data-derived filters perform better in that they tap on selected image features, but require a per-sensor effort by an analyst or a machine learning expert. This contribution addresses filter shaping as a data-driven process that is ‘placed in the hands of many end-users’ with extensive domain knowledge but no expertise in machine learning. The focus is on interactive machine learning technologies as a means to achieve self programming and specialization of image filters that learn to search images by their content, sequential order, and temporal attributes. We describe and assess the performance of two interactive algorithms designed and implemented for a real case study in process monitoring for nuclear safeguards. Experiments show that interactive machine learning helps detect safeguards relevant event while significantly reducing the number of false positives.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 299.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 379.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 379.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Versino, C., Stringa, E., Gonçalves, J.G.M.: Review of Surveillance Images using Classification Techniques in a Relevance Feedback Context. In: Proc. Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance (VS-PETS2003), pp. 46–53 (2003)

    Google Scholar 

  2. Fails, J.A., Olsen, D.R.: Light Widgets: Interacting in Every-day Spaces. In: Intelligent User Interfaces, IUI 2002, pp. 63–69. ACM, New York (2002)

    Chapter  Google Scholar 

  3. Fails, J.A., Olsen, D.R.: Interactive Machine Learning. In: Intelligent User Interfaces, IUI 2003. ACM, New York (2003)

    Google Scholar 

  4. Fails, J.A., Olsen, D.R.: A Design Tool for Camera-based Interaction. In: Human Factors in Computing Systems, CHI 2003, pp. 449–456. ACM, New York (2003)

    Chapter  Google Scholar 

  5. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)

    Google Scholar 

  6. http://www.rulequest.com/see5-info.html

  7. Stumpf, S., Sullivan, E., Fitzhenry, E., Oberst, I., Wong, W.-K., Burnett, M.: Integrating rich user feedback into intelligent user interfaces. In: Proc. 13th International Conference on Intelligent user Interfaces, pp. 50–59 (2008)

    Google Scholar 

  8. Stumpf, S., Rajaram, V., Li, L., Wong, W.K., Burnetta, M., Dietterich, T., Sullivan, E., Herlocker, J.: Interacting meaningfully with machine learning systems: Three experiments. International Journal of Human-Computer Studies 67(8), 639–662 (2009)

    Article  Google Scholar 

  9. Ware, M., Frank, E., Holmes, G., Hall, M., Witten, I.H.: Interactive machine learning: letting users build classifiers. International Journal of Human-Computer Studies 56(3), 281–292 (2002)

    Google Scholar 

  10. Zhou, X., Huang, T.S.: Relevance feedback in image retrieval: A comprehensive review. Multimedia Systems 8(6), 1432–1882 (2003)

    Article  Google Scholar 

  11. Tong, S., Koller, D.: Support vector machine active learning with applications to text classification. In: 17th International Conference on Machine Learning, pp. 401–412 (2000)

    Google Scholar 

  12. Tong, S., Chang, E.: Support vector machine active learning for image retrieval. In: Proc. of ACM Int.Conf. on Multimedia, pp. 107–118 (2001)

    Google Scholar 

  13. Vapnik, V.: Statistical Learning Theory. Wiley, Chichester (1998)

    MATH  Google Scholar 

  14. Tao, D., Tang, X., Li, X., Wu, X.: Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE Trans. Pattern Recognition and Machine Intelligence 28(7), 1088–1099 (2006)

    Article  Google Scholar 

  15. Markou, M., Singh, S.: Novelty Detection: A Review – Part 1: Statistical Approaches. Signal Processing 83 (2003)

    Google Scholar 

  16. Markou, M., Singh, S.: Novelty Detection: A Review – Part 2: Neural network based approaches. Signal Processing 83, 2499–2521 (2003)

    Article  MATH  Google Scholar 

  17. Mcivor, A.M.: Background subtraction techniques. In: Proc. of Image and Vision Computing, pp. 147–153 (2000)

    Google Scholar 

  18. Stauffer, C., Grimson, E.: Learning Patterns of Activity Using Real-Time Tracking. IEEE Trans. Pattern Recognition and Machine Intelligence 22(8), 747–757 (2000)

    Article  Google Scholar 

  19. Ahmed Elgammal, A., Duraiswami, R., Harwood, D., Davis, L.S.: Background and Foreground Modeling Using Nonparametric Kernel Density Estimation for Visual Surveillance. Proceedings of the IEEE 90(7), 1151–1163 (2002)

    Article  Google Scholar 

  20. Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: Principles and practice of background maintenance. In: Proc. 7th IEEE Int. Conf. on Computer Vision, pp. 255–261 (1999)

    Google Scholar 

  21. Messelodi, S., Modena, C.M., Zanin, M.: A Computer Vision System for the Detection and Classification of Vehicles at Urban Road Intersections. Pattern Analysis & Applications 8(1), 17–31 (2005)

    Article  MathSciNet  Google Scholar 

  22. Liu, Y., Yao, H., Gao, W., Chen, X., Zhao, D.: Nonparametric Background Generation. In: Proc. 18th Int. Conf. on Pattern Recognition, vol. 4, pp. 916–919 (2006)

    Google Scholar 

  23. Morris, B.T., Trivedi, M.M.: A Survey of Vision-Based Trajectory Learning and Analysis for Surveillance. IEEE Trans. Circuits and Systems for Video Technology 18(8), 1114–1127 (2008)

    Article  Google Scholar 

  24. Makris, D., Ellis, T.: Learning semantic scene models from observing activity in visual surveillance. IEEE Trans. Systems, Man, Cybernetics B 35(3), 397–408 (2005)

    Article  Google Scholar 

  25. Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM J. Computing Surveys 38(4), 13 (2006)

    Article  Google Scholar 

  26. Hu, W., Tan, T., Wang, L., Maybank, S.: A survey of visual surveillance of object motion and behaviors. IEEE Trans. System, Man, Cybernetics C 34(3), 334–352 (2004)

    Article  Google Scholar 

  27. Ewerth, R., Freisleben, B.: Semi-supervised learning for semantic video retrieval. In: Proc. 6th ACM Int. Conf. on Image and Video Retrieval, pp. 154–161 (2007)

    Google Scholar 

  28. Dorado, A., Calic, J., Izquierdo, E.: A rule-based video annotation system. IEEE Trans. on Circuits and Systems for Video Technology 14(5), 622–633 (2004)

    Article  Google Scholar 

  29. Iyengar, G., Nock, H.J.: Discriminative model fusion for semantic concept detection and annotation in video. In: Proc. 11th ACM Int. Conf. on Multimedia, pp. 255–258 (2003)

    Google Scholar 

  30. Qi, G.-J., Hua, X.-S., Rui, Y., Tang, J., Mei, T., Zhang, H.-J.: Correlative Multi-Label Video Annotation. In: Proc. 15th Int. Conf. on Multimedia, pp. 17–26 (2007)

    Google Scholar 

  31. Wang, M., Hua, X.-S., Hong, R., Tang, J., Qi, G.-J., Song, Y.: Unified Video Annotation via Multigraph Learning. IEEE Trans. Circuits and Systems for Video Technology 19(5), 733–746 (2009)

    Article  Google Scholar 

  32. GARS, http://www.canberra.com/pdf/Products/Systems_pdf/GARS-SS.pdf

  33. Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)

    Article  Google Scholar 

  34. Vaccaro, R., Dellepiane, S., Nicchiotti, G.: Content based database system: A solution to multimedia data management. In: Proc. of the Image and Video Content-Based Retrieval Workshop, pp. 89–95 (1998)

    Google Scholar 

  35. Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petrovic, D., Steele, D., Yanker, P.: Query by image and video content: The QBIC system. IEEE Computer, Los Alamitos (1995)

    Google Scholar 

  36. Brunelli, R., Mich, O.: Image retrieval by examples. IEEE Transactions on Multimedia 2, 164–171 (2000)

    Article  Google Scholar 

  37. Meilhac, C., Nastar, C.: Relevance feedback and category search in image databases. In: Proc IEEE International Conference on Multimedia Computing and Systems, vol. 1, pp. 512–517 (1999)

    Google Scholar 

  38. Cover, T., Hart, P.: Nearest Neighbor Pattern Classification. IEEE Trans. on Information Theory 13, 21–27 (1967)

    Article  MATH  Google Scholar 

  39. Hertz, J., Krogh, A., Palmer, R.G.: Introduction to the Theory of Neural Computation. Addison Wesley, Reading (1991)

    Google Scholar 

  40. Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L. (eds.): Feature Extraction, Foundations and Applications. Series Studies in Fuzziness and Soft Computing, Physica-Verlag. Physica-Verlag, Springer (2006)

    Google Scholar 

  41. Sullivan, G.J., Wiegand, T.: Video Compression—From Concepts to the H.264/AVC Standard. Proc. IEEE 93(1), 18–31 (2005)

    Article  Google Scholar 

  42. Mitra, S., Acharya, T.:  Gesture Recognition: A Survey. IEEE Trans. on Systems, Man, and Cybernetics, Part C: Applications and Reviews 37(3), 311–324 (2007)

    Article  Google Scholar 

  43. Brdiczka, O., Langet, M., Maisonnasse, J., Crowley, J.L.: Detecting Human Behavior Models From Multimodal Observation in a Smart Home. IEEE Trans. on Automation Science and Engineering (2003)

    Google Scholar 

  44. Ferrando, S., Gera, G., Massa, M., Regazzoni, C.: A New Method for Real Time Abandoned Object Detection and Owner Tracking. In: IEEE International Conference on Image Processing, pp. 3329–3332 (2006)

    Google Scholar 

  45. Trucco, E., Plakas, K.: Video Tracking: A Concise Survey. IEEE Journal of Oceanic Engineering 31(2), 520–529 (2006)

    Article  Google Scholar 

  46. Lombardi, P., Versino, C.: Tracking Nuclear Material at Low Frame Rate and Numerous False Detections. In: Proceedings of ICVS 2007 Vision Systems in the Real World: Adaptation, Learning, Evaluation - Conference, Bielefeld, Germany (2007)

    Google Scholar 

  47. Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

  48. Guédon, Y.: Estimating hidden semi-Markov chains from discrete sequences. Journal Comput. Graphical Statistics 12(3), 604–639 (2003)

    Article  Google Scholar 

  49. http://canberraeurisys.com.cn/PDF/gemini_surv.pdf

  50. Lombardi, P., Versino, C., Gonçalves, J.G.M., Heppleston, M., Tourin, L.: MM: The Markov Model Tool for Image Reviews. In: Proc. of the Institute of Nuclear Materials Management (2008)

    Google Scholar 

  51. IMS Research lowers video surveillance forecast but outlook still positive (March 2009), http://www.imsresearch.com/press_release_details.html&press_id=865

  52. http://www.securitysystemsnews.com/index.php?=p=article&id=ss200906FZVYeV

  53. Crowley, J.L., Hall, D., Emonet, R.: Autonomic Computer Vision Systems. In: Proc. 5th Int. Conf. On Computer Vision Systems (ICVS 2007), p. 182 (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. European Commission Joint Research Centre Nuclear Security Unit, Via Fermi 2749, 21027, Ispra, Italy

    Cristina Versino & Paolo Lombardi

  2. Computer Vision and Media Lab, DIS, University of Pavia, via Ferrata 1, 27100, Pavia, Italy

    Paolo Lombardi

Authors
  1. Cristina Versino
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paolo Lombardi
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Computer Engineering , Nanyang Technological University, 639798, Singapore

    Weisi Lin  & Dacheng Tao  & 

  2. Intelligent Systems Laboratory Systems Research Institute , Polish Academy of Sciences, Poland

    Janusz Kacprzyk

  3. Department of Computing , Hong Kong Polytechnic University, Hung Hom, Hong Kong

    Zhu Li

  4. School of Electronic Engineering and Computer Science, Queen Mary, University of London, London, U.K.

    Ebroul Izquierdo

  5. TCL-Thomson Electronics , Santa Clara, California

    Haohong Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Versino, C., Lombardi, P. (2011). Filtering Surveillance Image Streams by Interactive Machine Learning. In: Lin, W., Tao, D., Kacprzyk, J., Li, Z., Izquierdo, E., Wang, H. (eds) Multimedia Analysis, Processing and Communications. Studies in Computational Intelligence, vol 346. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19551-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-19551-8_10

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-19550-1

  • Online ISBN: 978-3-642-19551-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation