Skip to main content
SpringerLink
Log in

Requirements for multimedia metadata schemes in surveillance applications for security

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Surveillance for security requires communication between systems and humans, involves behavioural and multimedia research, and demands an objective benchmarking for the performance of system components. Metadata representation schemes are extremely important to facilitate (system) interoperability and to define ground truth annotations for surveillance research and benchmarks. Surveillance places specific requirements on these metadata representation schemes. This paper offers a clear and coherent terminology, and uses this to present these requirements and to evaluate them in three ways: their fitness in breadth for surveillance design patterns, their fitness in depth for a specific surveillance scenario, and their realism on the basis of existing schemes. It is also validated that no existing metadata representation scheme fulfils all requirements. Guidelines are offered to those who wish to select or create a metadata scheme for surveillance for security.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Alexander C (1977) A Pattern Language: Towns, Buildings, Construction

  2. Annesley J, Colombo A, Orwell J, Velastin S (2007) A profile of MPEG-7 for visual surveillance, IEEE Int. Conf. AVSS, 482–487

  3. Bouma H, Vogels J, Aarts O, Kruszynski C, Wijn R, Burghouts G (2013) Behavioral profiling in CCTV cameras by combining multiple subtle suspicious observations of different surveillance operators, Proc. SPIE 8745

  4. Burghouts GJ, Marck J (2011) Reasoning about threats: from observables to situation assessment. IEEE Trans Syst Man Cybern 41(5):608–616

    Article  Google Scholar 

  5. Buschmann F, Meunier R, Rohnert H, Sommerlad P (1996) Pattern-Oriented Software Architecture, Volume 1: A System of Patterns. John Wiley & Sons

  6. CAVIAR: Context aware vision using image-based active recognition. http://homepages.inf.ed.ac.uk/rbf/CAVIAR/

  7. CVML: Computer vision markup language. http://homepages.inf.ed.ac.uk/tlist/cvml/spec.html

  8. Doermann D, Mihalcik D (2000) Tools and techniques for video performance evaluation. ICPR 4:167–170

    Google Scholar 

  9. Dutch top sector high tech systems & materials: Roadmap security, passive sensors. http://www.htsm.nl/Roadmaps/Security

  10. ETISIO: Video understanding evaluation. http://www-sop.inria.fr/orion/ETISEO/

  11. Fisher RB (2004) The PETS04 surveillance ground-truth data sets. Proc. 6th IEEE Int. Workshop on Performance Evaluation of Tracking and Surveillance, pp 1–5

  12. Francois AR, Nevatia R, Hobbs J, Bolles RC, Smith JR (2005) VERL: an ontology framework for representing and annotating video events. Multimedia, IEEE 12(4):76–86

    Google Scholar 

  13. HUMAN ML: Human markup language. https://www.oasis-open.org/committees/download.php/60/HM.Primary-Base-Spec-1.0.html

  14. I-LIDS: Imagery library for intelligent detection systems. Home Office, UK

  15. INCOSE, a consensus of the INCOSE fellows. http://www.incose.org

  16. Kester LJHM (2008) Designing networked adaptive interactive hybrid systems. IEEE Multisensor Fusion and Integration for Intelligent Systems, 2008, MFI 2008, pp 516–521

  17. Kipp M (2013) Anvil: the video research annotation tool. http://www.anvil-software.org/ accessed January 4th 2013

  18. Kipp M (2013) Anvil 4.0 Annotation of video and spoken language

  19. La Vigne NG (2011) Evaluating the use of public surveillance cameras for crime control and prevention

  20. Lenat DB, Guha RV (1990) Building large knowledge-based systems: representation and inference in the CYC project. Addison–Wesley, Reading

    Google Scholar 

  21. List T, Fisher RB (2004) CVML-an XML-based computer vision markup language. Int Conf Pattern Recog (ICPR) 1:789–792

    Google Scholar 

  22. Lyon D (2007) Surveillance studies: an overview. Polity Press, Cambridge

    Google Scholar 

  23. Mariano VY, Min J, Park J-H, Kasturi R, Mihalcik D, Li H et al (2002) Performance evaluation of object detection algorithms. ICPR 3:965–969

    Google Scholar 

  24. Masolo C, Borgo S, Gangemi A, Guarino N, Oltramari A (2003) Ontology library (final). IST Project 2001–33052 WonderWeb Deliverable D18

  25. Milestone. http://www.milestonesys.com/

  26. MPEG-7: Moving pictures expert group

  27. Neely H (2010) Modeling Threat Behaviors in Surveillance Video Metadata for Detection using an Analogical Reasoner, IEEE Aerospace conference

  28. Nghiem AT, Bremond F, Thonnat M, Valentin V (2007) ETISEO, performance evaluation for video surveillance systems. IEEE Conference On Advanced Video and Signal Based Surveillance, AVSS 2007, pp 476–481

  29. Niles I, Pease A (2001) Towards a Standard Upper Ontology. In: Welty C, Smith B (eds) Proceedings of the 2nd International Conference on Formal Ontology in Information Systems (FOIS-2001), Ogunquit, Maine, October 17–19, 2001

  30. Noldus. www.noldus.com

  31. Online resource pickpocket video

  32. ONVIF: Open network video interface forum. http://www.onvif.org/Home.aspx

  33. Over P, Awad G, Fiscus J, Antonishek B, Michel M, Smeaton AF, et al (2011) Proceedings of TRECVID 2010—An overview of the goals, tasks, data, evaluation mechanisms, and metrics, Gaithersburg, Md., USA

  34. PETS: Performance evaluation of tracking and surveillance. http://pets2012.net

  35. PSIA: Physical security interoperability alliance. http://www.psialliance.org/

  36. SanMiguel JC, Martinez JM, Garcia A (2009) An ontology for event detection and its application in surveillance video, IEEE Int. Conf. AVSS, pp 220–225

  37. Schallauer P, Bailer W, Hofmann A, Mörzinger R (2009) SAM: An interoperable metadata model for multimodal surveillance applications. Proc. SPIE, 7344

  38. Sowa JF (1976) Conceptual graphs for a database interface. IBM J Res Dev 20(4):336–357

    Article  MATH  MathSciNet  Google Scholar 

  39. Sowa JF (1984) Conceptual graphs. Information Processing in Mind and Machine, 39–44

  40. Steinberg AN, Bowman CL, White FE (1999) Revisions to the JDL data fusion model. Environmental Research Institute of Michigan Arlington VA

  41. Surveillance of Unattended Baggage and the Identification and Tracking of the Owner (SUBITO) consortium (December 2011), SUBITO Deliverable D100.2: Final Report

  42. Suzić R (2005) A generic model of tactical plan recognition for threat assessment. Proc. SPIE

  43. TRECVID: TREC video retrieval evaluation. http://trecvid.nist.gov/

  44. UK Home Office, Invitation to Tender Efficient Archive Retrieval & Auto Searching (EARS) CONTEST Project http://www.homeoffice.gov.uk/publications/science-research-statistics/research-statistics/home-office-science/eoi-ears-hos, Accessed June 2012

  45. VIPER. http://viper-toolkit.sourceforge.net/

  46. Westermann U, Jain R (2007) Toward a common event model for multimedia applications. IEEE Multimedia 14(1):19–29

    Article  Google Scholar 

Download references

Acknowledgements

This work was performed as independent research of the applied research programme Dutch Top sector High Tech Systems & Materials: Roadmap Security, Passive Sensors. [9]. The authors thank Aart Beukers (Eye-D Security Experts) and the Amsterdam police for kindly providing the instruction video.

Author information

Authors and Affiliations

  1. TNO, The Hague, The Netherlands

    J. van Rest, M. Grootjen, R. Wijn, O. Aarts, M. L. Roelofs, G. J. Burghouts, H. Bouma, L. Alic & W. Kraaij

  2. Radboud University, Nijmegen, The Netherlands

    F. A. Grootjen

Authors
  1. J. van Rest
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. F. A. Grootjen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Grootjen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Wijn
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. O. Aarts
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. M. L. Roelofs
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. G. J. Burghouts
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. H. Bouma
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. L. Alic
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. W. Kraaij
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. van Rest.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MPG 7337 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

van Rest, J., Grootjen, F.A., Grootjen, M. et al. Requirements for multimedia metadata schemes in surveillance applications for security. Multimed Tools Appl 70, 573–598 (2014). https://doi.org/10.1007/s11042-013-1575-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1575-9

Keywords

Search

Navigation