Advertisement

Multimedia Systems

, Volume 11, Issue 4, pp 320–339 | Cite as

An MPEG-7 scheme for semantic content modelling and filtering of digital video

  • Marios C. AngelidesEmail author
  • Harry Agius
Regular Paper

Abstract

Part 5 of the MPEG-7 standard specifies Multimedia Description Schemes (MDS); that is, the format multimedia content models should conform to in order to ensure interoperability across multiple platforms and applications. However, the standard does not specify how the content or the associated model may be filtered. This paper proposes an MPEG-7 scheme which can be deployed for digital video content modelling and filtering. The proposed scheme, COSMOS-7, produces rich and multi-faceted semantic content models and supports a content-based filtering approach that only analyses content relating directly to the preferred content requirements of the user. We present details of the scheme, front-end systems used for content modelling and filtering and experiences with a number of users.

Keywords

Multimedia Semantic content modelling MPEG-7 Content-based filtering Digital video 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Day, Y.F., Khokhar, A., Dagtas, S., Ghafoor, A.: A multi-level abstraction and modeling in video databases. Multimedia Syst. 7(5), 409–423 (1999)CrossRefGoogle Scholar
  2. 2.
    Al-Safadi, L., Getta, J.: Semantic content-based retrieval for video documents. In: Rahman, S.M. (ed.) Design and Management of Multimedia Information Systems: Opportunities & Challenges, pp. 165–200. IDEA Group Publishing, Hershey, PA (2001)Google Scholar
  3. 3.
    Löffler, J., Biatov, K., Eckes, C., Köhler, J.: iFinder: an MPEG-7-based retrieval system for distributed multimedia content. In: Proceedings of the 10th ACM International Conference on Multimedia (MM02), pp. 431–435. Juan-les-Pins, France (2002)Google Scholar
  4. 4.
    Jaimes, A., Omura, K., Nagamine, T., Hirata, K.: Memory cues for meeting video retrieval. In: Proceedings of ACM CARPE'04, pp. 74–85, New York, NY, USA (2004)Google Scholar
  5. 5.
    Bolle, R.M., Yeo, B.-L., Yeung, M.M.: Video query: research directions. IBM J. Res. Dev. 42(2) (1998)Google Scholar
  6. 6.
    Naphande, M.R., Huang, T.S.: A probabilistic framework for semantic video indexing, filtering, and retrieval. IEEE Trans. Multimedia 3(1), 141–151 (2001)CrossRefGoogle Scholar
  7. 7.
    Davis, M.: Media Streams: An iconic visual language for video annotation. Telektronikk 4.93, 59–71 (1993)Google Scholar
  8. 8.
    Tusch, R., Kosch, H., Böszörmenyi, L.: VIDEX: an integrated generic video indexing approach. In: Proceedings of the ACM Multimedia'00, pp. 448–451. Los Angeles, CA (2000)Google Scholar
  9. 9.
    Vendrig, J.W., M.: Interactive adaptive movie annotation. In: Proceedings of the 2002 IEEE International Conference on Multimedia and Expo, vol. 1, pp. 93–96 (2002)Google Scholar
  10. 10.
    Zhao, R., Grosky, W.I.: Bridging the semantic gap in image retrieval. In: Shih, T.K. (ed.). Distributed Multimedia Databases: Techniques and Applications, pp. 14–36. IDEA Group Publishing, Hershey, PA (2002)Google Scholar
  11. 11.
    Rowe, L.A., Boreczky, J.S., Eads, C.A.: Indices for user access to large video databases. In: Proceedings of the Storage and Retrieval for Image and Video Database II, Proceedings of the SPIE, vol. 2185, pp. 150–161 (1994)Google Scholar
  12. 12.
    Hartley, E., Parkes, A.P., Hutchison, A.D.: A conceptual framework to support content-based multimedia applications. Lecture Notes in Computer Science 1629 (2000)Google Scholar
  13. 13.
    Adami, N., Bugatti, A., Leonardi, R., Migliorati, P., Rossi, L.A.: The ToCAI description scheme for indexing and retrieval of multimedia documents. Multimedia Tools Appl. 14(2), 153–173 (2001)CrossRefzbMATHGoogle Scholar
  14. 14.
    Golshani, F., Dimitrova, N.: A language for content-based video retrieval. Multimedia Tools Appl. 6(3), 289–312 (1998)CrossRefGoogle Scholar
  15. 15.
    ISO/IEC: Information Technology −Multimedia Content Description Interface – Part 5: Multimedia Description Schemes. Geneva, Switzerland, International Organisation for Standardisation (2002)Google Scholar
  16. 16.
    Tseng, B.L., Lin, C.-Y., Smith, J.R.: Using MPEG-7 and MPEG-21 for personalizing video. IEEE Multimedia 11(1), 42–53 (2004)CrossRefGoogle Scholar
  17. 17.
    Correia, P., Pereira, F.: The role of analysis in content-based video coding and indexing. Signal Process. 66, 125–142, (1998)CrossRefzbMATHGoogle Scholar
  18. 18.
    Kosch, H.: MPEG-7 and multimedia database systems. ACM SIGMOD Record 31(2), 34–39 (2002)CrossRefGoogle Scholar
  19. 19.
    Westermann, U., Klas, W.: An analysis of XML database solutions for the management of MPEG-7 media descriptions. ACM Comput. Surveys 35(4), 331–373 (2003)CrossRefGoogle Scholar
  20. 20.
    Kang, J.-H., Kim, C.-S., Ko, E.-J.: An XQuery engine for digital library systems. In: Proceedings of the 3rd ACM/IEEE-CS Joint Conference on Digital Libraries, p. 400. Houston, TX (2003)Google Scholar
  21. 21.
    Chu, Y., Chia, L.-T., Bhowmick, S.S.: Looking at mapping, indexing & querying of MPEG-7 descriptors in RDBMS with SM3. In: Proceedings of ACM MMDB '04, pp. 55–64. Washington, DC, USA (2004)Google Scholar
  22. 22.
    Döller, M., Kosch, H., Dörflinger, B., Bachlechner, A., Blaschke, G.: Demonstration of an MPEG-7 multimedia data cartridge. In: Proceedings of the 10th ACM International Conference on Multimedia (MM02), pp. 85–86. Juan-les-Pins, France (2002)Google Scholar
  23. 23.
    Lin, C.-Y., Tseng, B.L., Naphade, M., Natsev, A., Smith, J.R.: MPEG-7 video automatic labeling system. In: Proceedings of the ACM MM'03, pp. 98–99. Berkeley, CA, USA (2003)Google Scholar
  24. 24.
    Lin, C.-Y., Tseng, B.L., Naphade, M., Natsev, A., Smith, J.R.: VideoAL: a novel end-to-end MPEG-7 video automatic labeling system. In: Proceedings of the IEEE International Conference on Image Processing 2003, vol. 3, pp. III-53–56. Barcelona, Spain, IEEE Press, Piscataway, NJ (2003)Google Scholar
  25. 25.
    Tseng, B.L., Ching-Yung, L., Smith, J.R.: Video personalization and summarization system. In: Proceedings of the IEEE Workshop on Multimedia Signal Processing, pp. 424–427 (2002)Google Scholar
  26. 26.
    IBM: IBM VideoAnnEx website. http://www.research.ibm.com/VideoAnnEx/ (2005)
  27. 27.
    Ryu, J., Sohn, Y., Kim, M.: MPEG-7 metadata authoring tool. In: Proceedings of the 10th ACM International Conference on Multimedia (MM02), pp. 267–270. Juan-les-Pins, France (2002)Google Scholar
  28. 28.
    Lee, J.-H., Kim, H.-J., Kim, W.-Y.: Video image retrieval system based on MPEG-7 (VIRS). In: Proceedings of the International Conference on Information Technology: Research and Education (ITRE '03), pp. 79–83 (2003)Google Scholar
  29. 29.
    Benitez, A.B., Chang, S.-F.: Extraction, description and application of multimedia using MPEG-7. In: Proceedings of the 37th Asilomar Conference on Signals, Systems & Computers, pp. 92–96. IEEE Press (2003)Google Scholar
  30. 30.
    Tran-Thuong, T., Roisin, C.: Multimedia modeling using MPEG-7 for authoring multimedia integration. In: Proceedings of the 5th ACM SIGMM International Workshop on Multimedia Information Retrieval, pp. 171–178. Berkeley, CA, ACM Press, New York, NY (2003)Google Scholar
  31. 31.
    Goularte, R., Cattelan, R.G., Camacho-Guerrero, J.A., Inácio Jr., V.R., Pimentel, M.D.G.C.: Interactive multimedia annotations: enriching and extending content. In: Proceedings of DocEng'04, pp. 84–86. Milwaukee, Wisconsin, USA (2004)Google Scholar
  32. 32.
    Costa, M., Correia, N., Guimarães, N.: Annotations as multiple perspectives of video content. In: Proceedings of ACM Multimedia'02, pp. 283–286. Juan-les-Pins, France (2002)Google Scholar
  33. 33.
    Eidenberger, H.: Distance Measures for MPEG-7-based Retrieval. In: Proceedings of ACM MIR'03, pp. 130–137. Berkeley, CA, USA (2003)Google Scholar
  34. 34.
    Goularte, R., Moreira, E.D.S., Pimentel, M.D.G.C.: Structuring interactive TV documents. In: Proceedings of DocEng'03, pp. 42–51. Grenoble, France (2003)Google Scholar
  35. 35.
    Pfeiffer, S., Srinivasan, U.: TV Anytime as an application scenario for MPEG-7. In: Proceedings of the ACM Multimedia Workshop, pp. 89–92. Marina Del Rey, CA (2000)Google Scholar
  36. 36.
    Salembier, P., Qian, R., O'Connor, N., Correia, P., Sezan, I., van Beek, P.: Description schemes for video programs, users and devices. Signal Process.: Image Commun. 16, 211–234 (2000)CrossRefGoogle Scholar
  37. 37.
    Rehm, E.: Representing internet streaming media metadata using MPEG-7 multimedia description schemes. In: Proceedings of the ACM Multimedia Workshop, pp. 93–98. Marina Del Rey, CA, USA (2000)Google Scholar
  38. 38.
    van Setten, M., Oltmans, E.: Demonstration of a distributed MPEG-7 video search and retrieval application in the educational domain. In: Proceedings of ACM Multimedia'01, pp. 595–596. Ottawa, Canada (2001)Google Scholar
  39. 39.
    Vakali, A., Hacid, M.-S., Elmagarmid, A.: MPEG-7 based description schemes for multi-level video content classification. Image Vis. Comput. 22, 367–378 (2004)CrossRefGoogle Scholar
  40. 40.
    Kuo, P.-J., Aoki, T., Yasuda, H.: Building personal digital photograph libraries: An approach with ontology-based MPEG-7 dozen dimensional digital content architecture. In: Proceedings of IEEE Computer Graphics International (CGI'04) (2004)Google Scholar
  41. 41.
    Hunter, J.: Enhancing the semantic interoperability of multimedia through a core ontology. IEEE Trans. Circuits Syst. Video Technol. 13(1), 49–58 (2003)CrossRefGoogle Scholar
  42. 42.
    Graves, A., Lalmas, M.: Video retrieval using an MPEG-7 based inference network. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Tampere, pp. 339–346. Finland, ACM Press, New York, NY (2002)Google Scholar
  43. 43.
    Magalhães, J., Pereira, F.: Using MPEG standards for multimedia customization. Signal Process.: Image Commun. 19, 437–456 (2004)CrossRefGoogle Scholar
  44. 44.
    Martínez, J.M., González, C., Fernández, O., Garcia, C., de Ramón, J.: Towards universal access to content using MPEG-7. In: Proceedings of the 10th ACM International Conference on Multimedia (MM02), pp. 199–202. Juan-les-Pins, France (2002)Google Scholar
  45. 45.
    Ferman, A.M., Beek, J.H.E.P.V., Sezan, M.I.: Content-based filtering and personalization using structured metadata. In: Proceedings of the 2nd ACM/IEEE-CS Joint Conference on Digital Libraries, p. 393. Portland, Oregon (2002)Google Scholar
  46. 46.
    Ferman, A.M., van Beek, P., Errico, J.H., Sezan, M.I.: Multimedia content recommendation engine with automatic inference of user preferences. In: Proceedings of the IEEE International Conference on Image Processing, vol. 3, pp. 49–52 (2003)Google Scholar
  47. 47.
    Rovira, M., González, J., López, A., Mas, J., Puig, A., Fabregat, J., Fernàndez, G.: IndexTV: A MPEG-7 Based Personalized Recommendation System for Digital TV. In: Proceedings of the 2004 IEEE International Conference on Multimedia and Expo (ICME), pp. 823–826 (2004)Google Scholar
  48. 48.
    Tseng, B.L., Lin, C.-Y., Smith, J.R.: Video personalization and summarization system for usage environment. J. Vis. Commun. Image Recognit. 15, 370–392 (2004)CrossRefGoogle Scholar
  49. 49.
    Echigo, T., Masumitsu, K., Teraguchi, M., Etoh, M., Sekihuchi, S.: Personalized delivery of digest video managed on MPEG-7. In: Proceedings of the 2001 International Conference on Information Technology: Coding and Computing, pp. 216–220. Las Vegas, NV, USA, IEEE Press, Piscataway, NJ (2001)Google Scholar
  50. 50.
    Jaimes, A., Echigo, T., Teraguchi, M., Satoh, F.: Learning personalized video highlights from detailed MPEG-7 metadata. In: Proceedings of the 2002 IEEE International Conference on Image Processing (ICIP-02), vol. 1, pp. 133–136. Rochester, NY, IEEE Press, Piscataway, NJ (2002)Google Scholar
  51. 51.
    Fonseca, P.M., Pereira, F.: Automatic video summarization based on MPEG-7 descriptions. Signal Process.: Image Commun. 19, 685–699 (2004)CrossRefGoogle Scholar
  52. 52.
    Agius, H.W., Angelides, M.C.: COSMOS – Content oriented semantic modelling overlay scheme. Comput. J. 42(3), 153–176 (1999)CrossRefGoogle Scholar
  53. 53.
    Agius, H.W., Angelides, M.C.: A method for developing interactive multimedia from their semantic content. Data Knowledge Eng. 34(2), 165–187 (2000)CrossRefzbMATHGoogle Scholar
  54. 54.
    Agius, H.W., Angelides, M.C.: Modelling content for semantic-level querying of multimedia. Multimedia Tools Appl. 15(1), 5–37 (2001)CrossRefzbMATHGoogle Scholar
  55. 55.
    Appan, P., Sundaram, H.: Networked multimedia event exploration. In: Proceedings of the ACM Multimedia'04, pp. 40–47. New York, NY (2004)Google Scholar
  56. 56.
    Dasiopoulou, S., Papastathis, V.K., Mezaris, V., Kompatsiaris, I., Strintzis, M.G.: An ontology framework for knowledge-assisted semantic video analysis and annotation. In: Proceedings of the 4th International Workshop on Knowledge Markup and Semantic Annotation (SemAnnot 2004) at the 3rd International Semantic Web Conference (ISWC 2004) (2004)Google Scholar
  57. 57.
    Adali, S., Candan, K.S., Chen, S.-S., Erol, K., Subrahmanian, V.S.: The advanced video information system: Data structures and query processing. Multimedia Syst. 4(4), 172–186 (1996)CrossRefGoogle Scholar
  58. 58.
    Davis, M., King, S., Good, N., Sarvas, R.: From context to content: leveraging context to infer media metadata. In: Proceedings of the ACM Multimedia'04, pp. 188–195. New York, NY (2004)Google Scholar
  59. 59.
    Goldfarb, C.S.: HyTime: A standard for structured hypermedia interchange. Computer 24(8), 81–84 (1991)CrossRefGoogle Scholar
  60. 60.
    Newcomb, S.R., Kipp, N.A., Newcomb, V.T.: The “HyTime” hypermedia/time-based document structuring language. Commun. ACM 34(11), 67–83 (1991)CrossRefGoogle Scholar
  61. 61.
    Newcomb, S.R.: Multimedia interchange using SGML/HyTime: Part II: Principles and Applications. IEEE Multimedia 2(3), 60–64 (1995)CrossRefMathSciNetGoogle Scholar
  62. 62.
    Kretz, F., Colaïtis, F.: Standardizing hypermedia information objects. IEEE Commun. Mag. 30(5), 60–70 (1992)CrossRefGoogle Scholar
  63. 63.
    Meyer-Boudnik, T., Effelsberg, W.: MHEG explained. IEEE Multimedia 2(1), 26–38 (1995)CrossRefGoogle Scholar
  64. 64.
    Vieira, M.T.P., Santos, M.T.P.: Content-based search on an MHEG-5 standard-based multimedia database. In: Proceedings of the Eighth International Workshop on Database and Expert Systems Applications, pp. 154–159. Toulouse, France (1997)Google Scholar
  65. 65.
    Echiffre, M., Marchisio, C., Marchisio, P., Panicciari, P., Del Rossi, S.: MHEG-5-aims, concepts, and implementation issues. IEEE Multimedia 5(1), 84–91 (1998)CrossRefGoogle Scholar
  66. 66.
    Allen, J.F.: Maintaining knowledge about temporal intervals. Commun. ACM 26(11), 832–843 (1983)CrossRefzbMATHGoogle Scholar
  67. 67.
    NIST: NIST MPEG-7 Validation Service. http://m7itb.nist.gov/M7Validation.html (2005)
  68. 68.
    Wenyin, L., Chen, Z., Lin, F., Zhang, H., Ma, W.-Y.: Ubiquitous media agents: a framework for managing personally accumulated multimedia files. Multimedia Syst. 9(2), 144–156 (2003)CrossRefGoogle Scholar
  69. 69.
    Wallace, M.S.G.: Towards a context aware mining of user interests for consumption of multimedia documents. In: Proceedings of the 2002 IEEE International Conference on Multimedia and Expo, vol. 1, pp. 733–736 (2002)Google Scholar
  70. 70.
    Angelides, M.C.: Multimedia content modelling and personalization. IEEE Multimedia 10(4), 12–15 (2003)CrossRefGoogle Scholar
  71. 71.
    Eirinaki, M., Vazirgiannis, M.: Web mining for web personalization. ACM Trans. Internet Technol. 3(1), 1–27 (2003)CrossRefGoogle Scholar
  72. 72.
    Kuflik, T., Shoval, P.: Generation of user profiles for information filtering – research agenda. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 313–315 (2000)Google Scholar
  73. 73.
    van Meteren, R., van Someren, M.: Using content-based filtering for recommendation. In: Proceedings of the Machine Learning in the New Information Age: MLnet/ECML2000 Workshop, Barcelona, Spain (2000)Google Scholar
  74. 74.
    Good, N., Schafer, J., Konstan, J., Borchers, A., Sarwar, B., Herlocker, J., Riedl, J.: Combining collaborative filtering with personal agents for better recommendations. In: Proceedings of the 16th National Conference on Artificial Intelligence, pp. 439–446 (1999)Google Scholar
  75. 75.
    Specht, T.K.G., Kahabka, T.: Information filtering and personalisation in databases using Gaussian curves. In: Proceedings of the IEEE Databases Engineering and Applications Symposium. Yokohama, Japan, (2000)Google Scholar
  76. 76.
    Lienhart, R., Pfeiffer, S., Effelsberg, W.: The MoCA workbench: Support for creativity in movie content analysis. In: Proceedings of the 3rd IEEE International Conference on Multimedia Computing and Systems, pp. 314–321. Hiroshima, Japan (1996)Google Scholar
  77. 77.
    Okoli, C., Pawlowski, S.D.: The Delphi method as a research tool: an example, design considerations and applications. Inf. Manage. 42(1), 15–29 (2004)Google Scholar
  78. 78.
    Rowe, G., Wright, G., McColl, A.: Judgment change during Delphi-like procedures: the role of majority influence, expertise, and confidence. Technol. Forecasting Soc. Change 72(4), 377–399 (2005)CrossRefGoogle Scholar
  79. 79.
    Ambler, S.W.: Agile Modelling: Effective Practices for eXtreme Programming and the Unified Process John Wiley & Sons, New York, NY (2002)Google Scholar
  80. 80.
    Germain, É., Robillard, P.N.: Engineering-based processes and agile methodologies for software development: a comparative case study. J. Syst. Softw. 75(1/2), 17–27 (2005)CrossRefGoogle Scholar
  81. 81.
    Vetro, A.: MPEG-21 digital item adaptation: Enabling universal multimedia access. IEEE Multimedia 11(1), 84–87 (2004)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag 2006

Authors and Affiliations

  1. 1.School of Information Systems, Computing and MathematicsBrunel University, St John's, UxbridgeMiddlesexUK

Personalised recommendations