Skip to main content
SpringerLink
Log in

Modeling Content for Semantic-Level Querying of Multimedia

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Many semantic content-based models have been developed for modeling video and audio in order to enable information retrieval based on semantic content. The level of querying of the media depends upon the semantic aspects modeled. This paper proposes a semantic content-based model for semantic-level querying that makes full use of the explicit media structure, objects, spatial relationships between objects, events and actions involving objects, temporal relationships between events and actions, and integration between syntactic and semantic information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. S. Adah, K.S. Candan, S.-S. Chen, K. Erol, and V.S. Subrahmanian, “The Advanced Video Information System: Data structures and query processing,” Multimedia Systems, Vol. 4, No. 4, pp. 172–186, 1996.

    Google Scholar 

  2. H.W. Agius and M.C. Angelides, “Integrating logical video and audio segments with content-related information in instructional multimedia systems,” Information and Software Technology, Vol. 39, No. 10, pp. 679–694, 1997.

    Google Scholar 

  3. T.G. Aguierre Smith and G. Davenport, “The stratification system: A design environment for random access video,” in Proceedings of the Third International Workshop on Network and Operating Systems Support for Digital Audio and Video, Springer-Verlag: New York, NY, 1992, pp. 250–261.

    Google Scholar 

  4. M.C. Angelides, “Developing hybrid intelligent tutoring and hypertext systems,” The New Review of Hypermedia and Multimedia, Vol. 1, pp. 67–106, 1995.

    Google Scholar 

  5. R. Barber, W. Equitz, C. Faloutsos, M. Flickner, W. Niblack, D. Petkovic, and P. Yanker, “Query by content for large on-line image collections,” in A Guided Tour of Multimedia Systems and Applications, B. Furht and M. Milenkovic (Eds.), IEEE Computer Society Press: Los Alamitos, CA, 1995, pp. 357–378.

    Google Scholar 

  6. A. Brink, S. Marcus, and V.S. Subrahmanian, “Heterogeneous multimedia reasoning,” Computer, Vol. 28, No. 9, pp. 33–39, 1995.

    Google Scholar 

  7. V. Burrill, T. Kirste, and J. Weiss, “Time-varying sensitive regions in dynamic multimedia objects:Apragmatic approach to content-based retrieval from video,” Information and Software Technology, Vol. 36, No. 4, pp. 213–223, 1994.

    Google Scholar 

  8. M. Carrer, L. Ligresti, G. Ahanger, and T.D.C. Little, “An annotation engine for supporting video database population,” Multimedia Tools and Applications, Vol. 5, No. 3, pp. 233–258, 1997.

    Google Scholar 

  9. M. Christel, T. Kanade, M. Mauldin, R. Reddy, M. Sirbu, S. Stevens, and H. Wactlar, “Informedia Digital Video Library,” Communications of the ACM, Vol. 38, No. 4, pp. 57–58, 1995.

    Google Scholar 

  10. A. Csinger, K.S. Booth, and D. Poole, “AI meets authoring: User models for intelligent multimedia,” Artificial Intelligence Review, Vol. 8, No. 5/6, pp. 447–468, 1995.

    Google Scholar 

  11. M. Davis, “Media streams: An iconic visual language for video annotation,” in Proceedings of the IEEE Symposium on Visual Languages, Bergen, IEEE Computer Society Press: Los Alamitos, CA, 1993, pp. 196–202.

    Google Scholar 

  12. D. Fischer and C. Richards, “The presentation of time in interactive animated systems diagrams,” in Multimedia Systems and Applications, R.A. Earnshaw and J.A. Vince (Eds.), Academic Press: London, 1995, pp. 141–159.

    Google Scholar 

  13. M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, “Query by image and video content: The QBIC system,” Computer, Vol. 28, No. 9, pp. 23–32, 1995.

    Google Scholar 

  14. J.C. Fritzsche, “Continuous media described by time-dependent data,” Multimedia Systems, Vol. 3, No. 5/6, pp. 278–285, 1995.

    Google Scholar 

  15. S. Gibbs, C. Breiteneder, and D. Tsichritzis, “Modeling time-based media,” in The Handbook of Multimedia Information Management, W.I. Grosky, R. Jain and R. Mehrotra (Eds.), Prentice Hall PTR: Upper Saddle River, NJ, 1997, pp. 13–38.

    Google Scholar 

  16. B.A. Goodman, “Multimedia explanations for intelligent training systems,” in Intelligent Multimedia Interfaces, M.T. Maybury (Ed.), The AAAI Press/The MIT Press: Menlo Park, CA, 1993, pp. 148–171.

    Google Scholar 

  17. N. Hirzalla, B. Falchuk, and A. Karmouch, “A temporal model for interactive multimedia scenarios,” IEEE MultiMedia, Vol. 2, No. 3, pp. 24–31, 1995.

    Google Scholar 

  18. T. Kanade, “Immersion into visual media: New applications of image understanding,” IEEE Expert, Vol. 11, No. 1, pp. 73–80, 1996.

    Google Scholar 

  19. T.D.C. Little, G. Ahanger, H.-J. Chen, R.J. Folz, J.F. Gibbon, A. Krishnamurthy, P. Lumba, M. Ramanathan, and D. Venkatesh, “Selection and dissemination of digital video via the Virtual Video Browser,” Multimedia Tools and Applications, Vol. 1, No. 2, pp. 149–172, 1995.

    Google Scholar 

  20. T.D.C. Little, G. Ahanger, R.J. Folz, J.F. Gibbon, F.W. Reeve, D.H. Shelleng, and D. Venkatesh, “A digital video-on-demand service supporting content-based queries,” in Proceedings of the First ACM International Conference on Multimedia, Anaheim, CA, August, ACM Press: New York, NY, 1993, pp. 427–436.

    Google Scholar 

  21. W.E. Mackay, “EVA: An experimental video annotator for symbolic analysis of video data,” SIGCHI Bulletin, Vol. 21, No. 1, pp. 68–71, 1989.

    Google Scholar 

  22. W.E. Mackay and G. Davenport, “Virtual video editing in interactive multimedia applications,” Communications of the ACM, Vol. 32, No. 7, pp. 802–810, 1989.

    Google Scholar 

  23. E. Oomoto and K. Tanaka, “OVID: Design and implementation of a video-object database system,” IEEE Transactions on Knowledge and Data Engineering, Vol. 5, No. 4, pp. 629–643, 1993.

    Google Scholar 

  24. A.P. Parkes, “CLORIS: A prototype video-based intelligent computer-assisted instruction system,” in Proceedings of RIAO '88, 1988, pp. 24–50.

    Google Scholar 

  25. A.P. Parkes and J.A. Self, “Towards ‘interactive video’: A video-based intelligent tutoring environment,” in Intelligent Tutoring Systems: At the Crossroad of Artificial Intelligence and Education, C. Frasson and G. Gauthier (Eds.), Ablex Publishing Corporation: Norwood, NJ, 1990, pp. 56–82.

    Google Scholar 

  26. R.J. Price, “Multimedia information systems,” in Multimedia: Technology and Applications, J.A. Waterworth (Ed.), Ellis Horwood: New York, NY, 1991, pp. 114–150.

    Google Scholar 

  27. J. Siemer and M.C. Angelides, “A comprehensive method for the evaluation of complete intelligent tutoring systems.” Decision Support Systems, Vol. 22, No. 1, pp. 85–102, 1998.

    Google Scholar 

  28. J. Siemer and M.C. Angelides, “Towards an intelligent tutoring system architecture that supports remedial tutoring,” Artificial Intelligence Review, Vol. 12, No. 6, pp. 469–511, 1998.

    Google Scholar 

  29. S.W. Smoliar and H.J. Zhang, “Content-based video indexing and retrieval,” IEEE MultiMedia, Vol. 1, No. 2, pp. 62–72, 1994.

    Google Scholar 

  30. Y. Tanaka “IntelligentPad as meme media and its application to multimedia databases,” Information and Software Technology, Vol. 38, No. 3, pp. 201–211, 1996.

    Google Scholar 

  31. Y. Tanaka, “Meme media and a world-wide meme pool,” in ACM Multimedia '96 Proceedings, Proceedings of the Fourth ACM International Multimedia Conference, Boston, MA, 18–22 November, ACM Press/ Addison-Wesley: New York, NY, 1996, pp. 175–186.

    Google Scholar 

  32. Y. Tonomura, A. Akutsu, Y. Taniguchi, and G. Suzuki, “Structured video computing,” IEEE MultiMedia, Vol. 1, No. 3, pp. 34–43, 1994.

    Google Scholar 

  33. H.D. Wactlar, T. Kanade, M.A. Smith, and S.M. Stevens, “Intelligent access to digital video: Informedia project,” Computer, Vol. 29, No. 5, pp. 46–52, 1996.

    Google Scholar 

  34. R. Weiss, A. Duda, and D.K. Gifford, “Composition and search with a video algebra,” IEEE MultiMedia, Vol. 2, No. 1, pp. 12–25, 1995.

    Google Scholar 

  35. B.P. Woolf, “Intelligent multimedia tutoring systems,” Communications of the ACM, Vol. 39, No. 4, pp. 30–31, 1996.

    Google Scholar 

  36. B.P. Woolf and W. Hall, “Multimedia pedagogues: Interactive systems for teaching and learning,” Computer, Vol. 28, No. 5, pp. 74–80, 1995.

    Google Scholar 

  37. A. Yoshitaka, S. Kishida, M. Hirakawa, and T. Ichikawa, “Knowledge-assisted content-based retrieval for multimedia databases,” IEEE MultiMedia, Vol. 1, No. 4, pp. 12–21, 1994.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Systems and Computing, Brunel University, Uxbridge, Middlesex, UB8 3PH, UK

    Harry W. Agius & Marios C. Angelides

Authors
  1. Harry W. Agius
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marios C. Angelides
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Agius, H.W., Angelides, M.C. Modeling Content for Semantic-Level Querying of Multimedia. Multimedia Tools and Applications 15, 5–37 (2001). https://doi.org/10.1023/A:1011386102507

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1011386102507

Search

Navigation