An Integrated Approach to Semantic Evaluation and Content-Based Retrieval of Multimedia Documents
We present an overview of a large combined querying and retrieval system that performs content-based on-line searches in a large database of multimedia documents (currently text, tables and colour images). Queries are submitted as sentences in natural language and are transformed into the language of the target database. The documents are analyzed semantically for their information content; in a data fusion step the individual pieces of information extracted from these documents are aggregated into cognitively adequate result documents.
There is no pre-indexing necessary when new documents are stored into the system. This retains a high degree of flexibility with respect to the questions that may be asked. It implies, however, that both huge amounts of data must be evaluated rapidly and that intelligent caching strategies must be employed. It is therefore mandatory that the system be equipped with dedicated high-speed hardware processors.
The complete system is currently available as a prototype; the paper outlines its architecture and gives examples of some real sample queries in the knowledge domain of weather data documents.
KeywordsNatural Language Processing Semantic Representation Multimedia Document Semantic Evaluation Retrieval Module
Unable to display preview. Download preview PDF.
- 1.A. Del Bimbo, M. Campanai, and P. Nesi. A three-dimensional iconic environment for image database querying. IEEE Trans. on Software Eng., 19(10):997–1011, October 1993.Google Scholar
- 2.A. Del Bimbo and P. Pala. Visual image retrieval by elastic matching of user sketches. IEEE Trans. on Patt. Anal. and Mach. Intell., 19(2):121–132, February 1997.Google Scholar
- 3.J. Biskup, J. Freitag, Y. Karabulut, and B. Sprick. A mediator for multimedia systems. In Proceedings 3rd International Workshop on Multimedia Information Systems, Como, Italia, Sept. 1997.Google Scholar
- 4.J. Biskup, J. Freitag, Y. Karabulut, and B. Sprick. Query evaluation in an object-oriented multimedia mediator. In Proceedings 4th International Conference on Object-Oriented Information Systems, Brisbane, Australia, Berlin, Nov. 1997. Springer.Google Scholar
- 6.Norbert Bröker, Udo Hahn, and Susanne Schacht. Concurrent lexicalized dependency parsing: The ParseTalk model. In Proceedings of the 15th International Conference on Computational Linguistics (COLING 94), 1994.Google Scholar
- 9.Michael Eimermacher. Wortorientiertes Parsen. PhD thesis, TU Berlin, Berlin, 1988.Google Scholar
- 10.M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkhani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker. Query by image and video content: The QBIC system. IEEE Computer, 28(9), September 1995.Google Scholar
- 11.Michael Gansen, Frank Richter, Oliver Weiß, and Tobias G. Noll. A datapath generator for full custom macros of iterative logic arrays. Proceedings of the IEEE 1997 International Conference on Application Specific Systems, Architectures, and Processors, pages 438–447, July 1997.Google Scholar
- 12.Ingo Glöckner. DFS-an axiomatic approach to fuzzy quantification. Technical Report TR97-06, Technische Fakultät, Universität Bielefeld, 1997.Google Scholar
- 13.Sven Hartrumpf. Redundanzarme Lexika durch Vererbung. Master’s thesis, Universität Koblenz-Landau, Koblenz, June 1996.Google Scholar
- 14.Sven Hartrumpf. Partial evaluation for efficient access to inheritance lexicons. In Proceedings of the 2nd International Conference on Recent Advances in Natural Language Processing (RANLP-97), pages 43–50, Tzigov Chark, Bulgaria, September 1997.Google Scholar
- 16.Hermann Helbig. Der MESNET Primer-Die Darstellungsmittel der Mehrschichtigen Erweiterten Semantischen Netze. Technische Dokumentation, FernUniversität Hagen, Hagen, Germany, January 1997.Google Scholar
- 17.Hermann Helbig, Carsten Gnörlich, and Dirk Menke. Realization of a user-friendly access to networked information retrieval systems. In Proceedings of the AAAI Spring Symposium on Natural Language Processing for the World Wide Web, pages 62–71, Stanford, CA, 1997.Google Scholar
- 18.Hermann Helbig and Sven Hartrumpf. Word class functions for syntactic-semantic analysis. In Proceedings of the 2nd International Conference on Recent Advances in Natural Language Processing (RANLP-97), pages 312–317, Tzigov Chark, Bulgaria, September 1997.Google Scholar
- 19.Hermann Helbig and Andreas Mertens. Word Agent Based Natural Language Processing. In Loe Boves and Anton Nijholt, editors, Proceedings of the 8th Twente Workshop on Language Technology-Speech and Language Engineering, Twente, 1 and 2 December 1994, pages 65–74, Enschede, 1994. Universiteit Twente, Fakulteit Informatica.Google Scholar
- 20.Hermann Helbig and Marion Schulz. Knowledge representation with MESNET: A multilayered extended semantic network. In Proceedings of the AAAI Spring Symposium on Ontological Engineering, pages 64–72, Stanford, CA, 1997.Google Scholar
- 21.Christiane Henning and Tobias G. Noll. Architecture and implementation of a bitserial sorter for weighted median filtering. Proceedings of the 1998 Custom Integrated Circuits Conference, Santa Clatra, CA, May 1998.Google Scholar
- 22.T. Hermes, C. Klauck, J. Kreyß, and J. Zhang. Image retrieval for information systems. In Proc. SPIE’s Symp. on Electronic Imaging, San Jose, February 1995.Google Scholar
- 23.S. Iyengar and R. Kashyap. Special section on image databases. IEEE Trans. on Software Eng., 14(5):608–688, May 1988.Google Scholar
- 24.Jörg Jensch, Reinhard Lüling, and Norbert Sensen. A data layout strategy for parallel web servers. In Proceedings of EuroPar’ 98, 1998.Google Scholar
- 25.A. Knoll, R. Schröder, and A. Wolfram. Fusion of data from fuzzy integral-based active and passive colour stereo vision systems for correspondence identification. In Proceedings of the VIII European Signal Processing Conference (EUSIPCO-96), Trieste, Italy, Sept. 10–13 1996.Google Scholar
- 26.U. Manber and S. Wu. GLIMPSE: A tool to search through entire file systems. Tr 93-34, Department of Computer Science, University of Arizona, Tucson, Arizona, 1993.Google Scholar
- 27.Marion Schulz. Eine Werkbank zur interaktiven Erstellung semantikbasierter Computerlexika. PhD thesis, FernUniversität Hagen, Hagen, 1998.Google Scholar
- 28.Marion Schulz and Hermann Helbig. COLEX: Ein Computerlexikon für die automatische Sprachverarbeitung. Informatik-Bericht 210, FernUniversität Hagen, Hagen, Germany, December 1996.Google Scholar
- 29.Virage Inc. http://www.virage.com
- 30.Stefan Wermter, Ellen Riloff, and Gabriele Scheler, editors. Connectionist, Statistical, and Symbolic Approaches to Learning for Natural Language Processing, volume 1040 of LNAI. Springer, Berlin, 1996.Google Scholar
- 32.Wolfgang Wilhelm and Tobias G. Noll. A new mapping technique for automated design of highly efficient multiplexed fir digital filters. Proceedings of the IEEE International Symposium on Circuits and Systems, pages 2252–2255, June 1997.Google Scholar