Abstract
In a hypermedia authoring task, an author often wants to set up meaningful connections between different media, such as text and photographs. To facilitate this task, it is helpful to have a software agent dynamically adapt the presentation of a media database to the user’s authoring activities, and look for opportunities for annotation and retrieval. However, potential connections are often missed because of differences in vocabulary or semantic connections that are “obvious” to people but that might not be explicit.
ARIA (Annotation and Retrieval Integration Agent) is a software agent that acts an assistant to a user writing e-mail or Web pages. As the user types a story, it does continuous retrieval and ranking on a photo database. It can use descriptions in the story to semi-automatically annotate pictures. To improve the associations beyond simple keyword matching, we use natural language parsing techniques to extract important roles played by text, such as “who, what, where, when”. Since many of the photos depict common everyday situations such as weddings or recitals, we use a common sense knowledge base, Open Mind, to fill in semantic gaps that might otherwise prevent successful associations.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Budzik, J. and Hammond, K. J.: User Interactions with Everyday Applications as Context for Just-in-Time Information Access, ACM Conf. Intelligent User Interfaces (IUI 2000), ACM Press, New York, (Jan. 2000), pp. 44–51.
Fellbaum, C. (Ed.): WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA. (1998).
Jackendoff, R.: Semantic structures. Cambridge, MA: MIT Press, (1990).
Kuchinsky, A., Pering, C., Creech, M. L., Freeze, D., Serra, B., and Gwizdka, J.: FotoFile: a consumer multimedia organization and retrieval system, ACM Conference on Human-Computer Interface, (CHI-99) Pages 496–503, Pittsburgh, (May 1999).
Lenat, D.: The dimensions of context-space, Cycorp technical report, (1998), www.cyc.com.
Lieberman, H., Rosenzweig, E., and Singh, P.: Aria: An Agent For Annotating And Retrieing Images. IEEE Computer, (July 2001), pp. 57–61.
Minsky, M.: Commonsense-Based Interfaces. Communications of the ACM. Vol. 43, No. 8 (August, 2000), Pages 66–73
Mueller, E. T.: A calendar with common sense. In Proceedings of the 2000 International Conference on Intelligent User Interfaces, 198-201. New York: Association for Computing Machinery. (2000).
Peat, H. J. and Willett, P.: The limitations of term co-occurrence data for query expansion in document retrieval systems. Journal of the ASIS, 42(5), (1991), 378–383.
Salton G. and Buckley C.: On the Use of Spreading Activation Methods in Automatic Information Retrieval, In Proc. 11th Ann. Int. ACM SIGIR Conf. on R&D in Information Retrieval (ACM), (1988), 147–160.
Singh, P.: The Public Acquisition of Commonsense Knowledge. AAAI Spring Symposium, Stanford University, Palo Alto, CA, (2002).
Voorhees, E.: Query expansion using lexical-semantic relations. In Proceedings of ACM SIGIR Intl. Conf. on Research and Development in Information Retrieval. (1994) 61–69.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lieberman, H., Liu, H. (2002). Adaptive Linking between Text and Photos Using Common Sense Reasoning. In: De Bra, P., Brusilovsky, P., Conejo, R. (eds) Adaptive Hypermedia and Adaptive Web-Based Systems. AH 2002. Lecture Notes in Computer Science, vol 2347. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47952-X_2
Download citation
DOI: https://doi.org/10.1007/3-540-47952-X_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43737-6
Online ISBN: 978-3-540-47952-9
eBook Packages: Springer Book Archive