Abstract
Question answering systems aim to meet users’ information needs by returning exact answers in response to a question. Traditional open domain question answering systems are built around a single pipeline architecture. In an attempt to exploit multiple resources as well as multiple answering strategies, systems based on a multi-stream architecture have recently been introduced. Such systems face the challenging problem of having to select a single answer from pools of answers obtained using essentially different techniques. We report on experiments aimed at understanding and evaluating the effect of different options for answer selection in a multi-stream question answering system. We examine the impact of local tiling techniques, assignments of weights to streams based on past performance and/or question type, as well redundancy-based ideas. Our main finding is that redundancy-based ideas in combination with naively learned stream weights conditioned on question type work best, and improve significantly over a number of baselines.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chu-Carrol, J., Prager, J., Welty, C., Czuba, K., Ferrucci, D.: A multi-strategy and multi-source approach to question answering. In: Proceedings of TREC 2002, pp. 281–288 (2003)
Clarke, C.L.A., Cormack, G.V., Kemkes, G., Laszlo, M., Lynam, T.R., Terra, E.L., Tilker, P.L.: Statistical selection of exact answers. In: Proceedings of TREC 2002, pp. 823–831 (2003)
Croft, W.B.: Combining approaches to information retrieval. In: Croft, W.B. (ed.) Advances in Information Retrieval, pp. 1–36. Kluwer Academic Publishers, Dordrecht (2000)
de Chalendar, G., Dalmas, T., Elkateb-Gara, F., Ferret, O., Hurault-Plantet, M., Grau, B., Illouz, G., Monceaux, L., Robba, I., Vilnat, A.: The Question Answering System QALC at LIMSI, Experiments in Using Web and WordNet. In: Proceedings of TREC 2002, pp. 407–416 (2003)
Banko, M., et al.: AskMSR: Question answering using the Worldwide Web. In: Proc. EMNLP 2002 (2002)
Fleischman, M., Hovy, E.H., Echihabi, A.: Offline strategies for online question answering: Answering questions before they are asked. In: Proceedings ACL 2003, pp. 1–7 (2003)
Fox, E.A., Shaw, J.A.: Combination of multiple searches. In: Proceedings TREC-2, pp. 243–252 (1994)
Greenwood, M., Roberts, I., Gaizauskas, R.: The University of Sheffield TREC 2002 Q&A system. In: Proceedings of TREC 2002, pp. 823–831 (2003)
Hiemstra, D., Kraaij, W., Pohlmann, R., Westerveld, T.: Translation resources, merging strategies, and relevance feedback for cross-language information retrieval. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 102–115. Springer, Heidelberg (2001)
Hovy, E., Hermjakob, H., Junk, M., Lin, C.-Y.: Question answering in Webclopedia. In: Proceedings TREC-9 (2000)
Jijkoun, V., Kamps, J., Mishne, G., Monz, C., de Rijke, M., Schlobach, S., Tsur, O.: The University of Amsterdam at TREC 2003. In: TREC 2003 Notebook Papers (2003)
Jijkoun, V., Mishne, G., de Rijke, M.: How frogs built the Berlin Wall. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 523–534. Springer, Heidelberg (2004) (to appear)
Magnini, B., Negri, M., Prevete, R., Tanev, H.: Is it the right answer? Exploiting web redundancy for answer validation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 425–432 (2002)
Moldovan, D., Harabagiu, S., Paşca, M., Mihalcea, R., Girju, R., Goodrum, R., Rus, V.: The structure and performance of an open domain question answering system. In: Proceedings ACL 2000, pp. 563–570 (2000)
Moldovan, D., Paşca, M., Harabagiu, S., Surdeanu, M.: Performance issues and error analysis in an open-domain question answering system. ACM Transactions on Information Systems 21, 133–154 (2003)
Monz, C., de Rijke, M.: Tequesta: The University of Amsterdam’s textual question answering system. In: Proceedings TREC 2001, pp. 519–528 (2002)
Prager, J., Brown, E., Coden, A., Radev, D.: Question-answering by predicitive annotation. In: Proceedings SIGIR 2000, pp. 184–191 (2000)
Soubbotin, M.M., Soubbotin, S.M.: Use of patterns for detection of likely answer strings: A systematic approach. In: Proceedings TREC 2002, pp. 325–331 (2003)
Text REtreival Conference (TREC), http://trec.nist.gov
Xu, J., Licuanan, A., May, J., Miller, S., Weischedel, R.: TREC 2002 QA at BBN: Answer Selection and Confidence Estimation. In: Proceedings of TREC 2002, pp. 96–101 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jijkoun, V., de Rijke, M. (2004). Answer Selection in a Multi-stream Open Domain Question Answering System. In: McDonald, S., Tait, J. (eds) Advances in Information Retrieval. ECIR 2004. Lecture Notes in Computer Science, vol 2997. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24752-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-540-24752-4_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21382-6
Online ISBN: 978-3-540-24752-4
eBook Packages: Springer Book Archive