Abstract
This paper proposes a word embedding-based reranking algorithm with a boosting. The algorithm converts queries and documents into sets of word embeddings represented by vectors and reranks documents according to a similarity defined with the word embeddings as in Latent Semantic Indexing (LSI) and Supervised Semantic Indexing (SSI). Compared with LSI and SSI, our method uses top-n irrelevant documents of a relevant document of each query for training a reranking model. Furthermore, we also propose application of a boosting to the reranking model. Our method uses the weights of training samples decided by AdaBoost as coefficients for updating model, therefore, highly weighted samples are aggressively learned. We evaluate the proposed method with datasets created from English and Japanese Wikipedia respectively. The experimental results show that our method achieves better mean average precision than LSI and SSI.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We used https://dumps.wikimedia.org/jawiki/20160407/jawiki-20160407-pages-articles.xml.bz2 and https://dumps.wikimedia.org/enwiki/20160901/enwiki-20160901-pages-articles.xml.bz2. Retrieved October 14, 2016.
- 2.
https://github.com/taku910/mecab. Retrieved October 14, 2016.
- 3.
https://www.elastic.co/downloads/past-releases/elasticsearch-1-7-1, Retrieved October 14, 2016.
- 4.
https://tedlab.mit.edu/~dr/SVDLIBC/. Retrieved October 14, 2016.
- 5.
We used following options: −d 50 −i 5 −e 1e−30 −a las2 −k 1e−6.
References
Bai, B., Weston, J., Grangier, D., Collobert, R., Sadamasa, K., Qi, Y., Chapelle, O., Weinberger, K.: Supervised semantic indexing. In: Proceedings of the 18th CIKM (CIKM 2009 ), pp. 187–196 (2009)
Berger, A., Lafferty, J.: Information retrieval as statistical translation. In: Proceedings of the 22nd SIGIR (SIGIR 1999), pp. 222–229 (1999)
Bespalov, D., Bai, B., Qi, Y., Shokoufandeh, A.: Sentiment classification based on supervised latent n-gram analysis. In: Proceedings of the 20th CIKM (CIKM 2011), pp. 375–382 (2011)
Blei, D., Ng, A., Jordan, M.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., Hullender, G.: Learning to rank using gradient descent. In: Proceedings of the 22nd International Conference on Machine Learning (ICML 2005), pp. 89–96 (2005)
Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. 41(6), 391–407 (1990)
Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. Technical report UCB/EECS-2010-24. EECS Department, University of California, Berkeley, March 2010
Freund, Y., Iyer, R., Schapire, R.E., Singer, Y.: An efficient boosting algorithm for combining preferences. J. Mach. Learn. Res. 4, 933–969 (2003)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997)
Grangier, D., Bengio, S.: A discriminative kernel-based model to rank images from text queries. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 30, 1371–1384 (2008)
Iwakura, T.: A boosted semi-markov perceptron. In: Proceedings of the 17th CoNLL, pp. 47–55 (2013)
Joachims, T.: Optimizing search engines using click through data. In: Proceedings of the 8th KDD (KDD 2002), pp. 133–142 (2002)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 26, pp. 3111–3119. Curran Associates, Inc. (2013)
Min, K., Zhang, Z., Wright, J., Ma, Y.: Decomposing background topics from keywords by principal component pursuit. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management (CIKM 2010), pp. 269–278 (2010)
Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Wang, Y., Sun, H., Zhang, R.: AdaMF: adaptive boosting matrix factorization for recommender system. In: Li, F., Li, G., Hwang, S., Yao, B., Zhang, Z. (eds.) WAIM 2014. LNCS, vol. 8485, pp. 43–54. Springer, Cham (2014). doi:10.1007/978-3-319-08010-9_7
Weston, J., Bengio, S., Usunier, N.: Large scale image annotation: learning to rank with joint word-image embeddings. In: European Conference on Machine Learning (2010)
Xu, J., Li, H.: AdaRank: a boosting algorithm for information retrieval. In: Proceedings of the 30th SIGIR (SIGIR 2007), pp. 391–398 (2007)
Yin, W., Schütze, H., Xiang, B., Zhou, B.: ABCNN: attention-based convolutional neural network for modeling sentence pairs. TACL 4, 259–272 (2016)
Zeiler, M.D.: ADADELTA: an adaptive learning rate method. CoRR abs/1212.5701 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Makino, T., Iwakura, T. (2017). A Boosted Supervised Semantic Indexing for Reranking. In: Sung, WK., et al. Information Retrieval Technology. AIRS 2017. Lecture Notes in Computer Science(), vol 10648. Springer, Cham. https://doi.org/10.1007/978-3-319-70145-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-70145-5_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70144-8
Online ISBN: 978-3-319-70145-5
eBook Packages: Computer ScienceComputer Science (R0)