Advertisement

Indiscriminateness in Representation Spaces of Terms and Documents

  • Vincent Claveau
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10772)

Abstract

Examining the properties of representation spaces for documents or words in Information Retrieval (IR) – typically \(\mathbb {R}^n\) with n large – brings precious insights to help the retrieval process. Recently, several authors have studied the real dimensionality of the datasets, called intrinsic dimensionality, in specific parts of these spaces [14]. They have shown that this dimensionality is chiefly tied with the notion of indiscriminateness among neighbors of a query point in the vector space. In this paper, we propose to revisit this notion in the specific case of IR. More precisely, we show how to estimate indiscriminateness from IR similarities in order to use it in representation spaces used for documents and words [7, 18]. We show that indiscriminateness may be used to characterize difficult queries; moreover we show that this notion, applied to word embeddings, can help to choose terms to use for query expansion.

Keywords

Intrinsic dimensionality Indiscriminability RSV scores Distributional thesauri Query expansion 

References

  1. 1.
    Amati, G., Rijsbergen, C.J.V.: Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Trans. Inf. Syst. 20, 357–389 (2002)CrossRefGoogle Scholar
  2. 2.
    Amsaleg, L., Oussama, C., Furon, T., Girard, S., Houle, M.E., Kawarabayashi, K.I.: Estimating local intrinsic dimensionality. In: 21st Conference on Knowledge Discovery and Data Mining, KDD 2015, Sidney, Australia, August 2015. https://hal.inria.fr/hal-01159217
  3. 3.
    Bellogín, A., de Vries, A.P.: Understanding similarity metrics in neighbour-based recommender systems. In: Proceedings of the 2013 Conference on the Theory of Information Retrieval, ICTIR 2013, pp. 13:48–13:55. ACM, New York (2013). http://doi.acm.org/10.1145/2499178.2499186
  4. 4.
    Beygelzimer, A., Kakade, S., Langford, J.: Cover trees for nearest neighbors. In: Proceedings of International Conference on Machine Learning (ICML), pp. 97–104 (2006)Google Scholar
  5. 5.
    Clauset, A., Shalizi, C.R., Newman, M.E.J.: Power-law distributions in empirical data. SIAM Rev. 51(4), 661–703 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Claveau, V., Kijak, E.: Direct vs. indirect evaluation of distributional thesauri. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 1837–1848. The COLING 2016 Organizing Committee, Osaka, December 2016. http://aclweb.org/anthology/C16-1173
  7. 7.
    Claveau, V., Kijak, E., Ferret, O.: Improving distributional thesauri by exploring the graph of neighbors. In: International Conference on Computational Linguistics, COLING 2014, Dublin, August 2014. https://hal.archives-ouvertes.fr/hal-01027545
  8. 8.
    Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. 41, 391 (1990)CrossRefGoogle Scholar
  9. 9.
    Fang, H., Tao, T., Zhai, C.: Diagnostic evaluation of information retrieval models. ACM Trans. Inf. Syst. 29, 7 (2011)CrossRefGoogle Scholar
  10. 10.
    Hersh, W., Buckley, C., Leone, T.J., Hickam, D.: OHSUMED: an interactive retrieval evaluation and new large test collection for research. In: Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 1994, pp. 192–201. Springer-Verlag, New York Inc., New York (1994). http://dl.acm.org/citation.cfm?id=188490.188557
  11. 11.
    Hoffman, M., Bach, F.R., Blei, D.M.: Online learning for latent dirichlet allocation. In: Lafferty, J., Williams, C., Shawe-Taylor, J., Zemel, R., Culotta, A. (eds.) Advances in Neural Information Processing Systems, vol. 23, pp. 856–864. Curran Associates, Inc. (2010). http://papers.nips.cc/paper/3902-online-learning-for-latent-dirichlet-allocation.pdf
  12. 12.
    Houle, M.E., Ma, X., Nett, M., Oria, V.: Dimensional testing for multi-step similarity search. In: Proceedings of the 12th IEEE International Conference on Data Mining (ICDM), pp. 299–308 (2012)Google Scholar
  13. 13.
    Houle, M.E., Nett, M.: Rank cover trees for nearest neighbor search. In: Brisaboa, N., Pedreira, O., Zezula, P. (eds.) SISAP 2013. LNCS, vol. 8199, pp. 16–29. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-41062-8_3 CrossRefGoogle Scholar
  14. 14.
    Houle, M., Kashima, H., Nett, M.: Generalized expansion dimension. In: Proceedings of the 12th IEEE International Conference on Data Mining Workshops (ICDMW), pp. 587–594 (2012)Google Scholar
  15. 15.
    Levina, E., Bickel, P.J.: Maximum likelihood estimation of intrinsic dimension. In: Advances in Neural Information Processing Systems (NIPS) (2004)Google Scholar
  16. 16.
    Lv, Y., Zhai, C.: Lower-bounding term frequency normalization. In: Proceedings of the 20th ACM International Conference on Information and Knowledge Managementm, CIKM 2011, pp. 7–16. ACM, New York (2011). http://doi.acm.org/10.1145/2063576.2063584
  17. 17.
    Metzler, D., Croft, W.: Combining the language model and inference network approaches to retrieval. Inf. Process. Manag. 40(5), 735–750 (2004). Special Issue on Bayesian Networks and Information RetrievalCrossRefGoogle Scholar
  18. 18.
    Mikolov, T., Yih, W.t., Zweig, G.: Linguistic regularities in continuous space word representations. In: 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2013), Atlanta, Georgia, pp. 746–751 (2013)Google Scholar
  19. 19.
    Ponte, J.M., Croft, W.B.: A language modeling approach to information retrieval. In: Proceedings of the 21st Annual international ACM SIGIR Conference on Research and Development in information Retrieval (SIGIR 1998), pp. 275–281 (1998)Google Scholar
  20. 20.
    Robertson, S.E., Walker, S., Hancock-Beaulieu, M.: Okapi at TREC-7: automatic ad hoc, filtering, VLC and interactive track. In: Proceedings of the 7th Text Retrieval Conference, TREC-7, pp. 199–210 (1998)Google Scholar
  21. 21.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  22. 22.
    Scholkopf, B., Smola, A.J., Muller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)CrossRefGoogle Scholar
  23. 23.
    Singhal, A.: Modern information retrieval: a brief overview. Bull. IEEE Comput. Soc. Tech. Committee Data Eng. 24, 35–43 (2001)Google Scholar
  24. 24.
    Strohman, T., Metzler, D., Turtle, H., Croft, W.: Indri: a language-model based search engine for complex queries (extended version). Technical report CIIR (2005)Google Scholar
  25. 25.
    Turtle, H., Croft, W.: Evaluation of an inference network-based retrieval model. ACM Trans. Inf. Syst. 9(3), 187–222 (1991)CrossRefGoogle Scholar
  26. 26.
    Venna, J., Kaski, S.: Local multidimensional scaling. Neural Netw. 19, 889–899 (2006)CrossRefzbMATHGoogle Scholar
  27. 27.
    de Vries, T., Chawla, S., Houle, M.E.: Density-preserving projections for large-scale local anomaly detection. Knowl. Inf. Syst. 32(1), 25–52 (2012)CrossRefGoogle Scholar
  28. 28.
    Zhai, C., Lafferty, J.D.: A study of smoothing methods for language models applied to ad hoc information retrieval. In: Proceedings of the SIGIR Conference, pp. 334–342 (2001)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Univ. Rennes, CNRS, IRISARennesFrance

Personalised recommendations