Encyclopedia of Database Systems

Living Edition
| Editors: Ling Liu, M. Tamer Özsu

Language Models

  • Djoerd HiemstraEmail author
Living reference work entry
DOI: https://doi.org/10.1007/978-1-4899-7993-3_923-2

Synonyms

Definition

A language model assigns a probability to a piece of unseen text, based on some training data. For example, a language model based on a big English newspaper archive is expected to assign a higher probability to “a bit of text” than to “aw pit tov tags,” because the words in the former phrase (or word pairs or word triples if so-called N-gram models are used) occur more frequently in the data than the words in the latter phrase. For information retrieval, typical usage is to build a language model for each document. At search time, the top ranked document is the one whose language model assigns the highest probability to the query.

Historical Background

The term language models originates from probabilistic models of language generation developed for automatic speech recognition systems in the early 1980s [9]. Speech recognition systems use a language model to complement the results of the acoustic modelwhich models the relation between words (or...

This is a preview of subscription content, log in to check access.

Recommended Reading

  1. 1.
    Allan J, Aslam J, Belkin N, Buckley C, Callan J, Croft B, Dumais S, Fuhr N, Harman D, Harper DJ, Hiemstra D, Hofmann T, Hovy E, Kraaij W, Lafferty J, Lavrenko V, Lewis D, Liddy L, Manmatha R, McCallum A, Ponte J, Prager J, Radev D, Resnik P, Robertson S, Rosenfeld R, Roukos S, Sanderson M, Schwartz R, Singhal A, Smeaton A, Turtle H, Voorhees E, Weischedel E, Xu J, Zhai CX, editors. Challenges in information retrieval and language modeling. SIGIR Forum. 2003;37(1):31–47.Google Scholar
  2. 2.
    Balog K, Azzopardi L, Rijke M. Formal models for expert finding in enterprise corpora. In: Proceedings of 29th annual international ACM SIGIR conference on research and development in information retrieval; 2006. p. 43–50.Google Scholar
  3. 3.
    Basharin GP, Langville AN, Naumov VA. The life and work of A.A. Markov. Linear Algebra Appl. 2004;386:3–26.MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Berger A, Lafferty J. Information retrieval as statistical translation. In: Proceedings of 22nd ACM conference on research and development in information retrieval; 1999. p. 222–9.Google Scholar
  5. 5.
    Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. J Machine Learn Res. 2003;3(5):993–1022.zbMATHGoogle Scholar
  6. 6.
    Hiemstra D, Jong F. Disambiguation strategies for cross-language information retrieval. Lecture notes in computer science. In: Proceedings of the European conference on digital libraries, vol. 1696. Berlin: Springer; 1999. p. 274–93.Google Scholar
  7. 7.
    Hiemstra D, Kraaij W. Twenty-one at TREC-7: ad-hoc and cross-language track. In: Proceedings of 7th text retrieval conference TREC-7. NIST special publication 500-242. 1998. p. 227–38.Google Scholar
  8. 8.
    Hofmann T. Probabilistic latent semantic indexing. In: Proceedings of 22nd annual international ACM SIGIR conference on research and development in information retrieval; 1999. p. 50–57.Google Scholar
  9. 9.
    Jelinek F. Statistical methods for speech recognition. Cambridge, MA: MIT Press; 1997.Google Scholar
  10. 10.
    Jin H, Schwartz R, Sista S, Walls F. Topic tracking for radio, TV broadcast and newswire. In: Proceedings of DARPA broadcast news workshop; 1999.Google Scholar
  11. 11.
    Kraaij W, Westerveld T, Hiemstra D. The importance of prior probabilities for entry page search. In: Proceedings of 25th ACM conference on research and development in information retrieval (SIGIR’02); 2002. p. 27–34.Google Scholar
  12. 12.
    Kraft DH, Bruce Croft W, Harper DJ, Zobel J. Proceedings of 24th ACM conference on research and development in information retrieval. Association for Computing Machinery; 2001.Google Scholar
  13. 13.
    Lavrenko V, Croft WB. Relevance models in information retrieval. In: Bruce Croft W, Lafferty J, editors. Language modeling for information retrieval. Kluwer: Dordecht; 2003. p. 11–56.CrossRefGoogle Scholar
  14. 14.
    Miller DRH, Leek T, Schwartz RM. A hidden Markov model information retrieval system. In: Proceedings of 22nd ACM conference research and development in information retrieval; 1999. p. 214–21.Google Scholar
  15. 15.
    Ponte JM, Bruce CW. A language modeling approach to information retrieval. In: Proceedings of 21st ACM conference research and development in information retrieval; 1998. p. 275–81.Google Scholar
  16. 16.
    Schwartz RM, Sista S, Leek T. Unsupervised topic discovery. In: Proceedings of language models for information retrieval workshop; 2001.Google Scholar
  17. 17.
    Shannon CE. A mathematical theory of communication. Bell Syst Tech J. 1948;27(379–423):623–56.MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Spitters M, Kraaij W. Language models for topic tracking. In: Bruce Croft W, Lafferty J, editors. Language modeling for information retrieval. Dordecht: Kluwer; 2003. p. 95–124.Google Scholar
  19. 19.
    Xu J, Weischedel R. A probabilistic approach to term translation for cross-lingual retrieval. In: Bruce Croft W, Lafferty J, editors. Language modeling for information retrieval. Dordecht: Kluwer; 2003. p. 125–40.CrossRefGoogle Scholar
  20. 20.
    Zhai C, Lafferty J. Model-based feedback in the language modeling approach to information retrieval. In: Proceedings of ACM international conference on information and knowledge management; 2001. p. 403–10.Google Scholar

Copyright information

© Springer Science+Business Media LLC 2017

Authors and Affiliations

  1. 1.University of TwenteEnschedeThe Netherlands

Section editors and affiliations

  • Giambattista Amati
    • 1
  1. 1.Fondazione Ugo BordoniRomeItaly