Abstract
Language modeling is a very important step in several NLP applications. Most of the current language models are based on probabilistic methods. In this paper, we propose a new language modeling approach based on the possibility theory. Our goal is to suggest a method for estimating the possibility of a word-sequence and to test this new approach in a machine translation system. We propose a word-sequence possibilistic measure, which can be estimated from a corpus. We proceeded in two ways: first, we checked the behavior of the new approach compared with the existing work. Second, we compared the new language model with the probabilistic one used in statistical MT systems. The results, in terms of the METEOR metric, show that the possibilistic-language model is better than the probabilistic one. However, in terms of BLEU and TER scores, the probabilistic model remains better.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
A linguistic event is the succession of one or several words.
- 2.
This is possible for close languages which share several words such as English and French.
References
Bengio, Y., Schwenk, H., Senécal, J.S., Morin, F., Gauvain, J.L.: Neural probabilistic language models. In: Holmes, D.E., Jain, L.C. (eds.) Innovations in Machine Learning. Studies in Fuzziness and Soft Computing, vol. 194, pp. 137–186. Springer, Heidelberg (2006). https://doi.org/10.1007/3-540-33486-6_6
Brown, P.F., Cocke, J., Pietra, S.A.D., Pietra, V.J.D., Jelinek, F., Lafferty, J.D., Mercer, R.L., Roossin, P.S.: A Statistical Approach to Machine Translation, vol. 16, pp. 79–85. MIT Press, Cambridge (1990)
Callison-Burch, C., Fordyce, C., Koehn, P., Monz, C., Schroeder, J.: Further meta-evaluation of machine translation. In: Proceedings of the Third Workshop on Statistical Machine Translation (StatMT 2008), pp. 70–106. Association for Computational Linguistics, Stroudsburg (2008)
Chen, S.F., Goodman, J.: An empirical study of smoothing techniques for language modeling. In: Proceedings of the 34th Annual Meeting on Association for Computational Linguistics (ACL 1996), pp. 310–318. Association for Computational Linguistics, Stroudsburg (1996)
Clark, J.H., Dyer, C., Lavie, A., Smith, N.A.: Better hypothesis testing for statistical machine translation: controlling for optimizer instability. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (HLT 2011) (Short Papers - Volume 2), pp. 176–181. Association for Computational Linguistics, Stroudsburg (2011)
Deligne, S., Bimbot, F.: Language modeling by variable length sequences: theoretical formulation and evaluation of multigrams. In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP-1995), vol. 1, pp. 169–172. IEEE (1995)
Denkowski, M., Lavie, A.: Meteor 1.3: automatic metric for reliable optimization and evaluation of machine translation systems. In: Proceedings of the Sixth Workshop on Statistical Machine Translation, pp. 85–91. Association for Computational Linguistics (2011)
Dubois, D., Prade, H.: Possibility theory. Scholarpedia 2(10), 2074 (2007)
Dubois, D.: Possibility theory and statistical reasoning. Comput. Stat. Data Anal. 51(1), 47–69 (2006)
Heafield, K.: Kenlm: Faster and smaller language model queries. In: Proceedings of the Sixth Workshop on Statistical Machine Translation (WMT 2011), pp. 187–197. Association for Computational Linguistics, Stroudsburg (2011)
Jelinek, F.: Continuous speech recognition by statistical-methods. vol. 64, pp. 532–556. IEEE Institute of Electrical Electronics Engineers, Inc., New York (1976)
Kneser, R., Ney, H.: Improved backing-off for m-gram language modeling. In: 1995 International Conference on Acoustics, Speech, and Signal Processing (ICASSP-1995), vol. 1, pp. 181–184. IEEE (1995)
Koehn, P.: Europarl: a parallel corpus for statistical machine translation. In: Proceedings of the Conference on Tenth Machine Translation Summit (AAMT), Phuket, pp. 79–86 (2005)
Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E.: Moses: open source toolkit for statistical machine translation. In: Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions (ACL 2007), pp. 177–180. Association for Computational Linguistics, Stroudsburg (2007)
Kuhn, R., De Mori, R.: A cache-based natural language model for speech recognition. IEEE Trans. Pattern Anal. Mach. Intell. 12(6), 570–583 (1990)
Lau, R., Rosenfeld, R., Roukos, S.: Trigger-based language models: a maximum entropy approach. In: Proceedings of the 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing: Speech Processing (ICASSP 1993), vol. 2, pp. 45–48. IEEE Computer Society, Washington, D.C. (1993)
Negoita, C., Zadeh, L., Zimmermann, H.: Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets Syst. 1, 3–28 (1978)
Oger, S., Linarès, G.: Web-based possibilistic language models for automatic speech recognition. Comput. Speech Lang. 28(4), 923–939 (2014)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics (ACL 2002), pp. 311–318. Association for Computational Linguistics, Stroudsburg (2002)
Rosenfeld, R.: Two decades of statistical language modeling: where do we go from here? p. 2000 (2000)
Saporta, G.: Probabilités, analyse des données et statistique. Editions Technip (2011)
Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of Association for Machine Translation in the Americas, pp. 223–231 (2006)
Srihari, R., Baltus, C.: Combining statistical and syntactic methods in recognizing handwritten sentences. In: AAAI Symposium: Probabilistic Approaches to Natural Language, pp. 121–127 (1992)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Menacer, M.A., Boumerdas, A., Zakaria, C., Smaili, K. (2018). A New Language Model Based on Possibility Theory. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2016. Lecture Notes in Computer Science(), vol 9623. Springer, Cham. https://doi.org/10.1007/978-3-319-75477-2_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-75477-2_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-75476-5
Online ISBN: 978-3-319-75477-2
eBook Packages: Computer ScienceComputer Science (R0)