Skip to main content
SpringerLink
Log in

Near-term advances in quantum natural language processing

  • Regular Submission
  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

This paper describes experiments showing that some tasks in natural language processing (NLP) can already be performed using quantum computers, though so far only with small datasets. We demonstrate various approaches to topic classification. The first uses an explicit word-based approach, in which word-topic weights are implemented as fractional rotations of individual qubits, and a phrase is classified based on the accumulation of these weights onto a scoring qubit, using entangling quantum gates. This is compared with more scalable quantum encodings of word embedding vectors, which are used to compute kernel values in a quantum support vector machine: this approach achieved an average of 62% accuracy on classification tasks involving over 10000 words, which is the largest such quantum computing experiment to date. We describe a quantum probability approach to bigram modeling that can be applied to understand sequences of words and formal concepts, investigate a generative approximation to these distributions using a quantum circuit Born machine, and introduce an approach to ambiguity resolution in verb-noun composition using single-qubit rotations for simple nouns and 2-qubit entangling gates for simple verbs. The smaller systems presented have been run successfully on physical quantum computers, and the larger ones have been simulated. We show that statistically meaningful results can be obtained, but the quality of individual results varies much more using real datasets than using artificial language examples from previous quantum NLP research. Related NLP research is compared, partly with respect to contemporary challenges including informal language, fluency, and truthfulness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

The datasets that were analyzed in the current study may be found at the following public domain resources:\(\bullet \)Lambeq: CQCL Github repository at https://github.com/CQCL/lambeq\(\bullet \)IMDb: https://ai.stanford.edu/~amaas/data/sentiment/\(\bullet \)University of South Florida Free Association Norms: Available through the supplementary material at https://doi.org/10.3758/BF03195588

References

  1. Widdows, D.: A mathematical model for context and word-meaning. In: International and Interdisciplinary Conference on Modeling and Using Context, pp. 369–382 (2003). Springer

  2. Dirac, P.: The Principles of Quantum Mechanics, 4th edition, 1958, reprinted, 1982nd edn. Clarendon Press, Oxford (1930)

    Google Scholar 

  3. Orrell, D.: Quantum Economics and Finance: An Applied Mathematics Introduction. Panda Ohana Publishing, New York (2020)

    Google Scholar 

  4. Busemeyer, J.R., Bruza, P.D.: Quantum Models of Cognition and Decision. Cambridge University Press, (2012)

  5. Schrödinger, E.: Discussion of probability relations between separated systems. In: Mathematical Proceedings of the Cambridge Philosophical Society, vol. 31, pp. 555–563 (1935). Cambridge University Press

  6. Cooke, H.P., Treddenick, H. (eds.): Aristotle: Categories. Prior Analytics. Loeb Classical Library. Harvard University Press, On Interpretation (1938)

    Google Scholar 

  7. Piedeleu, R., Kartsaklis, D., Coecke, B., Sadrzadeh, M.: Open system categorical quantum semantics in natural language processing (2015). arXiv:1502.00831

  8. Coecke, B., de Felice, G., Meichanetzidis, K., Toumi, A.: Foundations for near-term quantum natural language processing (2020). arXiv:2012.03755

  9. Géron, A.: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. O’Reilly Media, (2019)

  10. Nielsen, M.A., Chuang, I.: Quantum Computation and Quantum Information. American Association of Physics Teachers, Cambridge University Press Edition, 2016 (2002)

  11. Van Rijsbergen, C.J.: The Geometry of Information Retrieval. Cambridge University Press, (2004)

  12. Widdows, D.: Geometry and Meaning. CSLI Publications, Stanford (2004)

    Google Scholar 

  13. Sordoni, A., Nie, J.-Y., Bengio, Y.: Modeling Term Dependencies with Quantum Language Models for IR. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR’13, pp. 653–662 (2013)

  14. Cohen, T., Widdows, D., Vine, L.D., Schvaneveldt, R., Rindflesch, T.C.: Many paths lead to discovery: Analogical retrieval of cancer therapies. In: Sixth International Symposium on Quantum Interaction (2012)

  15. Birkhoff, G., von Neumann, J.: The logic of quantum mechanics. Ann. Math. 37, 823–843 (1936)

    Article  MathSciNet  Google Scholar 

  16. Widdows, D., Kitto, K., Cohen, T.: Quantum mathematics in artificial intelligence. Journal of Artificial Intelligence Research. 72, 1307–1341 (2021)

    Article  MathSciNet  Google Scholar 

  17. Orrell, D., Houshmand, M.: Quantum propensity in economics. Frontiers in Artificial Intelligence 4 (2022). https://doi.org/10.3389/frai.2021.772294

  18. Pothos, E.M., Busemeyer, J.R.: Quantum cognition. Annu. Rev. Psychol. 73, 749–778 (2022)

    Article  Google Scholar 

  19. Wright, K., Beck, K.M., Debnath, S., Amini, J., Nam, Y., Grzesiak, N., Chen, J.-S., Pisenti, N., Chmielewski, M., Collins, C., et al.: Benchmarking an 11-qubit quantum computer. Nat. Commun. 10(1), 1–6 (2019)

    Article  Google Scholar 

  20. IonQ Aria: IonQ Aria Furthers Lead As World’s Most Powerful Quantum Computer. accessed 2022-05-28 (2022). https://ionq.com/news/february-23-2022-ionq-aria-furthers-lead

  21. ANIS, M.S., Abby-Mitchell, Abraham, H., AduOffei, Agarwal, R., Agliardi, G., other authors: Qiskit: An Open-source Framework for Quantum Computing (2021). https://doi.org/10.5281/zenodo.2573505

  22. Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., Brown, D.: Text classification algorithms: A survey. Information 10(4), 150 (2019)

    Article  Google Scholar 

  23. Schuld, M., Killoran, N.: Quantum machine learning in feature hilbert spaces. Phys. Rev. Lett. 122(4), 040504 (2019)

    Article  Google Scholar 

  24. Schuld, M.: Supervised quantum machine learning models are kernel methods (2021). arXiv:2101.11020

  25. Alvarez-Rodriguez, U., Sanz, M., Lamata, L., Solano, E.: The forbidden quantum adder. Scientific reports. 5(1), 1–3 (2015)

    Google Scholar 

  26. Widdows, D.: Nonlinear addition of qubit states using entangled quaternionic powers of single-qubit gates (2022). arXiv:2204.13787

  27. Alexander, A., Widdows, D.: Quantum text encoding for classification tasks. In: 2022 IEEE/ACM 7th Symposium on Edge Computing, pp. 355–361 (2022)

  28. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space (2013). arXiv:1301.3781

  29. Havlicek, V., Corcoles, A., Temme, K., other authors.: Supervised learning with quantum-enhanced feature spaces. Nature. 567(7747), 212–567 (2019)

  30. Kartsaklis, D., Fan, I., Yeung, R., Pearson, A., Lorenz, R., Toumi, A., de Felice, G., Meichanetzidis, K., Clark, S., Coecke, B.: lambeq: An Efficient High-Level Python Library for Quantum NLP (2021). arXiv:2110.04236

  31. Lorenz, R., Pearson, A., Meichanetzidis, K., Kartsaklis, D., Coecke, B.: QNLP in practice: Running compositional models of meaning on a quantum computer (2021). arXiv:2102.12846

  32. Maas, A.L., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pp. 142–150. ACL, Portland, Oregon, USA (2011)

  33. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., Le, Q.V.: Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems 32 (2019)

  34. Chen, J.-S., Nielsen, E., Ebert, M., Inlek, V., Wright, K., Chaplin, V., Maksymov, A., Páez, E., Poudel, A., Maunz, P., et al.: Benchmarking a trapped-ion quantum computer with 29 algorithmic qubits (2023). arXiv:2308.05071

  35. Ruskanda, F.Z., Abiwardani, M.R., Al Bari, M.A., Bagaspati, K.A., Mulyawan, R., Syafalni, I., Larasati, H.T.: Quantum representation for sentiment classification. In: 2022 IEEE International Conference on Quantum Computing and Engineering (QCE), pp. 67–78 (2022). IEEE

  36. Ganguly, S., Morapakula, S.N., Coronado, L.M.P.: Quantum natural language processing based sentiment analysis using lambeq toolkit. In: 2022 Second International Conference on Power, Control and Computing Technologies (ICPC2T), pp. 1–6 (2022). IEEE

  37. Martinez, V., Leroy-Meline, G.: A multiclass q-nlp sentiment analysis experiment using discocat (2022). arXiv:2209.03152

  38. Stein, J., Christ, I., Kraus, N., Mansky, M.B., Müller, R., Linnhof Popien, C.: Applying qnlp to sentiment analysis in finance (2023). arXiv:2307.11788

  39. Shannon, C.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423623656 (1948)

    Article  MathSciNet  Google Scholar 

  40. Manning, C.D., Schütze, H.: Foundations of Statistical Natural Language Processing. The MIT Press, Cambridge, Massachusetts (1999)

    Google Scholar 

  41. Bradley, T.-D.: At the interface of algebra and statistics. PhD thesis, City University of New York (2020)

  42. Araujo, I.F., Park, D.K., Petruccione, F., da Silva, A.J.: A divide-and-conquer algorithm for quantum state preparation. Nat. Sci. Rep. 11(1), 6329 (2021)

    Google Scholar 

  43. Spall, J.C.: An overview of the simultaneous perturbation method for efficient optimization. J. Hopkins APL Tech. Dig. 19(4), 482–492 (1998)

    Google Scholar 

  44. Nelson, D.L., McEvoy, C.L., Schreiber, T.A.: The University of South Florida free association, rhyme, and word fragment norms. Behavior Research Methods, Instruments, & Computers 36(3), 402–407 (2004)

    Article  Google Scholar 

  45. Bruza, P., Kitto, K., Nelson, D., McEvoy, C.: Is there something quantum like about the human mental lexicon? J. Math. Psychol. 53(5), 362–377 (2009)

    Article  MathSciNet  Google Scholar 

  46. Chandrasekaran, D., Mago, V.: Evolution of semantic similarity-a survey. ACM Computing Surveys (CSUR) 54(2), 1–37 (2021)

    Article  Google Scholar 

  47. IonQ Benchmarking: Algorithmic Qubits: A Better Single-Number Metric. https://ionq.com/posts/february-23-2022-algorithmic-qubits, Accessed 2022-09-19 (2022)

  48. Leskovec, J., Faloutsos, C.: Sampling from large graphs. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 631–636 (2006)

  49. Aitchison, J.: Words in the Mind: An Introduction to the Mental Lexicon, 3rd edn. Blackwell, (2002)

  50. Widdows, D.: Unsupervised methods for developing taxonomies by combining syntactic and statistical information. In: Proceedings of North American Chapter of the Association for Computational Linguistics, Edmonton, Canada (2003)

  51. Baroni, M., Bernardi, R., Zamparelli, R., et al.: Frege in space: A program for compositional distributional semantics. Linguistic Issues in language technology 9(6), 5–110 (2014)

    Google Scholar 

  52. Schütze, H.: Automatic word sense discrimination. Comput. Linguist. 24(1), 97–124 (1998)

    Google Scholar 

  53. Baroni, M., Zamparelli, R.: Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2010)

  54. Coecke, B., Sadrzadeh, M., Clark, S.: Mathematical foundations for a compositional distributional model of meaning (2010). arXiv:1003.4394

  55. Borujeni, S.E., Nannapaneni, S., Nguyen, N.H., Behrman, E.C., Steck, J.E.: Quantum circuit representation of Bayesian networks. Expert Syst. Appl. 176, 114768 (2021)

    Article  Google Scholar 

  56. Johri, S., Debnath, S., Mocherla, A., Singk, A., Prakash, A., Kim, J., Kerenidis, I.: Nearest centroid classification on a trapped ion quantum computer. npj Quantum Information 7(1), 1–11 (2021)

  57. Wang, K., Xiao, L., Yi, W., Ran, S.-J., Xue, P.: Experimental realization of a quantum image classifier via tensor-network-based machine learning. Photonics Research 9(12), 2332–2340 (2021)

    Article  Google Scholar 

  58. Johri, S., Zhu, E., Bacon, D., Esencan, M., Kim, J., Muir, M., Murgai, N., Nguyen, J., Pisenti, N., Schouela, A., et al.: Generative quantum learning of joint probability distribution functions. Bulletin of the American Physical Society (2022)

  59. Coecke, B., de Felice, G., Meichanetzidis, K., Toumi, A.: Quantum Natural Language Processing: “We did it! On an actual quantum computer!”. https://medium.com/cambridge-quantum-computing/quantum-natural-language-processing-748d6f27b31d (2020)

  60. Partee, B.H.: Montague Grammar. Academic Press Inc, Cambridge (1976)

    Google Scholar 

  61. Lewis, D.: General semantics. In: Montague Grammar, pp. 1–50. Academic Press, Inc., Cambridge (1976)

  62. Wiebe, N., Bocharov, A., Smolensky, P., Troyer, M., Svore, K.M.: Quantum language processing. arXiv preprint arXiv:1902.05162 (2019)

  63. Palangi, H., Huang, Q., Smolensky, P., He, X., Deng, L.: Grammatically interpretable learned representations in deep NLP models. In: Advances in Neural Information Processing Systems Workshop (2017)

  64. McCoy, R.T., Linzen, T., Dunbar, E., Smolensky, P.: Tensor product decomposition networks: Uncovering representations of structure learned by neural networks. Proceedings of the Society for Computation in Linguistics 3(1), 474–475 (2020)

    Google Scholar 

  65. Panahi, A., Saeedi, S., Arodz, T.: word2ket: Space-efficient word embeddings inspired by quantum entanglement (2019). arXiv:1911.04975

  66. Floridi, L., Chiriatti, M.: GPT-3: Its nature, scope, limits, and consequences. Mind. Mach. 30(4), 681–694 (2020)

    Article  Google Scholar 

  67. Sobieszek, A., Price, T.: Playing games with AIs: The limits of GPT-3 and similar large language models. Mind. Mach. 32(2), 341–364 (2022)

    Article  Google Scholar 

  68. Chomsky, N.: Syntactic Structures. Mouton de Gruyter, The Hague (1957)

    Book  Google Scholar 

  69. Chomsky, N.: Aspects of the Theory of Syntax vol. 11. MIT press, (1965)

  70. Jackendoff, R.: Foundations of Language. Oxford Universiry Press, (2002)

  71. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)

  72. Yang, C.-H.H., Qi, J., Chen, S.Y.-C., Chen, P.-Y., Siniscalchi, S.M., Ma, X., Lee, C.-H.: Decentralizing feature extraction with quantum convolutional neural network for automatic speech recognition. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6523–6527 (2021). IEEE

  73. Wu, Y., Mao, W., Feng, J.: AI for online customer service: Intent recognition and slot filling based on deep learning technology. Mobile Networks and Applications, 1–13 (2021)

Download references

Author information

Authors and Affiliations

  1. IonQ Inc., College Park, Maryland, USA

    Dominic Widdows, Aaranya Alexander, Daiwei Zhu, Chase Zimmerman & Arunava Majumder

Authors
  1. Dominic Widdows
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aaranya Alexander
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daiwei Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chase Zimmerman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Arunava Majumder
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dominic Widdows.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Widdows, D., Alexander, A., Zhu, D. et al. Near-term advances in quantum natural language processing. Ann Math Artif Intell (2024). https://doi.org/10.1007/s10472-024-09940-y

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10472-024-09940-y

Keywords

Mathematics Subject Classification (2010)

Search

Navigation