Abstract
Quantum machine learning (QML) is a promising field that combines the power of quantum computing with machine learning. Variational quantum circuits, where parameters of circuits are learned classically, have been widely used in many recent applications of QML. This is an instance of a hybrid quantum-classical framework, where both classical and quantum components are present. However, applying these techniques to applications involving massive data is a challenging task. One way to overcome this, is using the concept of classical-quantum transfer learning with the help of a dressed quantum circuit, introduced recently, where the underlying neural architecture is pre-trained classically, but at the final steps (decision layer), a quantum circuit is used, followed by quantum measurements and post-processing to classify images with high precision. In this paper, we applied hybrid classical-quantum transfer learning to another task of massive data processing, i.e., natural language processing (NLP). We show how to (binary) classify short texts (e.g., SMS) with classical-quantum transfer learning, which was originally applied to image processing only. Our quantum network uses pre-trained Bidirectional Encoder Representations from the Transformers (BERT) model, and its variational quantum circuit is fine-tuned for text processing. We evaluated the performance of our hybrid neural architecture using the receiver operating characteristic (ROC) curve, which is typically used in the evaluation of classification problems. The results indicate high precision as well as lower loss function. To our knowledge, our work is the first application of quantum transfer learning to the area of NLP. Finally, a comparison with a tool that uses learning but in a different way than transfer learning is presented.
Similar content being viewed by others
Availability of data and materials
All materials including codes for experimentation and data sets and relevant explanations are available at: https://github.com/mehdinasiri1373/Classical-to-quantum-transfer-learning-for-Natural-Language-Processing-NLP-.git
References
Abramsky S, Hardy L (2012) Logical bell inequalities. Phys Rev A 85:062114. https://doi.org/10.1103/PhysRevA.85.062114
Adhikary S, Dangwal S, Bhowmik D (2020) Supervised learning with a quantum classifier using multi-level systems. Quantum Inf Process 19(3):89. https://doi.org/10.1007/s11128-020-2587-9. Accessed 2023-05-31
Azevedo V, Silva C, Dutra I (2022) Quantum transfer learning for breast cancer detection. Quantum Mach Intell 4. https://doi.org/10.1007/s42484-022-00062-4
Benedetti M, Realpe-Gómez J, Perdomo-Ortiz A (2018) Quantum-assisted Helmholtz machines: a quantum-classical deep learning framework for industrial datasets in near-term devices. Quantum Sci Technol 3(3):034007. https://doi.org/10.1088/2058-9565/aabd98. Accessed 2023-05-31
Bergholm V, Izaac JA, Schuld M, Gogolin C, Killoran N (2018) Pennylane: automatic differentiation of hybrid quantum-classical computations. arXiv:1811.04968
Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein J, Doran C, Solorio T (eds) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, vol 1 (Long and Short Papers), pp 4171–4186. Association for Computational Linguistics, ???
Georgiou GM, Koutsougeras C (1992) Complex domain backpropagation. IEEE Trans Circuits Syst II: Analog Digit Signal Process 39(5):330–334. https://doi.org/10.1109/82.142037newpage
Kartsaklis D, Fan I, Yeung R, Pearson A, Lorenz R, Toumi A, Felice G, Meichanetzidis K, Clark S, Coecke B (2021) lambeq: an efficient high-level Python library for quantum NLP. arXiv:2110.04236
Li Q, Wang B, Zhu Y, Lioma C, Liu Q (2023) Adapting pre-trained language models for quantum natural language processing. https://doi.org/10.48550/arXiv.2302.13812_2302.13812. arXiv:2302.13812
Lorenz R, Pearson A, Meichanetzidis K, Kartsaklis D, Coecke B (2023) QNLP in practice: running compositional models of meaning on a quantum computer. J Artif Intell Res 76:1305–1342
Mari A, Bromley TR, Izaac J, Schuld M, Killoran N (2020) Transfer learning in hybrid classical-quantum neural networks. Quantum 4:340. https://doi.org/10.22331/q-2020-10-09-340
Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359. https://doi.org/10.1109/TKDE.2009.191
Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359. https://doi.org/10.1109/TKDE.2009.191
Raina R, Battle A, Lee H, Packer B, Ng AY (2007) Self-taught learning: transfer learning from unlabeled data. In: Proceedings of the 24th international conference on machine learning. ICML ’07, pp 759–766. Association for Computing Machinery, New York, USA. https://doi.org/10.1145/1273496.1273592
Scardapane S, Van Vaerenbergh S, Hussain A, Uncini A (2018) Complex-valued neural networks with nonparametric activation functions. IEEE Trans Emerg Topics Comput Intell 4:140–150. https://doi.org/10.1109/TETCI.2018.2872600
Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: Proceedings of the 27th international conference on neural information processing systems - vol 2. NIPS’14, pp 3320–3328. MIT Press, Cambridge, MA, USA
Author information
Authors and Affiliations
Contributions
E. Ardeshir-Larijani and M Nasiri wrote the main manuscript. Mehdi Nasiri has done the experimentation and coding. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ardeshir-Larijani, E., Nasiri Fatmehsari, M.M. Hybrid classical-quantum transfer learning for text classification. Quantum Mach. Intell. 6, 19 (2024). https://doi.org/10.1007/s42484-024-00147-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s42484-024-00147-2