Abstract
Traditional quantum system control methods often face different constraints, and are easy to cause both leakage and stochastic control errors under the condition of limited resources. Reinforcement learning has been proved as an efficient way to complete the quantum system control task. To learn a satisfactory control strategy under the condition of limited resources, a quantum system control method based on enhanced reinforcement learning (QSC-ERL) is proposed. The states and actions in reinforcement learning are mapped to quantum states and control operations in quantum systems. By using new enhanced neural networks, reinforcement learning can quickly achieve the maximization of long-term cumulative rewards, and a quantum state can be evolved accurately from an initial state to a target state. According to the number of candidate unitary operations, the three-switch control is used for simulation experiments. Compared with other methods, the QSC-ERL achieves close to 1 fidelity learning control of quantum systems, and takes fewer episodes to quantum state evolution under the condition of limited resources.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Enquiries about data availability should be directed to the authors.
References
Abualigah L, Diabat A, Mirjalili S, Abd Elaziz M, Gandomi AH (2021a) The arithmetic optimization algorithm. Comput Methods Appl Mech Eng 376:113609. https://doi.org/10.1016/j.cma.2020.113609
Abualigah L, Diabat A, Sumari P, Gandomi A (2021b) Applications, deployments, and integration of internet of drones (iod): a review. IEEE Sens J. https://doi.org/10.1109/JSEN.2021.3114266
Abualigah L, Elsayed Abd Elaziz M, Sumari P, Geem ZW, Gandomi A (2021c) Reptile search algorithm (rsa): a nature-inspired meta-heuristic optimizer. Expert Syst Appl 191:116158. https://doi.org/10.1016/j.eswa.2021.116158
Abualigah L, Yousri D, Abd Elaziz M, Ewees AA, Al-qaness MA, Gandomi AH (2021d) Aquila optimizer: a novel meta-heuristic optimization algorithm. Comput Ind Eng 157:107250. https://doi.org/10.1016/j.cie.2021.107250
An Z, Zhou D (2019) Deep reinforcement learning for quantum gate control. EPL (Europhysics Letters) 126(6):60002
An Z, Song HJ, He QK, Zhou D (2021) Quantum optimal control of multilevel dissipative quantum systems with reinforcement learning. Phys Rev A 103(1):012404
Bukov M (2018) Reinforcement learning for autonomous preparation of floquet-engineered states: inverting the quantum kapitza oscillator. Phys Rev B 98(22):224305
Bukov M, Day AG, Sels D, Weinberg P, Polkovnikov A, Mehta P (2018) Reinforcement learning in different phases of quantum control. Phys Rev X 8(3):031086
Cárdenas-López FA, Lamata L, Retamal JC, Solano E (2018) Multiqubit and multilevel quantum reinforcement learning with quantum technologies. PLoS ONE 13(7):e0200455
Chakrabarti R, Rabitz H (2007) Quantum control landscapes. Int Rev Phys Chem 26(4):671–735
Chen C, Dong D, Li HX, Chu J, Tarn TJ (2013) Fidelity-based probabilistic q-learning for control of quantum systems. IEEE Trans Neural Netw Learn Syst 25(5):920–933
Chu S (2002) Cold atoms and quantum control. Nature 416(6877):206–210
Chunlin C, Frank J, Daoyi D (2012) Hybrid control of uncertain quantum systems via fuzzy estimation and quantum reinforcement learning. In: Proceedings of the 31st Chinese Control Conference, IEEE, pp 7177–7182
D’Alessandro D, Dahleh M (2001) Optimal control of two-level quantum systems. IEEE Trans Autom Control 46(6):866–876
Dong D, Chen C, Li H, Tarn TJ (2008) Quantum reinforcement learning. IEEE Trans Syst Man Cybern Part B (Cybernetics) 38(5):1207–1220
Dong D, Chen C, Tarn TJ, Pechen A, Rabitz H (2008) Incoherent control of quantum systems with wavefunction-controllable subspaces via quantum reinforcement learning. IEEE Trans Syst Man Cybern Part B (Cybernetics) 38(4):957–962
Fang W, Pang L, Yi W (2020) Survey on the application of deep reinforcement learning in image processing. J Artif Intell 2(1):39–58
Fösel T, Tighineanu P, Weiss T, Marquardt F (2018) Reinforcement learning with neural networks for quantum feedback. Phys Rev X 8(3):031084
Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J et al (2018) Recent advances in convolutional neural networks. Pattern Recogn 77:354–377
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Hu B, Zhao H, Yang Y, Zhou B, Raj ANJ (2020) Multiple faces tracking using feature fusion and neural network in video. Intell Autom Soft Comput 26(6):1549–1560
Li Z, Zhang J, Zhang K, Li Z (2018) Visual tracking with weighted adaptive local sparse appearance model via spatio-temporal context learning. IEEE Trans Image Process 27(9):4478–4489
Ma H, Chen C (2020) Several developments in learning control of quantum systems. In: 2020 IEEE international conference on systems, man, and cybernetics (SMC), IEEE, pp 4165–4172
Meng F, Cong S (2022) Control design for state transition of open quantum system. J Phys Conf Series 2183:012005
Michael MH, Silveri M, Brierley R, Albert VV, Salmilehto J, Jiang L, Girvin SM (2016) New class of quantum error-correcting codes for a bosonic mode. Phys Rev X 6(3):031006
Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533
Niu MY, Boixo S, Smelyanskiy VN, Neven H (2019) Universal quantum control through deep reinforcement learning. npj Quant Inform 5(1):1–8
Palittapongarnpim P, Wittek P, Sanders BC (2017) Robustness of learning-assisted adaptive quantum-enhanced metrology in the presence of noise. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC), IEEE, pp 294–299
Patsch S, Maniscalco S, Koch CP (2020) Simulation of open-quantum-system dynamics using the quantum zeno effect. Phys Rev Res 2(2):023133
Rabitz H, de Vivie-Riedle R, Motzkus M, Kompa K (2000) Whither the future of controlling quantum phenomena? Science 288(5467):824–828
Roslund J, Rabitz H (2009) Gradient algorithm applied to laboratory quantum control. Phys Rev A 79(5):053417
Singh SP, Sutton RS (1996) Reinforcement learning with replacing eligibility traces. Mach Learn 22(1):123–158
Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT press
Sutton RS, McAllester DA, Singh SP, Mansour Y (2000) Policy gradient methods for reinforcement learning with function approximation. In: Advances in neural information processing systems, pp 1057–1063
Torosov BT, Shore BW, Vitanov NV (2021) Coherent control techniques for two-state quantum systems: a comparative study. Phys Rev A 103(3):033110
Tsubouchi M, Momose T (2008) Rovibrational wave-packet manipulation using shaped midinfrared femtosecond pulses toward quantum computation: optimization of pulse shape by a genetic algorithm. Phys Rev A 77(5):052326
Vedaie SS, Palittapongarnpim P, Sanders BC (2018) Reinforcement learning for quantum metrology via quantum control. In: 2018 IEEE photonics society summer topical meeting series (SUM), IEEE, pp 163–164
Vrajitoarea A, Huang Z, Groszkowski P, Koch J, Houck AA (2020) Quantum control of an oscillator using a stimulated josephson nonlinearity. Nat Phys 16(2):211–217
Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292
Xu F, Zhang X, Xin Z, Yang A (2019) Investigation on the chinese text sentiment analysis based on convolutional neural networks in deep learning. Comput Mater Contin 58(3):697–709
Yu S, Albarrán-Arriagada F, Retamal JC, Wang YT, Liu W, Ke ZJ, Meng Y, Li ZP, Tang JS, Solano E et al (2019) Reconstruction of a photonic qubit state with reinforcement learning. Adv Quant Technol 2(7–8):1800074
Zhang XM, Wei Z, Asad R, Yang XC, Wang X (2019) When does reinforcement learning stand out in quantum control? A comparative study on state preparation. npj Quant Inform 5(1):1–7
Zhang Y, Wang Z (2020) Hybrid malware detection approach with feedback-directed machine learning. Inf Sci 63(139103):1–139103
Acknowledgements
The authors would like to thank the anonymous reviewers and editors for their comments that improved the quality of this paper. This work is supported by the National Natural Science Foundation of China (62071240, 61802175), and the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD).
Funding
The authors have not disclosed any funding.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical statement
Articles do not rely on clinical trials.
Human and animal participants
All submitted manuscripts contain research which do not involve human participants and/or animal experimentation.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liu, W., Wang, B., Fan, J. et al. A quantum system control method based on enhanced reinforcement learning. Soft Comput 26, 6567–6575 (2022). https://doi.org/10.1007/s00500-022-07179-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-022-07179-5