Skip to main content
SpringerLink
Log in

Reinforcement learning for real-time process control in high-temperature superconductor manufacturing

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

With high efficiency and low energy loss, high-temperature superconductors (HTS) have demonstrated their profound applications in various fields, such as medical imaging, transportation, accelerators, microwave devices, and power systems. The high-field applications of HTS tapes have raised the demand for producing cost-effective tapes with long lengths in superconductor manufacturing. However, achieving the uniform and enhanced performance of a long HTS tape is challenging due to the unstable growth conditions in the manufacturing process. Although it is confirmed that the process parameters during the advanced metal organic chemical vapor deposition (A-MOCVD) process influence the uniformity of the produced HTS tapes, the high-dimensional process parameter signals and their complicated interactions make it difficult to develop an effective control policy. In this paper, we propose a local measure for the uniformity of HTS tapes to provide instant feedback for our control policy. Then, we model the manufacturing of HTS tapes as a Markov decision process (MDP) with continuous state and action spaces to assess the instant reward in real time in our feedback control model. As our MDP involves continuous and high-dimensional state and action spaces, a neural fitted Q-iteration (NFQ) algorithm is adopted to solve the MDP with artificial neural network (ANN) function approximation. The collinearity of process parameters can restrict our capability of adjusting the process parameters, which is addressed by the principal component analysis (PCA) in our method. The control policy adjusts the PCA of process parameters using the NFQ algorithm. Based on our case studies on real A-MOCVD dataset, the obtained control policy increases the average uniformity of tapes by 5.6% and performs especially well on sample HTS tapes with a low uniformity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Wang Y, Zheng Y (2014) Review of research and measurement for application properties of hts tapes. Sci China Technol Sci 57(8):1568–1577

    Article  Google Scholar 

  2. Ballarino A (2000) Application of high temperature superconductors to accelerators. Tech Rep

  3. Hull JR (2003) Applications of high-temperature superconductors in power technology. Rep Prog Phys 66(11):1865

    Article  Google Scholar 

  4. Lancaster MJ (2006) Passive microwave device applications of high-temperature superconductors. Cambridge University Press

    Google Scholar 

  5. Kalsi SS (2011) Applications of high temperature superconductors to electric power equipment. Wiley

    Book  Google Scholar 

  6. Fietz WH, Heller R, Schlachter SI, Goldacker W (2011) Application of high temperature superconductors for fusion. Fusion Eng Des 86(6–8):1365–1368

    Article  Google Scholar 

  7. Majkic G, Galstyan E, Selvamanickam V (2014) High performance 2G-HTS wire using a novel MOCVD system. IEEE Trans Appl Supercond 25(3):1–4

    Article  Google Scholar 

  8. Selvamanickam V, Heydari M, Gharahcheshmeh AX, Zhang Y, Galstyan E (2015) Critical current density above 15 ma cm- 2 at 30 k, 3 t in 2.2 \(\mu \)m thick heavily-doped (gd, y) ba2cu3ox superconductor tapes. Supercond Sci Technol 28(7):072002

  9. Wang T, Wang K, Tu R, Zhang S, Yang M, Qizhong L, Shi J, Li H, Goto T, Zhang L (2017) Thickness dependence of structure and superconductivity of the smba 2 cu 3 o 7 film by laser cvd. RSC Adv 7(89):56166–56172

    Article  Google Scholar 

  10. Majkic G, Pratap R, Xu A, Galstyan E, Selvamanickam V (2018) Over 15 ma/cm2 of critical current density in 4.8 \(\mu \)m thick, zr-doped (gd, y) ba2cu3ox superconductor at 30 k, 3t. Sci Rep 8(1):6982

  11. Gömöry F, Šouc J, Adámek M, Ghabeli A, Solovyov M, Vojenčiak M (2019) Impact of critical current fluctuations on the performance of a coated conductor tape. Supercond Sci Technol 32(12):124001

    Article  Google Scholar 

  12. Pitel J (2013) Differences between two definitions of the critical current of HTS coils. Supercond Sci Technol 26(12)

  13. Zou S, Gu C, Qu T, Chen S, Li X, Han Z (2014) Examination and analysis of critical current uniformity of long HTS tapes by the MCorder. IEEE Trans Appl Supercond 25(3):1–4

    Google Scholar 

  14. Higashikawa K, Guo X, Inoue M, Jiang Z, Badcock R, Long N, Kiss T (2016) Characterization of critical current distribution in Roebel cable strands based on reel-to-reel scanning hall-probe microscopy. IEEE Trans Appl Supercond 27(4):1–4

    Google Scholar 

  15. Vysotsky VS, Funaki K, Takeo M (2000) Current non-uniformity in multistrand superconducting cables. Experimental studies and its influence on stability of superconducting magnets. IEEE Trans Appl Supercond 10(1):1190–1195

  16. Lvovsky Y, Stautner EW, Zhang T (2013) Novel technologies and configurations of superconducting magnets for MRI. Supercond Sci Technol 26(9)

  17. Dew-Hughes D (2001) The critical current of superconductors: an historical review. Low Temp Phys 27(9):713–722

    Article  Google Scholar 

  18. Sato K-I (2008) Present status and future perspective of high-temperature superconductors. SEI TECHNICAL REVIEW-ENGLISH EDITION- 66:55

    Google Scholar 

  19. George Bednorz J, Alex Müller K (1986) Possible high t c superconductivity in the ba- la- cu- o system. Zeitschrift für Physik B Condensed Matter 64(2):189–193

    Article  Google Scholar 

  20. Wu M-K, Ashburn JR, Torng ClJ , Hor PH, Meng RL, Gao L, Huang ZJ, Wang YQ, Chu aCW, (1987) Superconductivity at 93 k in a new mixed-phase y-ba-cu-o compound system at ambient pressure. Phys Rev Lett 58(9):908

  21. Maeda H, Tanaka Y, Fukutomi M, Asano T (1988) A new high-Tc oxide superconductor without a rare earth element. Jpn J Appl Phys 27(2A):L209

    Article  Google Scholar 

  22. Mezzetti E, Gerbaldo R, Ghigo G, Gozzelino L, Minetti B, Camerlingo C, Monaco A, Cuttone G, Rovelli A (1999) Control of the critical current density in YBa 2 Cu 3 O 7- \(\delta \) films by means of intergrain and intragrain correlated defects. Phys Rev B 60(10):7623

    Article  Google Scholar 

  23. Matsumoto K, Horide T, Ichinose A, Horii S, Yoshida Y, Mukaida M (2005) Critical current control in yba2cu3o7-\(\delta \) films using artificial pinning centers. Jpn J Appl Phys 44(1L):L246

    Article  Google Scholar 

  24. Markelov A, Valikov A, Chepikov V, Petrzhik A, Massalimov B, Degtyarenko P, Uzkih R, Soldatenko A, Molodyk A, Sim K et al (2019) 2G HTS wire with enhanced engineering current density attained through the deposition of HTS layer with increased thickness. Prog Supercond Cryogenics 21(4):29–33

  25. Yamane H, Kurosawa H, Hirai T, Watanabe K, Iwasaki H, Kobayashi N, Muto Y (1989) Effect of deposition temperature on the superconducting properties of Y-Ba-Cu-O films prepared by CVD. J Cryst Growth 98(4):860–866

    Article  Google Scholar 

  26. Hu X, Rossi L, Stangl A, Sinclair JW, Kametani F, Abraimov D, Polyanskii A, Coulter JY, Jaroszynski J, Larbalestier DC (2016) An experimental and analytical study of periodic and aperiodic fluctuations in the critical current of long coated conductors. IEEE Trans Appl Supercond 27(4):1–5

    Google Scholar 

  27. Kashima N, Niwa T, Mori M, Nagaya S, Muroga T, Miyata S, Watanabe T, Yamada Y, Izumi T, Shiohara Y (2005) Fabrication of coated conductors by multiple-stage CVD. IEEE Trans Appl Supercond 15(2):2763–2766

    Article  Google Scholar 

  28. Zhang G, Janssens SD, Vanacken J, Timmermans M, Vacík J, Ataklti GW, Decelle W, Gillijns W, Goderis B, Haenen K et al (2011) Role of grain size in superconducting boron-doped nanocrystalline diamond thin films grown by CVD. Phys Rev B 84(21)

  29. Peng S, Li M, Lin Y, Feng Q, Fu W, Galstyan E, Chen S, Jain R (2022) Dynamic uniformity modeling in superconductor manufacturing via vector autoregression analysis. In: IIE annual conference. Proceedings, Institute of industrial and systems engineers (IISE), pp 1–6

  30. Li M, Peng S, Lin Y, Feng Q, Fu W, Galstyan E, Chen S, Jain R (2022) A spatial point process-based approach for dropout events modeling in high-temperature superconductor manufacturing. In: IIE annual conference. Proceedings, Institute of industrial and systems engineers (IISE), pp 1–6

  31. Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press

    MATH  Google Scholar 

  32. Ernst D, Geurts P, Wehenkel L (2005) Tree-based batch mode reinforcement learning. J Mach Learn Res 6(Apr):503–556

  33. Riedmiller M (2005) Neural fitted q iteration–first experiences with a data efficient neural reinforcement learning method. In: European conference on machine learning, Springer, pp 317–328

  34. Kubiczek K, Grilli F, Kario A, Godfrin A, Zermeno VMR, Stępień M, Kampik M (2018) Length uniformity of the angular dependences of i c and n of commercial REBCO tapes with artificial pinning at 77 k. IEEE Trans Appl Supercond 29(1):1–9

    Article  Google Scholar 

  35. Peng S et al (2021) Reinforcement learning with gaussian processes for condition-based maintenance. Comput Ind Eng 158:107321

    Article  Google Scholar 

  36. Riedmiller M, Braun H (1993) A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In: IEEE international conference on neural networks, IEEE, pp 586–591

  37. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980

  38. Paidpilli M, Pratap R, Kochat M, Galstyan E, Goel C, Majkic G, Selvamanickam V (2021) Growth of high-performance 4–5\(\mu \)m thick film REBCO tapes doped with hafnium using advanced MOCVD. IEEE Trans Appl Supercond 31(5):1–5

    Article  Google Scholar 

Download references

Acknowledgements

This paper is based upon work supported by the USA National Science Foundation (NSF) under Grant no. 1728321. and the Advanced Manufacturing Institute (AMI) at the University of Houston.

Funding

This paper is based upon work supported by the Advanced Manufacturing Institute (AMI) at the University of Houston. Feng has received research support from the USA National Science Foundation (NSF) under Grant no. 1728321.

Author information

Authors and Affiliations

  1. Department of Industrial Engineering, University of Houston, Houston, TX, 77204, USA

    Qianmei Feng, Shenglin Peng & Ying Lin

  2. Advanced Manufacturing Institute, University of Houston, Houston, TX, 77204, USA

    Qianmei Feng, Ying Lin, Siwei Chen, Mahesh Paidpilli, Chirag Goel, Eduard Galstyan & Venkat Selvamanickam

  3. Department of Mechanical Engineering, University of Houston, Houston, TX, 77204, USA

    Siwei Chen, Mahesh Paidpilli, Chirag Goel, Eduard Galstyan & Venkat Selvamanickam

  4. Texas Center for Superconductivity, Houston, TX, 77204, USA

    Siwei Chen, Mahesh Paidpilli, Chirag Goel, Eduard Galstyan & Venkat Selvamanickam

  5. Princeton Plasma Physics Laboratory, Princeton, NJ, 08540, USA

    Siwei Chen

Authors
  1. Qianmei Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shenglin Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Siwei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mahesh Paidpilli
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chirag Goel
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Eduard Galstyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Venkat Selvamanickam
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation and data collection were performed by Siwei Chen, Mahesh Paidpilli, Chirag Goel, Eduard Galstyan, and Venkat Selvamanickam. Data analysis and validation were conducted by Shenglin Peng. The first draft of the manuscript was written by Shenglin Peng, and Qianmei Feng and Ying Lin commented on and revised previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Qianmei Feng.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feng, Q., Peng, S., Lin, Y. et al. Reinforcement learning for real-time process control in high-temperature superconductor manufacturing. Int J Adv Manuf Technol 129, 2215–2225 (2023). https://doi.org/10.1007/s00170-023-12369-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-023-12369-y

Keywords

Search

Navigation