Skip to main content
SpringerLink
Log in

Quantum mechanics-based deep learning framework considering near-zero variance data

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

With the development of automation technology, big data is collected during operation processes, and among various machine learning analysis techniques using such data, deep neural network (DNN) has high analysis performance. However, most industrial data has low-variance or near-zero variance data from the refined processes in the collected data itself. This reduces deep learning analysis performance, which is affected by data quality. To overcome this, in this study, the weight learning pattern of an applied DNN is modeled as a stochastic differential equation (SDE) based on quantum mechanics. Through the drift and diffuse terms of quantum mechanics, the patterns of the DNN and data are quickly acquired, and the data with near-zero variance is effectively analyzed simultaneously. To demonstrate the superiority of the proposed framework, DNN analysis was performed using data with near-zero variance issues, and it was proved that the proposed framework is effective in processing near-zero variance data compared with other existing algorithms.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

The datasets used in our study are available in the UCR repository https://doi.org/10.24432/C5CW21 and Mendeley Data https://doi.org/10.17632/k22zxz29kr.1

References

  1. Oh E, Lee H (2020) An Imbalanced Data Handling Framework for Industrial Big Data Using a Gaussian Process Regression-Based Generative Adversarial Network. Symmetry 12:669. https://doi.org/10.3390/sym12040669

  2. Yu Z, Chu R, Wang W, Sheng M (2020) CRPN: A cascaded classification and regression DNN framework for seismic phase picking*. Earthquake Sci 33:53–61. https://doi.org/10.29382/eqs-2020-0053-01

  3. Montavon G, Samek W, Muller KR (2018) Methods for interpreting and understanding deep neural networks. Digital Signal Process 73:1–15. https://doi.org/10.1016/j.dsp.2017.10.011

  4. Nielsen RH (1992) Theory of the Backpropagation Neural Network. Neural Netw Perception 65–93. https://doi.org/10.1016/B978-0-12-741252-8.50010-8

  5. Bansal A, Kauffman RJ, Weitz RR (1993) Comparing the Modeling Performance of Regression and Neural Networks as Data Quality Varies: A Business Value Approach. J Manage Inf Syst 10:11–32. https://doi.org/10.1080/07421222.1993.11517988

  6. Jacobs J, Romanosky S, Adjerid I, Baker W (2020) Improving vulnerability remediation through better exploit prediction. J Cybersec 6:1–12. https://doi.org/10.1093/cybsec/tyaa015

  7. Ridgway GR, Litvak V, Flandin G, Friston KJ, Penny WD (2012) The problem of low variance voxels in statistical parametric mapping; a new hat avoids a ’haircut’. NeuroImage 59:2131–2141. https://doi.org/10.1016/j.neuroimage.2011.10.027

  8. Xiong H, Pandey G, Steinbach M, Kumar V (2006) Enhancing data analysis with noise removal. IEEE Trans Knowl Data Eng 18:304–319. https://doi.org/10.1109/TKDE.2006.46

  9. Reimold M, Slifstein M, Heinz A, Schauenburg WM, Bares R (2006) Effect of spatial smoothing on t-maps: arguments for going back from t-maps to masked contrast images. J Cerebral Blood Flow & Metabolism 26:751–759. https://doi.org/10.1038/sj.jcbfm.9600231

  10. Mehta P, Bukov M, Wang C, Day AGR, Richardson C, Fisher CK, Schwab DJ (2019) A high-bias, low-variance introduction to Machine Learning for physicists. Phys Reports 810:1–124. https://doi.org/10.1016/j.physrep.2019.03.001

  11. Jolliffe IT (1982) A Note on the Use of Principal Components in Regression. J Royal Stat Soc Series C: Appl Stat 31:300–303. https://doi.org/10.2307/2348005

  12. Holmes TH, He X (2016) Human immunophenotyping via low-variance, low-bias, interpretive regression modeling of small, wide data sets: Application to aging and immune response to influenza vaccination. J Immunological Methods 437:1–12. https://doi.org/10.1016/j.jim.2016.05.004

  13. Rodríguez-Fdez I, Mucientes M, Bugarín A (2013) An instance selection algorithm for regression and its application in variance reduction. 2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp 1–8. https://doi.org/10.1109/FUZZ-IEEE.2013.6622486

  14. Jolly S, Gupta N (2021) Understanding and Implementing Machine Learning Models with Dummy Variables with Low Variance. In: International conference on innovative computing and communications, pp 477–487. https://doi.org/10.1007/978-981-15-5113-0_37

  15. Moon J, Yu J, Sohn K (2022) An ensemble approach to anomaly detection using high- and low-variance principal components. Comput Electrical Eng 99:107773. https://doi.org/10.1016/j.compeleceng.2022.107773

  16. Blundell C, Cornebise J, Kavukcuoglu K, Wierstra D (2015) Weight Uncertainty in Neural Network. In: Proceedings of the 32nd international conference on machine learning (PMLR) 37:1613–1622

  17. Pawlowski N, Brock A, Lee MCH, Rajchl M, Glocker B (2017) Implicit Weight Uncertainty in Neural Networks. Machine Learn 2. https://doi.org/10.48550/arXiv.1711.01297

  18. Ma Q, Wang Y, Ao Y, Wang Q, Lu W (2022) UB-Net: Improved Seismic Inversion Based on Uncertainty Backpropagation. IEEE Trans Geosci Remote Sens 60:11–11. https://doi.org/10.1109/TGRS.2022.3174911

  19. Gernoth KA, Clark JW (1995) A modified backpropagation algorithm for training neural networks on data with error bars. Comput Phys Commun 88:1–22. https://doi.org/10.1016/0010-4655(95)00012-5

  20. Hofer V, Krempl G (2013) Drift mining in data: A framework for addressing drift in classification. Comput Stat & Data Anal 57:377–391. https://doi.org/10.1016/j.csda.2012.07.007

  21. Zliobaite I, Bifet A, Pfahringer B, Holmes G (2014) Active Learning With Drifting Streaming Data. IEEE Trans Neural Netw Learn Syst 25:27–39. https://doi.org/10.1109/TNNLS.2012.2236570

  22. Wu D, Lisser A (2023) A deep learning approach for solving linear programming problems. Neurocomputing 520:15–24. https://doi.org/10.1016/j.neucom.2022.11.053

  23. Nazemi A, Omidi F (2013) An efficient dynamic model for solving the shortest path problem. Transportation Res Part C: Emerging Technolog 26:1–19. https://doi.org/10.1016/j.trc.2012.07.005

  24. Lee H, Banerjee A (2009) Design, Simulation and Analysis of Logistics with Uncertainties. In: International conference on value chain sustainability, pp 19–21

  25. Lee H, Banerjee A (2009) A Modular Petri Net based Architecture to Model Manufacturing Systems Exhibiting Resource and Timing Uncertainties. In: 5th Annual IEEE conference on automation science and engineering, pp 525–530. https://doi.org/10.1109/COASE.2009.5234151

  26. Ballentine LE (2014) Quantum Mechanics: A Modern Development, in World Scientific Publishing Company. second ed. Singapura, pp 29–37

  27. Karatzas I, Shreve SE (1998) Brownian Motion. Brownian Motion and Stochastic Calculus 113:47–127. https://doi.org/10.1007/978-1-4612-0949-2_2

  28. Ito K (1946) On a Stochastic Integral Equation. Proceedings of the Japan Academy, pp 32–35. https://doi.org/10.2183/pjab1945.22.32

  29. Ito K (1967) On stochastic differential equations, American Mathematical Society, 4ed edn. Providence, Rhode Island, pp 1–51

  30. Pourahmadi M (1984) Taylor Expansion of and Some Applications. The American Mathematical Monthly 91:303–307. https://doi.org/10.1080/00029890.1984.11971411

  31. Eskov VM, Eskov VV, Gavrilenko TV (2014) Uncertainty in the quantum mechanics and biophysics of complex systems. Moscow University Phys Bullet 69:406–411. https://doi.org/10.3103/S002713491405004X

  32. Sen D (2014) The uncertainty relations in quantum mechanics. Current Sci 107:203–218

    Google Scholar 

  33. Oh E, Lee H (2022) Effective route generation framework using quantum mechanism-based multi-directional and parallel ant colony optimization. Comput & Industrial Eng 196:108308. https://doi.org/10.1016/j.cie.2022.108308

  34. Mandelstam L, Tamm I (1991) The Uncertainty Relation Between Energy and Time in Non-relativistic Quantum Mechanics. In: Bolotovskii BM, Frenkel VY, Peierls R (eds.) Selected Papers, Springer, Berlin, Heidelberg, pp 115–123. https://doi.org/10.1007/978-3-642-74626-0_8

  35. Xiao F (2022) CEQD: A Complex Mass Function to Predict Interference Effects. IEEE Transactions on cybernetics 52:7402–7414. https://doi.org/10.1109/TCYB.2020.3040770

  36. Oh E, Lee H (2023) Quantum mechanics-based missing value estimation framework for industrial data. Expert Syst Appl 236:121385. https://doi.org/10.1016/j.eswa.2023.121385

  37. Tavakoli M, Mood A, Vranken DV, Baldi P (2022) Quantum Mechanics and Machine Learning Synergies: Graph Attention Neural Networks to Predict Chemical Reactivity. Machine Learn Deep Learn 62:2121–2132. https://doi.org/10.1021/acs.jcim.1c01400

  38. Ventura D, Martinez T (1998) An Artificial Neuron with Quantum Mechanical Properties. Artif Neural Nets Genetic Algorithms pp 482–485. https://doi.org/10.1007/978-3-7091-6492-1_106

  39. Rai BK, Sresht V, Yang Q, Unwalla R, Tu M, Mathiowetz AM, Bakken GA (2022) TorsionNet: A Deep Neural Network to Rapidly Predict Small-Molecule Torsional Energy Profiles with the Accuracy of Quantum Mechanics. Mach Learn Deep Learn 62:785–800. https://doi.org/10.1021/acs.jcim.1c01346

  40. Cao H, Cao F, Wang D (2015) Quantum artificial neural networks with applications. Inf Sci 290:1–6. https://doi.org/10.1016/j.ins.2014.08.033

  41. Lantz V, Abiri N, Carlsson G, Pistol ME (2021) Deep learning for inverse problems in quantum mechanics. Quantum Chemistry 121:1–6. https://doi.org/10.1002/qua.26599

  42. Helwig N (2018) Condition monitoring of hydraulic systems. UCI Machine Learning Repository. https://doi.org/10.24432/C5CW21

  43. Pestana D (2020) Diesel Engine Faults Features Dataset (3500-DEFault). Mendeley Data v1. https://doi.org/10.17632/k22zxz29kr.1

  44. Zhang F, O’Donnell LJ (2020) Support Vector Regression, in: Mechelli A, Vieira S (Eds.) Machine Learning, Academic Press, pp 123–140. https://doi.org/10.1016/B978-0-12-815739-8.00007-9

  45. Buhmann MD (2000) Radial basis functions. Acta Numerica 9:1–38. https://doi.org/10.1017/S0962492900000015

Download references

Acknowledgements

This research was supported by The Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, S. Korea (grant number:NRF-2021R1A2C1008647).

Author information

Authors and Affiliations

  1. School of Industrial Engineering, Kumoh National Institute of Technology, Daehak-ro, Daehak-ro, Gumi, Gyeongsangbuk-do, 39177, South Korea

    Eunseo Oh & Hyunsoo Lee

Authors
  1. Eunseo Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyunsoo Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hyunsoo Lee.

Ethics declarations

Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Oh, E., Lee, H. Quantum mechanics-based deep learning framework considering near-zero variance data. Appl Intell (2024). https://doi.org/10.1007/s10489-024-05465-3

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-05465-3

Keywords

Search

Navigation