Skip to main content

Advertisement

SpringerLink
Log in

Machine learning modeling for the prediction of materials energy

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Machine learning (ML) is a fast-evolving field of artificial intelligence that has been applied in many domains due to the increasing availability of computerized databases, including materials science; for instance, validating crystal descriptors for energy prediction poses difficult problems. This work investigates machine learning models to substitute the laboratory crystal energy prediction using two- and three-body distribution functions as structural and atomic descriptors. To achieve this, ML algorithms were used notably ElasticNet, Bayesian Ridge, Random Forest, Support Vector Machine, and Deep Neural Networks to model structural descriptors. Moreover, a non-conventional Deep Neural Networks topology was developed and implemented to model atomic descriptors. Five-fold cross-validation procedure was performed on each model; quality assessment metrics were else used for testing and evaluation in order to identify the most robust descriptors. Finally, the best result of energy prediction was achieved by combining both two- and three-body atomic distribution functions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Availability of data and material

The data utilized in this study is open for provision on request.

References

  1. Artem R (2015-2021) Oganov’s Lab. https://uspex-team.org/en/lab. Accessed 26 Sept 2021

  2. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M et al (2016) \(\{\)TensorFlow\(\}\): a system for \(\{\)Large-Scale\(\}\) machine learning. In: 12th USENIX symposium on operating systems design and implementation (OSDI 16), 265–283

  3. Adamović S, Miškovic V, Maček N, Milosavljević M, Šarac M, Saračević M, Gnjatović M (2020) An efficient novel approach for iris recognition based on stylometric features and machine learning techniques. Future Gener Comput Syst 107:144–157

    Article  Google Scholar 

  4. Agarap AF (2018) Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375

  5. Agrawal A, Meredig B, Wolverton C, Choudhary A (2016) A formation energy predictor for crystalline materials using ensemble data mining. In: 2016 IEEE 16th international conference on data mining workshops (ICDMW). 1276–1279. IEEE

  6. Alfatni MSM, Shariff ARM, Bejo SK, Saaed OMB, Mustapha A (2018) Real-time oil palm ffb ripeness grading system based on ann, knn and svm classifiers. In: IOP conference series: earth and environmental science, 169, 012067. IOP Publishing

  7. Altun H, Bilgil A, Fidan B (2007) Treatment of multi-dimensional data to enhance neural network estimators in regression problems. Expert Syst Appl 32(2):599–605

  8. Assaf AG, Tsionas M, Tasiopoulos A (2019) Diagnosing and correcting the effects of multicollinearity: Bayesian implications of ridge regression. Tourism Manag 71:1–8

    Article  Google Scholar 

  9. Banaei H, Honarvar AR (2017) Web page rank estimation in search engine based on seo parameters using machine learning techniques. Int J Comput Sci Netw Sec 17:95–100

    Google Scholar 

  10. Bhattacharjee D, Basu DK, Nasipuri M, Kundu M (2010) Human face recognition using fuzzy multilayer perceptron. Soft Comput 14(6):559–570

    Article  Google Scholar 

  11. Bradley AP (1997) The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recogn 30(7):1145–1159

    Article  Google Scholar 

  12. Chauhan V, Jaiswal A, Khan J (2015) Web page ranking using machine learning approach. In: 2015 fifth international conference on advanced computing and communication technologies. 575–580. IEEE

  13. Cortes C, Vapnik V (1995) Support-vector networks. Machine learning 20(3):273–297

    Article  Google Scholar 

  14. Curtarolo S, Morgan D, Persson K, Rodgers J, Ceder G (2003) Predicting crystal structures with data mining of quantum calculations. Phys Rev Lett 91(13)

  15. Das S, Kolya AK (2017) Sense gst: text mining and sentiment analysis of gst tweets by naive bayes algorithm. In: 2017 third international conference on research in computational intelligence and communication networks (ICRCICN). 239–244. IEEE

  16. De Breuck PP, Hautier G, Rignanese GM (2021) Materials property prediction for limited datasets enabled by feature selection and joint learning with modnet. npj Comput Mater 7(1):1–8

  17. Dobrescu A, Valerio Giuffrida M, Tsaftaris SA (2019) Understanding deep neural networks for regression in leaf counting. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 0

  18. Efendi A (2017) Effrihan: a simulation study on bayesian ridge regression models for several collinearity levels. In: AIP conference proceedings, 1913, 020031. AIP Publishing LLC

  19. Faber F, Lindmaa A, von Lilienfeld OA, Armiento R (2015) Crystal structure representations for machine learning models of formation energies. Int J Quant Chem 115(16):1094–1101

  20. Fedorov AV, Shamanaev IV (2017) Crystal structure representation for neural networks using topological approach. Mole Inform 36(8):1600162

  21. Fischer CC, Tibbetts KJ, Morgan D, Ceder G (2006) Predicting crystal structure by merging data mining with quantum mechanics. Nature Mater 5(8):641–646

  22. Fonseca E, Gong R, Bogdanov D, Slizovskaia O, Gómez Gutiérrez E, Serra X (2017) Acoustic scene classification by ensembling gradient boosting machine and convolutional neural networks. In: Virtanen T, Mesaros A, Heittola T, Diment A, Vincent E, Benetos E, Martinez B, editors. Detection and classification of acoustic scenes and events 2017 workshop (DCASE2017); 2017 Nov 16; Munich, Germany. Tampere (Finland): Tampere University of Technology; 2017. 37-41. Tampere University of Technology

  23. Frias-Martinez E, Sanchez A, Velez J (2006) Support vector machines versus multi-layer perceptrons for efficient off-line signature recognition. Eng Appl Artif Intel 19(6):693–704

  24. Gholami H, Mohamadifar A, Sorooshian A, Jansen JD (2020) Machine-learning algorithms for predicting land susceptibility to dust emissions: the case of the Jazmurian basin, Iran. Atmosph Pollut Res 11(8):1303–1315

  25. Glass CW, Oganov AR, Hansen N (2006) Uspex-evolutionary crystal structure prediction. Comput Phys Commun 175(11–12):713–720

  26. Grossi E, Buscema M (2007) Introduction to artificial neural networks. Eur J Gastroenterol Hepatol 19(12):1046–1054

  27. Guo G, Zhang N (2019) A survey on deep learning based face recognition. Comput Vis Image Understand 189

  28. Hasnip PJ, Refson K, Probert MI, Yates JR, Clark SJ, Pickard CJ (2014) Density functional theory in the solid state. Philosoph Trans Royal Soc A Math Phys Eng Sci 372(2011):20130270

  29. Hautier G (2013) Data mining approaches to high-throughput crystal structure and compound prediction. In: Prediction and calculation of crystal structures, 139–179. Springer

  30. Hegde G, Bowen RC (2017) Machine-learned approximations to density functional theory hamiltonians. Sci Rep 7:42669

  31. Himanen L, Jäger MO, Morooka EV, Canova FF, Ranawat YS, Gao DZ, Rinke P, Foster AS (2020) Dscribe: library of descriptors for machine learning in materials science. Comput Phys Commun 247

  32. Hofmann DW, Apostolakis J (2003) Crystal structure prediction by data mining. J Mole Struct 647(1–3):17–39

  33. Huang J, Ling CX (2005) Using auc and accuracy in evaluating learning algorithms. IEEE Trans Knowl Data Eng 17(3):299–310

  34. Hunter JD (2007) Matplotlib: a 2d graphics environment. Comput Sci Eng 9(03):90–95

  35. Iqbal T, Qureshi S (2020) The survey: text generation models in deep learning. J King Saud Univ-Comput Inf Sci

  36. Jain A, Hautier G, Ong SP, Persson K (2016) New opportunities for materials informatics: resources and data mining techniques for uncovering hidden relationships. J Mater Res 31(8):977–994

  37. Jana R, Bhattacharyya S, Das S (2019) Epileptic seizure prediction from eeg signals using densenet. In: 2019 IEEE symposium series on computational intelligence (SSCI), 604–609. IEEE

  38. Jha D, Choudhary K, Tavazza F, Liao Wk, Choudhary A, Campbell C, Agrawal A (2019) Enhancing materials property prediction by leveraging computational and experimental data using deep transfer learning. Nat Commun 10(1):1–12

  39. Karamad M, Magar R, Shi Y, Siahrostami S, Gates ID, Farimani AB (2020) Orbital graph convolutional neural network for material property prediction. Phys Rev Mater 4(9)

  40. Kasuya E (2019) On the use of r and r squared in correlation and regression. Technical report, Wiley Online Library

  41. Liang H, Stanev V, Kusne AG, Takeuchi I (2020) Cryspnet: crystal structure predictions via neural network. arXiv preprint arXiv:2003.14328

  42. Liaw A, Wiener M et al (2002) Classification and regression by randomforest. R News 2(3):18–22

  43. Liu Y, Sun L, Du C, Wang X (2020) Near-infrared prediction of edible oil frying times based on bayesian ridge regression. Optik 218:164950

  44. Liu Y, Zhao T, Ju W, Shi S (2017) Materials discovery and design using machine learning. J Mater 3(3):159–177

  45. Louis SY, Zhao Y, Nasiri A, Wang X, Song Y, Liu F, Hu J (2020) Graph convolutional neural networks with global attention for improved materials property prediction. Phys Chem Chem Phys 22(32):18141–18148

  46. Lyakhov AO, Oganov AR, Stokes HT, Zhu Q (2013) New developments in evolutionary structure prediction algorithm uspex. Comput Phys Commun 184(4):1172–1182

  47. Meriem M, Chahrazed T, Aouache M (2020) Low-contrast x-ray enhancement using a fuzzy gamma reasoning model. Med Biol Eng Comput 58(6):1177–1197

  48. Milanova I, Sarvanoska K, Srbinoski V, Gjoreski H (2019) Automatic text generation in macedonian using recurrent neural networks. In: International conference on ICT innovations, 1–12. Springer

  49. Müller IM (2021) Feature selection for energy system modeling: identification of relevant time series information. Energy and AI 100057

  50. Mustapha A, Hussain A, Samad SA (2011) A new approach for noise reduction in spine radiograph images using a non-linear contrast adjustment scheme based adaptive factor. Sci Res Essays 6(20):4246–4258

  51. Oganov AR, Glass CW (2006) Crystal structure prediction using ab initio evolutionary techniques: Principles and applications. J Chem Phys 124(24):244704

  52. Oganov AR, Ma Y, Glass CW, Valle M (2007) Evolutionary crystal structure prediction: overview of the uspex method and some of its applications. Psi-k Newsletter 84:142–171

  53. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch

  54. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830

  55. Podryabinkin EV, Tikhonov EV, Shapeev AV, Oganov AR (2019) Accelerating crystal structure prediction by machine-learning interatomic potentials with active learning. Phys Rev B 99(6):064114

  56. Pourmahyabadi M, Nejad SM (2009) Prediction of photonic crystal fiber characteristics by neuro-fuzzy system. Optic Commun 282(20):4081–4086

  57. Pozdnyakov S, Oganov AR, Mazitov A, Frolov T, Kruglov I, Mazhnik E (2019) Fast general two-and three-body interatomic potential. arXiv preprint arXiv:1910.07513

  58. Price SL (2014) Predicting crystal structures of organic compounds. Chem Soc Rev 43(7):2098–2111

  59. Rai P, Prabhumoye S, Khattri P, Sandhu LRS, Kamath SS (2014) A prototype of an intelligent search engine using machine learning based training for learning to rank. In: Advanced computing, networking and informatics, Vol 1, 67–75. Springer

  60. Réda C, Kaufmann E, Delahaye-Duriez A (2020) Machine learning applications in drug development. Comput Struct Biotechnol J 18:241–252

  61. Ryan K, Lengyel J, Shatruk M (2018) Crystal structure prediction via deep learning. J Am Chem Soc 140(32):10158–10168

  62. Salminen J, Corporan J, Marttila R, Salenius T, Jansen BJ (2019) Using machine learning to predict ranking of webpages in the gift industry: factors for search-engine optimization. In: Proceedings of the 9th international conference on information systems and technologies, 1–8

  63. Sammut C, Webb GI (2011) Encyclopedia of machine learning. Springer

  64. Schmidt J, Marques MR, Botti S, Marques MA (2019) Recent advances and applications of machine learning in solid-state materials science. npj Comput Mater 5(1):1–36

  65. Schütt KT, Glawe H, Brockherde F, Sanna A, Müller KR, Gross EK (2014) How to represent crystal structures for machine learning: towards fast prediction of electronic properties. Phys Rev B 89(20):205118

  66. Segal MR (2004) Machine learning benchmarks and random forest regression

  67. Seko A, Togo A, Tanaka I (2018) Descriptors for machine learning of materials data. Nanoinformatics. Springer, Singapore, pp 3–23

  68. Shalev-Shwartz S, Ben-David S (2014) Understanding machine learning. || cambridge university press, New York, NY, pp 10013–2473

  69. Singh B, Sihag P, Singh K (2017) Modelling of impact of water quality on infiltration rate of soil by random forest regression. Model Earth Syst Environ 3(3):999–1004

  70. Sizochenko N, Majumdar D, Roszak S, Leszczynski J (2016) Application of quantum mechanics and molecular mechanics in chemoinformatics. Handbook Comput Chem 1–23

  71. Smola A, Vishwanathan S (2008) Introduction to machine learning. Cambridge University, UK 32(34),2008

  72. Sutskever I, Martens J, Hinton GE (2011) Generating text with recurrent neural networks. In: ICML

  73. Svetnik V, Liaw A, Tong C, Culberson JC, Sheridan RP, Feuston BP (2003) Random forest: a classification and regression tool for compound classification and qsar modeling. J Chem Inf Comput Sci 43(6):1947–1958

  74. Tharwat A (2020) Classification assessment methods. Appl Comput Inform

  75. Thompson HP, Day GM (2014) Which conformations make stable crystal structures? mapping crystalline molecular geometries to the conformational energy landscape. Chem Sci 5(8):3173–3182

  76. Trafalis TB, Ince H (2000) Support vector machine for regression and applications to financial forecasting. In: Proceedings of the IEEE-INNS-ENNS international joint conference on neural networks. IJCNN 2000. Neural computing: new challenges and perspectives for the new millennium. 6:348–353. IEEE

  77. Vazquez-Fernandez E, Gonzalez-Jimenez D (2016) Face recognition for authentication on mobile devices. Image Vis Comput 55:31–33

  78. Wang K, Li D, Fei Z, Ma X, Zeng X (2019) Discovery of a new crystal structure of libef3 and its thermodynamic and optical properties. Comput Mater Sci 169

  79. Wauters M, Vanhoucke M (2014) Support vector machine regression for project control forecasting. Autom Constr 47:92–106

  80. Wu DL, Ng WW, Chan PP, Ding HL, Jing BZ, Yeung DS (2010) Access control by rfid and face recognition based on neural network. In: 2010 international conference on machine learning and cybernetics. 2:675–680. IEEE

  81. Xu Y, Du J, Dai LR, Lee CH (2013) An experimental study on speech enhancement based on deep neural networks. IEEE Signal Process Lett 21(1):65–68

  82. Yang W, Siriwardane EMD, Dong R, Li Y, Hu J (2021) Crystal structure prediction of materials with high symmetry using differential evolution. arXiv preprint arXiv:2104.09764

  83. Yang Y, Yang Y (2020) Hybrid prediction method for wind speed combining ensemble empirical mode decomposition and bayesian ridge regression. IEEE Access 8:71206–71218

    Article  Google Scholar 

  84. Zafar R, Dass SC, Malik AS (2017) Electroencephalogram-based decoding cognitive states using convolutional neural network and likelihood ratio based score fusion. PloS One 12(5)

  85. Zhang L, Zhou W, Jiao L (2004) Wavelet support vector machine. IEEE Trans Syst Man Cybern Part B Cybern 34(1):34–39

  86. Zhang Z, Schott JA, Liu M, Chen H, Lu X, Sumpter BG, Fu J, Dai S (2019) Prediction of carbon dioxide adsorption via deep learning. Angewandte Chemie 131(1):265–269

    Article  Google Scholar 

  87. Zhu Q, Oganov AR, Zhou XF (2014) Crystal structure prediction and its application in earth and materials sciences. Predict Calc Crystal Struct. 223–256

  88. Zulfiqar M, Syed F, Khan MJ, Khurshid K (2019) Deep face recognition for biometric authentication. In: 2019 international conference on electrical, communication, and computer engineering (ICECCE), 1–6. IEEE

  89. Zuo H, Jiang Y, Yang Y, Hu J (2021) Prediction of properties of metal alloy materials based on machine learning

Download references

Acknowledgements

The authors would like to offer special thanks to Professor Artem Oganov (Skolkovo Institute of Science and Technology, Moscow) for database provision and helpful supervision. We also extend thanks to both Sergei Pozdniakov (Skolkovo Institute of Science and Technology, Moscow) and Hammouda Elbez (IRCICA, University of Lille) for their assistance in features extraction and the PyTorch framework, respectively.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. LRDSI Laboratory, Faculty of Science, University Blida 1, Soumaa, BP 270, Blida, Algeria

    Meriem Mouzai & Saliha Oukid

  2. Division Télécom, Centre de Développement des Technologies Avancées (CDTA), PO. Box 17, Baba-Hassen, 16303, Algiers, Algeria

    Aouache Mustapha

Authors
  1. Meriem Mouzai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saliha Oukid
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aouache Mustapha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meriem Mouzai.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Code availability

The code is available from authors upon reasonable request and with permission of LRDSI Laboratory, Faculty of Science, University Blida 1.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mouzai, M., Oukid, S. & Mustapha, A. Machine learning modeling for the prediction of materials energy. Neural Comput & Applic 34, 17981–17998 (2022). https://doi.org/10.1007/s00521-022-07416-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07416-w

Keywords

Search

Navigation