Abstract
Manufacturing advanced materials and products with a specific property or combination of properties is often warranted. To achieve that it is crucial to find out the optimum recipe or processing conditions that can generate the ideal combination of these properties. Most of the time, a sufficient number of experiments are needed to generate a Pareto front. However, manufacturing experiments are usually costly and even conducting a single experiment can be a time-consuming process. So, it's critical to determine the optimal location for data collection to gain the most comprehensive understanding of the process. Sequential learning is a promising approach to actively learn from the ongoing experiments, iteratively update the underlying optimization routine, and adapt the data collection process on the go. This paper presents a novel data-driven Bayesian optimization framework that utilizes sequential learning to efficiently optimize complex systems with multiple conflicting objectives. Additionally, this paper proposes a novel metric for evaluating multi-objective data-driven optimization approaches. This metric considers both the quality of the Pareto front and the amount of data used to generate it. The proposed framework is particularly beneficial in practical applications where acquiring data can be expensive and resource intensive. To demonstrate the effectiveness of the proposed algorithm and metric, the algorithm is evaluated on a manufacturing dataset. The results indicate that the proposed algorithm can achieve the actual Pareto front while processing significantly less data. Our data-driven framework can facilitate more efficient manufacturing choices, which not only minimizes resource usage but also promotes reduced energy consumption and thereby aids in pollution prevention.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets generated and analysed during the current study are available from the corresponding author on reasonable request.
References
Aboutaleb, A. M., Tschopp, M. A., Rao, P. K., & Bian, L. (2017). Multi-objective accelerated process optimization of part geometric accuracy in additive manufacturing. Journal of Manufacturing Science and Engineering. https://doi.org/10.1115/1.4037319
Al Hazza, M. H. F., Adesta, E. Y., Riza, M., & Suprianto, M. Y. (2012). Power consumption optimization in CNC turning process using multi objective genetic algorithm. Advanced Materials Research, 576, 95–98. https://doi.org/10.4028/www.scientific.net/amr.576.95
Alizadeh Afrouzy, Z., Nasseri, S. H., Mahdavi, I., & Paydar, M. M. (2016). A fuzzy stochastic multi-objective optimization model to configure a supply chain considering new product development. Applied Mathematical Modelling, 40(17–18), 7545–7570. https://doi.org/10.1016/j.apm.2016.03.015
Alvarado-Iniesta, A., Guillen-Anaya, L. G., Rodríguez-Picón, L. A., & Ñeco-Caberta, R. (2018). Multi-objective optimization of an engine mount design by means of memetic genetic programming and a local exploration approach. Journal of Intelligent Manufacturing, 31(1), 19–32. https://doi.org/10.1007/s10845-018-1432-9
Amini, S., & Barsoum, M. W. (2010). On the effect of texture on the mechanical and damping properties of nanocrystalline MG-matrix composites reinforced with Max Phases. Materials Science and Engineering: A, 527(16–17), 3707–3718. https://doi.org/10.1016/j.msea.2010.01.073
Aria, M., & Cuccurullo, C. (2017). Bibliometrix : An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959–975. https://doi.org/10.1016/j.joi.2017.08.007
Arróyave, R., Talapatra, A., Duong, T., Son, W., Gao, H., & Radovic, M. (2016). Does aluminum play well with others? intrinsic al-a alloying behavior in 211/312 max phases. Materials Research Letters, 5(3), 170–178. https://doi.org/10.1080/21663831.2016.1241319
Aryal, S., Sakidja, R., Barsoum, M. W., & Ching, W.-Y. (2014). A genomic approach to the stability, elastic, and electronic properties of the Max Phases. Physica Status Solidi (b), 251(8), 1480–1497. https://doi.org/10.1002/pssb.201451226
Asadollahi-Yazdi, E., Gardan, J., & Lafon, P. (2018). Multi-objective optimization of Additive Manufacturing Process. IFAC-PapersOnLine, 51(11), 152–157. https://doi.org/10.1016/j.ifacol.2018.08.250
Ashraf, M., Chetty, G., & Tran, D. (2013). Feature selection techniques on thyroid, hepatitis, and breast cancer datasets. International Journal on Data Mining and Intelligent Information Technology Applications, 3(1), 1. https://doi.org/10.4156/ijmia.vol3.issue1.1
Audet, C., Bigeon, J., Cartier, D., Le Digabel, S., & Salomon, L. (2021). Performance indicators in multiobjective optimization. European Journal of Operational Research, 292(2), 397–422. https://doi.org/10.1016/j.ejor.2020.11.016
Balekelayi, N., Woldesellasse, H., & Tesfamariam, S. (2022). Comparison of the performance of a surrogate based Gaussian process, NSGA2 and PSO multi-objective optimization of the operation and fuzzy structural reliability of water distribution system: Case study for the City of Asmara, Eritrea. Water Resources Management, 36(15), 6169–6185. https://doi.org/10.1007/s11269-022-03347-2
Bandyopadhyay, A., & Traxel, K. D. (2018). Invited review article: Metal-Additive Manufacturing—modeling strategies for application-optimized designs. Additive Manufacturing, 22, 758–774. https://doi.org/10.1016/j.addma.2018.06.024
Barsoum, M. W. (2000). The MN+1AXN phases: A new class of solids. Progress in Solid State Chemistry, 28(1–4), 201–281. https://doi.org/10.1016/s0079-6786(00)00006-6
Barsoum, M. W. (2013). MAX Phases. https://doi.org/10.1002/9783527654581
Barsoum, M. W., & Radovic, M. (2011). Elastic and mechanical properties of the Max Phases. Annual Review of Materials Research, 41(1), 195–227. https://doi.org/10.1146/annurev-matsci-062910-100448
Biswas, A., Morozovska, A. N., Ziatdinov, M., Eliseev, E. A., & Kalinin, S. V. (2021). Multi-objective bayesian optimization of ferroelectric materials with interfacial control for memory and Energy Storage Applications. Journal of Applied Physics, 130(20), 204102. https://doi.org/10.1063/5.0068903
Cao, L., Li, J., Hu, J., Liu, H., Wu, Y., & Zhou, Q. (2021). Optimization of surface roughness and dimensional accuracy in LPBF additive manufacturing. Optics & Laser Technology, 142, 107246. https://doi.org/10.1016/j.optlastec.2021.107246
Chepiga, T., Zhilyaev, P., Ryabov, A., Simonov, A. P., Dubinin, O. N., Firsov, D. G., Kuzminova, Y. O., & Evlashin, S. A. (2023). Process parameter selection for production of stainless steel 316L using efficient multi-objective bayesian optimization algorithm. Materials, 16(3), 1050. https://doi.org/10.3390/ma16031050
Cohn, D. A., Ghahramani, Z., & Jordan, M. I. (1996). Active learning with statistical models. Journal of Artificial Intelligence Research, 4, 129–145. https://doi.org/10.1613/jair.295
Daulton, S., Balandat, M., & Bakshy, E. (2020). Differentiable expected hypervolume improvement for parallel multi-objective Bayesian optimization. Advances in Neural Information Processing Systems, 33, 9851–9864.
Davins-Valldaura, J., Moussaoui, S., Pita-Gil, G., & Plestan, F. (2016). Parego extensions for multi-objective optimization of expensive evaluation functions. Journal of Global Optimization, 67(1–2), 79–96. https://doi.org/10.1007/s10898-016-0419-3
Davis, J., Edgar, T., Porter, J., Bernaden, J., & Sarli, M. (2012). Smart Manufacturing, manufacturing intelligence and demand-dynamic performance. Computers & Chemical Engineering, 47, 145–156. https://doi.org/10.1016/j.compchemeng.2012.06.037
Davletov, E. T., Tsyganok, V. V., Khlebnikov, V. A., Pershin, D. A., Shaykin, D. V., & Akimov, A. V. (2020). Machine learning for achieving Bose-Einstein condensation of thulium atoms. Physical Review A. https://doi.org/10.1103/physreva.102.011302
Dong, G., Marleau-Finley, J., & Zhao, Y. F. (2019). Investigation of electrochemical post-processing procedure for ti-6al-4v lattice structure manufactured by Direct Metal Laser Sintering (DMLS). The International Journal of Advanced Manufacturing Technology, 104(9–12), 3401–3417. https://doi.org/10.1007/s00170-019-03996-5
Duquesnoy, M., Liu, C., Dominguez, D. Z., Kumar, V., Ayerbe, E., & Franco, A. A. (2023). Machine learning-assisted multi-objective optimization of battery manufacturing from synthetic data generated by physics-based simulations. Energy Storage Materials, 56, 50–61. https://doi.org/10.1016/j.ensm.2022.12.040
Fahle, S., Prinz, C., & Kuhlenkötter, B. (2020). Systematic review on machine learning (ML) methods for manufacturing processes – identifying artificial intelligence (AI) methods for Field Application. Procedia CIRP, 93, 413–418. https://doi.org/10.1016/j.procir.2020.04.109
Fountas, N. A., Kechagias, J. D., Manolakos, D. E., & Vaxevanidis, N. M. (2020). Single and multi-objective optimization of FDM-based additive manufacturing using metaheuristic algorithms. Procedia Manufacturing, 51, 740–747. https://doi.org/10.1016/j.promfg.2020.10.104
Geng, H., Chen, T., Ma, Y., Zhu, B., & Yu, B. (2023). PTPT: Physical design tool parameter tuning via multi-objective bayesian optimization. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 42(1), 178–189. https://doi.org/10.1109/tcad.2022.3167858
Golab, M., Massey, S., & Moultrie, J. (2022). How generalisable are material extrusion additive manufacturing parameter optimisation studies? A systematic review. Heliyon. https://doi.org/10.1016/j.heliyon.2022.e11592
Greenhill, S., Rana, S., Gupta, S., Vellanki, P., & Venkatesh, S. (2020). Bayesian optimization for adaptive experimental design: A review. IEEE Access, 8, 13937–13948. https://doi.org/10.1109/access.2020.2966228
Greinacher, S., Overbeck, L., Kuhnle, A., Krahe, C., & Lanza, G. (2020). Multi-objective optimization of Lean and Resource Efficient Manufacturing Systems. Production Engineering, 14(2), 165–176. https://doi.org/10.1007/s11740-019-00945-9
Harifi, S., Khalilian, M., Mohammadzadeh, J., & Ebrahimnejad, S. (2020). Optimization in solving inventory control problem using nature inspired emperor penguins colony algorithm. Journal of Intelligent Manufacturing, 32(5), 1361–1375. https://doi.org/10.1007/s10845-020-01616-8
Hu, B., Wang, Z., Du, C., Zou, W., Wu, W., Tang, J., Ai, J., Zhou, H., Chen, R., & Shan, B. (2023). Multi-objective bayesian optimization accelerated design of TPMS structures. International Journal of Mechanical Sciences, 244, 108085. https://doi.org/10.1016/j.ijmecsci.2022.108085
Iqbal, A., Zhang, H.-C., Kong, L. L., & Hussain, G. (2013). A rule-based system for trade-off among energy consumption, tool life, and productivity in machining process. Journal of Intelligent Manufacturing, 26(6), 1217–1232. https://doi.org/10.1007/s10845-013-0851-x
Ishibuchi, H., Masuda, H., Tanigaki, Y., & Nojima, Y. (2015). Modified distance calculation in generational distance and inverted generational distance. Lecture Notes in Computer Science. https://doi.org/10.1007/978-3-319-15892-1_8
Jiang, H., Chen, J., & Liu, T. (2014a). Multi-objective design of an FBG sensor network using an improved strength pareto evolutionary algorithm. Sensors and Actuators a: Physical, 220, 230–236. https://doi.org/10.1016/j.sna.2014.10.020
Jiang, S., Ong, Y.-S., Zhang, J., & Feng, L. (2014b). Consistencies and contradictions of performance metrics in multiobjective optimization. IEEE Transactions on Cybernetics, 44(12), 2391–2404. https://doi.org/10.1109/tcyb.2014.2307319
Jin, Z., Lim, D. D., Zhao, X., Mamunuru, M., Roham, S., & Gu, G. X. (2023). Machine learning enabled optimization of showerhead design for semiconductor deposition process. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-023-02082-8
Jones, D. R., Schonlau, M., & Welch, W. J. (1998). Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4), 455–492. https://doi.org/10.1023/a:1008306431147
Karlsson, I., Bandaru, S., & Ng, A. H. (2021). Online knowledge extraction and preference guided multi-objective optimization in manufacturing. IEEE Access, 9, 145382–145396. https://doi.org/10.1109/access.2021.3123211
Khatamsaz, D., Vela, B., Singh, P., Johnson, D. D., Allaire, D., & Arróyave, R. (2022). Multi-objective materials bayesian optimization with active learning of design constraints: Design of ductile refractory multi-principal-element alloys. Acta Materialia, 236, 118133. https://doi.org/10.1016/j.actamat.2022.118133
Knowles, J. (2006). Parego: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Transactions on Evolutionary Computation, 10(1), 50–66. https://doi.org/10.1109/tevc.2005.851274
Konakovic Lukovic, M., Tian, Y., & Matusik, W. (2020). Diversity-guided multi-objective bayesian optimization with batch evaluations. Advances in Neural Information Processing Systems, 33, 17708–17720.
Kübler, F., Böhner, J., & Steinhilper, R. (2015). Resource efficiency optimization of manufacturing processes using evolutionary computation: A turning case. Procedia CIRP, 29, 822–827. https://doi.org/10.1016/j.procir.2015.02.208
Kumar, S., Gopi, T., Harikeerthana, N., Gupta, M. K., Gaur, V., Krolczyk, G. M., & Wu, C. S. (2022). Machine learning techniques in Additive Manufacturing: A State of the art review on design, processes and production control. Journal of Intelligent Manufacturing, 34(1), 21–55. https://doi.org/10.1007/s10845-022-02029-5
Kumar, Y., & Singh, H. (2014). Multi-response optimization in dry turning process using Taguchi’s approach and utility concept. Procedia Materials Science, 5, 2142–2151. https://doi.org/10.1016/j.mspro.2014.07.417
Li, L., Liu, F., Chen, B., & Li, C. B. (2015). Multi-objective optimization of cutting parameters in sculptured parts machining based on Neural Network. Journal of Intelligent Manufacturing, 26(5), 891–898. https://doi.org/10.1007/s10845-013-0809-z
Lin, W., Yu, D., Zhang, C., Zhang, S., Tian, Y., Liu, S., & Luo, M. (2017). Multi-objective optimization of machining parameters in multi-pass turning operations for low-carbon manufacturing. Proceedings of the Institution of Mechanical Engineers, Part b: Journal of Engineering Manufacture, 231(13), 2372–2383. https://doi.org/10.1177/0954405416629098
Malakshan, S. R., Saadabadi, M. S., Mostofa, M., Soleymani, S., & Nasrabadi, N. M. (2023). Joint Super-Resolution and head pose estimation for extreme low-resolution faces. IEEE Access, 11, 11238–11253. https://doi.org/10.1109/access.2023.3241606
Natarajan, E., Kaviarasan, V., Lim, W. H., Tiang, S. S., Parasuraman, S., & Elango, S. (2019). Non-dominated sorting modified teaching–learning-based optimization for multi-objective machining of polytetrafluoroethylene (PTFE). Journal of Intelligent Manufacturing, 31(4), 911–935. https://doi.org/10.1007/s10845-019-01486-9
Nti, I. K., Adekoya, A. F., Weyori, B. A., & Nyarko-Boateng, O. (2021). Applications of artificial intelligence in engineering and manufacturing: A systematic review. Journal of Intelligent Manufacturing, 33(6), 1581–1601. https://doi.org/10.1007/s10845-021-01771-6
Okabe, T., Yaochu Jin, & Sendhoff, B. (n.d.). A critical survey of performance indices for multi-objective optimisation. The 2003 Congress on Evolutionary Computation, 2003. CEC ’03. https://doi.org/10.1109/cec.2003.1299759
Palizhati, A., Torrisi, S. B., Aykol, M., Suram, S. K., Hummelshøj, J. S., & Montoya, J. H. (2022). Agents for sequential learning using multiple-fidelity data. Scientific Reports. https://doi.org/10.1038/s41598-022-08413-8
Radovic, M., & Barsoum, M. W. (2013). MAX phases: Bridging the gap between metals and ceramics. American Ceramics Society Bulletin, 92(3), 20–27.
Raju, M., Gupta, M. K., Bhanot, N., & Sharma, V. S. (2018). A hybrid PSO–BFO evolutionary algorithm for optimization of fused deposition modelling process parameters. Journal of Intelligent Manufacturing, 30(7), 2743–2758. https://doi.org/10.1007/s10845-018-1420-0
Ramezankhani, M., Crawford, B., Narayan, A., Voggenreiter, H., Seethaler, R., & Milani, A. S. (2021). Making costly manufacturing smart with transfer learning under limited data: A case study on composites autoclave processing. Journal of Manufacturing Systems, 59, 345–354. https://doi.org/10.1016/j.jmsy.2021.02.015
Ramírez-Márquez, C., Contreras-Zarazúa, G., Martín, M., & Segovia-Hernández, J. G. (2019). Safety, economic, and environmental optimization applied to three processes for the production of solar-grade silicon. ACS Sustainable Chemistry & Engineering, 7(5), 5355–5366. https://doi.org/10.1021/acssuschemeng.8b06375
Rao, R. V., Rai, D. P., & Balic, J. (2016). Multi-objective optimization of machining and micro-machining processes using non-dominated sorting teaching–learning-based optimization algorithm. Journal of Intelligent Manufacturing, 29(8), 1715–1737. https://doi.org/10.1007/s10845-016-1210-5
Rao, S. S. (2019). Engineering Optimization Theory and Practice. https://doi.org/10.1002/9781119454816
Riquelme, N., Von Lucken, C., & Baran, B. (2015). Performance metrics in multi-objective optimization. 2015 Latin American Computing Conference (CLEI). https://doi.org/10.1109/clei.2015.7360024
Ruane, P., Walsh, P., & Cosgrove, J. (2023). Using simulation optimization to improve the performance of an automated manufacturing line. Procedia Computer Science, 217, 630–639. https://doi.org/10.1016/j.procs.2022.12.259
Ryu, J., Kim, S., & Wan, H. (2009). Pareto front approximation with adaptive weighted sum method in multiobjective simulation optimization. Proceedings of the 2009 Winter Simulation Conference (WSC). https://doi.org/10.1109/wsc.2009.5429562
Shao, G., Brodsky, A., & Miller, R. (2015). Modeling and optimization of manufacturing process performance using Modelica graphical representation and process analytics formalism. Journal of Intelligent Manufacturing, 29(6), 1287–1301. https://doi.org/10.1007/s10845-015-1178-6
Simon, V. V. (2020). Multi-objective optimization of the manufacture of face-milled hypoid gears on numerical controlled machine tool. Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture, 235(6–7), 1120–1130.
Simpson, T., Mistree, F., Korte, J., & Mauery, T. (1998). Comparison of response surface and kriging models for multidisciplinary design optimization. 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization. https://doi.org/10.2514/6.1998-4755
Solomou, A., Zhao, G., Boluki, S., Joy, J. K., Qian, X., Karaman, I., Arróyave, R., & Lagoudas, D. C. (2018). Multi-objective Bayesian Materials Discovery: Application on the discovery of precipitation strengthened niti shape memory alloys through micromechanical modeling. Materials & Design, 160, 810–827. https://doi.org/10.1016/j.matdes.2018.10.014
Sun, Z. M. (2011). Progress in research and development on max phases: A family of layered ternary compounds. International Materials Reviews, 56(3), 143–166. https://doi.org/10.1179/1743280410y.0000000001
Talapatra, A., Boluki, S., Duong, T., Qian, X., Dougherty, E., & Arróyave, R. (2018). Autonomous Efficient Experiment Design for Materials Discovery with bayesian model averaging. Physical Review Materials. https://doi.org/10.1103/physrevmaterials.2.113803
Talapatra, A., Duong, T., Son, W., Gao, H., Radovic, M., & Arróyave, R. (2016). High-throughput combinatorial study of the effect of M site alloying on the solid solution behavior of M2 AlC MAX phases. Physical Review B. https://doi.org/10.1103/physrevb.94.104106
Tong, S. (2001). Active learning: theory and applications. USA: Stanford University.
van Eck, N. J., & Waltman, L. (2009). Software survey: VOSviewer, a computer program for Bibliometric mapping. Scientometrics, 84(2), 523–538. https://doi.org/10.1007/s11192-009-0146-3
Van Veldhuizen, D. A. (1999). Multiobjective evolutionary algorithms: classifications, analyses, and new innovations. Air Force Institute of Technology.
Wang, B., Tao, F., Fang, X., Liu, C., Liu, Y., & Freiheit, T. (2021). Smart manufacturing and intelligent manufacturing: A comparative review. Engineering, 7(6), 738–757. https://doi.org/10.1016/j.eng.2020.07.017
Yuan, C., Li, G., Kamarthi, S., Jin, X., & Moghaddam, M. (2022). Trends in intelligent manufacturing research: A Keyword co-occurrence network based review. Journal of Intelligent Manufacturing, 33(2), 425–439. https://doi.org/10.1007/s10845-021-01885-x
Zerka, F., Urovi, V., Bottari, F., Leijenaar, R. T. H., Walsh, S., Gabrani-Juma, H., Gueuning, M., Vaidyanathan, A., Vos, W., Occhipinti, M., Woodruff, H. C., Dumontier, M., & Lambin, P. (2021). Privacy preserving distributed learning classifiers—sequential learning with small sets of data. Computers in Biology and Medicine, 136, 104716. https://doi.org/10.1016/j.compbiomed.2021.104716
Zitzler, E., & Thiele, L. (1999). Multiobjective Evolutionary Algorithms: A comparative case study and the strength pareto approach. IEEE Transactions on Evolutionary Computation, 3(4), 257–271. https://doi.org/10.1109/4235.797969
Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C. M., & da Fonseca, V. G. (2003). Performance assessment of Multiobjective Optimizers: An analysis and review. IEEE Transactions on Evolutionary Computation, 7(2), 117–132. https://doi.org/10.1109/tevc.2003.810758
Funding
This research has been generously supported by the United States Envirnmental Protection Agency (EPA) under the Pollution Prevention (P2) practices grant.
Author information
Authors and Affiliations
Contributions
Conceptualization: Imtiaz Ahmed, Hamed Khosravi, Taofeeq Olajire; Methodology: Hamed Khosravi, Taofeeq Olajire, Imtiaz Ahmed; Formal analysis and investigation: Hamed Khosravi, Taofeeq Olajire; Writing—original draft preparation: Hamed Khosravi, Taofeeq Olajire, Ahmed Shoyeb Raihan; Writing—review and editing: Imtiaz Ahmed, Ahmed Shoyeb Raihan; Supervision: Imtiaz Ahmed.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Khosravi, H., Olajire, T., Raihan, A.S. et al. A data driven sequential learning framework to accelerate and optimize multi-objective manufacturing decisions. J Intell Manuf (2024). https://doi.org/10.1007/s10845-024-02337-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10845-024-02337-y