Abstract
Product development is the process of creating and bringing a new or improved product to market. Formulation trials constitute a crucial stage in product development, often involving the exploration of numerous variables and product properties. Traditional methods of formulation trials involve time-consuming experimentation, trial and error, and iterative processes. In recent years, machine learning (ML) has emerged as a promising avenue to streamline this complex journey by enhancing efficiency, innovation, and customization. One of the paramount challenges in ML for product development is the models’ lack of interpretability and explainability. This challenge poses significant limitations in gaining user trust, meeting regulatory requirements, and understanding the rationale behind ML-driven decisions. Moreover, formulation trials involve the exploration of relationships and similarities among previous preparations; however, data related to formulation are typically stored in tables and not in a network-like manner. To cope with the above challenges, we propose a general methodology for fast product development leveraging graph ML models, explainability techniques, and powerful data visualization tools. Starting from tabular formulation trials, our model simultaneously learns a latent graph between items and a downstream task, i.e. predicting consumer-appealing properties of a formulation. Subsequently, explainability techniques based on graphs, perturbation, and sensitivity analysis effectively support the R&D department in identifying new recipes for reaching a desired property. We evaluate our model on two datasets derived from a case study based on food design plus a standard benchmark from the healthcare domain. Results show the effectiveness of our model in predicting the outcome of new formulations. Thanks to our solution, the company has drastically reduced the labor-intensive experiments in real laboratories and the waste of materials.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
https://marvl.infotech.monash.edu/webcola/, March 2024.
- 2.
https://www.pyg.org/, June 2024.
References
Material prediction for design automation using graph representation learning. In: International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, vol. Volume 3A: 48th Design Automation Conference (DAC) (08 2022). https://doi.org/10.1115/DETC2022-88049, https://doi.org/10.1115/DETC2022-88049
Altmann, A., Toloşi, L., Sander, O., Lengauer, T.: Permutation importance: a corrected feature importance measure. Bioinformatics 26(10), 1340–1347 (2010). https://doi.org/10.1093/bioinformatics/btq134
Dileo, M., Zignani, M., Gaito, S.: Temporal graph learning for dynamic link prediction with text in online social networks. Mach. Learn. 113(6), 1–20 (2023). https://doi.org/10.1007/s10994-023-06475-x
Franz, M., et al.: Cytoscape.js 2023 update: a graph theory library for visualization and analysis. Bioinformatics 39(1), btad031 (2023)
Gaudelet, T., et al.: Utilizing graph machine learning within drug discovery and development. Briefings Bioinf. 22(6), bbab159 (2021). https://doi.org/10.1093/bib/bbab159
Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: ICML Proceedings of Machine Learning Research, vol. 70, pp. 1263–1272. PMLR (2017)
He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., Wang, M.: LightGCN: simplifying and powering graph convolution network for recommendation. In: SIGIR, pp. 639–648. ACM (2020)
Kazi, A., Cosmo, L., Ahmadi, S., Navab, N., Bronstein, M.M.: Differentiable graph module (DGM) for graph convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 45(2), 1606–1617 (2023)
Kazi, A., Farghadani, S., Aganj, I., Navab, N.: IA-GCN: interpretable attention based graph convolutional network for disease prediction. In: Cao, X., Xu, X., Rekik, I., Cui, Z., Ouyang, X. (eds.) Machine Learning in Medical Imaging: 14th International Workshop, MLMI 2023, Held in Conjunction with MICCAI 2023, Vancouver, BC, Canada, October 8, 2023, Proceedings, Part I, pp. 382–392. Springer Nature Switzerland, Cham (2024). https://doi.org/10.1007/978-3-031-45673-2_38
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (2015). http://arxiv.org/abs/1412.6980
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (Poster). OpenReview.net (2017)
Krahe, C., Bräunche, A., Jacob, A., Stricker, N., Lanza, G.: Deep learning for automated product design. Procedia CIRP 91, 3–8 (2020). https://doi.org/10.1016/j.procir.2020.01.135, https://www.sciencedirect.com/science/article/pii/S2212827120307769, enhancing design through the 4th Industrial Revolution Thinking
Langville, A.N., Meyer, C.D.: A survey of eigenvector methods for web information retrieval. SIAM Rev. 47(1), 135–161 (2005). https://doi.org/10.1137/S0036144503424786
Li, C.T., Tsai, Y.C., Liao, J.C.: Graph neural networks for tabular data learning. In: 2023 IEEE 39th International Conference on Data Engineering (ICDE), pp. 3589–3592 (2023). https://doi.org/10.1109/ICDE55515.2023.00275
Li, F., et al.: Developing a dynamic graph network for interpretable analysis of multi-modal MRI data in parkinson’s disease diagnosis. In: EMBC, pp. 1–4. IEEE (2023)
Marinescu, R.V., et al.: TADPOLE challenge: accurate alzheimer’s disease prediction through crowdsourced forecasting of future data. Predict. Intell. Med. 11843, 1–10 (2019)
Merchant, A., Batzner, S., Schoenholz, S.S., Aykol, M., Cheon, G., Cubuk, E.D.: Scaling deep learning for materials discovery. Nature 624(7990), 80–85 (2023). https://doi.org/10.1038/s41586-023-06735-9
Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Networks 20(1), 61–80 (2009). https://doi.org/10.1109/TNN.2008.2005605
Shi, Y., Huang, Z., Feng, S., Zhong, H., Wang, W., Sun, Y.: Masked label prediction: Unified message passing model for semi-supervised classification. In: IJCAI, pp. 1548–1554. ijcai.org (2021)
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: ICLR (Poster), OpenReview.net (2018)
Verdone, A., Scardapane, S., Panella, M.: Explainable spatio-temporal graph neural networks for multi-site photovoltaic energy production. Appl. Energy 353, 122151 (2024). https://doi.org/10.1016/j.apenergy.2023.122151, https://www.sciencedirect.com/science/article/pii/S0306261923015155
Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. ACM Trans. Graph. 38(5), 146:1–146:12 (2019)
Ying, Z., Bourgeois, D., You, J., Zitnik, M., Leskovec, J.: GNNExplainer: generating explanations for graph neural networks. In: NeurIPS, pp. 9240–9251 (2019)
Zhu, Y., et al.: A survey on graph structure learning: Progress and opportunities (2022)
Acknowledgments
We would like to thank the members of the PVM Lainate team for the opportunity given to us by this collaboration, which fostered the development of this novel methodology. A special thanks to Marco Violi and Maria Giovanna Esposito for their time and assistance in understanding and analyzing the data.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Dileo, M., Olmeda, R., Pindaro, M., Zignani, M. (2024). Graph Machine Learning for Fast Product Development from Formulation Trials. In: Bifet, A., Krilavičius, T., Miliou, I., Nowaczyk, S. (eds) Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track. ECML PKDD 2024. Lecture Notes in Computer Science(), vol 14949. Springer, Cham. https://doi.org/10.1007/978-3-031-70378-2_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-70378-2_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70377-5
Online ISBN: 978-3-031-70378-2
eBook Packages: Computer ScienceComputer Science (R0)
