Skip to main content

Machine Learning Data Suitability and Performance Testing Using Fault Injection Testing Framework

  • Conference paper
  • First Online:
Engineering of Computer-Based Systems (ECBS 2023)


Creating resilient machine learning (ML) systems has become necessary to ensure production-ready ML systems that acquire user confidence seamlessly. The quality of the input data and the model highly influence the successful end-to-end testing in data-sensitive systems. However, the testing approaches of input data are not as systematic and are few compared to model testing. To address this gap, this paper presents the Fault Injection for Undesirable Learning in input Data (FIUL-Data) testing framework that tests the resilience of ML models to multiple intentionally-triggered data faults. Data mutators explore vulnerabilities of ML systems against the effects of different fault injections. The proposed framework is designed based on three main ideas: The mutators are not random; one data mutator is applied at an instance of time, and the selected ML models are optimized beforehand. This paper evaluates the FIUL-Data framework using data from analytical chemistry, comprising retention time measurements of anti-sense oligonucleotide. Empirical evaluation is carried out in a two-step process in which the responses of selected ML models to data mutation are analyzed individually and then compared with each other. The results show that the FIUL-Data framework allows the evaluation of the resilience of ML models. In most experiments cases, ML models show higher resilience at larger training datasets, where gradient boost performed better than support vector regression in smaller training sets. Overall, the mean squared error metric is useful in evaluating the resilience of models due to its higher sensitivity to data mutation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions


  1. Chapter 16 quantitative analysis by gas chromatography sources of errors, accuracy and precision of chromatographic measurements. In: Guiochon, G., Guillemin, C.L. (eds.) For Laboratory Analyses and On-Line Process Control, Journal of Chromatography Library, vol. 42, pp. 661–687. Elsevier (1988).

  2. Breck, E., Cai, S., Nielsen, E., Salib, M., Sculley, D.: The ml test score: a rubric for ml production readiness and technical debt reduction. In: 2017 IEEE International Conference on Big Data (Big Data), pp. 1123–1132 (2017).

  3. D’Archivio, A.: Artificial neural network prediction of retention of amino acids in reversed-phase HPLC under application of linear organic modifier gradients and/or pH gradients. Molecules 24(3), 632 (2019).,

  4. Enmark, M., Häggström, J., Samuelsson, J., Fornstedt, T.: Building machine-learning-based models for retention time and resolution predictions in ion pair chromatography of oligonucleotides. J. Chromatogr. A 1671, 462999 (2022).

    Article  Google Scholar 

  5. Fornstedt, T., Forssén, P., Westerlund, D.: Basic HPLC theory and definitions: retention, thermodynamics, selectivity, zone spreading, kinetics, and resolution. Anal. Sep. Sci. 5 Vol. Set 2, 1–22 (2015).

  6. Gangolli, A., Mahmoud, Q.H., Azim, A.: A systematic review of fault injection attacks on IoT systems. Electronics 11(13), 2023 (2022).,

  7. Ghiduk, A.S., Girgis, M.R., Shehata, M.H.: Higher order mutation testing: a systematic literature review. Comput. Sci. Rev. 25, 29–48 (2017).

    Article  MathSciNet  Google Scholar 

  8. Hellier, E., Edworthy, J., Lee, A.: An analysis of human error in the analytical measurement task in chemistry. Int. J. Cogn. Ergon. 5(4), 445–458 (2001).

    Article  Google Scholar 

  9. Jha, S., Banerjee, S.S., Cyriac, J., Kalbarczyk, Z.T., Iyer, R.K.: AVFI: fault injection for autonomous vehicles. In: 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), pp. 55–56. IEEE Computer Society (2018).

  10. Kaiser, R.E.: Errors in chromatography. Chromatographia 4, 479–490 (1971).

    Article  Google Scholar 

  11. Katzir, Z., Elovici, Y.: Quantifying the resilience of machine learning classifiers used for cyber security. Expert Syst. Appl. 92, 419–429 (2018).

    Article  Google Scholar 

  12. Kohlbacher, O., Quinten, S., Strum, M., Mayr, B.M., Huber, C.G.: Structure-activity relationships in chromatography: retention prediction of oligonucleotides with support vector regression. Angew. Chem. Int. Ed. Engl. 45(42), 7009–7012 (2006).

  13. Korany, M.A., Mahgoub, H., Fahmy, O.T., Maher, H.M.: Application of artificial neural networks for response surface modelling in HPLC method development. J. Adv. Res. 3(1), 53–63 (2012)

    Article  Google Scholar 

  14. Kuselman, I., et al.: House-of-security approach to measurement in analytical chemistry: quantification of human error using expert judgments. Accred. Qual. Assur. 18(6), 459–467 (2013).

    Article  Google Scholar 

  15. Kuselman, I., Pennecchi, F., Fajgelj, A., Karpov, Y.: Human errors and reliability of test results in analytical chemistry. Accred. Qual. Assur. 18, 3–9 (2013).

    Article  Google Scholar 

  16. Lotfi, R., Gholamrezaei, A., Kadłubek, M., Afshar, M., Ali, S.S., Kheiri, K.: A robust and resilience machine learning for forecasting agri-food production. Sci. Rep. 12(1), 21787 (2022).

    Article  Google Scholar 

  17. Lu, Y., Sun, W., Sun, M.: Towards mutation testing of reinforcement learning systems. J. Syst. Architect. 131, 102701 (2022).

    Article  Google Scholar 

  18. Ma, L., et al.: DeepMutation: mutation testing of deep learning systems. In: 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE), pp. 100–111. IEEE Computer Society, Los Alamitos, CA, USA (2018).

  19. Narayanan, N., Pattabiraman, K.: TF-DM: tool for studying ml model resilience to data faults. In: 2021 IEEE/ACM Third International Workshop on Deep Learning for Testing and Testing for Deep Learning (DeepTest), pp. 25–28. IEEE Computer Society, Los Alamitos, CA, USA (2021).

  20. Nurminen, J.K., et al.: Software framework for data fault injection to test machine learning systems. In: 2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), pp. 294–299 (2019).

  21. Papadakis, M., Kintis, M., Zhang, J., Jia, Y., Traon, Y.L., Harman, M.: Chapter six - mutation testing advances: an analysis and survey. In: Memon, A.M. (ed.) Advances in Computers, Advances in Computers, vol. 112, pp. 275–378. Elsevier (2019).

  22. Petritis, K., et al.: Use of artificial neural networks for the accurate prediction of peptide liquid chromatography elution times in proteome analyses. Anal. Chem. 75(5), 1039–1048 (2003).

    Article  Google Scholar 

  23. Riccio, V., Jahangirova, G., Stocco, A., Humbatova, N., Weiss, M., Tonella, P.: Testing machine learning based systems: a systematic mapping. Empir. Softw. Eng. 25(6), 5193–5254 (2020).

    Article  Google Scholar 

  24. Risum, A.B., Bro, R.: Using deep learning to evaluate peaks in chromatographic data. Talanta 204, 255–260 (2019).

    Article  Google Scholar 

  25. Sturm, M., Quinten, S., Huber, C.G., Kohlbacher, O.: A statistical learning approach to the modeling of chromatographic retention of oligonucleotides incorporating sequence and secondary structure data. Nucleic Acids Res. 35(12), 4195–4202 (2007).

    Article  Google Scholar 

  26. Tambon, F., Khomh, F., Antoniol, G.: A probabilistic framework for mutation testing in deep neural networks. Inf. Softw. Technol. 155(C), 107129 (2023).

  27. Tran, A., Hyne, R., Pablo, F., Day, W., Doble, P.: Optimisation of the separation of herbicides by linear gradient high performance liquid chromatography utilising artificial neural networks. Talanta 71(3), 1268–1275 (2007).

  28. Vairo, T., Pettinato, M., Reverberi, A.P., Milazzo, M.F., Fabiano, B.: An approach towards the implementation of a reliable resilience model based on machine learning. Process Saf. Environ. Prot. 172, 632–641 (2023).

    Article  Google Scholar 

  29. Webb, R., Doble, P., Dawson, M.: Optimisation of HPLC gradient separations using artificial neural networks (ANNs): application to benzodiazepines in post-mortem samples. J. Chromatogr. B 877(7), 615–620 (2009).

    Article  Google Scholar 

  30. Zhang, J.M., Harman, M., Ma, L., Liu, Y.: Machine learning testing: survey, landscapes and horizons. IEEE Trans. Software Eng. 48(1), 1–36 (2022).

    Article  Google Scholar 

  31. Zheng, A., Casari, A.: Feature Engineering for Machine Learning. O’Reilly Media, Inc. (2018)

    Google Scholar 

  32. Zhu, Q., Panichella, A., Zaidman, A.: A systematic literature review of how mutation testing supports quality assurance processes. Softw. Test. Verification and Reliab. 28(6), e1675 (2018).

    Article  Google Scholar 

Download references


This work has been funded by the Knowledge Foundation of Sweden (KKS) through the Synergy project - Improved Methods for Process and Quality Controls using Digital Tools (IMPAQCDT) grant number (20210021). In this project, we acknowledge Gergely Szabados, Jakob Häggström, and Patrik Forssén from the Department of Engineering and Chemical Sciences/Chemistry at Karlstad University for their contribution to the acquisition and preprocessing of data.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Bestoun S. Ahmed .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rahal, M., Ahmed, B.S., Samuelsson, J. (2024). Machine Learning Data Suitability and Performance Testing Using Fault Injection Testing Framework. In: Kofroň, J., Margaria, T., Seceleanu, C. (eds) Engineering of Computer-Based Systems. ECBS 2023. Lecture Notes in Computer Science, vol 14390. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49251-8

  • Online ISBN: 978-3-031-49252-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics