Skip to main content
SpringerLink
Log in

Certified Machine-Learning Models

  • Conference paper
  • First Online:
SOFSEM 2020: Theory and Practice of Computer Science (SOFSEM 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12011))

Abstract

The massive adoption of Machine Learning (ML) has deeply changed the internal structure, the design and the operation of software systems. ML has shifted the focus from code to data, especially in application areas where it is easier to collect samples that embody correct solutions to individual instances of a problem, than to design and code a deterministic algorithm solving it for all instances. There is an increasing awareness of the need to verify key non-functional properties of ML-based software applications like fairness and privacy. However, the traditional approach trying to verify these properties by code inspection is pointless, since ML models’ behavior mostly depends on the data and parameters used to train them. Classic software certification techniques cannot solve the issue as well. The Artificial Intelligence (AI) community has been working on the idea of preventing undesired behavior by controlling a priori the ML models’ training sets and parameters. In this paper, we take a different, online approach to ML verification, where novel behavioral monitoring techniques based on statistical testing are used to support a dynamic certification framework enforcing the desired properties on black-box ML models in operation. Our aim is to deliver a novel framework suitable for practical certification of distributed ML-powered applications in heavily regulated domains like transport, energy, healthcare, even when the certifying authority is not privy to the model training. To achieve this goal, we rely on three key ideas: (i) use test suites to define desired non-functional properties of ML models, (ii) Use statistical monitoring of ML models’ behavior at inference time to check that the desired behavioral properties are achieved, and (iii) compose monitors’ outcome within dynamic, virtual certificates for composite software applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Mitkov, R.: Anaphora resolution: the state of the art. School of Languages and European Studies, University of Wolverhampton, pp. 1–34 (1999)

    Google Scholar 

  2. Lewis, B.: In the game: the interface between Watson and Jeopardy! IBM J. Res. Dev. 56(34), 171–176 (2012)

    Article  Google Scholar 

  3. Dafoe, A.: AI Governance: A Research Agenda. Governance of AI Program, Future of Humanity Institute, University of Oxford, Oxford, UK (2018)

    Google Scholar 

  4. Guizzardi, R., Li, F.-L., Borgida, A., Mylopoulos, J.: An ontological interpretation of non-functional requirements. In: Frontiers in Artificial Intelligence and Applications, Proceedings of the 8th International Conference on Formal Ontology in Information Systems (FOIS) (2014)

    Google Scholar 

  5. Kesner, R.P., Gilbert, P.E., Wallenstein, G.V.: Testing neural network models of memory with behavioral experiments. Curr. Opin. Neurobiol. 10(2), 260–265 (2000)

    Article  Google Scholar 

  6. Schelter, S., Böse, J.-H., Kirschnick, J., Klein, T., Seufert, S.: Automatically tracking metadata and provenance of machine learning experiments. In: Proceedings of Workshop on ML Systems at NIPS 2017, Long Beach, CA, USA (2017)

    Google Scholar 

  7. Ardagna, C.A., Bellandi, V., Bezzi, M., Ceravolo, P., Damiani, E., Hebert, C.: Model-based big data analytics-as-a-service. Take Big Data to the Next Level. IEEE Transactions on Services Computing (Early Access) (2018)

    Google Scholar 

  8. Redavid, D., et al.: Semantic support for model based big data analytics-as-a-service (MBDAaaS). In: Proceedings of 12th International Conference on Complex, Intelligent, and Software Intensive Systems (CISIS-2018), pp. 1012–1021, Matsue, Japan (2018)

    Google Scholar 

  9. Di Martino, B., Esposito, A., Damiani, E.: Towards AI-powered multiple cloud management. IEEE Internet Comput. 23(1), 64–71 (2019)

    Article  Google Scholar 

  10. Khosravi, P., Liang, Y., Choi, Y., Van den Broeck, G.: What to expect of classifiers? reasoning about logistic regression with missing features. In: Proceedings of the ICML Workshop on Tractable Probabilistic Modeling (TPM), pp. 2716–2724, Macao, China (2019)

    Google Scholar 

  11. Biggio, B., Roli, F.: Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recogn. 84, 317–331 (2018)

    Article  Google Scholar 

  12. Bryson, J., Winfield, A.: Standardizing ethical design for artificial intelligence and autonomous systems. Computer 50(5), 116–119 (2017)

    Article  Google Scholar 

  13. Morley, J., Floridi, L., Kinsey, L., Elhalal, A.: From What to How: An Overview of AI Ethics Tools, Methods and Research to Translate Principles into Practices (2019). arXiv:1905.06876v1

  14. McNamara, D., Soon Ong, C., Williamson, R.C.: Costs and benefits of fair representation learning. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 263–270, ACM, Honolulu, HI, USA (2019)

    Google Scholar 

  15. Adel, T., Valera, I., Ghahramani, Z., Weller, A.: One-network adversarial fairness. In: Proceedings of 33rd AAAI Conference on Artificial Intelligence, Honolulu, HI, USA (2019)

    Google Scholar 

  16. Raff, E., Sylvester, J., Mills, S.: Fair forests: regularized tree induction to minimize model bias. In: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pp. 243–250, ACM, New Orleans, LA (2018)

    Google Scholar 

  17. Madras, D., Creager, E., Pitassi, T., Zemel, R.: Fairness through causal awareness: learning causal latent-variable models for biased data. In: Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, pp. 349–358, ACM, Atlanta, GA (2019)

    Google Scholar 

  18. Rafael, Y., et al.: Four ethical priorities for neurotechnologies and AI. Nat. News 551(7679), 159 (2017)

    Article  Google Scholar 

  19. Morris, J., Lee, G., Parker, K., Bundell, G.A., Lam, C.P.: Software component certification. Computer 34(9), 30–36 (2001)

    Article  Google Scholar 

  20. Damiani, E., Manã, A.: Toward ws-certificate. In: Proceedings of the 2009 ACM Workshop on Secure Web Services, pp. 1–2, ACM, Chicago, IS, USA (2009)

    Google Scholar 

  21. Damiani, E., Ardagna, C.A., El Ioini, N.: Open Source Systems Security Certification. Springer, Berlin (2009). https://doi.org/10.1007/978-0-387-77324-7

    Book  Google Scholar 

  22. Spanoudakis, G., Damiani, E., Maña, A.: Certifying services in cloud: the case for a hybrid, incremental and multi-layer approach. In: Proceedings of IEEE 14th International Symposium on High-Assurance Systems Engineering (HASE 2012), pp. 175–176, IEEE, Singapore (2012)

    Google Scholar 

  23. Anisetti, M., Ardagna, C.A., Damiani, E., Polegri, G.: Test-based security certification of composite services. ACM Trans. Web (TWEB) 13(1), 3 (2019)

    Google Scholar 

  24. Méry, D., Kumar Singh, N.: Trustable formal specification for software certification. In: Proceedings of Symposium on Leveraging Applications of Formal Methods, Verification and Validation, 2010. Lecture Notes in Computer Science, vol. 6416, pp. 312–326 (2010)

    Google Scholar 

  25. Denney, E., Pai, G.: Evidence arguments for using formal methods in software certification. In: Proceedings of IEEE Symposium on Software Reliability Engineering (ISSRE), pp. 375–380, IEEE, Pasadena, CA, USA (2013)

    Google Scholar 

  26. Armando, A., et al.: The AVISPA tool for the automated validation of internet security protocols and applications. In: Proceedings of CAV 2005: Computer Aided Verification. Lecture Notes in Computer Science, vol. 3576, pp. 281–285 (2005)

    Chapter  Google Scholar 

  27. Clarkson, M.R., Schneider, F.B.: Hyperproperties. J. Comput. Secur. 18(6), 1157–1210 (2010)

    Article  Google Scholar 

  28. Datta, A., Franklin, J., Garg, D., Jia, L., Kaynar, D.: On adversary models and compositional security. IEEE Secur. Priv. 9(3), 26–32 (2011)

    Article  Google Scholar 

  29. Fuchs, A., Gürgens, S.: Preserving confidentiality in component compositions. In: Proceedings of International Conference on Software Composition. Lecture Notes in Computer Science, vol. 8088, pp. 33–48 (2013)

    Chapter  Google Scholar 

  30. Mellado, D., Fernández-Medina, E., Piattini, M.: A common criteria based security requirements engineering process for the development of secure information systems. Comput. Stan. Interfaces 29(2), 244–253 (2007)

    Article  Google Scholar 

  31. Scott, S.L.: Multi-armed bandit experiments in the online service economy. Appl. Stochast. Models Bus. Ind. 31, 37–49 (2015)

    Article  MathSciNet  Google Scholar 

  32. Leite, R., Pavel, B., Vanschoren, J,: Selecting classification algorithms with active testing. In: Proceedings of MLDM 2012: Machine Learning and Data Mining in Pattern Recognition. Lecture Notes in Computer Science, vol. 7376, pp. 117–131 (2012)

    Chapter  Google Scholar 

  33. Antos, A., Grover, V., Szepesvári, C.: Active learning in multi-armed bandits. In: Freund, Y., Györfi, L., Turán, G., Zeugmann, T. (eds.) ALT 2008. LNCS (LNAI), vol. 5254, pp. 287–302. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87987-9_25

    Chapter  MATH  Google Scholar 

  34. Menke, J., Martinez, T.R.: Using permutations instead of student’s t distribution for p-values in paired-difference algorithm comparisons. In: Proceedings of 2004 IEEE International Joint Conference on Neural Networks, vol. 2, pp. 1331–1335, IEEE, Budapest, Hungary (2004)

    Google Scholar 

  35. Vanschoren, J.: Meta-learning: A survey (2018). arXiv:1810.03548

  36. Damiani, E., Tettamanzi, A., Liberali, V.: On-line evolution of FPGA-based circuits: a case study on hash functions. In: Proceedings of the First NASA/DoD Workshop on Evolvable Hardware, pp. 26–33, IEEE, Pasadena, CA, USA (1999)

    Google Scholar 

  37. Brennet, T., Dieterich, W., Ehret, B.: Evaluating the predictive validity of the COMPAS risk and needs assessment system. Crim. Justice Behav. 36(1), 21–40 (2008)

    Google Scholar 

  38. Lai, T.L., Robbins, H.: Asymptotically efficient adaptive allocation rules. Adv. Appl. Math. 6(1), 4–22 (1985)

    Article  MathSciNet  Google Scholar 

  39. Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., Huq, A.: Algorithmic decision making and the cost of fairness. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 797–806, ACM, Halifax, NS, Canada (2017)

    Google Scholar 

  40. Scott, S.L.: Applied stochastic models in business and industry. Appl. Stoch. Models Bus. Ind. 26, 639–658 (2010)

    Article  MathSciNet  Google Scholar 

  41. Anisetti, M., Ardagna, C.A., Gaudenzi, F., Damiani, E., Diomede, N., Tufarolo, P.: Moon cloud: a cloud platform for ICT security governance. In: Proceedings of IEEE Global Communications Conference (GLOBECOM 2018), pp. 1–7, IEEE, Abu Dhabi, UAE (2018)

    Google Scholar 

  42. Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating noise to sensitivity in private data analysis. TCC 2006: Theory of Cryptography. Lecture Notes in Computer Science, vol. 3876, pp. 265–284 (2006)

    Chapter  Google Scholar 

Download references

Acknowledgements

Research supported, in parts, by EC H2020 Project CONCORDIA GA 830927 and Università degli Studi di Milano under the program “Piano sostegno alla ricerca 2018”.

Author information

Authors and Affiliations

  1. Center on Cyber-Physical Systems, Khalifa University, Abu Dhabi, UAE

    Ernesto Damiani

  2. Computer Science Department, Università degli Studi di Milano, Milan, Italy

    Ernesto Damiani & Claudio A. Ardagna

Authors
  1. Ernesto Damiani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claudio A. Ardagna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ernesto Damiani .

Editor information

Editors and Affiliations

  1. University of Macedonia, Thessaloniki, Greece

    Alexander Chatzigeorgiou

  2. University of Bergamo, Bergamo, Italy

    Riccardo Dondi

  3. Cyprus University of Technology, Limassol, Cyprus

    Herodotos Herodotou

  4. Carnegie Mellon University Qatar, Doha, Qatar

    Christos Kapoutsis

  5. Open University of Cyprus, Nicosia, Cyprus

    Yannis Manolopoulos

  6. University of Cyprus, Nicosia, Cyprus

    George A. Papadopoulos

  7. Paris Dauphine University, Paris, France

    Florian Sikora

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Damiani, E., Ardagna, C.A. (2020). Certified Machine-Learning Models. In: Chatzigeorgiou, A., et al. SOFSEM 2020: Theory and Practice of Computer Science. SOFSEM 2020. Lecture Notes in Computer Science(), vol 12011. Springer, Cham. https://doi.org/10.1007/978-3-030-38919-2_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-38919-2_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-38918-5

  • Online ISBN: 978-3-030-38919-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation