Practical and Open Source Best Practices for Ethical Machine Learning

Abstract

The acknowledged operational, ethical, legal and governance risks involved in applying Machine Learning (ML) have generated a need for a clear and thoughtful repository of best practices on how to responsibly govern, manage and implement “responsible ML”. The Foundation for Best Practices in Machine Learning (a non-profit foundation) seeks to promote responsible ML through creating an open-sourced, freely accessible repository of best practices and associated guides. Its model and organisational guides look at both the technical and institutional requirements needed to promote responsible ML. Both blueprints touch on subjects such as “Fairness & Non-Discrimination”, “Representativeness & Specification”, “Product Traceability”, “Explainability” amongst other topics. Where the organisational guide relates to organisation-wide process and responsibilities (i.e. the necessity of setting proper product definitions and risk portfolios); the model guide details issues ranging from cost function specification and optimisation to selection function characterization, from disparate impact metrics to local explanations and counterfactuals. It also addresses issues concerning thorough product management. These guidelines have been developed principally by senior ML engineers, data scientists, data science managers, and legal professionals for ML engineers, data scientists, data science managers, compliance professionals, legal practitioners, and, more broadly, management. The Foundation’s philosophy is that (a) context is key, (b) responsible ML starts with prudent MLOps and product management, and (c) responsible ML needs to be supported by all aspects of an organisation’s structure.