Skip to main content

Modelling Machine Learning Models

Part of the Studies in Applied Philosophy, Epistemology and Rational Ethics book series (SAPERE,volume 44)

Abstract

Machine learning (ML) models make decisions for governments, companies, and individuals. Accordingly, there is the increasing concern of not having a rich explanatory and predictive account of the behaviour of these ML models relative to the users’ interests (goals) and (pre-)conceptions (ontologies). We argue that the recent research trends in finding better characterisations of what a ML model does are leading to the view of ML models as complex behavioural systems. A good explanation for a model should depend on how well it describes the behaviour of the model in simpler, more comprehensible, or more understandable terms according to a given context. Consequently, we claim that a more contextual abstraction is necessary (as is done in system theory and psychology), which is very much like building a subjective mind modelling problem. We bring some research evidence of how this partial and subjective modelling of machine learning models can take place, suggesting that more machine learning is the answer.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-96448-5_16
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-96448-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Hardcover Book
USD   159.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.

References

  • Balasubramanian, V., Ho, S.-S., Vovk, V.: Conformal Prediction for Reliable Machine Learning: Theory, Adaptations and Applications. Newnes, Oxford (2014)

    MATH  Google Scholar 

  • Bella, A., Ferri, C., Hernández-Orallo, J., Ramírez-Quintana, M.J.: Using negotiable features for prescription problems. Computing 91(2), 135–168 (2011)

    CrossRef  Google Scholar 

  • Bella, A., Ferri, C., Hernández-Orallo, J., Ramírez-Quintana, M.J.: On the effect of calibration in classifier combination. Appl. Intell. 38(4), 566–585 (2013)

    CrossRef  Google Scholar 

  • Blanco-Vega, R., Ferri-Ramírez, C., Hernández-Orallo, J., Ramírez-Quintana, M.: Estimating the class probability threshold without training data. In: Workshop on ROC Analysis in Machine Learning, p. 9 (2006)

    Google Scholar 

  • Blanco-Vega, R., Hernández-Orallo, J., Ramírez-Quintana, M.: Analysing the trade-off between comprehensibility and accuracy in mimetic models. In: Discovery Science, pp. 35–39 (2004)

    Google Scholar 

  • Boden, M.A.: Computer Models of Mind: Computational Approaches in Theoretical Psychology. Cambridge University Press, New York (1988)

    Google Scholar 

  • Chart, D.: A Theory of Understanding: Philosophical and Psychological Perspectives. Routledge, New York (2000)

    Google Scholar 

  • Core, M., Lane, H.C., van Lent, M., Gomboc, D., Solomon, S., Rosenberg, M.: Building explainable artificial intelligence systems. In: Proceedings of the 18th Innovative Applications of Artificial Intelligence Conference (2006)

    Google Scholar 

  • Cousot, P., Cousot, R.: Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: Proceedings of the 4th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages, pp. 238–252 (1977)

    Google Scholar 

  • Cui, Z., Chen, W., He, Y., Chen, Y.: Optimal action extraction for random forests and boosted trees. In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 179–188 (2015)

    Google Scholar 

  • Domingos, P.: Knowledge discovery via multiple models. Intell. Data Anal. 2(1–4), 187–202 (1998)

    CrossRef  Google Scholar 

  • Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv (2017)

    Google Scholar 

  • Elkan, C.: The foundations of cost-sensitive learning. In: International Joint Conference on Artificial Intelligence, vol. 17, pp. 973–978 (2001)

    Google Scholar 

  • Fenton, N.E., Neil, M.: Software metrics: successes, failures and new directions. J. Syst. Softw. 47(2), 149–157 (1999)

    CrossRef  Google Scholar 

  • Ferri, C., Hernández-Orallo, J., Ramírez-Quintana, M.J.: From ensemble methods to comprehensible models. In: 5th International Conference on Discovery Science, pp. 165–177 (2002)

    Google Scholar 

  • Ferri, C., Hernández-Orallo, J., Martínez-Usó, A., Ramírez-Quintana, M.: Identifying dominant models when the noise context is known. In: First Workshop on Generalization and Reuse of Machine Learning Models Over Multiple Contexts (2014)

    Google Scholar 

  • Ferri, C., Hernández-Orallo, J., Modroiu, R.: An experimental comparison of performance measures for classification. Pattern Recogn. Lett. 30(1), 27–38 (2009)

    CrossRef  Google Scholar 

  • Hernández-Orallo, J., Flach, P.A., Ferri, C.: A unified view of performance metrics: translating threshold choice into expected classification loss. J. Mach. Learn. Res. 13, 2813–2869 (2012)

    MathSciNet  MATH  Google Scholar 

  • Hernández-Orallo, J.: The Measure of All Minds: Evaluating Natural and Artificial Intelligence. Cambridge University Press, New York (2017)

    CrossRef  Google Scholar 

  • Hernández-Orallo, J., Martínez-Usó, A., Prudêncio, R.B., Kull, M., Flach, P., Farhan Ahmed, C., Lachiche, N.: Reframing in context: a systematic approach for model reuse in machine learning. AI Commun. 29(5), 551–566 (2016)

    MathSciNet  CrossRef  Google Scholar 

  • Huang, L., Joseph, A.D., Nelson, B., Rubinstein, B.I., Tygar, J.D.: Adversarial machine learning. In: Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence, pp. 43–58 (2011)

    Google Scholar 

  • Jain, S., Osherson, D., Royer, J.S., Sharma, A.: Systems that Learn, 2nd edn. MIT Press, Cambridge (1999)

    Google Scholar 

  • Japkowicz, N., Shah, M.: Evaluating Learning Algorithms: A Classification Perspective. Cambridge University Press, New York (2011)

    CrossRef  Google Scholar 

  • Kamiran, F., Calders, T.: Classifying without discriminating. In: 2nd International Conference on Computer, Control and Communication, pp. 1–6 (2009)

    Google Scholar 

  • Langley, P., et al.: Selection of relevant features in machine learning. In: Proceedings of the AAAI Fall Symposium on Relevance, vol. 184, pp. 245–271 (1994)

    Google Scholar 

  • Lichman, M.: UCI machine learning repository (2013)

    Google Scholar 

  • Lyu, Q., Chen, Y., Li, Z., Cui, Z., Chen, L., Zhang, X., Shen, H.: Extracting actionability from machine learning models by sub-optimal deterministic planning. arXiv preprint arXiv:1611.00873 (2016)

  • Martínez-Plumed, F., Prudêncio, R.B.C., Usó, A.M., Hernández-Orallo, J.: Making sense of item response theory in machine learning. In: European Conference on Artificial Intelligence, pp. 1140–1148 (2016)

    Google Scholar 

  • Ribeiro, M.T., Singh, S., Guestrin, C.: Why should I trust you?: Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144 (2016)

    Google Scholar 

  • Samek, W., Wiegand, T., Müller, K.-R.: Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models, ArXiv e-prints (2017)

    Google Scholar 

  • Tickle, A.B., Andrews, R., Golea, M., Diederich, J.: The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks. Trans. Neur. Netw. 9(6), 1057–1068 (1998)

    CrossRef  Google Scholar 

  • Turney, P.D.: The management of context-sensitive features: a review of strategies. arXiv preprint cs/0212037 (2002)

    Google Scholar 

  • Weller, A.: Challenges for transparency. arXiv preprint arXiv:1708.01870 (2017)

  • Yang, Q., Yin, J., Ling, C., Pan, R.: Extracting actionable knowledge from decision trees. IEEE Trans. Knowl. Data Eng. 19(1), 43–56 (2007)

    CrossRef  Google Scholar 

Download references

Acknowledgements

This material is based upon work supported by the Air Force Office of Scientific Research under award number FA9550-17-1-0287, the EU (FEDER), and the Spanish MINECO under grant TIN 2015-69175-C4-1-R, the Generalitat Valenciana PROMETEOII/2015/013. F. Martínez-Plumed was also supported by INCIBE (Ayudas para la excelencia de los equipos de investigación avanzada en ciberseguridad). J. H-Orallo also received a Salvador de Madariaga grant (PRX17/00467) from the Spanish MECD for a research stay at the CFI, Cambridge, and a BEST grant (BEST/2017/045) from the GVA for another research stay at the CFI.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Raül Fabra-Boluda .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Fabra-Boluda, R., Ferri, C., Hernández-Orallo, J., Martínez-Plumed, F., Ramírez-Quintana, M.J. (2018). Modelling Machine Learning Models. In: Müller, V. (eds) Philosophy and Theory of Artificial Intelligence 2017. PT-AI 2017. Studies in Applied Philosophy, Epistemology and Rational Ethics, vol 44. Springer, Cham. https://doi.org/10.1007/978-3-319-96448-5_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-96448-5_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-96447-8

  • Online ISBN: 978-3-319-96448-5

  • eBook Packages: Computer ScienceComputer Science (R0)