Skip to main content
SpringerLink
Log in

Perlocution vs Illocution: How Different Interpretations of the Act of Explaining Impact on the Evaluation of Explanations and XAI

  • Conference paper
  • First Online:
Explainable Artificial Intelligence (xAI 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1901))

Included in the following conference series:

  • 790 Accesses

Abstract

This article discusses the concepts of illocutionary, perlocutionary, and locutionary acts, and their role in understanding explanations. Illocutionary acts concern the speaker’s intended meaning, perlocutionary acts refer to the listener’s reaction, and locutionary acts are about the speech act itself. We suggest a new way to categorise established definitions of explanation based on these speech act principles. This method enhances our grasp of how explanations work. We found that if you define explanation as a perlocutionary act, it requires subjective judgements. This makes it hard to assess an explanation objectively before the listener receives it. On the other hand, we claim that existing legal systems prefer explanations based on illocutionary acts. We propose that the exact meaning of explanation depends on the situation. Some kinds of definitions suit specific circumstances better. For example, in educational settings, a perlocutionary approach often works best, while legal settings call for an illocutionary approach. Additionally, we show how current measures of explainability can be grouped based on their theoretical support and the speech act they rely on. This categorisation helps us pinpoint which measures are best for assessing the results of Explainable AI (XAI) tools in legal or other settings. In simpler terms, we are explaining how to evaluate and improve XAI and explanations in different situations, such as in education and law. By understanding where and when to apply different explainability measures, we can make better and more specialised XAI tools. This will lead to significant improvements in AI explainability.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The exact mechanisms and implications of mental models in various aspects of cognition and decision-making are still areas of active investigation.

  2. 2.

    https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32016R0679.

  3. 3.

    https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206.

  4. 4.

    https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32019R1150.

  5. 5.

    Prototypes are instances of the ground-truth considered similar to a specific input-output for which the similarity explains the model’s behaviour.

  6. 6.

    https://www.unicef.org/esa/sites/unicef.org.esa/files/2019-08/ThinkPiece_9_LearnerCentredEducation.pdf.

References

  1. Achinstein, P.: The Nature of Explanation. Oxford University Press, Oxford (1983). https://books.google.it/books?id=0XI8DwAAQBAJ

  2. Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018). https://doi.org/10.1109/ACCESS.2018.2870052

    Article  Google Scholar 

  3. Antoniadi, A.M., et al.: Current challenges and future opportunities for XAI in machine learning-based clinical decision support systems: a systematic review. Appl. Sci. 11(11), 5088 (2021). https://doi.org/10.3390/app11115088, https://www.mdpi.com/2076-3417/11/11/5088

  4. Arrieta, A.B., et al.: Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 58, 82–115 (2020). https://doi.org/10.1016/j.inffus.2019.12.012

    Article  Google Scholar 

  5. Austin, J., Urmson, J., Sbisà, M.: How to Do Things with Words. William James lectures, Clarendon Press (1975). https://books.google.it/books?id=XnRkQSTUpmgC

  6. Beckers, S.: Causal explanations and XAI. In: Schölkopf, B., Uhler, C., Zhang, K. (eds.) 1st Conference on Causal Learning and Reasoning, CLeaR 2022, Sequoia Conference Center, Eureka, CA, USA, 11–13 April 2022. Proceedings of Machine Learning Research, vol. 177, pp. 90–109. PMLR (2022). https://proceedings.mlr.press/v177/beckers22a.html

  7. Bibal, A., Lognoul, M., de Streel, A., Frénay, B.: Legal requirements on explainability in machine learning. Artif. Intell. Law 29(2), 149–169 (2021). https://doi.org/10.1007/s10506-020-09270-4

    Article  Google Scholar 

  8. Brandes, D., Ginnis, P.: A Guide to Student-centred Learning. Stanley Thornes (1996). https://books.google.ch/books?id=MTJSGGTAN3MC

  9. Bromberger, S.: Why-questions. In: Colodny, R.G. (ed.) Mind and Cosmos - Essays in Contemporary Science and Philosophy, pp. 86–111. University of Pittsburgh Press (1966)

    Google Scholar 

  10. Buçinca, Z., Lin, P., Gajos, K.Z., Glassman, E.L.: Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems. In: Paternò, F., Oliver, N., Conati, C., Spano, L.D., Tintarev, N. (eds.) 25th International Conference on Intelligent User Interfaces, IUI 2020, Cagliari, Italy, 17–20 March 2020, pp. 454–464. ACM (2020). https://doi.org/10.1145/3377325.3377498

  11. Chen, T., Guestrin, C.: Xgboost: A scalable tree boosting system. In: Krishnapuram, B., Shah, M., Smola, A.J., Aggarwal, C.C., Shen, D., Rastogi, R. (eds.) Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016, pp. 785–794. ACM (2016). https://doi.org/10.1145/2939672.2939785

  12. Colombo, M.: Experimental philosophy of explanation rising: the case for a plurality of concepts of Explanation. Cogn. Sci. 41(2), 503–517 (2017). https://doi.org/10.1111/cogs.12340

    Article  Google Scholar 

  13. DG, E.: Understanding algorithmic decision-making: opportunities and challenges (2019). https://www.europarl.europa.eu/thinktank/en/document/EPRS_STU(2019)624261

  14. Dhurandhar, A., et al.: Explanations based on the missing: towards contrastive explanations with pertinent negatives. In: Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, pp. 3–8, 2018. Montréal, Canada, pp. 590–601 (2018). https://proceedings.neurips.cc/paper/2018/hash/c5ff2543b53f4cc0ad3819a36752467b-Abstract.html

  15. Dieber, J., Kirrane, S.: A novel model usability evaluation framework (muse) for explainable artificial intelligence. Inf. Fusion 81, 143–153 (2022). https://doi.org/10.1016/j.inffus.2021.11.017

    Article  Google Scholar 

  16. Endsley, M.R.: Toward a theory of situation awareness in dynamic systems. Hum. Factors 37(1), 32–64 (1995). https://doi.org/10.1518/001872095779049543

    Article  Google Scholar 

  17. Gary, M.S., Wood, R.E.: Mental models, decision rules, and performance heterogeneity. Strateg. Manage. J. 32(6), 569–594 (2011). https://doi.org/10.1002/smj.899

    Article  Google Scholar 

  18. Geelan, D.: Teacher Explanations, pp. 987–999. Springer, Dordrecht (2012). https://doi.org/10.1007/978-1-4020-9041-7_65

  19. Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M.A., Kagal, L.: Explaining explanations: an overview of interpretability of machine learning. In: Bonchi, F., Provost, F.J., Eliassi-Rad, T., Wang, W., Cattuto, C., Ghani, R. (eds.) 5th IEEE International Conference on Data Science and Advanced Analytics, DSAA 2018, Turin, Italy, 1–3 October 2018, pp. 80–89. IEEE (2018). https://doi.org/10.1109/DSAA.2018.00018

  20. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. 51(5), 93:1-93:42 (2019). https://doi.org/10.1145/3236009

    Article  Google Scholar 

  21. Hacker, P., Passoth, J.: Varieties of AI explanations under the law. From the GDPR to the AIA, and beyond. In: Holzinger, A., Goebel, R., Fong, R., Moon, T., Müller, K., Samek, W. (eds.) xxAI 2020. LNCS, vol. 13200, pp. 343–373. Springer, Cham (2020). https://doi.org/10.1007/978-3-031-04083-2_17

    Chapter  Google Scholar 

  22. Heider, F.: The Psychology of Interpersonal Relations. Psychology Press, London (1982)

    Google Scholar 

  23. Hempel, C.G., Oppenheim, P.: Studies in the logic of explanation. Philos. Sci. 15(2), 135–175 (1948). https://doi.org/10.1086/286983

    Article  Google Scholar 

  24. Hilton, D.J.: Mental models and causal explanation: judgements of probable cause and explanatory relevance. Think. Reason. 2(4), 273–308 (1996). https://doi.org/10.1080/135467896394447

    Article  Google Scholar 

  25. Hilton, D.J., Slugoski, B.R.: Knowledge-based causal attribution: the abnormal conditions focus model. Psychol. Rev. 93(1), 75 (1986)

    Article  Google Scholar 

  26. Hoffman, R.R., Mueller, S.T., Klein, G., Litman, J.: Metrics for explainable AI: challenges and prospects. CoRR abs/1812.04608 (2018). http://arxiv.org/abs/1812.04608

  27. Holland, J., Holyoak, K., Nisbett, R., Thagard, P.: Induction: Processes of Inference, Learning, and Discovery. Bradford books, MIT Press (1986). https://books.google.it/books?id=Z6EFBaLApE8C

  28. Holzinger, A., Carrington, A.M., Müller, H.: Measuring the quality of explanations: the system causability scale (SCS). Künstliche Intell. 34(2), 193–198 (2020). https://doi.org/10.1007/s13218-020-00636-z

    Article  Google Scholar 

  29. Horne, Z., Muradoglu, M., Cimpian, A.: Explanation as a cognitive process. Trends Cogn. Sci. 23(3), 187–199 (2019). https://doi.org/10.1016/j.tics.2018.12.004, https://www.sciencedirect.com/science/article/pii/S1364661318302857

  30. Jansen, P., Balasubramanian, N., Surdeanu, M., Clark, P.: What’s in an explanation? characterizing knowledge and inference requirements for elementary science exams. In: Calzolari, N., Matsumoto, Y., Prasad, R. (eds.) COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, 11–16 December 2016, Osaka, Japan, pp. 2956–2965. ACL (2016). https://aclanthology.org/C16-1278/

  31. Lakkaraju, H., Kamar, E., Caruana, R., Leskovec, J.: Interpretable & explorable approximations of black box models. CoRR abs/1707.01154 (2017). http://arxiv.org/abs/1707.01154

  32. Liao, Q.V., Gruen, D.M., Miller, S.: Questioning the AI: informing design practices for explainable AI user experiences. In: Bernhaupt, R., et al. (eds.) CHI Conference on Human Factors in Computing Systems, CHI 2020, Honolulu, HI, USA, 25–30 April 2020, pp. 1–15. ACM (2020). https://doi.org/10.1145/3313831.3376590

  33. Liao, Q.V., Varshney, K.R.: Human-centered explainable AI (XAI): from algorithms to user experiences. CoRR abs/2110.10790 (2021). https://arxiv.org/abs/2110.10790

  34. Lim, B.Y., Dey, A.K., Avrahami, D.: Why and why not explanations improve the intelligibility of context-aware intelligent systems. In: Jr., D.R.O., Arthur, R.B., Hinckley, K., Morris, M.R., Hudson, S.E., Greenberg, S. (eds.) Proceedings of the 27th International Conference on Human Factors in Computing Systems, CHI 2009, Boston, MA, USA, 4–9 April 2009, pp. 2119–2128. ACM (2009). https://doi.org/10.1145/1518701.1519023

  35. Lombrozo, T.: The structure and function of explanations. Trends Cogn. Sci. 10(10), 464–470 (2006). https://doi.org/10.1016/j.tics.2006.08.004, https://www.sciencedirect.com/science/article/pii/S1364661306002117

  36. Lundberg, S.M., et al.: From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2(1), 56–67 (2020). https://doi.org/10.1038/s42256-019-0138-9

    Article  Google Scholar 

  37. Madumal, P., Miller, T., Sonenberg, L., Vetere, F.: A grounded interaction protocol for explainable artificial intelligence. In: Elkind, E., Veloso, M., Agmon, N., Taylor, M.E. (eds.) Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2019, Montreal, QC, Canada, 13–17 May 2019, pp. 1033–1041. International Foundation for Autonomous Agents and Multiagent Systems (2019). http://dl.acm.org/citation.cfm?id=3331801

  38. Mayes, G.R.: Theories of explanation (2001). https://iep.utm.edu/explanat/

  39. Miller, T.: Explanation in artificial intelligence: insights from the social sciences. Artif. Intell. 267, 1–38 (2019). https://doi.org/10.1016/j.artint.2018.07.007

    Article  MathSciNet  MATH  Google Scholar 

  40. Mohseni, S., Block, J.E., Ragan, E.D.: Quantitative evaluation of machine learning explanations: A human-grounded benchmark. In: Hammond, T., Verbert, K., Parra, D., Knijnenburg, B.P., O’Donovan, J., Teale, P. (eds.) 26th International Conference on Intelligent User Interfaces, IUI 2021, College Station, TX, USA, 13–17 April 2021, pp. 22–31. ACM (2021). https://doi.org/10.1145/3397481.3450689

  41. Nguyen, A., Martínez, M.R.: On quantitative aspects of model interpretability. CoRR abs/2007.07584 (2020). https://arxiv.org/abs/2007.07584D

  42. Poursabzi-Sangdeh, F., Goldstein, D.G., Hofman, J.M., Vaughan, J.W., Wallach, H.M.: Manipulating and measuring model interpretability. In: Kitamura, Y., Quigley, A., Isbister, K., Igarashi, T., Bjørn, P., Drucker, S.M. (eds.) CHI Conference on Human Factors in Computing Systems, CHI 2021, Virtual Event / Yokohama, Japan, 8–13 May 2021, pp. 237:1–237:52. ACM (2021). https://doi.org/10.1145/3411764.3445315

  43. Rebanal, J.C., Combitsis, J., Tang, Y., Chen, X.A.: Xalgo: a design probe of explaining algorithms’ internal states via question-answering. In: Hammond, T., Verbert, K., Parra, D., Knijnenburg, B.P., O’Donovan, J., Teale, P. (eds.) 26th International Conference on Intelligent User Interfaces, IUI 2021, College Station, TX, USA, 13–17 April 2021, pp. 329–339. ACM (2021). https://doi.org/10.1145/3397481.3450676

  44. Ribera, M., Lapedriza, À.: Can we do better explanations? A proposal of user-centered explainable AI. In: Trattner, C., Parra, D., Riche, N. (eds.) Joint Proceedings of the ACM IUI 2019 Workshops co-located with the 24th ACM Conference on Intelligent User Interfaces (ACM IUI 2019), Los Angeles, USA, March 20, 2019. CEUR Workshop Proceedings, vol. 2327, p. 38. CEUR-WS.org (2019). http://ceur-ws.org/Vol-2327/IUI19WS-ExSS2019-12.pdf

  45. Rosenfeld, A.: Better metrics for evaluating explainable artificial intelligence. In: Dignum, F., Lomuscio, A., Endriss, U., Nowé, A. (eds.) 20th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2021, Virtual Event, United Kingdom, 3–7 May 2021, pp. 45–50. ACM (2021). https://doi.org/10.5555/3463952.3463962, https://www.ifaamas.org/Proceedings/aamas2021/pdfs/p45.pdf

  46. Salmon, W.: Scientific Explanation and the Causal Structure of the World. Book collections on Project MUSE, Princeton University Press (1984). https://books.google.it/books?id=2ug9DwAAQBAJ

  47. Schweisfurth, M.: Learner-centred education in international perspective: whose pedagogy for whose development? Education, Poverty and International Development, Taylor & Francis (2013). https://books.google.ch/books?id=dT4jLusPp9AC

  48. Searle, J.R.: Austin on locutionary and illocutionary acts. Philos. Rev. 77(4), 405–424 (1968). http://www.jstor.org/stable/2183008

  49. Sellars, W.: Science, Perception and Reality. Humanities Press, New York (1963)

    Google Scholar 

  50. Sovrano, F., Sapienza, S., Palmirani, M., Vitali, F.: A survey on methods and metrics for the assessment of explainability under the proposed AI act. In: Erich, S. (ed.) Legal Knowledge and Information Systems - JURIX 2021: The Thirty-fourth Annual Conference, Vilnius, Lithuania, 8–10 December 2021. Frontiers in Artificial Intelligence and Applications, vol. 346, pp. 235–242. IOS Press (2021). https://doi.org/10.3233/FAIA210342, https://doi.org/10.3233/FAIA210342

  51. Sovrano, F., Sapienza, S., Palmirani, M., Vitali, F.: Metrics, explainability and the European AI act proposal. J 5(1), 126–138 (2022). https://doi.org/10.3390/j5010010

  52. Sovrano, F., Vitali, F.: From philosophy to interfaces: an explanatory method and a tool inspired by Achinstein’s theory of explanation. In: Hammond, T., Verbert, K., Parra, D., Knijnenburg, B.P., O’Donovan, J., Teale, P. (eds.) 26th International Conference on Intelligent User Interfaces, IUI 2021, College Station, TX, USA, 13–17 April 2021, pp. 81–91. ACM (2021). https://doi.org/10.1145/3397481.3450655

  53. Sovrano, F., Vitali, F.: An objective metric for explainable AI: how and why to estimate the degree of explainability. CoRR abs/2109.05327 (2021). https://arxiv.org/abs/2109.05327

  54. Sovrano, F., Vitali, F.: Explanatory artificial intelligence (YAI): human-centered explanations of explainable AI and complex data. Data Min. Knowl. Disc. (2022). https://doi.org/10.1007/s10618-022-00872-x

  55. Sovrano, F., Vitali, F.: How to quantify the degree of explainability: experiments and practical implications. In: 31th IEEE International Conference on Fuzzy Systems, FUZZ-IEEE 2022, Padova, 18–23 July 2022, pp. 1–9. IEEE (2022)

    Google Scholar 

  56. Sovrano, F., Vitali, F., Palmirani, M.: Modelling GDPR-compliant explanations for trustworthy AI. In: Kő, A., Francesconi, E., Kotsis, G., Tjoa, A.M., Khalil, I. (eds.) EGOVIS 2020. LNCS, vol. 12394, pp. 219–233. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58957-8_16

    Chapter  Google Scholar 

  57. Szymanski, M., Millecamp, M., Verbert, K.: Visual, textual or hybrid: the effect of user expertise on different explanations. In: Hammond, T., Verbert, K., Parra, D., Knijnenburg, B.P., O’Donovan, J., Teale, P. (eds.) 26th International Conference on Intelligent User Interfaces, IUI 2021, College Station, TX, USA, 13–17 April 2021, pp. 109–119. ACM (2021). https://doi.org/10.1145/3397481.3450662

  58. Thagard, P.: Analogy, explanation, and education. J. Res. Sci. Teach. 29(6), 537–544 (1992). https://doi.org/10.1002/tea.3660290603, https://onlinelibrary.wiley.com/doi/abs/10.1002/tea.3660290603

  59. Van Fraassen, B.C.: The Scientific Image. Clarendon Library of Logic and Philosophy, Clarendon Press (1980). https://books.google.it/books?id=VLz2F1zMr9QC

  60. Vilone, G., Rizzo, L., Longo, L.: A comparative analysis of rule-based, model-agnostic methods for explainable artificial intelligence. In: Longo, L., Rizzo, L., Hunter, E., Pakrashi, A. (eds.) Proceedings of The 28th Irish Conference on Artificial Intelligence and Cognitive Science, Dublin, Republic of Ireland, 7–8 December 2020. CEUR Workshop Proceedings, vol. 2771, pp. 85–96. CEUR-WS.org (2020). http://ceur-ws.org/Vol-2771/AICS2020_paper_33.pdf

  61. Wachter, S., Mittelstadt, B., Floridi, L.: Why a right to explanation of automated decision-making does not exist in the general data protection regulation. Int. Data Priv. Law 7(2), 76–99 (2017)

    Article  Google Scholar 

  62. Wachter, S., Mittelstadt, B., Russell, C.: Counterfactual explanations without opening the black box: automated decisions and the GDPR. Harvard J. Law Technol. 31(2), 841 (2018). https://doi.org/10.2139/ssrn.3063289

    Article  Google Scholar 

  63. Wang, X., Yin, M.: Are explanations helpful? A comparative study of the effects of explanations in AI-assisted decision-making. In: Hammond, T., Verbert, K., Parra, D., Knijnenburg, B.P., O’Donovan, J., Teale, P. (eds.) 26th International Conference on Intelligent User Interfaces, IUI 2021, College Station, TX, USA, 13–17 April 2021, pp. 318–328. ACM (2021). https://doi.org/10.1145/3397481.3450650

  64. Warren, G., Keane, M.T., Byrne, R.M.J.: Features of explainability: How users understand counterfactual and causal explanations for categorical and continuous features in XAI. In: Heyninck, J., Meyer, T., Ragni, M., Thimm, M., Kern-Isberner, G. (eds.) Proceedings of the Workshop on Cognitive spects of Knowledge Representation Co-located with the 31st International Join Conference on Artificial Intelligence (IJCAI-ECAI 2022), Vienna, Austria, 23 July 2022. CEUR Workshop Proceedings, vol. 3251. CEUR-WS.org (2022). https://ceur-ws.org/Vol-3251/paper1.pdf

  65. Williams, J.J., Lombrozo, T., Rehder, B.: The hazards of explanation: overgeneralization in the face of exceptions. J. Exp. Psychol.: General 142(4), 1006 (2013)

    Article  Google Scholar 

  66. Zhou, J., Gandomi, A.H., Chen, F., Holzinger, A.: Evaluating the quality of machine learning explanations: a survey on methods and metrics. Electronics 10(5) (2021). https://doi.org/10.3390/electronics10050593, https://www.mdpi.com/2079-9292/10/5/593

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Bologna, Bologna, Italy

    Francesco Sovrano & Fabio Vitali

  2. Department of Informatics, University of Zurich, Zurich, Switzerland

    Francesco Sovrano

Authors
  1. Francesco Sovrano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fabio Vitali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francesco Sovrano .

Editor information

Editors and Affiliations

  1. Technological University Dublin, Dublin, Ireland

    Luca Longo

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sovrano, F., Vitali, F. (2023). Perlocution vs Illocution: How Different Interpretations of the Act of Explaining Impact on the Evaluation of Explanations and XAI. In: Longo, L. (eds) Explainable Artificial Intelligence. xAI 2023. Communications in Computer and Information Science, vol 1901. Springer, Cham. https://doi.org/10.1007/978-3-031-44064-9_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-44064-9_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-44063-2

  • Online ISBN: 978-3-031-44064-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation