Skip to main content

Towards the Design of Ethically-Aware Pedagogical Conversational Agents

Part of the Lecture Notes in Networks and Systems book series (LNNS,volume 158)


Pedagogical conversational agents (PCA) aimed at offering personalized support could greatly enhance the student experience by adapting their support to the student’s learning needs and habits. But, as they become more autonomous, there is a greater need to consider the potential moral consequences of their choices and to take into consideration an holistic view of students. In face-to-face setting, the individual and holistic understanding that human teachers can have about their students and classrooms is key in ensuring that the teachers’ decisions strive for the best learning experience. However, furnishing PCAs with such a holistic understanding of complex dimensions that often involve relations between multiple elements poses a challenge. Furthermore, ethical aspects should also be considered in order to avoid decisions that can lead to discriminatory and unfair collateral results and end up favouring certain students over others; therefore, the PCA should be aware of how decisions aimed at satisfying certain needs could negatively affect other dimensions. The current position paper states the need to furnish PCAs with ethical awareness and discusses the nature of ethical dilemmas in the context of PCAs in online learning environments. Through this discussion, the paper proposes some first steps towards the design of ethically-aware PCAs.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   259.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. 1.

    We understand an ethical dilemma as a scenario involving a decision in which each possible choice results in both beneficial and detrimental consequences. Namely, in ethical dilemmas there are no solutions that are only (or clearly more) beneficial for the actors involved; instead, they always involve a choice between options that have a certain amount of “bad” consequences.


  1. Dede, C., Richards, J., Saxberg, B.: Learning Engineering for Online Education: Theoretical Contexts and Design-based Examples. Routledge, New York (2018)

    CrossRef  Google Scholar 

  2. Sullivan, C.A.: Vill. L. Rev. 63, 395 (2018)

    Google Scholar 

  3. King, A.G., Mrkonich, M.J.: Okla. L. Rev. 68, 555 (2015)

    Google Scholar 

  4. Gumbus, A., Grodzinsky, F.: ACM SIGCAS Comput. Soc. 45(3), 118 (2016)

    CrossRef  Google Scholar 

  5. Yapo, A., Weiss, J.: Proceedings of the 51st Hawaii International Conference on System Sciences (2018)

    Google Scholar 

  6. Favaretto, M., De Clercq, E., Elger, B.S.: J. Big Data 6(1), 12 (2019)

    CrossRef  Google Scholar 

  7. Angwin, J., Larson, J., Mattu, S., Kirchner, L., ProPublica, 23 May 2016 (2016)

    Google Scholar 

  8. Taylor, L.: Big Data Soc. 4(2), 2053951717736335 (2017)

    CrossRef  MathSciNet  Google Scholar 

  9. Veale, M., Binns, R.: Big Data Soc. 4(2), 2053951717743530 (2017)

    CrossRef  Google Scholar 

  10. Gunkel, D.J.: The Machine Question: Critical Perspectives on AI, Robots, and Ethics. MIT Press, Cambridge (2012)

    CrossRef  Google Scholar 

  11. Floridi, L., Sanders, J.W.: Minds Mach. 14(3), 349 (2004)

    CrossRef  Google Scholar 

  12. Sullins, J.P.: Ethics Inf. Technol. 7, 139 (2005)

    CrossRef  Google Scholar 

  13. Wallach, W., Franklin, S., Allen, C.: Topics Cogn. Sci. 2(3), 454 (2010)

    CrossRef  Google Scholar 

  14. Muntean, I., Howard, D.: In: Seibt, J., Hakli, R., Norskov, M. (eds.) Sociable Robots and the Future of Social Relations: Proceedings of Robo-Philosophy 2014, pp. 217–230. IOS Press (2014)

    Google Scholar 

  15. Anderson, M., Anderson, S.L., Armen, C.: IEEE Intell. Syst. 21, 56 (2006)

    CrossRef  Google Scholar 

  16. Wallach, W., Allen, C., Smit, I.: AI Soc. 22(4), 565 (2008)

    CrossRef  Google Scholar 

  17. Cointe, N., Bonnet, G., Boissier, O.: In: AAMAS, pp. 1106–1114 (2016)

    Google Scholar 

  18. Casas-Roma, J., Nelson, M.J., Arnedo-Moreno, J., Gaudl, S.E., Saunders, R.: In: Proceedings of the 11th International Conference on Agents and Artificial Intelligence, AIIDE 2012, vol. 1, pp. 244–251 (2019)

    Google Scholar 

  19. Casas-Roma, J., Arnedo-Moreno, J.: In: Proceedings of the Digital Games Research Association Conference, DiGRA 2019 (2019)

    Google Scholar 

  20. Casas-Roma, J., Arnedo-Moreno, J.: Categorizing morality systems through the lens of fallout. In: Proceedings of the Catalan Conference of Artificial Intelligence (CCIA), pp. 19–28 (2019)

    Google Scholar 

  21. Fitzpatrick, R., Walsh, M., Nitsche, M.: In: Aesthetics of Play, Bergen, Norway, 14–15 October 2005 (2005)

    Google Scholar 

  22. Russell, A.: In: AI Game Programming Wisdom, vol. 3. Charles River Media, Rockland (2006)

    Google Scholar 

  23. Heron, M., Belford, P.: Comput. Games J. 3(1), 34 (2014)

    CrossRef  Google Scholar 

Download references


This work has been partially supported by European Commission through the project “colMOOC: Integrating Conversational Agents and Learning Analytics in MOOCs” (588438-EPP-1-2017-1-EL-EPPKA2-KA) and by a UOC postdoctoral stay.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Joan Casas-Roma .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Casas-Roma, J., Conesa, J. (2021). Towards the Design of Ethically-Aware Pedagogical Conversational Agents. In: Barolli, L., Takizawa, M., Yoshihisa, T., Amato, F., Ikeda, M. (eds) Advances on P2P, Parallel, Grid, Cloud and Internet Computing. 3PGCIC 2020. Lecture Notes in Networks and Systems, vol 158. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-61104-0

  • Online ISBN: 978-3-030-61105-7

  • eBook Packages: EngineeringEngineering (R0)