Skip to main content

Mental Health Chatbots, Moral Bio-Enhancement, and the Paradox of Weak Moral AI

  • 240 Accesses

Part of the Advances in Neuroethics book series (AIN)


From a philosophical perspective, the idea of a weak moral artificial intelligence (AI), which aims at info-enhancement alone without affecting moral motivation, seems paradoxical. By being safe and avoiding the problem of undermining human agency and the infringement of privacy, a model of weak moral AI as a Socratic assistant, if technically feasible, seems to fall short of delivering a moral act and diminishes what morality means for humans. However, the recent use of chatbots for mental health and therapy implies that the argument against weak moral AI based on motivation can be extremely theoretical, leading to a hasty conclusion. The very nature of moral info-enhancement also shows its therapeutic potential in the brain and mental health. This reminds us that technological innovation and empirical findings might enlighten us about a new paradigm that cannot be wholly foreseen or captured by a philosophical argument that is stripped of context.


  • Weak moral AI
  • Info-enhancement
  • Motivation
  • Chatbots
  • Moral therapy

This chapter revises and updates portions of a Chinese paper by Jie Yin. (尹洁, “弱道德人工智能可行吗——从精神医学用途到道德增强?”, 医学与哲学, July 2020, Vol. 41. No. 13: 1–7).

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-74188-4_10
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   119.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-74188-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book
USD   159.99
Price excludes VAT (USA)


  1. O’Brolcháin F, Gordijn B. Ethics of brain–computer interfaces for enhancement purposes. In: Clausen J, Levy N, editors. Handbook of neuroethics. Dordrecht: Springer; 2015. p. 1207–26.

    Google Scholar 

  2. Harris J. Moral enhancement and freedom. Bioethics. 2011;25(2):102–11.

    CrossRef  Google Scholar 

  3. Huang PH. Moral enhancement, self-governance, and resistance. J Med Philos. 2018;43(5):547–67.

    CrossRef  PubMed  Google Scholar 

  4. Wiseman H. The myth of the moral brain: the limits of moral enhancement. Cambridge: The MIT Press; 2016.

    CrossRef  Google Scholar 

  5. Savulescu J, Maslen H. Moral enhancement and artificial intelligence: moral AI? In: Romportl J, Zackova E, Kelemen J, editors. Beyond artificial intelligence. The disappearing human-machine divide. Dordrecht: Springer; 2015. p. 79–95.

    Google Scholar 

  6. Persson I, Savulescu J. Moral hard-wiring and moral enhancement. Bioethics. 2017;31(4):286–95.

    CrossRef  PubMed  PubMed Central  Google Scholar 

  7. Lara F, Deckers J. Artificial intelligence as a socratic assistant for moral enhancement. Neuroethics. 2020;13:275–87.

    CrossRef  Google Scholar 

  8. Rosati CS. Moral motivation. In: Zalta EN, editor. The Stanford Encyclopedia of Philosophy. Stanford, CA: Stanford University; 2016. Accessed 30 Aug 2019.

    Google Scholar 

  9. Grenne JD, et al. The neural bases of cognitive conflict and control in moral judgment. Neuron. 2004;44:389–400.

    CrossRef  Google Scholar 

  10. Glannon W. Brain, body and mind: neuroethics with a human face. New York: Oxford University Press; 2011.

    CrossRef  Google Scholar 

  11. Coakley M. Motivation ethics. London: Bloomsbury; 2017.

    CrossRef  Google Scholar 

  12. Stonestreet EL. Notre dame philosophical reviews. 2017. Accessed 30 Aug 2019.

  13. Kant I. Practical philosophy (trans. Gregor M). New York: Cambridge University Press; 1999.

    Google Scholar 

  14. De Jesus A. Chatbots for mental health and therapy—comparing 5 current apps and use cases. 2019. Accessed 28 May 2020.

  15. Abd-alrazaq AA, et al. An overview of the features of chatbots in mental health: a scoping review. Int J Med Inform. 2019;132:103978.

    CrossRef  PubMed  Google Scholar 

  16. Vaidyam AN, et al. Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can J Psychiatry. 2019;64(7):456–64.

    CrossRef  Google Scholar 

  17. Borenstein J, Arkin R. Robotic nudges: the ethics of engineering a more socially just human being. Sci Eng Ethics. 2016;22:31–46.

    CrossRef  PubMed  Google Scholar 

  18. Casal P. On not taking men as they are: reflections on moral bioenhancement. J Med Ethics. 2015;41(4):340–2.

    CrossRef  Google Scholar 

  19. Carter S. Could moral enhancement interventions be medically indicated? Health Care Anal. 2017;25:338–53.

    CrossRef  Google Scholar 

  20. Persson I, Savulescu J. Unfit for the future: the need for moral enhancement. Oxford: Oxford University Press; 2012.

    CrossRef  Google Scholar 

  21. Daniels N. Normal functioning and the treatment-enhancement distinction. Camb Q Healthc Ethics. 2000;9(3):309–22.

    CAS  CrossRef  Google Scholar 

  22. Conrad P. Medicalisation and social control. Annu Rev Sociol. 1992;18:209–32.

    CrossRef  Google Scholar 

Download references


This work was supported by NSFC (National Natural Science Foundation of China). Grant Number 71843009.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Jie Yin .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Yin, J. (2021). Mental Health Chatbots, Moral Bio-Enhancement, and the Paradox of Weak Moral AI . In: Jotterand, F., Ienca, M. (eds) Artificial Intelligence in Brain and Mental Health: Philosophical, Ethical & Policy Issues. Advances in Neuroethics. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-74187-7

  • Online ISBN: 978-3-030-74188-4

  • eBook Packages: MedicineMedicine (R0)