Abstract
Artificial virtuous agents are artificial intelligence agents capable of virtuous behavior. Virtues are defined as an excellence in moral character, for example, compassion, honesty, etc. Developing virtues in AI comes under the umbrella of machine ethics research, which aims to embed ethical theories into artificial intelligence systems. We have recently suggested the use of affinity-based reinforcement learning to impart virtuous behavior. Such a technique uses policy regularization on reinforcement learning algorithms, and it has advantages such as interpretability and convergence properties. Hence, we evaluate the efficacy of affinity-based reinforcement learning to design artificial virtuous agents using a stochastic role-playing game environment. Our results show that virtuous behavior can indeed result in our Papers, Please environment, and that algorithmic convergence can be controlled by the relevant hyperparameters. We then discuss some insights from our empirical evaluation of this method and motivate future research directions.
Supported by University of Agder, Norway.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Achiam J, Held D, Tamar A, Abbeel P (2017) Constrained policy optimization. In: International conference on machine learning. PMLR, pp 22–31
Berberich N, Diepold K (2018) The virtuous machine - old ethics for new technology? arXiv: 1806.10322
Crawford J, Cowling M, Allen KA (2023) Leadership is needed for ethical ChatGPT: character, assessment, and learning using artificial intelligence (AI). J Univ Teach & Learn Pract 20(3):02
Formosa P, Ryan M, Staines D (2016) Papers, please and the systemic approach to engaging ethical expertise in videogames. Ethics Inf Technol 18(3):211–225. https://doi.org/10.1007/s10676-016-9407-z. https://link.springer.com/10.1007/s10676-016-9407-z
Garcıa J, Fernández F (2015) A comprehensive survey on safe reinforcement learning. J Mach Learn Res 16(1):1437–1480
Govindarajulu NS, Bringsjord S, Ghosh R, Sarathy V (2019) Toward the engineering of virtuous machines. In: Proceedings of the 2019 AAAI/ACM conference on AI, ethics, and society. ACM, Honolulu, HI, USA, pp 29–35. https://doi.org/10.1145/3306618.3314256. https://dl.acm.org/doi/10.1145/3306618.3314256
Maree C, Omlin CW (2022) Can interpretable reinforcement learning manage prosperity your way? AI 3(2):526–537
Moor J (2006) The nature, importance, and difficulty of machine ethics. IEEE Intell Syst 21(4):18–21. https://doi.org/10.1109/MIS.2006.80. https://ieeexplore.ieee.org/document/1667948/
Nay JL, Zagal JP (2017) Meaning without consequence: virtue ethics and inconsequential choices in games. In: Proceedings of the 12th international conference on the foundations of digital games. ACM, Hyannis, Massachusetts, pp 1–8. https://doi.org/10.1145/3102071.3102073. https://dl.acm.org/doi/10.1145/3102071.3102073
Ng AY, Harada D, Russell S (1999) Policy invariance under reward transformations: theory and application to reward shaping. In: ICML, vol 99. Citeseer, pp 278–287
OpenAI. ChatGPT. https://chat.openai.com
Persiani M, Hellström T (2022) Policy regularization for legible behavior. Neural Comput Appl :1–10
Pope L (2013) Papers, please. https://papersplea.se/
Rodriguez-Soto M, Serramia M, Lopez-Sanchez M, Rodriguez-Aguilar JA (2022) Instilling moral value alignment by means of multi-objective reinforcement learning. Ethics Inf Technol 24(1):9. https://doi.org/10.1007/s10676-022-09635-0. https://link.springer.com/10.1007/s10676-022-09635-0
Ross WD, Brown L (eds) (1980) Oxford world’s classics: Aristotle: the nicomachean ethics (revised edition). Oxford World’s Classics. https://doi.org/10.1093/actrade/9780199213610.book.1. http://www.oxfordscholarlyeditions.com/view/10.1093/actrade/9780199213610.book.1/actrade-9780199213610-book-1
Stenseke J (2021) Artificial virtuous agents: from theory to machine implementation. AI & Society. https://doi.org/10.1007/s00146-021-01325-7. https://link.springer.com/10.1007/s00146-021-01325-7
Stenseke J (2022) Artificial virtuous agents in a multi-agent tragedy of the commons. AI & Society. https://doi.org/10.1007/s00146-022-01569-x
Sutton RS, Barto AG (2018) Reinforcement learning: an introduction, 2nd edn. The MIT Press
Tirumala D, Galashov A, Noh H, Hasenclever L, Pascanu R, Schwarz J, Desjardins G, Czarnecki WM, Ahuja A, Teh YW et al (2022) Behavior priors for efficient reinforcement learning. J Mach Learn Res 23(1):9989–10056
Tolmeijer S, Kneer M, Sarasua C, Christen M, Bernstein A (2021) Implementations in machine ethics: a survey. ACM Comput Surv 53(6):1–38. https://doi.org/10.1145/3419633. https://dl.acm.org/doi/10.1145/3419633
Van Dis EAM, Bollen J, Zuidema W, Van Rooij R, Bockting CL (2023) ChatGPT: five priorities for research. Nature 614(7947):224–226. https://doi.org/10.1038/d41586-023-00288-7.www.nature.com/articles/d41586-023-00288-7
Vishwanath A, Bøhn ED, Granmo OC, Maree C, Omlin C (2022) Towards artificial virtuous agents: games, dilemmas and machine learning. AI and Ethics. https://doi.org/10.1007/s43681-022-00251-8. https://link.springer.com/10.1007/s43681-022-00251-8
Wallach W, Allen C (2010) Moral machines: teaching robots right from wrong. NY, first issued as an oxford University Press paperback edn. Oxford University Press, New York
Wirth C, Akrour R, Neumann G, Fürnkranz J et al (2017) A survey of preference-based reinforcement learning methods. J Mach Learn Res 18(136):1–46
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Vishwanath, A., Omlin, C. (2024). Exploring Affinity-Based Reinforcement Learning for Designing Artificial Virtuous Agents in Stochastic Environments. In: Farmanbar, M., Tzamtzi, M., Verma, A.K., Chakravorty, A. (eds) Frontiers of Artificial Intelligence, Ethics, and Multidisciplinary Applications. FAIEMA 2023. Frontiers of Artificial Intelligence, Ethics and Multidisciplinary Applications. Springer, Singapore. https://doi.org/10.1007/978-981-99-9836-4_3
Download citation
DOI: https://doi.org/10.1007/978-981-99-9836-4_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9835-7
Online ISBN: 978-981-99-9836-4
eBook Packages: Business and ManagementBusiness and Management (R0)