Skip to main content
SpringerLink
Log in

Augmented Utilitarianism for AGI Safety

  • Conference paper
  • First Online:
Artificial General Intelligence (AGI 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11654))

Included in the following conference series:

Abstract

In the light of ongoing progresses of research on artificial intelligent systems exhibiting a steadily increasing problem-solving ability, the identification of practicable solutions to the value alignment problem in AGI Safety is becoming a matter of urgency. In this context, one preeminent challenge that has been addressed by multiple researchers is the adequate formulation of utility functions or equivalents reliably capturing human ethical conceptions. However, the specification of suitable utility functions harbors the risk of “perverse instantiation” for which no final consensus on responsible proactive countermeasures has been achieved so far. Amidst this background, we propose a novel non-normative socio-technological ethical framework denoted Augmented Utilitarianism which directly alleviates the perverse instantiation problem. We elaborate on how augmented by AI and more generally science and technology, it might allow a society to craft and update ethical utility functions while jointly undergoing a dynamical ethical enhancement. Further, we elucidate the need to consider embodied simulations in the design of utility functions for AGIs aligned with human values. Finally, we discuss future prospects regarding the usage of the presented scientifically grounded ethical framework and mention possible challenges.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Importantly, this also applies to non-consequentialist frameworks such as deontological ethics [21].

  2. 2.

    AU is not be to confused with agent-relative consequentialism which is a normative agent-based framework, does not foresee a grounding in science and seems to assume a “pretheoretical grasp” [37] of its “better-than-relative-to” relation.

References

  1. Aliman, N.-M., Kester, L.: Hybrid strategies towards safe self-aware superintelligent systems. In: Iklé, M., Franz, A., Rzepka, R., Goertzel, B. (eds.) AGI 2018. LNCS (LNAI), vol. 10999, pp. 1–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-97676-1_1

    Chapter  Google Scholar 

  2. Aliman, N.M., Kester, L.: Transformative AI governance and AI-empowered ethical enhancement through preemptive simulations. Delphi Interdisc. Rev. Emerg. Technol. 2(1), 23–29 (2019)

    Article  Google Scholar 

  3. Arrhenius, G.: An impossibility theorem for welfarist axiologies. Econ. Philos. 16(2), 247–266 (2000)

    Article  Google Scholar 

  4. Awad, E., et al.: The moral machine experiment. Nature 563(7729), 59 (2018)

    Article  Google Scholar 

  5. Barrett, L.F.: The theory of constructed emotion: an active inference account of interoception and categorization. Soc. Cogn. Affect. Neurosci. 12(1), 1–23 (2017)

    Article  Google Scholar 

  6. Baucells, M., Bellezza, S.: Temporal profiles of instant utility during anticipation, event, and recall. Manag. Sci. 63(3), 729–748 (2016)

    Article  Google Scholar 

  7. Bentham, J.: An Introduction to the Principles of Morals and Legislation. Dover Publications, Mineola (1780)

    Google Scholar 

  8. Berridge, K.C., O’Doherty, J.P.: From experienced utility to decision utility. In: Neuroeconomics, pp. 335–351. Elsevier (2014)

    Google Scholar 

  9. Bogosian, K.: Implementation of moral uncertainty in intelligent machines. Mind. Mach. 27(4), 591–608 (2017)

    Article  Google Scholar 

  10. Bostrom, N.: Superintelligence: Paths, Dangers, Strategies, 1st edn. Oxford University Press Inc., New York (2014)

    Google Scholar 

  11. Busseri, M.A., Sadava, S.W.: A review of the tripartite structure of subjective well-being: implications for conceptualization, operationalization, analysis, and synthesis. Pers. Soc. Psychol. Rev. 15(3), 290–314 (2011)

    Article  Google Scholar 

  12. Calvo, R.A., Peters, D.: Positive Computing: Technology for Wellbeing and Human Potential. MIT Press, Cambridge (2014)

    Book  Google Scholar 

  13. Diener, E.: Subjective well-being: the science of happiness and a proposal for a national index. Am. Psychol. 55(1), 34 (2000)

    Article  MathSciNet  Google Scholar 

  14. Diener, E., Biswas-Diener, R.: Happiness: Unlocking the Mysteries of Psychological Wealth. Wiley, New York (2011)

    Google Scholar 

  15. Eckersley, P.: Impossibility and uncertainty theorems in AI value alignment (or why your AGI should not have a utility function). CoRR abs/1901.00064 (2018)

    Google Scholar 

  16. Everitt, T.: Towards safe artificial general intelligence. Ph.D. thesis, Australian National University (2018)

    Google Scholar 

  17. Everitt, T., Lea, G., Hutter, M.: AGI safety literature review. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, pp. 5441–5449. International Joint Conferences on Artificial Intelligence Organization, July 2018. https://doi.org/10.24963/ijcai.2018/768

  18. Frey, B.S., Stutzer, A.: Beyond Bentham-measuring procedural utility (2001)

    Google Scholar 

  19. Gilbert, D.T., Wilson, T.D.: Prospection: experiencing the future. Science 317(5843), 1351–1354 (2007)

    Article  Google Scholar 

  20. Goertzel, B.: Superintelligence: fears, promises and potentials. J. Evol. Technol. 24(2), 55–87 (2015)

    Google Scholar 

  21. Greaves, H.: Population axiology. Philos. Compass 12(11), e12442 (2017)

    Article  Google Scholar 

  22. Johnson, M.: Moral Imagination: Implications of Cognitive Science for Ethics. University of Chicago Press, Chicago (1994)

    Google Scholar 

  23. Kahneman, D., Diener, E., Schwarz, N.: Well-Being: Foundations of Hedonic Psychology. Russell Sage Foundation, New York (1999)

    Google Scholar 

  24. Kahneman, D., Wakker, P.P., Sarin, R.: Back to Bentham? explorations of experienced utility. Q. J. Econ. 112(2), 375–406 (1997)

    Article  Google Scholar 

  25. Kaminitz, S.C.: Contemporary procedural utility and Hume’s early idea of utility. J. Happiness Stud. 20, 1–14 (2019)

    Article  Google Scholar 

  26. Kaufman, S.B.: Self-actualizing people in the 21st century: integration with contemporary theory and research on personality and well-being. J. Humanist. Psychol. 0022167818809187 (2018). https://doi.org/10.1177/0022167818809187

  27. Koltko-Rivera, M.E.: Rediscovering the later version of Maslow’s hierarchy of needs: self-transcendence and opportunities for theory, research, and unification. Rev. Gen. Psychol. 10(4), 302–317 (2006)

    Article  Google Scholar 

  28. van Loon, A., Bailenson, J., Zaki, J., Bostick, J., Willer, R.: Virtual reality perspective-taking increases cognitive empathy for specific others. PloS ONE 13(8), e0202442 (2018)

    Article  Google Scholar 

  29. Lyubomirsky, S.: Why are some people happier than others? The role of cognitive and motivational processes in well-being. Am. Psychol. 56(3), 239 (2001)

    Article  Google Scholar 

  30. Maslow, A.H.: The Farther Reaches of Human Nature. Viking Press, New York (1971)

    Google Scholar 

  31. Meuhlhauser, L., Helm, L.: Intelligence explosion and machine ethics. In: Singularity Hypotheses: A Scientific and Philosophical Assessment, pp. 101–126 (2012)

    Google Scholar 

  32. Mossbridge, J., et al.: Emotionally-sensitive AI-driven android interactions improve social welfare through helping people access self-transcendent states. In: AI for Social Good Workshop at Neural Information Processing Systems 2018 Conference (2018)

    Google Scholar 

  33. Oosterwijk, S., Lindquist, K.A., Anderson, E., Dautoff, R., Moriguchi, Y., Barrett, L.F.: States of mind: emotions, body feelings, and thoughts share distributed neural networks. NeuroImage 62(3), 2110–2128 (2012)

    Article  Google Scholar 

  34. Parfit, D.: Reasons and Persons. Oxford University Press, Oxford (1984)

    Google Scholar 

  35. Peterson, C.: A Primer in Positive Psychology. Oxford University Press, Oxford (2006)

    Google Scholar 

  36. Rafal, R., Kenji, A.: Toward artificial ethical learners that could also teach you how to be a moral man. In: IJCAI 2015 Workshop on Cognitive Knowledge Acquisition and Applications (Cognitum 2015). IJCAI (2015)

    Google Scholar 

  37. Schroeder, M.: Teleology, agent-relative value, and ‘good’. Ethics 117(2), 265–295 (2007)

    Article  Google Scholar 

  38. Seligman, M.E.: Flourish: A Visionary New Understanding of Happiness and Well-Being. Simon and Schuster, New York (2012)

    Google Scholar 

  39. Seligman, M.E.P., Csikszentmihalyi, M.: positive psychology: an introduction. In: Csikszentmihalyi, M. (ed.) Flow and the Foundations of Positive Psychology, pp. 279–298. Springer, Dordrecht (2014). https://doi.org/10.1007/978-94-017-9088-8_18

    Chapter  Google Scholar 

  40. Werkhoven, P., Kester, L., Neerincx, M.: Telling autonomous systems what to do. In: Proceedings of the 36th European Conference on Cognitive Ergonomics, p. 2. ACM (2018)

    Google Scholar 

  41. Yampolskiy, R.V.: Utility function security in artificially intelligent agents. J. Exp. Theor. Artif. Intell. 26(3), 373–389 (2014)

    Article  Google Scholar 

  42. Yampolskiy, R.V.: Artificial Superintelligence: A Futuristic Approach. Chapman and Hall/CRC, Boca Raton (2015)

    Book  Google Scholar 

  43. Yampolskiy, R.V.: Personal universes: a solution to the multi-agent value alignment problem. arXiv preprint arXiv:1901.01851 (2019)

  44. Ziesche, S.: Potential synergies between the united nations sustainable development goals and the value loading problem in artificial intelligence. Maldives Nat. J. Res. 6, 47 (2018)

    Google Scholar 

Download references

Acknowledgements

We would like to thank Peter Werkhoven for a helpful discussion of our approach.

Author information

Authors and Affiliations

  1. Utrecht University, Utrecht, The Netherlands

    Nadisha-Marie Aliman

  2. TNO Netherlands, The Hague, The Netherlands

    Leon Kester

Authors
  1. Nadisha-Marie Aliman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leon Kester
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nadisha-Marie Aliman .

Editor information

Editors and Affiliations

  1. Temple University, Philadelphia, PA, USA

    Patrick Hammer

  2. University of Memphis, Memphis, TN, USA

    Pulin Agrawal

  3. SingularityNET, Tai Po, Hong Kong, China

    Ben Goertzel

  4. SingularityNET, Alamosa, CO, USA

    Matthew Iklé

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Aliman, NM., Kester, L. (2019). Augmented Utilitarianism for AGI Safety. In: Hammer, P., Agrawal, P., Goertzel, B., Iklé, M. (eds) Artificial General Intelligence. AGI 2019. Lecture Notes in Computer Science(), vol 11654. Springer, Cham. https://doi.org/10.1007/978-3-030-27005-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-27005-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-27004-9

  • Online ISBN: 978-3-030-27005-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation