Complex Value Systems in Friendly AI
A common reaction to first encountering the problem statement of Friendly AI (”Ensure that the creation of a generally intelligent, self-improving, eventually superintelligent system realizes a positive outcome”) is to propose a simple design which allegedly suffices; or to reject the problem by replying that ”constraining” our creations is undesirable or unnecessary. This paper briefly presents some of the reasoning which suggests that Friendly AI is solvable, but not simply or trivially so, and that a wise strategy would be to invoke detailed learning of and inheritance from human values as a basis for further normalization and reflection.
KeywordsFriendly AI machine ethics anthropomorphism
Unable to display preview. Download preview PDF.
- 1.Rinkworks: ComputerStupidities:Programming, http://www.rinkworks.com/stupid/cs_programming.shtml
- 2.Kurzweil, R.: The Singularity is Near: When Humans Transcend Biology. Viking, New York (2005)Google Scholar
- 3.Omohundro, S.: The basic AI drives. In: Wang, P., Goertzel, B., Franklin, S. (eds.) Proceedings of the First AGI Conference, pp. 483–492. IOS Press, Amsterdam (2008)Google Scholar
- 4.Schmidhuber, J.: Gödel machines: Fully Self-Referential Optimal Universal Self-Improvers. In: Goertzel, B., Pennachin, C. (eds.) Artificial General Intelligence, pp. 119–226. Springer, Heidelberg (2006)Google Scholar
- 5.Hibbard, B.: Super-intelligent machines. ACM SIGGRAPH Computer Graphics 35(1) (2001)Google Scholar
- 6.Hibbard, B.: Message to the SL4 email list, archived at (2004), http://yudkowsky.net/singularity/AIRisk_Hibbard.html
- 7.McDermott, D.: Artificial intelligence meets natural stupidity. SIGART Newsletter 57, 4–9 (1976)Google Scholar
- 8.Frankena, W.: Ethics, 2nd edn. Prentice Hall, Englewood Cliffs (1973)Google Scholar
- 9.Tarleton, N.: Coherent extrapolated volition: A meta-level approach to machine ethics, http://singinst.org/upload/coherent-extrapolated-volition.pdf