Skip to main content

Evolutionary Computation and AI Safety

Research Problems Impeding Routine and Safe Real-World Application of Evolution

  • 477 Accesses

Part of the Genetic and Evolutionary Computation book series (GEVO)

Abstract

Recent developments in artificial intelligence and machine learning have spurred interest in the growing field of AI safety, which studies how to prevent human-harming accidents when deploying AI systems. This paper thus explores the intersection of AI safety with evolutionary computation, to show how safety issues arise in evolutionary computation and how understanding from evolutionary computational and biological evolution can inform the broader study of AI safety.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-39958-0_10
  • Chapter length: 20 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   139.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-39958-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   179.99
Price excludes VAT (USA)
Hardcover Book
USD   179.99
Price excludes VAT (USA)
Fig. 10.1

References

  1. Google photos labeled black people ‘gorillas’. https://www.usatoday.com/story/tech/2015/07/01/google-apologizes-after-photos-identify-black-people-as-gorillas/29567465/. Accessed: 2019-05-01

  2. Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete problems in AI safety. arXiv preprint arXiv:1606.06565 (2016)

    Google Scholar 

  3. Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robotics and Autonomous Systems 57(5), 469–483 (2009)

    CrossRef  Google Scholar 

  4. Armstrong, S., Levinstein, B.: Low impact artificial intelligences. arXiv preprint arXiv:1705.10720 (2017)

    Google Scholar 

  5. Banzhaf, W., Nordin, P., Keller, R.E., Francone, F.D.: Genetic Programming — An Introduction. Morgan Kaufmann, San Francisco (1998)

    MATH  CrossRef  Google Scholar 

  6. Branke, J., Schmeck, H.: Designing evolutionary algorithms for dynamic optimization problems. In: Advances in Evolutionary Computing, pp. 239–262. Springer (2003)

    Google Scholar 

  7. Buss, D.: Evolutionary psychology: The new science of the mind. Psychology Press (2015)

    Google Scholar 

  8. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: A survey. ACM computing Surveys (CSUR) 41(3), 15 (2009)

    Google Scholar 

  9. Christiano, P., Shlegeris, B., Amodei, D.: Supervising strong learners by amplifying weak experts. arXiv preprint arXiv:1810.08575 (2018)

    Google Scholar 

  10. Cobbe, K., Klimov, O., Hesse, C., Kim, T., Schulman, J.: Quantifying generalization in reinforcement learning. arXiv preprint arXiv:1812.02341 (2018)

    Google Scholar 

  11. Everitt, T., Lea, G., Hutter, M.: Agi safety literature review. arXiv preprint arXiv:1805.01109 (2018)

    Google Scholar 

  12. Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., Song, D.: Robust physical-world attacks on deep learning models. arXiv preprint arXiv:1707.08945 (2017)

    Google Scholar 

  13. Eysenbach, B., Gu, S., Ibarz, J., Levine, S.: Leave no trace: Learning to reset for safe and autonomous reinforcement learning. arXiv preprint arXiv:1711.06782 (2017)

    Google Scholar 

  14. Gaier, A., Asteroth, A., Mouret, J.B.: Data-efficient design exploration through surrogate-assisted illumination. Evolutionary Computation 26(3), 381–410 (2018)

    CrossRef  Google Scholar 

  15. Garcıa, J., Fernández, F.: A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research 16(1), 1437–1480 (2015)

    MathSciNet  MATH  Google Scholar 

  16. Goldsby, H.J., Cheng, B.H.: Automatically discovering properties that specify the latent behavior of UML models. In: International Conference on Model Driven Engineering Languages and Systems, pp. 316–330. Springer (2010)

    Google Scholar 

  17. Goodhart, C.A.: Problems of monetary management: The UK experience. In: Monetary Theory and Practice, pp. 91–121. Springer (1984)

    Google Scholar 

  18. Hadfield-Menell, D., Hadfield, G.K.: Incomplete contracting and ai alignment. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 417–422. ACM (2019)

    Google Scholar 

  19. Hadfield-Menell, D., Milli, S., Abbeel, P., Russell, S.J., Dragan, A.: Inverse reward design. In: Advances in Neural Information Processing Systems, pp. 6765–6774 (2017)

    Google Scholar 

  20. Irving, G., Christiano, P., Amodei, D.: AI safety via debate. arXiv preprint arXiv:1805.00899 (2018)

    Google Scholar 

  21. Jakobi, N., Husbands, P., Harvey, I.: Noise and the reality gap: The use of simulation in evolutionary robotics. In: European Conference on Artificial Life, pp. 704–720. Springer (1995)

    Google Scholar 

  22. Jin, Y.: Surrogate-assisted evolutionary computation: Recent advances and future challenges. Swarm and Evolutionary Computation 1(2), 61–70 (2011)

    CrossRef  Google Scholar 

  23. Justesen, N., Torrado, R.R., Bontrager, P., Khalifa, A., Togelius, J., Risi, S.: Procedural level generation improves generality of deep reinforcement learning. arXiv preprint arXiv:1806.10729 (2018)

    Google Scholar 

  24. Kahn, G., Villaflor, A., Pong, V., Abbeel, P., Levine, S.: Uncertainty-aware reinforcement learning for collision avoidance. arXiv preprint arXiv:1702.01182 (2017)

    Google Scholar 

  25. Karpov, I.V., Valsalam, V.K., Miikkulainen, R.: Human-assisted neuroevolution through shaping, advice and examples. In: Proceedings of the 13th annual Conference on Genetic and evolutionary computation, pp. 371–378. ACM (2011)

    Google Scholar 

  26. Kashtan, N., Noor, E., Alon, U.: Varying environments can speed up evolution. Proceedings of the National Academy of Sciences 104(34), 13,711–13,716 (2007)

    CrossRef  Google Scholar 

  27. Kirschner, M., Gerhart, J.: Evolvability. Proceedings of the National Academy of Sciences 95(15), 8420–8427 (1998)

    CrossRef  Google Scholar 

  28. Koos, S., Mouret, J.B., Doncieux, S.: The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Transactions on Evolutionary Computation 17(1), 122–145 (2013)

    CrossRef  Google Scholar 

  29. Koza, J.R.: Genetic programming: On the programming of computers by means of natural selection. MIT press (1992)

    Google Scholar 

  30. Koza, J.R.: Genetic programming II: Automatic discovery of reusable subprograms. Cambridge, MA, USA 13(8), 32 (1994)

    Google Scholar 

  31. Kurakin, A., Goodfellow, I., Bengio, S.: Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533 (2016)

    Google Scholar 

  32. Langton, C.G.: Artificial life: An overview. Mit Press (1997)

    Google Scholar 

  33. Lehman, J., Chen, J., Clune, J., Stanley, K.O.: ES is more than just a traditional finite-difference approximator. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 450–457. ACM (2018)

    Google Scholar 

  34. Lehman, J., Chen, J., Clune, J., Stanley, K.O.: Safe mutations for deep and recurrent neural networks through output gradients. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 117–124. ACM (2018)

    Google Scholar 

  35. Lehman, J., Clune, J., Misevic, D., Adami, C., Altenberg, L., Beaulieu, J., Bentley, P.J., Bernard, S., Beslon, G., Bryson, D.M., et al.: The surprising creativity of digital evolution: A collection of anecdotes from the evolutionary computation and artificial life research communities. arXiv preprint arXiv:1803.03453 (2018)

    Google Scholar 

  36. Lehman, J., Miikkulainen, R.: Neuroevolution. Scholarpedia 8(6), 30,977 (2013)

    CrossRef  Google Scholar 

  37. Lehman, J., Stanley, K.O.: Evolving a diversity of virtual creatures through novelty search and local competition. In: Proceedings of the 13th annual Conference on Genetic and Evolutionary Computation, pp. 211–218. ACM (2011)

    Google Scholar 

  38. Lehman, J., Stanley, K.O.: Improving evolvability through novelty search and self-adaptation. In: 2011 IEEE Congress of Evolutionary Computation (CEC), pp. 2693–2700. IEEE (2011)

    Google Scholar 

  39. Leike, J., Krueger, D., Everitt, T., Martic, M., Maini, V., Legg, S.: Scalable agent alignment via reward modeling: A research direction. arXiv preprint arXiv:1811.07871 (2018)

    Google Scholar 

  40. Leike, J., Martic, M., Krakovna, V., Ortega, P.A., Everitt, T., Lefrancq, A., Orseau, L., Legg, S.: AI safety gridworlds. arXiv preprint arXiv:1711.09883 (2017)

    Google Scholar 

  41. Lenat, D.B.: Cyc: A large-scale investment in knowledge infrastructure. Communications of the ACM 38(11), 33–38 (1995)

    CrossRef  Google Scholar 

  42. Lewis, M.A., Fagg, A.H., Solidum, A.: Genetic programming approach to the construction of a neural network for control of a walking robot. In: Proceedings 1992 IEEE International Conference on Robotics and Automation, pp. 2618–2623. IEEE (1992)

    Google Scholar 

  43. Lipton, Z.C., Azizzadenesheli, K., Kumar, A., Li, L., Gao, J., Deng, L.: Combating reinforcement learning’s sisyphean curse with intrinsic fear. arXiv preprint arXiv:1611.01211 (2016)

    Google Scholar 

  44. Markou, M., Singh, S.: Novelty detection: A review - part 1: Statistical approaches. Signal Processing 83(12), 2481–2497 (2003)

    MATH  CrossRef  Google Scholar 

  45. Moldovan, T.M., Abbeel, P.: Safe exploration in Markov decision processes. arXiv preprint arXiv:1205.4810 (2012)

    Google Scholar 

  46. Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 427–436 (2015)

    Google Scholar 

  47. Nolfi, S., Floreano, D., Floreano, D.D.: Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines. MIT press (2000)

    Google Scholar 

  48. Ong, C.S., Huang, J.J., Tzeng, G.H.: Building credit scoring models using genetic programming. Expert Systems with Applications 29(1), 41–47 (2005)

    CrossRef  Google Scholar 

  49. Pollack, J.B., Lipson, H., Ficici, S., Funes, P., Hornby, G., Watson, R.A.: Evolutionary techniques in physical robotics. In: International Conference on Evolvable Systems, pp. 175–186. Springer (2000)

    Google Scholar 

  50. Pugh, J.K., Soros, L.B., Stanley, K.O.: Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI 3, 40 (2016)

    CrossRef  Google Scholar 

  51. Richter, H.: Detecting change in dynamic fitness landscapes. In: 2009 IEEE Congress on Evolutionary Computation, pp. 1613–1620. IEEE (2009)

    Google Scholar 

  52. Rocha, M., Cortez, P., Neves, J.: Evolution of neural networks for classification and regression. Neurocomputing 70(16–18), 2809–2816 (2007)

    CrossRef  Google Scholar 

  53. Ross, S.A.: The economic theory of agency: The principal’s problem. The American Economic Review 63(2), 134–139 (1973)

    Google Scholar 

  54. Saunders, W., Sastry, G., Stuhlmueller, A., Evans, O.: Trial without error: Towards safe reinforcement learning via human intervention. In: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, pp. 2067–2069. International Foundation for Autonomous Agents and Multiagent Systems (2018)

    Google Scholar 

  55. Secretan, J., Beato, N., D Ambrosio, D.B., Rodriguez, A., Campbell, A., Stanley, K.O.: Picbreeder: Evolving pictures collaboratively online. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1759–1768. ACM (2008)

    Google Scholar 

  56. Shaker, N., Togelius, J., Nelson, M.J.: Procedural content generation in games. Springer (2016)

    Google Scholar 

  57. Sims, K.: Evolving virtual creatures. In: Proceedings of the 21st annual conference on Computer graphics and interactive techniques, pp. 15–22. ACM (1994)

    Google Scholar 

  58. Soltoggio, A., Bullinaria, J.A., Mattiussi, C., Dürr, P., Floreano, D.: Evolutionary advantages of neuromodulated plasticity in dynamic, reward-based scenarios. In: Proceedings of the 11th international Conference on Artificial Life (Alife XI), CONF, pp. 569–576. MIT Press (2008)

    Google Scholar 

  59. Soltoggio, A., Stanley, K.O., Risi, S.: Born to learn: the inspiration, progress, and future of evolved plastic artificial neural networks. Neural Networks 108, 48–67 (2018)

    CrossRef  Google Scholar 

  60. Standish, R.K.: Open-ended artificial evolution. International Journal of Computational Intelligence and Applications 3(02), 167–175 (2003)

    CrossRef  Google Scholar 

  61. Stanley, K.O., Bryant, B.D., Miikkulainen, R.: Real-time neuroevolution in the NERO video game. IEEE Transactions on Evolutionary Computation 9(6), 653–668 (2005)

    CrossRef  Google Scholar 

  62. Stanley, K.O., Lehman, J., Soros, L.: Open-endedness: The last grand challenge you’ve never heard of. While open-endedness could be a force for discovering intelligence, it could also be a component of AI itself (2017)

    Google Scholar 

  63. Sutton, R.S.: Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In: Machine Learning Proceedings 1990, pp. 216–224. Elsevier (1990)

    Google Scholar 

  64. Takagi, H.: Interactive evolutionary computation: Fusion of the capabilities of EC optimization and human evaluation. Proceedings of the IEEE 89(9), 1275–1296 (2001)

    CrossRef  Google Scholar 

  65. Taylor, J.: Quantilizers: A safer alternative to maximizers for limited optimization. In: Workshops at the Thirtieth AAAI Conference on Artificial Intelligence (2016)

    Google Scholar 

  66. Taylor, T., Bedau, M., Channon, A., Ackley, D., Banzhaf, W., Beslon, G., Dolson, E., Froese, T., Hickinbotham, S., Ikegami, T., et al.: Open-ended evolution: Perspectives from the OEE workshop in York. Artificial Life 22(3), 408–423 (2016)

    CrossRef  Google Scholar 

  67. Vann, M.G.: Of rats, rice, and race: The great Hanoi rat massacre, an episode in French colonial history. French Colonial History 4(1), 191–203 (2003)

    CrossRef  Google Scholar 

  68. Wagner, G.P., Altenberg, L.: Perspective: Complex adaptations and the evolution of evolvability. Evolution 50(3), 967–976 (1996)

    CrossRef  Google Scholar 

  69. Wang, R., Lehman, J., Clune, J., Stanley, K.O.: Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions. arXiv preprint arXiv:1901.01753 (2019)

    Google Scholar 

  70. Watson, R.A., Ficici, S.G., Pollack, J.B.: Embodied evolution: Distributing an evolutionary algorithm in a population of robots. Robotics and Autonomous Systems 39(1), 1–18 (2002)

    CrossRef  Google Scholar 

  71. Whittaker, M., Crawford, K., Dobbe, R., Fried, G., Kaziunas, E., Mathur, V., West, S.M., Richardson, R., Schultz, J., Schwartz, O.: AI now report 2018. AI Now Institute at New York University (2018)

    Google Scholar 

  72. Wierstra, D., Schaul, T., Peters, J., Schmidhuber, J.: Natural evolution strategies. In: 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence), pp. 3381–3387. IEEE (2008)

    Google Scholar 

  73. Woolley, B.G., Stanley, K.O.: A novel human-computer collaboration: Combining novelty search with interactive evolution. In: Proceedings of the 2014 annual Conference on Genetic and Evolutionary Computation, pp. 233–240. ACM (2014)

    Google Scholar 

  74. Yao, X.: Evolving artificial neural networks. Proceedings of the IEEE 87(9), 1423–1447 (1999)

    CrossRef  Google Scholar 

  75. Yudkowsky, E.: Coherent extrapolated volition. Singularity Institute for Artificial Intelligence (2004)

    Google Scholar 

  76. Zafar, M.B., Valera, I., Gomez Rodriguez, M., Gummadi, K.P.: Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In: Proceedings of the 26th International Conference on World Wide Web, pp. 1171–1180. International World Wide Web Conferences Steering Committee (2017)

    Google Scholar 

  77. Zhang, C., Vinyals, O., Munos, R., Bengio, S.: A study on overfitting in deep reinforcement learning. arXiv preprint arXiv:1804.06893 (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Joel Lehman .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Lehman, J. (2020). Evolutionary Computation and AI Safety. In: Banzhaf, W., Goodman, E., Sheneman, L., Trujillo, L., Worzel, B. (eds) Genetic Programming Theory and Practice XVII. Genetic and Evolutionary Computation. Springer, Cham. https://doi.org/10.1007/978-3-030-39958-0_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-39958-0_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-39957-3

  • Online ISBN: 978-3-030-39958-0

  • eBook Packages: Computer ScienceComputer Science (R0)