Skip to main content

Using Generative Adversarial Networks to Develop a Realistic Human Behavior Simulator

Part of the Lecture Notes in Computer Science book series (LNAI,volume 11224)

Abstract

Simulation environments have proven to be very useful as testbeds for reinforcement learning (RL) algorithms. For settings where an actual human user is involved, these simulation environments allow one to test out the suitability of new RL approaches without having to include real users at first. It obviously does require the simulator to have a certain degree of realism, however, realistic simulators for the behavior of humans in the health domain are rarely seen. To generate realistic behavior, the simulator could be driven by data from real users, but this might lead to privacy issues. In this paper, we propose to use Generative Adversarial Networks (GANs) for generating realistic simulation environments. In this first step, we use an existing simulator that simulates daily activities of users and the GANs are used to generate realistic sensory data that accompanies such activities. After training, the original (potentially privacy sensitive) data can be thrown away and the simulator can simply be driven by the GAN models. Results show that a model trained on real data shows similar performance on the data artificially generated by the GAN.

Keywords

  • Simulation
  • Generative adversarial networks
  • Reinforcement learning
  • Deep learning
  • e-Health

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-03098-8_32
  • Chapter length: 8 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-03098-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.

References

  1. Alzantot, M., Chakraborty, S., Srivastava, M.B.: SenseGen: a deep learning architecture for synthetic sensor data generation. In: 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pp. 188–193 (2017)

    Google Scholar 

  2. Barto, A.G., Sutton, R.S., Watkins, C.J.C.H.: Learning and sequential decision making. In: Learning and Computational Neuroscience, pp. 539–602. MIT Press (1989)

    Google Scholar 

  3. Chevalier, G.: LSTM human activity recognition (2017). https://github.com/guillaume-chevalier/LSTM-Human-Activity-Recognition

  4. Christiano, P.F., et al.: Transfer from simulation to real world through learning deep inverse dynamics model. CoRR abs/1610.03518 (2016). http://arxiv.org/abs/1610.03518

  5. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. The MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  6. Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  7. El Hassouni, A., Hoogendoorn, M., van Otterlo, M., Barbaro, E.: Personalization of health interventions using cluster-based reinforcement learning. CoRR abs/1804.03592 (2018). http://arxiv.org/abs/1804.03592

  8. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735

    CrossRef  Google Scholar 

  9. Hoogendoorn, M., Funk, B.: Machine Learning for the Quantified Self: On the Art of Learning from Sensory Data. Springer, Heidelberg (2017). https://doi.org/10.1007/978-3-319-66308-1

    CrossRef  Google Scholar 

  10. Kwapisz, J.R., Weiss, G.M., Moore, S.A.: Activity recognition using cell phone accelerometers. SIGKDD Explor. Newsl. 12(2), 74–82 (2011). https://doi.org/10.1145/1964897.1964918

    CrossRef  Google Scholar 

  11. Malekzadeh, M., Clegg, R.G., Haddadi, H.: Replacement AutoEncoder: a privacy-preserving algorithm for sensory data analysis. In: 2018 IEEE/ACM Third International Conference on Internet-of-Things Design and Implementation (IoTDI), pp. 165–176, April 2018. https://doi.org/10.1109/IoTDI.2018.00025

  12. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1. MIT press, Cambridge (2018)

    Google Scholar 

  13. Tseng, H.H., Luo, Y., Cui, S., Chien, J.T., Ten Haken, R.K., Naqa, I.E.: Deep reinforcement learning for automated radiation adaptation in lung cancer. Med. Phys. 44(12), 6690–6705 (2017)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ali el Hassouni .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Hassouni, A.e., Hoogendoorn, M., Muhonen, V. (2018). Using Generative Adversarial Networks to Develop a Realistic Human Behavior Simulator. In: Miller, T., Oren, N., Sakurai, Y., Noda, I., Savarimuthu, B., Cao Son, T. (eds) PRIMA 2018: Principles and Practice of Multi-Agent Systems. PRIMA 2018. Lecture Notes in Computer Science(), vol 11224. Springer, Cham. https://doi.org/10.1007/978-3-030-03098-8_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03098-8_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03097-1

  • Online ISBN: 978-3-030-03098-8

  • eBook Packages: Computer ScienceComputer Science (R0)