Abstract
Simulation environments have proven to be very useful as testbeds for reinforcement learning (RL) algorithms. For settings where an actual human user is involved, these simulation environments allow one to test out the suitability of new RL approaches without having to include real users at first. It obviously does require the simulator to have a certain degree of realism, however, realistic simulators for the behavior of humans in the health domain are rarely seen. To generate realistic behavior, the simulator could be driven by data from real users, but this might lead to privacy issues. In this paper, we propose to use Generative Adversarial Networks (GANs) for generating realistic simulation environments. In this first step, we use an existing simulator that simulates daily activities of users and the GANs are used to generate realistic sensory data that accompanies such activities. After training, the original (potentially privacy sensitive) data can be thrown away and the simulator can simply be driven by the GAN models. Results show that a model trained on real data shows similar performance on the data artificially generated by the GAN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alzantot, M., Chakraborty, S., Srivastava, M.B.: SenseGen: a deep learning architecture for synthetic sensor data generation. In: 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pp. 188–193 (2017)
Barto, A.G., Sutton, R.S., Watkins, C.J.C.H.: Learning and sequential decision making. In: Learning and Computational Neuroscience, pp. 539–602. MIT Press (1989)
Chevalier, G.: LSTM human activity recognition (2017). https://github.com/guillaume-chevalier/LSTM-Human-Activity-Recognition
Christiano, P.F., et al.: Transfer from simulation to real world through learning deep inverse dynamics model. CoRR abs/1610.03518 (2016). http://arxiv.org/abs/1610.03518
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. The MIT Press, Cambridge (2016)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
El Hassouni, A., Hoogendoorn, M., van Otterlo, M., Barbaro, E.: Personalization of health interventions using cluster-based reinforcement learning. CoRR abs/1804.03592 (2018). http://arxiv.org/abs/1804.03592
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Hoogendoorn, M., Funk, B.: Machine Learning for the Quantified Self: On the Art of Learning from Sensory Data. Springer, Heidelberg (2017). https://doi.org/10.1007/978-3-319-66308-1
Kwapisz, J.R., Weiss, G.M., Moore, S.A.: Activity recognition using cell phone accelerometers. SIGKDD Explor. Newsl. 12(2), 74–82 (2011). https://doi.org/10.1145/1964897.1964918
Malekzadeh, M., Clegg, R.G., Haddadi, H.: Replacement AutoEncoder: a privacy-preserving algorithm for sensory data analysis. In: 2018 IEEE/ACM Third International Conference on Internet-of-Things Design and Implementation (IoTDI), pp. 165–176, April 2018. https://doi.org/10.1109/IoTDI.2018.00025
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1. MIT press, Cambridge (2018)
Tseng, H.H., Luo, Y., Cui, S., Chien, J.T., Ten Haken, R.K., Naqa, I.E.: Deep reinforcement learning for automated radiation adaptation in lung cancer. Med. Phys. 44(12), 6690–6705 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Hassouni, A.e., Hoogendoorn, M., Muhonen, V. (2018). Using Generative Adversarial Networks to Develop a Realistic Human Behavior Simulator. In: Miller, T., Oren, N., Sakurai, Y., Noda, I., Savarimuthu, B.T.R., Cao Son, T. (eds) PRIMA 2018: Principles and Practice of Multi-Agent Systems. PRIMA 2018. Lecture Notes in Computer Science(), vol 11224. Springer, Cham. https://doi.org/10.1007/978-3-030-03098-8_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-03098-8_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03097-1
Online ISBN: 978-3-030-03098-8
eBook Packages: Computer ScienceComputer Science (R0)