Abstract
It is a challenge task to enable a robot to dance according to different types of music. However, two problems have not been well resolved yet: (1) how to assign a dance to a certain type of music, and (2) how to ensure a dancing robot to keep in balance. To tackle these challenges, a robot automatic choreography system based on the deep learning technology is introduced in this paper. First, two deep learning neural network models are built to convert local and global features of music to corresponding features of dance, respectively. Then, an action graph is built based on the collected dance segments; the main function of the action graph is to generate a complete dance sequence based on the dance features generated by the two deep learning models. Finally, the generated dance sequence is performed by a humanoid robot. The experimental results shows that, according to the input music, the proposed model can successfully generate dance sequences that match the input music; also, the robot can maintain its balance while it is dancing. In addition, compared with the dance sequences in the training dataset, the dance sequences generated by the model has reached the level of artificial choreography in both diversity and innovation. Therefore, this method provides a promising solution for robotic choreography automation and design assistance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The complete dance performances can be seen on the website: https://dwz.cn/ocj9m2eP.
References
Eaton, M.: An approach to the synthesis of humanoid robot dance using non-interactive evolutionary techniques. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics, pp. 3305–3309. IEEE (2013)
Peng, H., Zhou, C., Hu, H., Chao, F., Li, J.: Robotic dance in social robotics–a taxonomy. IEEE Trans. Hum.-Mach. Syst. 45(3), 281–293 (2015). https://doi.org/10.1109/THMS.2015.2393558
Peng, H., Hu, H., Chao, F., Zhou, C., Li, J.: Autonomous robotic choreography creation via semi-interactive evolutionary computation. Int. J. Soc. Robot. 8(5), 649–661 (2016)
Manfrè, A., Infantino, I., Vella, F., Gaglio, S.: An automatic system for humanoid dance creation. Biol. Inspired Cogn. Architect. 15, 1–9 (2016)
Meng, Q., Tholley, I., Chung, P.W.: Robots learn to dance through interaction with humans. Neural Comput. Appl. 24(1), 117–124 (2014)
Tholley, I.S.: Towards a framework to make robots learn to dance. Ph.D. thesis, Ibrahim S. Tholley (2012)
Xia, G., Tay, J., Dannenberg, R., Veloso, M.: Autonomous robot dancing driven by beats and emotions of music. In: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, vol. 1, pp. 205–212. International Foundation for Autonomous Agents and Multiagent Systems (2012)
Qin, R., Zhou, C., Zhu, H., Shi, M., Chao, F., Li, N.: A music-driven dance system of humanoid robots. Int. J. Hum. Robot. 15(05), 1850023 (2018)
Chu, H., Urtasun, R., Fidler, S.: Song from PI: a musically plausible network for pop music generation. CoRR abs/1611.03477 (2016). http://arxiv.org/abs/1611.03477
Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: a neural image caption generator. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
Crnkovic-Friis, L., Crnkovic-Friis, L.: Generative choreography using deep learning. In: Proceedings of the Seventh International Conference on Computational Creativity (2016)
Choi, K., Fazekas, G., Cho, K., Sandler, M.B.: A tutorial on deep learning for music information retrieval. CoRR abs/1709.04396 (2017). http://arxiv.org/abs/1709.04396
Cho, K., van Merrienboer, B., Gülçehre, Ç., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. CoRR abs/1406.1078 (2014). http://arxiv.org/abs/1406.1078
McFee, B., et al.: Librosa: audio and music signal analysis in python. In: Proceedings of the 14th Python in Science Conference, pp. 18–25 (2015)
Ellis, D.P.: Beat tracking by dynamic programming. J. New Music Res. 36(1), 51–60 (2007)
Mikolov, T., Karafiát, M., Burget, L., Černockỳ, J., Khudanpur, S.: Recurrent neural network based language model. In: Eleventh Annual Conference of the International Speech Communication Association (2010)
Zhao, L., Safonova, A.: Achieving good connectivity in motion graphs. Graph. Models 71(4), 139–152 (2009)
Kovar, L., Gleicher, M., Pighin, F.: Motion graphs. In: ACM SIGGRAPH 2008 Classes, p. 51. ACM (2008)
Acknowledgment
This work was supported by the National Natural Science Foundation of China (No. 61673322, 61673326, and 91746103), the Fundamental Research Funds for the Central Universities (No. 20720190142), Natural Science Foundation of Fujian Province of China (No. 2017J01128 and 2017J01129), and the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement (No. 663830).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, R. et al. (2019). Towards Deep Learning Based Robot Automatic Choreography System. In: Yu, H., Liu, J., Liu, L., Ju, Z., Liu, Y., Zhou, D. (eds) Intelligent Robotics and Applications. ICIRA 2019. Lecture Notes in Computer Science(), vol 11743. Springer, Cham. https://doi.org/10.1007/978-3-030-27538-9_54
Download citation
DOI: https://doi.org/10.1007/978-3-030-27538-9_54
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-27537-2
Online ISBN: 978-3-030-27538-9
eBook Packages: Computer ScienceComputer Science (R0)