Abstract
Labanotation is a well-known notation system for effective dance recording and archiving. Using computer technology to generate Labanotation automatically is a challenging but meaningful task, while existing methods cannot fully utilize spatial characteristics of human motion and distinguish subtle differences between similar human movements. In this paper, we propose a method based on multi-model aggregation for Labanotation generation. Firstly, two types of feature are extracted, the joint feature and the Lie group feature, which reinforce the representation of human motion data. Secondly, a two-branch network architecture based on Long Short-Term Memory (LSTM) network and LieNet is introduced to conduct effective human movement recognition. LSTM is capable to model long-term dependencies in temporal domain, and LieNet is a powerful network for spatial analysis based on Lie group structure. Within the architecture, the joint feature and the Lie group feature are fed into LSTM model and LieNet model respectively for training. Furthermore, we utilize score fusion methods to fuse the output class scores of the two branches, which performs better than any of the single models, due to complementarity between LSTM and LieNet. In addition, skip connection is applied in the structure of LieNet, which simplifies the training procedure and improves the convergence behavior. Evaluations on standard motion capture dataset demonstrate the effectiveness of proposed method and its superiority compared with previous works.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Loke, L., Larssen, A.T., Robertson, T.: Labanotation for design of movement-based interaction. In: Australasian Conference on Interactive Entertainment (2005)
Guest, A.H.: Labanotation: The System of Analyzing and Recording Movement. Psychology Press, London (2014)
Venable, L.: Archives of the dance: Labanwriter: there had to be a better way. Dance Res. J. Soc. Dance Res. 9(2), 76–88 (1991)
Edward, F.: LED&LINTEL: a windows mini-editor and interpreter for Labanotation [EB/OL]. http://donhe.topcities.com/pubs/led.heml. Accessed 16 Nov 2016
Misi, G.: Labanatory [EB/OL]. http://labanatory.com/eng/software.html. Accessed 16 Nov 2016
Hachimura, K.: Digital archiving of dance by using motion-capture technology. In: New Directions in Digital Humanities for Japanese Arts and Cultures, pp. 167–182. Nakanishiya Publishing, Kyoto (2008)
Hachimura, K., Nakamura, M.: Method of generating coded description of human body motion from motion-captured data. In: 10th IEEE International Workshop on Robot & Human Interactive Communication, pp. 122–127. IEEE (2001)
Guo, H., Miao, Z., Zhu, F., Zhang, G., Li, S.: Automatic labanotation generation based on human motion capture data. In: Li, S., Liu, C., Wang, Y. (eds.) CCPR 2014. CCIS, vol. 483, pp. 426–435. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-45646-0_44
Choensawat, W., Nakamura, M., Hachimura, K.: GenLaban: a tool for generating labanotation from motion capture data. Multimedia Tools Appl. 74(23), 10823–10846 (2015)
Zhou, Z., Miao, Z., Wang, J.: A system for automatic generation of Labanotation from motion capture data. In: 13th IEEE International Conference of Signal Process, pp. 1031–1034. IEEE (2016)
Li, M., Miao, Z.: Automatic Labanotation generation from motion-captured data based on hidden Markov models. In: 4th Asia Conference of Pattern Recognition (2017)
Zhang, X., Miao, Z., Zhang, Q., Wang, J.: Skeleton-based automatic generation of Labanotation with neural networks. J. Electron. Imaging 28(2), 23–26 (2019)
Li, C., Wang, P., Wang, S., Hou, Y., Li, W.: Skeleton-based action recognition using LSTM and CNN. In: IEEE International Conference on Multimedia & Expo Workshops (2017)
Zhang, S., et al.: Fusing geometric features for skeleton-based action recognition using multilayer LSTM networks. IEEE Trans. Multimedia 20(9), 2330–2343 (2018)
Huang, Z., Wan, C., Probst, T., Van Gool, L.: Deep learning on lie groups for skeleton-based action recognition. In: Computer Vision & Pattern Recognition (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016, Part IV. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38
Vemulapalli, R., Chellappa, R.: Rolling rotations for recognizing human actions from 3D skeletal data. In: The IEEE Conference on Computer Vision and Pattern Recognition (2016)
Wang, J., Miao, Z.: A method of automatically generating Labanotation from human motion capture data. In: International Conference on Pattern Recognition, pp. 845–859 (2018)
NaturalPoint Corporation: OptiTrack Documentation Center [EB/OL]. http://wiki.optitrack.com/. Accessed 16 Nov 2016
Meredith, M., Maddock, S.: Motion capture file formats explained. Department of Computer Science, University of Sheffield (2001)
Acknowledgement
This work is supported by the NSFC 61672089, 61273274, 61572064, and National Key Technology R&D Program of China 2012BAH01F03.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Xie, N., Miao, Z., Wang, J. (2020). Skeleton-Based Labanotation Generation Using Multi-model Aggregation. In: Palaiahnakote, S., Sanniti di Baja, G., Wang, L., Yan, W. (eds) Pattern Recognition. ACPR 2019. Lecture Notes in Computer Science(), vol 12047. Springer, Cham. https://doi.org/10.1007/978-3-030-41299-9_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-41299-9_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-41298-2
Online ISBN: 978-3-030-41299-9
eBook Packages: Computer ScienceComputer Science (R0)