Skip to main content

Advertisement

SpringerLink
Log in

Salsa dance learning evaluation and motion analysis in gamified virtual reality environment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Learning couple dance such as salsa is challenging as it requires to understand and assimilate all the dance skills (guidance, rhythm, style) correctly. Salsa is traditionally learned by attending a dancing class with a teacher and practice with a partner, the difficulty to access such classes though, and the variability of dance environment can impact the learning process. Understanding how people learn using a virtual reality platform could bring interesting knowledge in motion analysis and can be the first step toward a complementary learning system at home. In this paper, we propose an interactive learning application in the form of a virtual reality game, that aims to help the user to improve its salsa dancing skills. The application was designed upon previous literature and expert discussion and has different components that simulate salsa dance: A virtual partner with interactive control to dance with, visual and haptic feedback, and a game mechanic with dance tasks. This application is tested on a two-class panel of 20 regular and 20 non-dancers, and their learning is evaluated and analyzed through the extraction of Musical Motion Features and the Laban Motion Analysis system. Both motion analysis frameworks were compared prior and after training and show a convergence of the profile of non-dancer toward the profile of regular dancers, which validates the learning process. The work presented here has profound implications for future studies of motion analysis, couple dance learning, and human-human interaction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Alborno P, Piana S, Mancini M, Niewiadomski R, Volpe G, Camurri A (2016) Analysis of intrapersonal synchronization in full-body movements displaying different expressive qualities. In: Proceedings of the international working conference on advanced visual interfaces, AVI ’16. ACM, New York, pp 136–143. https://doi.org/10.1145/2909132.2909262

  2. Alexiadis DS, Kelly P, Daras P, O’Connor NE, Boubekeur T, Moussa MB (2011) Evaluating a dancer’s performance using kinect-based skeleton tracking. In: Proceedings of the 19th ACM international conference on multimedia - MM ’11, p 659. https://doi.org/10.1145/2072298.2072412. https://core.ac.uk/download/pdf/11310464.pdf. http://dl.acm.org/citation.cfm?doid=2072298.2072412

  3. Aristidou A, Charalambous P, Chrysanthou Y (2015) Emotion analysis and classification: understanding the performers’ emotions using the LMA entities. Comput Graph Forum 34(6):262–276. https://doi.org/10.1111/cgf.12598

    Article  Google Scholar 

  4. Aristidou A, Chrysanthou Y (2013) Motion indexing of different emotional states using lma components. In: SIGGRAPH Asia 2013 technical briefs, SA ’13. ACM, New York, pp 21:1–21:4. https://doi.org/10.1145/2542355.2542381

  5. Aristidou A, Cohen-Or D, Hodgins JK, Chrysanthou Y, Shamir A (2018) Deep motifs and motion signatures. ACM Trans Graph 37(6):187:1–187:13. https://doi.org/10.1145/3272127.3275038

    Article  Google Scholar 

  6. Aristidou A, Stavrakis E, Charalambous P, Chrysanthou Y, Loizidou-Himona S (2015) Folk dance evaluation using laban movement analysis. J Comput Cult Herit 8(4):20:1–20:19. https://doi.org/10.1145/2755566

    Article  Google Scholar 

  7. Aristidou A, Stavrakis E, Papaefthimiou M, Papagiannakis G, Chrysanthou Y (2018) Style-based motion analysis for dance composition. Vis Comput 34(12):1725–1737. https://doi.org/10.1007/s00371-017-1452-z

    Article  Google Scholar 

  8. Aristidou A, Zeng Q, Stavrakis E, Yin K, or Daniel C, Chrysanthou Y, Chen B (2017) Emotion control of unstructured dance movements. In: Proceedings of the ACM SIGGRAPH / eurographics symposium on computer animation, SCA ’17. ACM, New York, pp 9:1–9:10. https://doi.org/10.1145/3099564.3099566

  9. Bastanfard A, Takahashi H, Nakajima M (2004) Toward e-appearance of human face and hair by age, expression and rejuvenation. In: Proceedings of the 2004 international conference on cyberworlds, CW ’04. IEEE Computer Society, USA, pp 306–311. https://doi.org/10.1109/CW.2004.65

  10. Bellini R, Kleiman Y, Cohen-Or D (2018) Dance to the beat: enhancing dancing performance in video. Computational Visual Media 4:197–208. https://doi.org/10.1007/s41095-018-0115-y

    Article  Google Scholar 

  11. Chan JC, Leung H, Tang JK, Komura T (2011) A virtual reality dance training system using motion capture technology. IEEE Trans Learn Technol 4(2):187–195. https://doi.org/10.1109/TLT.2010.27

    Article  Google Scholar 

  12. Cuykendall S, Soutar-Rau E, Schiphorst T (2016) POEME: a poetry engine powered by your movement. In: Proceedings of the TEI ’16: tenth international conference on tangible, embedded, and embodied interaction , pp 635–640. https://doi.org/10.1145/2839462.2856339. http://dl.acm.org/citation.cfm?doid=2839462.2856339. http://doi.acm.org/10.1145/2839462.2856339

  13. Cuykendall S, Soutar-Rau E, Schiphorst T, Dipaola S (2016) If words could dance: moving from body to data through kinesthetic evaluation. In: Proceedings of the 2016 ACM conference on designing interactive systems - DIS ’16, pp 234–238. https://doi.org/10.1145/2901790.2901822. http://dl.acm.org/citation.cfm?doid=2901790.2901822

  14. DanceVirtual: (2018). http://salsa.dance-virtual.com/

  15. Deng L, Leung H, Gu N, Yang Y (2011) Real-time mocap dance recognition for an interactive dancing game. In: Computer animation and virtual worlds, vol 22, pp 229–237. https://doi.org/10.1002/cav.397

  16. dos Santos A, Yacef K, Martinez-Maldonado R (2017) Let’s dance: how to build a user model for dance students using wearable technology. In: Proceedings of the 25th conference on user modeling, adaptation and personalization. ACM Press, New York, pp 183–191. https://doi.org/10.1145/3079628.3079673. http://dl.acm.org/citation.cfm?doid=3079628.3079673

  17. Forbes K, Fiume E (2005) An efficient search algorithm for motion data using weighted PCA. In: Proceedings of the ACM SIGGRAPH/eurographics symposium on computer animation, SCA ’05, pp 67–76

  18. Fourati N, Pelachaud C (2015) Relevant body cues for the classification of emotional body expression in daily actions. In: Proceedings of the sixth international conference on affective computing and intelligent interaction (ACII2015), pp 267–273

  19. Hajarian M, Bastanfard A, Mohammadzadeh J, Khalilian M (2019) A personalized gamification method for increasing user engagement in social networks. Social Network Analysis and Mining 9:1–14

    Article  Google Scholar 

  20. Ho ESL, Chan JCP, Komura T, Leung H (2013) Interactive partner control in close interactions for real-time applications. ACM Transactions on Multimedia Computing,Communications, and Applications 9(3):1–19. https://doi.org/10.1145/2487268.2487274

    Article  Google Scholar 

  21. Kim TH, Park SI, Shin SY (2003) Rhythmic-motion synthesis based on motion-beat analysis. ACM Trans Graph 22(3):392–401. https://doi.org/10.1145/882262.882283

    Article  Google Scholar 

  22. Kyan M, Sun G, Li H, Zhong L, Muneesawang P, Dong N, Elder B, Guan L (2015) An approach to ballet dance training through MS Kinect and visualization in a CAVE virtual reality environment. ACM Transactions on Intelligent Systems and Technology 6(2):1–37. https://doi.org/10.1145/2735951. http://dl.acm.org/citation.cfm?doid=2753829.2735951

    Article  Google Scholar 

  23. Laban R, Ullmann L (2011) The mastery of movement, 4th edn. Dance Books Ltd, Binsted

    Google Scholar 

  24. Lee J, Chai J, Reitsma PSA, Hodgins JK, Pollard NS (2002) Interactive control of avatars animated with human motion data. In: Proceedings of the 29th annual conference on computer graphics and interactive techniques - SIGGRAPH ’02, p 491. https://doi.org/10.1145/566570.566607. http://portal.acm.org/citation.cfm?doid=566570.566607

  25. Merom D, Cumming R, Mathieu E, Anstey KJ, Rissel C, Simpson JM, Morton RL, Cerin E, Sherrington C, Lord SR (2013) Can social dancing prevent falls in older adults? A protocol of the Dance, Aging, Cognition, Economics (DAnCE) fall prevention randomised controlled trial. BMC Public Health 13(1):477. https://doi.org/10.1186/1471-2458-13-477. http://www.ncbi.nlm.nih.gov/pubmed/23675705. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC3691670. http://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-13-477

    Article  Google Scholar 

  26. Merom D, Grunseit A, Eramudugolla R, Jefferis B, Mcneill J, Anstey KJ (2016) Cognitive benefits of social dancing and walking in old age: the dancing mind randomized controlled trial. Frontiers in Aging Neuroscience 8:26. https://doi.org/10.3389/fnagi.2016.00026. http://www.ncbi.nlm.nih.gov/pubmed/26941640. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC4761858

    Article  Google Scholar 

  27. Merom D, Mathieu E, Cerin E, Morton RL, Simpson JM, Rissel C, Anstey KJ, Sherrington C, Lord SR, Cumming RG (2016) Social dancing and incidence of falls in older adults: a cluster randomised controlled trial. PLOS Medicine 13(8):e1002112. https://doi.org/10.1371/journal.pmed.1002112. http://www.ncbi.nlm.nih.gov/pubmed/27575534. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC5004860

    Article  Google Scholar 

  28. Mousas C (2018) Performance-driven dance motion control of a virtual partner character. In: 25th IEEE conference on virtual reality and 3D user interfaces, VR 2018 - Proceedings, pp 57–64. https://doi.org/10.1109/VR.2018.8446498

  29. Ozcimder K, Dey B, Lazier RJ, Trueman D, Leonard NE (2016) Investigating group behavior in dance: an evolutionary dynamics approach. In: Proceedings of the American control conference, vol 2016-July. IEEE, pp 6465–6470. https://doi.org/10.1109/ACC.2016.7526687. http://ieeexplore.ieee.org/document/7526687/

  30. Paez Granados DF, Kinugawa J, Hirata Y, Kosuge K (2016) Guiding human motions in physical human – robot interaction through COM motion control of a dance teaching robot. In: IEEE international conference on humanoid robots (humanoids). IEEE, pp 279–285. https://doi.org/10.1109/HUMANOIDS.2016.7803289. http://ieeexplore.ieee.org/document/7803289/

  31. Patil P, Kumar K, Gaud N, Semwal VB (2019) Clinical human gait classification: extreme learning machine approach. In: 2019 1st international conference on advances in science, engineering and robotics technology (ICASERT), pp 1–6

  32. Piana S (2016) Movement fluidity analysis based on performance and perception. In: CHI extended abstracts on human factors in computing systems, pp 1629–1636. https://doi.org/10.1145/2851581.2892478. http://dl.acm.org/citation.cfm?doid=2851581.2892478

  33. Powers RSU (2020) Brief histories of social dance. https://socialdance.stanford.edu/Syllabi/dance_histories.htm

  34. Raheb KE, Katifori V, Rc A (2016) HCI challenges in dance education. ICST Transactions on Ambient Systems 3(9):6–10. https://doi.org/10.4108/eai.23-8-2016.151642. http://eudl.eu/doi/10.4108/eai.23-8-2016.151642

    Article  Google Scholar 

  35. Rubner Y, Tomasi C, Guibas LJ (2000) The earth mover’s distance as a metric for image retrieval. Int J Comput Vision 40(2):99–121

    Article  Google Scholar 

  36. Seber GAF (2008) Multivariate observations. Wiley, New York

    MATH  Google Scholar 

  37. Semwal VB, Mondal K, Nandi GC (2015) Robust and accurate feature selection for humanoid push recovery and classification: deep learning approach. Neural Comput Applic 28:565–574

    Article  Google Scholar 

  38. Senecal S, Cuel L, Aristidou A, Magnenat-Thalmann N (2016) Continuous body emotion recognition system during theater performances. Comput Animat Virtual Worlds 27(3-4):311–320. https://doi.org/10.1002/cav.1714

    Article  Google Scholar 

  39. Senecal S, Nijdam N, Thalmann N (2019) Classification of salsa dance level using music and interaction based motion features. In: GRAPP 2019 - international conference on computer graphics theory and applications, pp 100–109. https://doi.org/10.5220/0007399701000109

  40. Senecal S, Nijdam NA, Thalmann NM (2018) Motion analysis and classification of salsa dance using music-related motion features. In: Proceedings of the 11th annual international conference on motion, interaction, and games, MIG ’18. ACM, New York, pp 11:1–11:10. https://doi.org/10.1145/3274247.3274514

  41. Shiratori T, Nakazawa A, Ikeuchi K (2006) Dancing-to-music character animation. Computer Graphics Forum 25(3):449–458. https://doi.org/10.1111/j.1467-8659.2006.00964.x

    Article  Google Scholar 

  42. Shum HP, Komura T, Shiraishi M, Yamazaki S (2008) Interaction patches for multi-character animation. ACM Transactions on Graphics (TOG) 27(5):114

    Article  Google Scholar 

  43. Tang T, Jia J, Mao H (2018) Dance with melody: an lstm-autoencoder approach to music-oriented dance synthesis. In: Proceedings of the 26th ACM international conference on multimedia, MM ’18. ACM, New York, pp 1598–1606. https://doi.org/10.1145/3240508.3240526

  44. Theodorou L, Healey PGT, Smeraldi F (2016) Exploring audience behaviour during contemporary dance performances. In: Proceedings of the 3rd international symposium on movement and computing - MOCO ’16. ACM Press, New York, pp 1–7. https://doi.org/10.1145/2948910.2948928. http://dl.acm.org/citation.cfm?doid=2948910.2948928

  45. van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605

    MATH  Google Scholar 

  46. Whyatt CP, Torres EB (2017) The social-dance. In: Proceedings of the 4th international conference on movement computing - MOCO ’17. ACM Press, New York, pp 1–8. https://doi.org/10.1145/3077981.3078055. http://dl.acm.org/citation.cfm?doid=3077981.3078055

  47. Won J, Lee K, O’Sullivan C, Hodgins JK, Lee J (2014) Generating and ranking diverse multi-character interactions. ACM Transactions on Graphics (TOG) 33 (6):219

    Article  Google Scholar 

Download references

Acknowledgements

This work is co-financed by the European project MINGEI. It has also been partly supported by the project that has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578 (RISE-Call: H2020-WIDESPREAD -01-2016-2017- TeamingPhase2) and the Government of the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development.

Author information

Authors and Affiliations

  1. University of Geneva, Geneva, Switzerland

    Simon Senecal, Niels A. Nijdam & Nadia Magnenat-Thalmann

  2. University of Cyprus, Nicosia, Cyprus

    Andreas Aristidou

  3. RISE Research Center, Nicosia, Cyprus

    Andreas Aristidou

Authors
  1. Simon Senecal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Niels A. Nijdam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Aristidou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nadia Magnenat-Thalmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon Senecal.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MP4 172 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Senecal, S., Nijdam, N.A., Aristidou, A. et al. Salsa dance learning evaluation and motion analysis in gamified virtual reality environment. Multimed Tools Appl 79, 24621–24643 (2020). https://doi.org/10.1007/s11042-020-09192-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09192-y

Keywords

Search

Navigation