Journal on Multimodal User Interfaces

, Volume 9, Issue 4, pp 403–411 | Cite as

Investigating the effects of motion-based Kinect game system on user cognition

  • Akihito NakaiEmail author
  • Aung Pyae
  • Mika Luimula
  • Satoshi Hongo
  • Hannu Vuola
  • Jouni Smed
Original Paper


Kinect, which is useful as a multimodal input device, offers multiple features such as voice recognition, facial tracking, object recognition, and gesture recognition. Games with such human-centred, interactive interfaces create an environment that bridges between artificial and natural cognitive capabilities, enabling them to work together more efficiently. This study specifically examines a usability evaluation method that uses Kansei engineering knowledge to identify and remedy difficulties related to user interface development. As a practical application of the suggested evaluation method, we used a particular serious game, “The Glider,” a prototype developed by Serious Games Finland. It exploits Kinect as its game controller. Serious games are usually aimed at promoting specific effects that can be achieved through playing the game. However, such effects can also present tradeoffs in terms of usability. Controlling motions in “The Glider” has been developed in close cooperation with medical doctors, physiotherapists, and patients in terms of well-being and light exercise. However, modulating the difficulty balance and usability of the game is also a very important aspect of game development. As described in this paper, our goal is to evaluate the control motion of the game from a usability perspective. Evaluation consists of state and appearance observations conducted with 12 Japanese subjects. Observed data were tagged into several categories to quantize their behaviours and speeches, and were analysed using both cognitive bias evaluation and statistical evaluation. Results show that difficulties for subjects in control motions of the game were curved out, which is a key to balancing usability and difficulty while maintaining the genuine purpose of the serious game.


Skeletal motion interface Usability test Kansei engineering Serious game 



The authors thank all participants in the game development, doctors and physiotherapists in Carea, developers at Serious Games Finland, and students who helped us conduct experiments. Additionally, we would like to thank Mr. Fusao Sato, Director Hiroshi Yoshimura from the Sendai—Finland Well-being Center, and professors of the Sendai National College of Technology for their fruitful cooperation.


  1. 1.
    Nagamachi M (1995) Kansei engineering: a new ergonomic consumer-oriented technology for product development. Int J Ind Ergon 15(1):3–11MathSciNetCrossRefGoogle Scholar
  2. 2.
    Pranav M, Pattie M, Liyan C (2009) WUW-wear Ur world: a wearable gestural interface. In: CHI EA 09 extended abstracts on human factors in computing systems, pp 4111–4116Google Scholar
  3. 3.
  4. 4.
    Choh N et al. (2008) Demonstration of interactive teleolfaction with movie. In: Proc. ACE 08 the 2008 international conference on advances in computer entertainment technology, pp 395–395Google Scholar
  5. 5.
    Allan O, Regina BA (2012) Creation and visualization of context aware augmented reality interfaces. In: Proc. AVI 12 the international conference on advanced visual interfaces, pp 324–327Google Scholar
  6. 6.
    Buchenau M, Fulton J (2000) Experience prototyping. In: Symposium on designing interactive systems 2000 (Brooklyn, NY). ACM Press, Neu York, pp 424–433Google Scholar
  7. 7.
    Leo KH, Tan BY (2010) User-tracking mobile floor projection virtual reality game system for paediatric gait and dynamic balance training. In: Proc. iCREATE10 the 4th international convention on rehabilitation engineering and assistive technology (article no. 25)Google Scholar
  8. 8.
    Theng YL, Chua PH, Pham TP (2012) Wii as entertainment and socialization aids for mental and social health of elderly. In: Proc. CHI12 extended abstracts. ACM, New York, pp 691–702Google Scholar
  9. 9.
    Matsuguma H, Fujioka S, Nakajima A, Kaneko K (2011) Standing growing blooming trees rehabilium-serious game for rehabilium. In: Asia digital art and design associationGoogle Scholar
  10. 10.
    Biswas KK, Basu SK (2011) Gesture recognition using microsoft Kinect. In: Proc. 5th international conference on automation, robotics and applications (ICARA)Google Scholar
  11. 11.
    Thibaut W, Sofien B, Hao L, Mark P (2011) Kinect-based facial animation. In: Proc. SA 11 SIGGRAPH Asia 2011 emerging technologies (article no. 1)Google Scholar
  12. 12.
    Wu D, Zhu F, Shao L, Zhang H (2012) One shot learning gesture recognition with Kinect sensor. In: Proc. MM 12 the 20th ACM international conference on multimedia, pp 1303–1304Google Scholar
  13. 13.
    Chao S, Bing-Kun B, Changsheng X, Tao M (2012) Kinect-based visual communication system. In: Proc. the 4th international conference on internet multimedia computing and service, pp 55–59Google Scholar
  14. 14.
    Xia L, Chen CC, Aggarwal JK (2011) Human detection using depth information by Kinect. In: International workshop on human activity understanding from 3D data in conjunction with CVPR (HAU3D). Colorado Springs COGoogle Scholar
  15. 15.
    Huotari K, Hamari J (2012) Defining gamification a service marketing perspective. In: Proc. the 16th international academic MindTrek conference, pp 17–22Google Scholar
  16. 16.
    Burke JW et al. (2010) Designing engaging, playable games for rehabilitation. In: International conference series on disability, virtual reality and associated technologies (ICDVRAT)Google Scholar
  17. 17.
    Chang YJ, Cheng SF, Huang JD (2011) A kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. Res Dev Disabil 32:2566–2570CrossRefGoogle Scholar
  18. 18.
    Anderson F, Annett M, Bischof WF (2010) Lean on Wii: physical rehabilitation with virtual reality and wii peripherals. Ann Rev CyberTherapy Telemed 8:181–184Google Scholar
  19. 19.
    Pyae A, Yuen TB, Gossage M (2013) Understanding stroke patients needs for designing user-centered rehabilitative games. In: Proc. the 7th annual international conference on computer games multimedia and allied technologies, pp 151–156Google Scholar
  20. 20.
    Shiizuka H, Hashizume A (2011) The role of Kansei/affective engineering and its expected in aging society. In: Watada J et al (eds) Intelligent decision technologies, SIST10. Springer, Berlin, pp 329–339Google Scholar
  21. 21.
    KEIO Univ. Fukuda Lab, Ergonomics guide- the way to study Kansei scientificaly, Scientistsya, p 354 (ISBN 10-4860790367)Google Scholar
  22. 22.
    Kaiho H, Harada E, Protocol analysis what would you read from speech data, Sinyosya, p 249 (ISBN 10-4788504693)Google Scholar
  23. 23.
    Csapo A, Baranyi P (2012) A unified terminology for the structure and semantics of CogInfoCom channels. Acta Polytech Hungar 9(1):85105Google Scholar
  24. 24.
    Levy L, Novak J (2009) Game development essentials QA & Testing. Course Technology. ISBN 10-1435439473Google Scholar
  25. 25. Accessed 27 Aug 2015

Copyright information

© OpenInterface Association 2015

Authors and Affiliations

  • Akihito Nakai
    • 1
    Email author
  • Aung Pyae
    • 2
    • 5
  • Mika Luimula
    • 2
  • Satoshi Hongo
    • 3
  • Hannu Vuola
    • 4
  • Jouni Smed
    • 5
  1. 1.Research Institute of Electrical CommunicationTohoku UniversitySendaiJapan
  2. 2.Turku University of Applied ScienceTurkuFinland
  3. 3.Sendai National College of TechnologyNatoriJapan
  4. 4.Serious Games FinlandHelsinkiFinland
  5. 5.University of TurkuTurkuFinland

Personalised recommendations