Multimedia Tools and Applications

, Volume 62, Issue 1, pp 259–286 | Cite as

Playful training with augmented reality games: case studies towards reality-oriented system design

  • Tetsuo YamabeEmail author
  • Tatsuo Nakajima


In this paper, we propose a reality-oriented augmentation approach to support training activities. The approach aims at adding new value and playful features to traditional training environments with keeping their original look-and-feel. For example, a game monitoring service enables to automatically record game events so that players can review a gaming process and strategy for soul-searching, or replay most impressive scenes to share the experience with others after the game finishes. Even several services are running on background, digital devices and services are seamlessly integrated to the game environment in unobtrusive way so that players can concentrate on training as usual. The concept can be applied to both traditional games (e.g., poker and the game of Go) and non-gaming activities (e.g., calligraphy and drumming). We developed four case studies on the concept: Augmented Reality Go, EmoPoker, Augmented Calligraphy and AR Drum Kit. We discuss design issues in the reality-oriented augmentation process based on user study results.


Augmented reality Gamification Tangible interaction Training support Design guideline 


  1. 1.
    Al-Shihabi T, Mourant RR (2001) A framework for modeling human-like driving behaviors for autonomous vehicles in driving simulators. In: Proceedings of the fifth international conference on autonomous agents (AGENTS ’01)Google Scholar
  2. 2.
    Azuma RT (1997) A survey of augmented reality. In: Presence: teleoperators and virtual environments, pp 355–385Google Scholar
  3. 3.
    Bouchner P, Novotny S (2011) Development of advanced driving simulator: steering wheel and brake pedal feedback. In: Proceedings of the 2nd international conference on circuits, systems, control, signals (CSCS ’11)Google Scholar
  4. 4.
    Cao Y, v.d.Sluis F, Theune M, Rieks o.d. Akker, Nijholt, A (2010) Evaluating informative auditory and tactile cues for in-vehicle information systems. In: Proceedings of the 2nd international conference on automotive user interfaces and interactive vehicular applications (AutomotiveUI ’10)Google Scholar
  5. 5.
    Cooper N, Keatley A, Dahlquist M, Mann S, Slay H, Zucco J, Smith R, Thomas B (2004) Augmented reality Chinese checkers. In: ACE ’04: proceedings of the 2004 ACM SIGCHI international conference on advances in computer entertainment technologyGoogle Scholar
  6. 6.
    Egglestone SR, Walker B, Marshall J, Benford S, McAuley D (2011) Analysing the playground: sensitizing concepts to inform systems that promote playful interaction. Hum-Comput Interact 6946:452–469 (2011)Google Scholar
  7. 7.
    Endsley MR (1995) Toward a theory of situation awareness in dynamic systems. Human Factors: The Journal of the Human Factors and Ergonomics Society (HFS) 37(1):32–64CrossRefGoogle Scholar
  8. 8.
    Floerkemeier C, Mattern F (2006) Smart playing cards – enhancing the gaming experience with RFID. In: Proceedings of the third international workshop on pervasive gaming applications - PerGames 2006 at PERVASIVE 2006, pp 27–36Google Scholar
  9. 9.
    Froehlich P, Schatz R, Leitner P, Mantler S, Baldauf M (2010) Augmenting the driver’s view with realtime safety-related information. In: Proceedings of the 1st augmented human international conference (AH ’10)Google Scholar
  10. 10.
    Gorry GA, and Scott Morton MS (1971) A framework for management information systems. Sloan Manage Rev 13(1):21–36Google Scholar
  11. 11.
    Hetzner S, Steiner CM, Dimitrova V, Brna P, Conlan O (2011) Adult self-regulated learning through linking experience in simulated and real world: a holistic approach. In: Proceedings of the 6th European conference on technology enhanced learning: towards ubiquitous learning (EC-TEL’11), pp 166–180Google Scholar
  12. 12.
    Hinske S, Langheinrich M (2009) W41K: digitally augmenting traditional game environments. In: TEI ’09: proceedings of the 3rd international conference on tangible and embedded interactio. ACM, pp 99–106Google Scholar
  13. 13.
    Holland S, Bouwer AJ, Dalgelish M, Hurtig TM (2010) Feeling the beat where it counts: fostering multi-limb rhythm skills with the haptic drum kit. In: TEI ’10: proceedings of the fourth international conference on tangible, embedded, and embodied interaction. ACM Request PermissionsGoogle Scholar
  14. 14.
    Hsiao KF, Chen NS (2011) The development of the AR-fitness system in education. In: Proceedings of the 6th international conference on E-learning and games, edutainment technologies (Edutainment’11), pp 2–11Google Scholar
  15. 15.
    Huang K, Do EYL, Starner T (2008) PianoTouch: a wearable haptic piano instruction system for passive learning of piano skills. In: ISWC ’08: proceedings of the 2008 12th IEEE international symposium on wearable computers. IEEE Computer SocietyGoogle Scholar
  16. 16.
    Iimura T (2002) A note for MA: space/time in the garden of Ryoan-Ji. Millenn Film J 38:51–63Google Scholar
  17. 17.
    Ishii H, Wisneski C, Orbanes J, Chun B, Paradiso J (1999) PingPongPlus: design of an athletic-tangible interface for computer-supported cooperative play. In: CHI’99: proceedings of the SIGCHI conference on human factors in computing systems: the CHI is the limit, pp 394–401Google Scholar
  18. 18.
    Johnson D, Wiles J (2003) Effective affective user interface design in games. Ergonomics 46(13 & 14):1332–1345CrossRefGoogle Scholar
  19. 19.
    Kallinen K, Salminen M, Ravaja N, Yanev K (2009) Psychophysiological responses to online poker game. In: IADIS’09: proceedings of game and entertainment technologies, pp 35–43Google Scholar
  20. 20.
    Kappel K, Grechenig T (2009) Show-me: water consumption at a glance to promote water conservation in the shower. In: Proceedings of the 4th international conference on persuasive technology (Persuasive ’09)Google Scholar
  21. 21.
    Kim S, Dey AK, Lee J, Forlizzi J (2011) Usability of car dashboard displays for elder drivers. In: Proceedings of the 2011 annual conference on human factors in computing systems (CHI ’11)Google Scholar
  22. 22.
    Kim T, Hong H, Magerko B (2010) Design requirements for ambient display that supports sustainable lifestyle. In: Proceedings of the 8th ACM conference on designing interactive systems (DIS ’10)Google Scholar
  23. 23.
    Lam H (2008) A framework of interaction costs in information visualization. IEEE Trans Vis Comput Graph 14(6):1149–1156CrossRefGoogle Scholar
  24. 24.
    Lee M, Kim JG (2009) Effects of heightened sensory feedback to presence and arousal in virtual driving simulators. In: Proceedings of the 8th international conference on virtual reality continuum and its applications in industry (VRCAI ’09)Google Scholar
  25. 25.
    Lee S, Lim Y, Lee K (2011) A long-term study of user experience towards interaction designs that support behavior change. In: Proceedings of the 2011 annual conference extended abstracts on human factors in computing systems (CHI EA ’11)Google Scholar
  26. 26.
    Liu Y, Lehdonvirta V, Alexandrova T, Nakajima T (2011) Drawing on mobile crowds via social media. ACM / Springer Multimedia Systems Journal (in press, SCI/EI, IF=1.176)Google Scholar
  27. 27.
    Magerkurth C, Cheok A, Mandryk R, Nilsen T (2005) Pervasive games: bringing computer entertainment back to the real world. In: Computers in Entertainment 3(3):4–4Google Scholar
  28. 28.
    Medenica Z, Kun AL, Paek T, Palinko O (2011) Augmented reality vs. street views: a driving simulator study comparing two emerging navigation aids. In: Proceedings of the 13th international conference on human computer interaction with mobile devices and services (MobileHCI ’11)Google Scholar
  29. 29.
    Milgram P, Takemura H, Utsumi A, Kishino F (1995) Augmented reality: a class of displays on the reality-virtuality continuum. In: Proceedings of the SPIE conference on telemanipulator and telepresence technologies, pp 282–292Google Scholar
  30. 30.
    Motokawa Y, Saito H (2006) Support system for guitar playing using augmented reality display. In: 2006 IEEE/ACM international symposium on mixed and augmented reality. IEEE, pp 243–244Google Scholar
  31. 31.
    Nacke L, Lindley C, Stellmach S (2008) Log who’s playing: psychophysiological game analysis made easy through event logging. In: Proceedings of the 2nd international conference on fun and gamesGoogle Scholar
  32. 32.
    Nakajima T, Lehdonvirta V (2011) Designing motivation using persuasive ambient mirrors. In: Personal and ubiquitous computing, pp 1–20Google Scholar
  33. 33.
    Olstam JJ, Lundgren J, Adlers M, Matstoms P (2008) A framework for simulation of surrounding vehicles in driving simulators. ACM Trans Model Comput Simul (TOMACS) 18(3):9:1–9:24Google Scholar
  34. 34.
    O’Regan JK, Noë A (2001) A sensorimotor account of vision and visual consciousness. Behav Brain Sci 24(5):939–73; discussion 973–1031Google Scholar
  35. 35.
    Power DJ (2008) A brief history of decision support systems. In: Handbook on decision support systems, vol 1, pp 121–140Google Scholar
  36. 36.
    Rasmussen J (1983) Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE Trans Syst Man Cybern 13(3):257–266CrossRefGoogle Scholar
  37. 37.
    Sadakata M, Hoppe D, Brandmeyer A, Timmers R, Desain P (2008) Real-time visual feedback for learning to perform short rhythms with expressive variations in timing and loudness. J New Music Res 37(14):207–220CrossRefGoogle Scholar
  38. 38.
    Schwarz N (2000) Emotion, cognition, and decision making. Cogn Emot 14(4):433–440CrossRefGoogle Scholar
  39. 39.
    Shirazi AS, Döring T, Parvahan P, Ahrens B, Schmidt A (2009) Poker surface: combining a multi-touch table and mobile phones in interactive card games. In: Proceedings of the 11th international conference on human-computer interaction with mobile devices and services (MobileHCI ’09)Google Scholar
  40. 40.
    Soga M, Nishino T, Taki H (2011) Proposal and development of motion navigator enabling learners to observe expert’s motion from expert’s viewpoint by augmented reality. In: Proceedings of the 15th international conference on knowledge-based and intelligent information and engineering systems, vol part III (KES’11), vol part III, pp 40–48Google Scholar
  41. 41.
    Varoudis T (2011) Ambient displays: influencing movement patterns. In: Proceedings of the 13th IFIP TC 13 international conference on human-computer interaction, vol part IV (INTERACT’11)Google Scholar
  42. 42.
    Wilfinger D, Meschtscherjakov A, Murer M, Tscheligi M (2010) Influences on user acceptance: informing the design of eco-friendly in-car interfaces. In: Proceedings of the 2nd international conference on automotive user interfaces and interactive vehicular applications (AutomotiveUI ’10)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.63-505 Faculty of Science and EngineeringWaseda UniversityTokyoJapan

Personalised recommendations