Multimedia Tools and Applications

, Volume 77, Issue 16, pp 21329–21370 | Cite as

Extracting intermediate-level design knowledge for speculating digital–physical hybrid alternate reality experiences

  • Fumiko Ishizawa
  • Mizuki Sakamoto
  • Tatsuo NakajimaEmail author


This paper reports a process to derive intermediate-level knowledge as a service design and analysis framework for designing digital services to offer alternate reality experiences, and analyzes the possible opportunities and pitfalls of the framework. The user experience felt by refining the meaning of real space through virtuality is defined as alternate reality experiences. Alternate reality experiences are typically achieved by modifying our eyesight or replacing our five senses to others, and they make our world interactive by implicitly influencing human attitudes and behaviors. First, the paper extracts observations for deriving the intermediate-level knowledge through the discussions raised in exploration workshops. In the workshops, the three digital services that utilize diverse strategies to offer alternate reality experiences are chosen. The workshops’ main focus is to examine how a person could have a sense of values in alternate reality experiences via the three digital services. Second, the paper shows how to derive the proposed service design and analysis framework from the extracted observations through expert analysis, then an overview of the framework is explained. Finally, the paper presents feasibility analysis of the proposed framework through a new digital service named Mindful Reminder as a case study for refining the service through focus group discussions. The approach described in the paper is to report a concrete process through which extracted observations can be converted into intermediate-level knowledge that can be used to design alternate reality experiences. Traditionally, the process for generating intermediate-level knowledge has not been well-documented; however, documenting the process is very important in theorizing the design of alternate reality experiences and helps effectively develop a variety of emerging advanced digital services that will offer alternate reality experiences in the future.


Service design Augmented reality Virtual reality Alternate reality experiences Research through design Design fiction Intermediate-level design knowledge 


  1. 1.
    Akasaki H, Ishizawa F, Sakamoto M, Nakajima T (2017) Building a Platform Society towards Sustainability based on Internet-of-Things. In: Proceedings of the 5th International Conference on Distributed, Ambient and Pervasive InteractionsGoogle Scholar
  2. 2.
    Akkil D, Lucero A, Kangas J, Jokela T, Salmimaa M, Raisamo R (2016) User Expectations of Everyday Gaze Interaction on Smartglasses. In: Proceeding of the 9th Nordic Conference on Human-Computer InteractionGoogle Scholar
  3. 3.
    Allison A (2006) Millennial Monsters: Japanese Toys and the Global Imagination. University of California Press, BerkeleyGoogle Scholar
  4. 4.
    Alexander C (1977) A Pattern Language: Towns, Buildings, Construction. Oxford University Press, OxfordGoogle Scholar
  5. 5.
    Bardzell J, Bardzell S (2013) What is critical about critical design? Proceedings of the SIGCHI conference on human factors in computing systemsGoogle Scholar
  6. 6.
    Baudrillard J (1998) The Consumer Society: Myths and Structures. Sage Publications Ltd., Thousand OaksGoogle Scholar
  7. 7.
    Bhaskar M (2016) Curation: The Power of Selection in a World of Excess. Piatkus, LondonGoogle Scholar
  8. 8.
    Blythe B (2014) Research Through Design Fiction: Narrative in Real and Imaginary Abstracts. Proceedings of the International Conference on Human factors in computing systems, InCrossRefGoogle Scholar
  9. 9.
    Boyd LB, Wilson SR, Pennebaker JW, Kosinski M, Stillwell DJ, Mihalcea R (2015) Values in Words: Using Language to Evaluate and Understand Personal Values. In: Proceedings of the 9th International AAAI Conference on Web and Social Media, p 31-40Google Scholar
  10. 10.
    Cassone VI (2017) Through the Ludic Glass. A Cultural Genealogy of Gamification. In: Proceedings of the 21th International Academic Mindtrek Conference, Tampere, Finland, 20–21 September 2017Google Scholar
  11. 11.
    Colley A, Väyrynen J, Häkkilä J (2015) Skiing in a blended virtuality: an in-the-wild experiment. In: Proceeding of the 19th International Academic Mindtrek ConferenceGoogle Scholar
  12. 12.
    Corbin JM (2008) Basics of Qualitative Research: Techniques ad Procedures for developing Grounded Thoery. Sage, Newcastle upon TyneCrossRefGoogle Scholar
  13. 13.
    Coulton P, Burnett D, Gradinar A (2016) Games as Speculative Design: Allowing Players to Consider Alternate Presents and Plausible Futures. In: Proceedings of the International Conference on DRS2016: Future-Focused ThinkingGoogle Scholar
  14. 14.
    Curedale R (2016) Affinity Diagrams: The tool to tame complexity. Design Community College Inc., Los AngelesGoogle Scholar
  15. 15.
    Dalsgaard P, Dindler C (2014) Between Theory and Practice: Bridging Concepts in HCI Research. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘14)Google Scholar
  16. 16.
    Desmet PMA, Pohlmeyer AE (2013) Positive Design: An Introduction to Design for Subjective Well-Being. Int J Des 7(3):5–19Google Scholar
  17. 17.
    Deterding S, Dixon D, Khaled R, Nacke N (2011) From Game Design Elements to Gamefulness: Defining "Gamification. In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, pp. 9-15Google Scholar
  18. 18.
    Dunne A (2006) Hertzian Tales: Electronic Products, Aesthetic Experience and Critical Design. MIT Press, CambridgeGoogle Scholar
  19. 19.
    Dunne A, Raby F (2013) Speculative Everything: Design, Fiction, and Social Dreaming. MIT Press, CambridgeGoogle Scholar
  20. 20.
    Eaton G, Ormerod TC (2001) Expert/Novices Differences in Case Analysis, RP0303. Lancaster University, LancasterGoogle Scholar
  21. 21.
    Forlano L, Mathew A (2014) From Design Fiction to Design Friction: Speculative and Participatory Design of Values-Embedded Urban Technology. J Urban Technol 21(4):7–24CrossRefGoogle Scholar
  22. 22.
    Forlizzi J, Zimmerman J (2013) Promoting Service Design as a Core Practice in Interaction Design. In: Proceeding of the 5th IASDR World Conference on Design ResearchGoogle Scholar
  23. 23.
    Gaver WW (1991) Technology affordances. In: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsGoogle Scholar
  24. 24.
    Gaver WW, Beaver J, Benford S (2003) Ambiguity as a resource for design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘03)Google Scholar
  25. 25.
    Gaver WW, Bowers J (2012) Annotated portfolios. Interactions 19(4):40–49CrossRefGoogle Scholar
  26. 26.
    Godin D, Zahedi M (2014) Aspects of Research through Design: A Literature Review. In: Proceedings of DRS 2014: Design’s Big DebatesGoogle Scholar
  27. 27.
    Gonzatto RF, van Amstel FMC, Merkle LE, Hartmann T (2013) The Ideology of the Future in Design Fiction. Digit Creat 24(1):36–45CrossRefGoogle Scholar
  28. 28.
    Gushima K, Akasaki H, Nakajima T (2017) Ambient Bot: Delivering Daily Casual Information through Eye Contact with an Intimate Virtual Creature. In: Proceedings of the 21th International Academic Mindtrek Conference, Tampere, Finland, 20–21 September 2017Google Scholar
  29. 29.
    Hekler EB, Klasnja P, Froehlich JE, Buman MP (2013) Mind the Theoretical Gus: Interpreting, Using and Developing Behavior Theory in HCI Research. In: Proceedings of the ACM SIGCHI conference on Human Factors i Computing Systems, pp 3307-3316Google Scholar
  30. 30.
    Ho H, Saeedi E, Kim SS, Shen TT, Parviz BA (2008) Contact lens with integrated inorganic semiconductor devices. In: Proceedings of the IEEE 21st International Conference on Micro Electro Mechanical Systems, 2008. MEMS 2008, pp 403-406Google Scholar
  31. 31.
    Höök K, Löwgren J (2012) Strong Concepts: Intermediate-Level Knowledge in Interaction Design Research. ACM Trans Comput Hum Interact 19(3).
  32. 32.
    Ikeuchi K, Otsuka T, Yoshii A, Sakamoto M, Nakajima T (2014) KinecDrone: Enhancing Somatic Sensation to Fly in the Sky with Kinect and AR.Drone. In: Proceedings of the 5th Augmented Human International ConferenceGoogle Scholar
  33. 33.
    Ikeuchi K, AlSada M, Nakajima T (2015) Providing Ambient Information as Comfortable Sound for Reducing Cognitive Overload. In: Proceedings of the 12th International Conference on Advances in Computer Entertainment TechnologyGoogle Scholar
  34. 34.
    Imagineers (2010) Walt Disney Imagineering. Disney Book Group, GlendaleGoogle Scholar
  35. 35.
    Ishizawa F, Nakajima T (2016) Alternative Reality: An Augmented Daily Urban World Inserting Virtual Scenes Temporally. In: Proceedings of the 10th International Conference on Ubiquitous Computing & Ambient IntelligenceGoogle Scholar
  36. 36.
    Ishizawa F, Nakajima T (2017) Guiding Human Behavior through Enhancing the Meaning of Real Spaces, In: Proceedings of the 6th International Conference on Design, User Experience and UsabilityGoogle Scholar
  37. 37.
    Kim H, Lee W (2014) Everyday Design as a Design Resource. Int J Des 8(11):1–11Google Scholar
  38. 38.
    Kirby D (2010) The Future is Now: Diegetic Prototypes and the Role of Popular Films in Generating Real-world Technological Development. Soc Stud Sci 40(1):41–70CrossRefGoogle Scholar
  39. 39.
    Lawson B (2005) How Designers Think: The Design Process Demystified, Fourth Edition. Architectural Press, New YorkGoogle Scholar
  40. 40.
    Leitner M, Subasi Ö (2016) Arty Portfolios: Manifesting Artistic Work in Interaction Design Research. In: Proceeding of the 9th Nordic Conference on Human-Computer InteractionGoogle Scholar
  41. 41.
    Lindley J, Coulton P (2016) Pushing the Limits of Design Fiction: The Case For Fictional Research Papers, In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp 4032-4033Google Scholar
  42. 42.
    Marshall P, Hornecker E (2013) Theories of Embodiment in HCI. In: The SAGE Handbook of Digital Technology Research, pp 144–158.
  43. 43.
    Montola M, Stemros J, Waern A (2009) Pervasive Games - Theory and Design. Morgan Kaufmann, BurlingtonGoogle Scholar
  44. 44.
    Nakajima T, Lehdonvirta V (2013) Designing Motivation in Persuasive Ambient Mirrors. Pers Ubiquit Comput 17(1):107–126CrossRefGoogle Scholar
  45. 45.
    Narumi T, Ban Y, Kajinami T, Tanikawa T, Hirose M (2012) Augmented Perception of Satiety: Controlling Food Consumption by Changing Apparent Size of Food with Augmented Reality. In: Proceedings of the Conference on Human Factors in Computing SystemsGoogle Scholar
  46. 46.
    Niedderer K, Ludden G, Clune SJ, Lockton D, Mackrill J, Morris A, Cain R, Gardiner E, Evans M, Gutteridge R, Hekkert PP (2016) Design for Behaviour Change as a Driver for Sustainable Innovation: Challenges and Opportunities for Implementation in the Private and Public Sectors. Int J Des 10(2):67–85Google Scholar
  47. 47.
    Norman D (2013) The Design of Everyday Things: Revised and Expanded Edition. Basic Books, New YorkGoogle Scholar
  48. 48.
    Ostrom E (1990) Governing the Commons: The Evolution of Institutions for Collective Action. CUP, CambridgeCrossRefGoogle Scholar
  49. 49.
    Paul CL (2008) A Modified Delphi Approach to a New Card Sorting Methodology. J Usability Stud 4(1):7–30Google Scholar
  50. 50.
    Saffer D (2005) The Role of Metaphor in Interaction Design. The School of Design, Carnegie Mellon University, PittsburghGoogle Scholar
  51. 51.
    Sakamoto M, Nakajima T, Alexandrova T (2015) Enhancing Values through Virtuality for Intelligent Artifacts that Influence Human Attitude and Behavior. Multimed Tools Appl, Springer 74(24):11537–11568CrossRefGoogle Scholar
  52. 52.
    Sakamoto M, Nakajima T (2015) Incorporating Fictionality into the Real World with Transmedia Storytelling. In: Proc. of the International Conference on Design, User Experience and UsabilityGoogle Scholar
  53. 53.
    Sakamoto M, Alexandrova T, Nakajima T (2016) Analyzing the Influence of Virtuality on Playful Social Interaction. Multimed Tools Appl, Springer 75(14):8289–8317CrossRefGoogle Scholar
  54. 54.
    Sakamoto M, Nakajima T, Akioka S (2017) Gamifying Collective Human Behavior with Gameful Digital Rhetoric. Multimed Tools Appl 76(10):12539–12581CrossRefGoogle Scholar
  55. 55.
    Sakamoto M, Nakajima T (2016) Making Citizens’ Activities Flourish through a Crowdsourcing-based Social Infrastructure. In: Konomi S, Rousso G (eds) Enriching Urban Spaces with Ambient Computing, the Internet of Things, and Smart City Design. IGI Global, HersheyGoogle Scholar
  56. 56.
    Salovaara A, Oulasvirta A, Jacucci G (2017) Evaluation of Prototypes and the Problem of Possible Future. In: Proceedings of international conference on Human Factors in Computing Systems 2017Google Scholar
  57. 57.
    Sanchez-Vives MV, Slater M (2005) From presence to consciousness through virtual reality. Nat Rev Neurosci 6(4):332–339CrossRefGoogle Scholar
  58. 58.
    Schouten B, Ferri G, de Lange M, Millenaar K (2016) Games as Strong Concepts for City-Making. In: Nijholt A (ed) Playable Cities: The City as a Digital Playground. Springer, BerlinGoogle Scholar
  59. 59.
    Schwalk M, Maier T (2016) Effect of Pressure and Vibration stimuli on the usability of Human-Machine Systems. In: Deml B, Stock P, Bruder R, Schlick CM (eds) Advanced in Ergonomic Design of Systems, Products and Processes. Springer, BerlinGoogle Scholar
  60. 60.
    Sengers P, Gaver WW (2006) Staying open to interpretation: engaging multiple meanings in design and evaluation. In: Proceedings of the 6th conference on Designing Interactive systems (DIS ‘06)Google Scholar
  61. 61.
    Stickdorn M, Schneider J (2012) This is Service Design Thinking: Basics, Tools, Cases. Wiley, HobokenGoogle Scholar
  62. 62.
    Spence C, Okajima K, Cheok AD, Petit O, Michel C (2016) Eating with Our Eyes: From Visual Hunger to Digital Satiation. Brain Cogn 110:53–63CrossRefGoogle Scholar
  63. 63.
    Suzuki E, Narumi T, Sakurai S, Tanikawa T, Hirose, M (2014) Illusion Cup: Interactive Controlling of Beverage Consumption Based on an Illusion of Volume Perception. In: Proceedings of the 5th Augmented Human International ConferenceGoogle Scholar
  64. 64.
    Suzuki K, Wakisaka S, Fujii N (2012) Substitutional Reality System: A Novel Experimental Platform for Experiencing Alternative Reality. Sci Rep 2(459).
  65. 65.
    Takahashi M, Irie K, Sakamoto M, Nakajima T (2015) Incorporating Fictionality into the Real Space: A Case of Enhanced TCG. In: Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable ComputersGoogle Scholar
  66. 66.
    Treanor M, Schweizer B, Bogost I, Mateas M (2011) Proceduralist Readings: How to Find Meaning in Games with Graphical Logics. In: Proceedings of Foundations of Digital GamesGoogle Scholar
  67. 67.
    Weick KE, Sutcliffe KM, Obstfeld D (2005) Organizing and the Process of Sensemaking. Organ Sci 16(4):409–421CrossRefGoogle Scholar
  68. 68.
    Wolfe AK, Malone EL, Heerwagen J, Dion J (2014) Behavioral Change and Building Performance: Strategies for Significant, Persistent, and Measurable Institutional Change. US Department of EnergyGoogle Scholar
  69. 69.
    Yamabe T, Nakajima T (2013) Playful Training with Augmented Reality Games: Case Studies Toward Reality-oriented System Design. Multimed Tools Appl 62(1):259–286CrossRefGoogle Scholar
  70. 70.
    Zimmerman J, Forlizzi J, Evenson S (2007) Research through Design as a Method for Interaction Design Research in HCI. Proceedings of the SIGCHI conference on Human factors in computing systemsGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Fumiko Ishizawa
    • 1
  • Mizuki Sakamoto
    • 1
  • Tatsuo Nakajima
    • 1
    Email author
  1. 1.Department of Computer Science and EngineeringWaseda UniversityTokyoJapan

Personalised recommendations