Cultural Computing – How to Investigate a Form of Unconscious User Experiences in Mixed Realities

  • Matthias Rauterberg
  • Jun Hu
  • Geert Langereis
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 333)


This paper presents a new direction of research in user experiences and cognitive science. The problem addressed is drawing on results from different disciplines: psychology, brain and cognitive sciences, physics, and interaction design. As main objective we plan the empirical validation of the claim: Cultural computing as enabling technology for social transformations. Cultural computing is based on a form of cultural translation that uses scientific methods to capture and represent essential aspects of a particular culture. Cultural computing will enable particular cognitive and emotional responses from users as reflections on and of their inner, subliminal consciousness. Cultural computing is not only integrating cultural aspects into the interaction but also allowing the user to experience an interaction that is closely related to the core aspects of his/her own culture. As such it is important to understand one’s cultural determinants and how to render them during the interaction. We will address individually and collectively the cultural determinants of the Western culture. Based on the given narrative ‘Alice Adventures in Wonderland’ we have already built the first demo version and plan to build an optimized version of a mixed reality installation to provide and investigate cultural user experiences. To address the research questions we propose three main research lines: (1) designing a mixed reality environment to provide certain unconscious user experiences; (2) inducing and measuring the changes in the individual’s unconscious knowledge structure; and (3) empirical validation of a fundamentally new synchronization mechanism for sharing individual changes collectively. We hypothesize that there eventually exists an interconnecting knowledge field at the foundation of reality that conserves and conveys information collectively. This knowledge field looks like a possible candidate for a required ‘supra-natural’ memory for cultural knowledge to tap into for social transformations.


cultural computing mixed reality social transformation user experience unconsciousness 


  1. 1.
    Nakatsu, R., Rauterberg, M.: Entertainment computing: Inaugural Editorial. Entertainment Computing 1(1), 1–7 (2009)CrossRefGoogle Scholar
  2. 2.
    Nijholt, A., Bos, D.P.-O., Reuderink, B.: Turning shortcomings into challenges: Brain-computer interfaces for games. Entertainment Computing 1(2), 85–94 (2009)CrossRefGoogle Scholar
  3. 3.
    Tosa, N., Matsuoka, S., Thomas, H.: Inter-culture computing: ZENetic computer. In: Elliott-Famularo, H. (ed.) ACM SIGGRAPH 2004 Emerging Technologies, p. 11. ACM, Los Angeles (2004)CrossRefGoogle Scholar
  4. 4.
    Tosa, N.: Expression of emotion, unconsciousness with art and technology. In: Hatano, G., Okada, N., Tanabe, H. (eds.) Affective Minds, pp. 183–205. Elsevier, Amsterdam (2000)Google Scholar
  5. 5.
    Hu, J., Bartneck, C.: Culture matters: A study on presence in an interactive movie. Cyberpsychology and Behavior 11(5), 529–536 (2008)CrossRefGoogle Scholar
  6. 6.
    Tosa, N.: Unconscious flow. Leonardo 33(5), 442 (2000)CrossRefGoogle Scholar
  7. 7.
    Tosa, N., Matsuoka, S.: Recreating our selves: ZENetic computer. In: International Conference on Information Visualisation - IV 2003, pp. 614–618. IEEE Computer Society, Los Alamitos (2003)CrossRefGoogle Scholar
  8. 8.
    Cheok, A.D., et al.: Confucius computer: Transforming the future through ancient philosophy. In: ACM SIGGRAPH 2008 New Tech Demos. ACM, Los Angeles (2008)Google Scholar
  9. 9.
    Tosa, N.: Cultural Computing: ZENetic Computer. In: Intelligent Agent 6.1.1, Kyoto (2006)Google Scholar
  10. 10.
    Nisbett, R.E., et al.: Culture and systems of thought: Holistic versus analytic cognition. Psychological Review 108(2), 291–310 (2001)CrossRefGoogle Scholar
  11. 11.
    Nisbett, R.E., Masuda, T.: Culture and point of view. Proceedings National Academy of Sciences 100(19), 11163–11170 (2003)CrossRefGoogle Scholar
  12. 12.
    Rauterberg, M.: Ambient culture: A possible future for entertainment computing. In: Lugmayr, A., Golebiowski, P. (eds.) Interactive TV: a shared experience - Adjunct Proceedings of EuroITV-2007, pp. 37–39. Tampere International Center for Signal Processing, Tampere (2007)Google Scholar
  13. 13.
    Rauterberg, M.: How to assess the user’s experience in cultural computing. In: Bosenick, T. (ed.) Usability Professionals, pp. 12–17. Fraunhofer Informationszentrum Raum und Bau, Gelsenkirchen (2006)Google Scholar
  14. 14.
    Rauterberg, M.: Usability in the future – explicit and implicit effects in cultural computing. In: Heinecke, A.M., Paul, H. (eds.) Mensch und Computer im Strukturwandel, pp. 29–36. München, Oldenbourg (2006)Google Scholar
  15. 15.
    Peng, K., Nisbett, R.E., Wong, N.Y.C.: Validity problems comparing values across cultures and possible solutions. Psychological Methods 2(4), 329–344 (1997)CrossRefGoogle Scholar
  16. 16.
    Merikle, P.M., Daneman, M.: Memory for unconsciously perceived events: Evidence from anesthetized patients. Consciousness and Cognition 5, 525–541 (1996)CrossRefGoogle Scholar
  17. 17.
    van den Noort, M., Hugdahl, K., Bosch, P.: Human machine interaction: The special role for human unconscious emotional information processing. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, pp. 598–605. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  18. 18.
    Drascic, D., Milgram, P.: Perceptual issues in augmented reality. In: Proceedings of SPIE - Stereoscopic Displays and Virtual Reality Systems III, pp. 123–134. The International Society for Optical Engineering, Bellingham (1996)Google Scholar
  19. 19.
    Qin, H., Rau, P.-L., Salvendy, G.: Player immersion in the computer game narrative. In: Ma, L., Rauterberg, M., Nakatsu, R. (eds.) ICEC 2007. LNCS, vol. 4740, pp. 458–461. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  20. 20.
    Milgram, P., Kishino, A.F.: Taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems E77-D(12), 1321–1329 (1994)Google Scholar
  21. 21.
    Busselle, R., Ryabovolova, A., Wilson, B.: Ruining a good story: Cultivation, perceived realism and narrative. Communications 29(3), 365–378 (2004)CrossRefGoogle Scholar
  22. 22.
    Carroll, L.: Alice’s adventures in wonderland. Macmillan, London (1865)Google Scholar
  23. 23.
    ALICE. Project ALICE: Not just another interactive installation (2010),
  24. 24.
    van Aart, J., et al.: Designing for experience: Arousing boredom to evoke predefined user behaviour. In: Desmet, P., et al. (eds.) Proceedings of the 6th Design & Emotion Conference, Hong Kong, pp. 1607–1611 (2008)Google Scholar
  25. 25.
    Schilder, P.: The image and appearance of the human body. International University Press, New York (1950)Google Scholar
  26. 26.
    Kooijmans, T., Rauterberg, M.: Cultural computing and the self concept: Towards unconscious metamorphosis. In: Ma, L., Rauterberg, M., Nakatsu, R. (eds.) ICEC 2007. LNCS, vol. 4740, pp. 171–181. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  27. 27.
    Sobol-Shikler, T., Robinson, P.: Classification of complex information: Inference of co-occuring affective states from their expressions in speech. IEEE Transactions on Pattern Analysis and Machine Intelligence (in Press)Google Scholar
  28. 28.
    Raichle, M.E.: The brain’s dark energy. Science 314, 1249–1250 (2006)CrossRefGoogle Scholar
  29. 29.
    Raichle, M.E.: A brief history of human brain mapping. Trends in Neurosciences 32(2), 118–126 (2008)CrossRefGoogle Scholar
  30. 30.
    Dijksterhuis, A., et al.: On making the right choice: the deliberation-without-attention effect. Science 311, 1005–1007 (2006)CrossRefGoogle Scholar
  31. 31.
    Nakatsu, R., Rauterberg, M., Salem, B.: Forms and theories of communication: From multimedia to Kansei mediation. Multimedia Systems 11(3), 304–312 (2006)CrossRefGoogle Scholar
  32. 32.
    Dijksterhuis, A., Nordgren, L.: A theory of unconscious thought. Perspectives on Psychology 1(2), 95–109 (2006)CrossRefGoogle Scholar
  33. 33.
    Dehaene, S., et al.: Imaging unconscious semantic priming. Nature 395, 597–600 (1998)CrossRefGoogle Scholar
  34. 34.
    Greenwald, A.G., Farnham, S.D.: Using the Implicit Association Test to measure selfesteem and self-concept. Journal of Personality and Social Psychology 79(6), 1022–1038 (2000)CrossRefGoogle Scholar
  35. 35.
    Barakova, E., Laurens, T.: Expressing and interpreting emotional movements in social games with robots. Personal and Ubiquitous Computing (January 16, 2010)Google Scholar
  36. 36.
    Taelman, J., et al.: Contactless EMG sensors for continuous monitoring of muscle activity to prevent musculoskeletal disorders. In: Proceedings of the first Annual Symposium of the IEEE-EMBS Benelux Chapter, pp. 223–226. IEEE Benelux chapter on Engineering in Medicine and Biology Society, Brussels (2006)Google Scholar
  37. 37.
    Ouwerkerk, M., Pasveer, F., Langereis, G.: Unobtrusive sensing of psychophysiological parameters. In: Westerink, J., et al. (eds.) Probing Experience: From Assessment of User Emotions and Behaviour to Development of Products, pp. 163–193. Springer, Dordrecht (2008)Google Scholar
  38. 38.
    REG. Random event generator design (2010),
  39. 39.
    Nelson, R.D., et al.: Correlations of continuous random data with major world events. Foundations of Physics Letters 15(6), 537–550 (2002)CrossRefGoogle Scholar
  40. 40.
    Nelson, R.: The Global Consciousness Project - Part 2. The Golden Thread, 6–10 (August 2002)Google Scholar
  41. 41.
    Nelson, R.: The Global Consciousness Project - Part 3. The Golden Thread, 30–32 (November 2002)Google Scholar
  42. 42.
    Nelson, R.: The Global Consciousness Project - Part 1. The Golden Thread, 8–12 (May 2002)Google Scholar
  43. 43.
    Greenwald, A.G., McGhee, D.E., Schwartz, J.K.L.: Measuring individual differences in implicit cognition: The Implicit Association Test. Journal of Personality and Social Psychology 74(6), 1464–1480 (1998)CrossRefGoogle Scholar
  44. 44.
    Naccache, L., Dehaene, S.: Unconscious semantic priming extends to novel unseen stimuli. Cognition 80, 215–229 (2001)CrossRefGoogle Scholar
  45. 45.
    Noort, M.v.d., Bosch, P., Hugdahl, K.: Understanding the unconscious brain: Evidence for a non-linear information processing. In: Proceedings of the International Conference on Cognitive Systems, pp. 2254–2258 (2005)Google Scholar
  46. 46.
    Di Maggio, P.: Culture and cognition. Annual Review of Sociology 23, 263–287 (1997)CrossRefGoogle Scholar
  47. 47.
    Jung, C.G.: Die Archetypen und das kollektive Unbewußte, Gesammelte Werke von C.G. Jung (Walter 1934)Google Scholar
  48. 48.
    Einstein, A., Podolsky, B., Rosen, N.: Can quantum-mechanical description of physical reality be considered complete? Physical Review 47(10), 777–780 (1935)MATHCrossRefGoogle Scholar
  49. 49.
    Aspect, A., Grangier, P., Roger, G.: Experimental realization of Einstein-Podolsky-Rosen-Bohm Gedankenexperiment: a new violation of Bell’s inequalities. Physical Review Letters 49(2), 91–94 (1982)CrossRefGoogle Scholar
  50. 50.
    Aspect, A.: Quantum mechanics: to be or not to be local. Nature 446, 866–867 (2007)CrossRefGoogle Scholar
  51. 51.
    Woolf, N.J., Hameroff, S.R.: A quantum approach to visual consciousness. Trends in Cognitive Sciences 5(11), 472–478 (2001)CrossRefGoogle Scholar
  52. 52.
    Sheldrake, R.: New science of life: The hypothesis of morphic resonance. Blond & Briggs (1981)Google Scholar
  53. 53.
    Noort, M.v.d.: The unconscious brain - the relative time and information theory of emotions. Citadel, Oegstgeest (2003)Google Scholar
  54. 54.
    Penrose, R.: The emperor’s new mind. Oxford University Press, Oxford (1989)Google Scholar
  55. 55.
    Penrose, R.: Shadows of the mind: A search for the missing science of consciousness. Oxford University Press, Oxford (1994)Google Scholar
  56. 56.
    Hameroff, S., et al.: Conduction pathways in microtubules, biological quantum computation, and consciousness. BioSystems 64, 149–168 (2002)CrossRefGoogle Scholar
  57. 57.
    Hameroff, S.R.: Quantum computation in brain microtubules? The Penrose-Hameroff ‘OrchOR’ model of consciousness. Philosophical Transactions Royal Society London A 356, 1869–1896 (1998)CrossRefMathSciNetGoogle Scholar
  58. 58.
    Penrose, R., Hameroff, S.R.: What gaps? Reply to Grush and Churchland. Journal of Consciousness Studies 2, 98–112 (1995)Google Scholar
  59. 59.
    Hameroff, S.R., Penrose, R.: Conscious events as orchestrated space-time selections. NeuroQuantology 1, 10–35 (2003)Google Scholar
  60. 60.
    Rauterberg, M.: Entertainment computing, social transformation and the quantum field. In: Nijholt, A., Reidsma, D., Hondorp, H. (eds.) Intelligent Technologies for Interactive Entertainment - INTETAIN 2009, pp. 1–8. Springer, Heidelberg (2009)CrossRefGoogle Scholar

Copyright information

© IFIP 2010

Authors and Affiliations

  • Matthias Rauterberg
    • 1
  • Jun Hu
    • 1
  • Geert Langereis
    • 1
  1. 1.Industrial DesignEindhoven University of TechnologyThe Netherlands

Personalised recommendations