Narrative Cognition in Interactive Systems: Suspense-Surprise and the P300 ERP Component

  • Luis Emilio Bruni
  • Sarune Baceviciute
  • Mohammed Arief
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8832)

Abstract

In this article we explore some of the methodological problems related to characterizing cognitive aspects of involvement with interactive narratives using well known EEG/ERP techniques. To exemplify this, we construct an experimental EEG-ERP set-up with an interactive narrative that considers the dialectical relation between suspense and surprise as a function of expectancy, which in turn can be correlated to the P300-ERP component. We address the difficulties of designing a coherent narrative with a suitable level of closure while meeting the requirements of the ERP experimental procedures. We stress the necessity of fine-tuning the highly specific ERP paradigms necessary for the investigation of user experience in interactive narratives and storytelling.

Keywords

Narrative cognition interactive narrative ERP P300 component EEG suspense surprise expectancy interactive storytelling 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Vecchiato, G., Astolfi, L., Tabarrini, A., Salinari, S., Mattia, D., Cincotti, F., Bianchi, L., Sorrentino, D., Aloise, F., Soranzo, R., Babiloni, F.: EEG Analysis of the Brain Activity during the Observation of Commercial, Political, or Public Service Announcements. Computational Intelligence and Neuroscience (2010)Google Scholar
  2. 2.
    Ravaja, N.: Contributions of Psychophysiology to Media Research: Review and Recommendations. Media Psychology, 193-235 (2004)Google Scholar
  3. 3.
    Ganglbauer, E., Schrammel, J., Deutsch, S., Tscheligi, M.: Applying psychophysiological methods for measuring user experience: possibilities, challenges and feasibility. In: User Experience Evaluation Methods in Product Development (UXEM 2009) (2009)Google Scholar
  4. 4.
    Kivikangas, J.M., Ekman, I., Chanel, G., Järvelä, S., Salminen, M., Cowley, B., Ravaja, N.: Review on psychophysiological methods in game research. In: Proc. of 1st Nordic DiGRA (2010)Google Scholar
  5. 5.
    Ryan, M.-L.: Narratology and Cognitive Science: A Problematic Relation. Style 44(4) (2010)Google Scholar
  6. 6.
    Bruni, L.E., Baceviciute, S.: On the Embedded Cognition of Non-verbal Narratives. Sign Systems Studies (in press)Google Scholar
  7. 7.
    Sanford, A.J., Emmott, C.: Mind, brain and narrative. Cambridge University Press (2012)Google Scholar
  8. 8.
    Speer, N.K., Reynolds, J.R., Swallow, K.M., Zacks, J.M.: Reading Stories Activates Neural Representations of Visual and Motor Experiences. Psychological Science 20(8), 989–999 (2009)CrossRefGoogle Scholar
  9. 9.
    Zacks, J.M., Speer, N.K., Swallow, K.M., Maley, C.J.: The brain’s cutting-room floor: segmentation of narrative cinema. Frontiers in Human Neuroscience 4, 1–15 (2010)CrossRefGoogle Scholar
  10. 10.
    Yarkoni, T., Speer, N.K., Zacks, J.M.: Neural substrates of narrative comprehension and memory. NeuroImage 41, 1408–1425 (2008)CrossRefGoogle Scholar
  11. 11.
    Mar, R.A.: The neuropsychology of narrative: story comprehension, story production and their interrelation. Neuropsychologia 42, 1414–1434 (2004)CrossRefGoogle Scholar
  12. 12.
    Duncan, C.C., Connolly, R.J.B.J.F., Fischer, C., Michie, P.T., Näätänen, R., Polich, J., Reinvang, I., Van Petten, C.: Event-related potentials in clinical research: guidelines for eliciting, recording, and quantifying mismatch negativity, P300, and N400. Clinical Neurophysiology 120(11), 1883–1908 (2009)CrossRefGoogle Scholar
  13. 13.
    McPherson, W.B., Holcomb, P.J.: An electrophysiological investigation of semantic priming with pictures of real objects. Psychophysiology 36, 53–65 (1999)CrossRefGoogle Scholar
  14. 14.
    Federmeier, K.D., Kutas, M.: Picture the difference: Electrophysiological investigations of picture processing in the two cerebral hemispheres. Neuropsychologia 40(7), 730–747 (2002)CrossRefGoogle Scholar
  15. 15.
    Wu, Y.C., Coulson, S.: Meaningful gestures: Electrophysiological indices of iconic gesture comprehension: Psychophysiology, vol. Psychophysiology 42, 654–667 (2005)CrossRefGoogle Scholar
  16. 16.
    Sitnikova, T., Kuperberg, G., Holcomb, P.: Semantic integration in videos of real–world events: An electrophysiological investigation. Psychophysiology 40(1), 160–164 (2003)CrossRefGoogle Scholar
  17. 17.
    Chao, L.L., Nielsen-Bohlman, L., Knight, R.T.: Auditory event-related potentials dissociate early and late memory processes. Electroencephalography & Clinical Neurophysiology 96, 157–168 (1995)CrossRefGoogle Scholar
  18. 18.
    Cohn, N., Paczynski, M., Holcomb, P., Jackendoff, R., Kuperberg, G.: Comics on the Brain: Structure and Meaning in Sequential Image Comprehension. Psychophysiology 48 (2011)Google Scholar
  19. 19.
    Cohn, N., Paczynski, M., Jackendoff, R., Holcomb, P., Kuperberg, G. (Pea)nuts and bolts of visual narrative: Structure and meaning in sequential image comprehension. Cognitive Psychology 65(1), 1–38 (2012)CrossRefGoogle Scholar
  20. 20.
    Batterink, L., Neville, H.: Implicit and explicit mechanisms of word learning in a narrative context: An event-related potential study. Journal of cognitive neuroscience 23(1), 3181–3196 (2011)CrossRefGoogle Scholar
  21. 21.
    Baretta, L., Lêda, M.B., Tomitch, N., MacNair, V., Kwan, L., Waldie, K.E.: Inference making while reading narrative and expository texts: an ERP study. Psychology & Neuroscience 2(2), 137–145 (2009)CrossRefGoogle Scholar
  22. 22.
    Thierry, G., Martin, C., Gonzalez-Diaz, V., Rezaie, R., Roberts, N., Davis, P.: Event-related potential characterisation of the Shakespearean functional shift in narrative sentence structure. Neuroimage 40(2), 923–931 (2008)CrossRefGoogle Scholar
  23. 23.
    Cohn, N.: Visual narrative structure. Cognitive Science 37(3), 413–452 (2013)CrossRefGoogle Scholar
  24. 24.
    Nijholt, A.: BCI for games: A ‘State of the art’ survey. In: Stevens, S.M., Saldamarco, S.J. (eds.) ICEC 2008. LNCS, vol. 5309, pp. 225–228. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  25. 25.
    Ryan, M.-L.: Narrative as virtual reality. Johns Hopkins University Press, Baltimore (2001)Google Scholar
  26. 26.
    Cupchik, G.: Suspense and disorientation: Two poles of emotionally charged literary uncertainty. In: Peter Vorderer, Hans J. Wulff, and Mike Friedrichsen (eds.) Conceptualizations, theoretical analyses, and empirical explorations, pp. 189-197. Lawrence Erlbaum Associates, Mahwah (1996)Google Scholar
  27. 27.
    Iwata, Y.: Creating Suspense and Surprise in Short Literary Fiction: A stylistic and narratological approach. University of Birmingham, Birmingham (2009)Google Scholar
  28. 28.
    Chatman, S.B.: Story and discourse: Narrative structure in fiction and film. Cornell University Press, Ithaca (1980)Google Scholar
  29. 29.
    Tan, E., Diteweg, G.: Suspense, predictive inference, and emotion in film viewing. In: Vorderer, P., Wulff, H.J., Friedrichsen, M. (eds.) Conceptualizations, theoretical analyses, and empirical explorations, pp. 149–188. Lawrence Erlbaum Associates, Mahwah (1996)Google Scholar
  30. 30.
    Mikos, L.: The experience of suspense: Between fear and pleasure. In: Vorderer, P., Wulff, H.J., Friedrichsen, M. (eds.) Conceptualizations, theoretical analyses, and empirical explorations, pp. 37–50. Lawrence Erlbaum Associates, Mahwah (1996)Google Scholar
  31. 31.
    de Beaugrande, R.: The story grammar and the grammar of stories. Journal of Pragmatics 6, 383–422 (1982)CrossRefGoogle Scholar
  32. 32.
    Carroll, N.: The Philosophy of Horror or Paradoxes of the Heart. Routledge, New York (1990)Google Scholar
  33. 33.
    de Wied, M.: The role of temporal expectancies in the production of film suspense. Poetics 23, 107–123 (1994)CrossRefGoogle Scholar
  34. 34.
    Wulff, H.: Suspense and the influence of cataphora on viewers’ expectations. In: Vorderer, P., Wulff, H.J., Friedrichsen Conceptualizations, M. (eds.) Theoretical Analyses, and Empirical Explorations, pp. 1–18. Lawrence Erlbaum Associates, Mahwah (1996)Google Scholar
  35. 35.
    Berlyne, D.E.: Conflict, arousal, and curiosity. McGraw-Hill, New York (1960)Google Scholar
  36. 36.
    Fazel-Rezai, R., Ahmad, W.: P300-based Brain-Computer Interface paradigm design. Recent Advances in Brain-Computer Interface Systems, pp. 83–98 (2011)Google Scholar
  37. 37.
    Kutas, M., Hillyard, S.: Reading Senseless Sentences: brain potentials reflect semantic incongruity. Science, 203–205 (1980)Google Scholar
  38. 38.
    Polich, J., Kok, A.: Cognitive and biological determinants of P300: an integrative review. Biological Psychology 41(2), 103–146 (1995)CrossRefGoogle Scholar
  39. 39.
    Marathe, A.R., Ries, A.J., McDowell, K.: Sliding HDCA: Single-trial EEG classification to overcome and quantify temporal variability. IEEE Transactions on Neural Systems and Rehabilitation Engineering 22(2), 201–211 (2014)CrossRefGoogle Scholar
  40. 40.
    Pernet, C.R., Sajda, P., Rousselet, G.A.: Single-trial analyses: why bother? Frontiers in Psychology 2, 1–2 (2011)CrossRefGoogle Scholar
  41. 41.
    Li, K., Raju, V.N., Sankar, R., Arbel, Y., Donchin, E.: Advances and challenges in signal analysis for single trial P300-BCI. Foundations of Augmented Cognition. Directing the Future of Adaptive Systems, pp. 87-94 (2011)Google Scholar
  42. 42.
    Zhang, D., Luo, W., Luo, Y.: Single-trial ERP analysis reveals facial expression category in a three-stage scheme. Brain research 1512 , 78-88 (2013)Google Scholar
  43. 43.
    Quitadamo, L., Cincotti, F., Mattia, D., Cardarilli, G., Marciani, M., Bianchi, L.: Finding the Best Number of Flashes in P300-based BCIs: a Selection-Cost Approach. In: TOBI Workshop 2010, Graz, Austria, p. 54 (2010)Google Scholar
  44. 44.
    Ortner, R., Prückl, R., Putz, V., Bruckner, J.S.M., Schnürer, A., Guger, C.: Accuracy of a P300 Speller for different conditions: A comparison. In: Proc. of 5th Int. BCI Conf. 2011, pp. 196–199 (2011)Google Scholar
  45. 45.
    g.Tec, P300 Spelling Device V2.12.00. g.Tec – medical engineering GmbH, Austria (2012) Google Scholar
  46. 46.
    Gilroy, S.W., Porteous, J., Charles, F., Cavazza, M., Soreq, E., Raz, G., Ikar, L., Or-Borichov, A., Ben-Arie, U., Klovatch, I., Hendler, T.: A brain-computer interface to a plan-based narrative. In: Proceedings of the Twenty-Third international joint conference on Artificial Intelligence, pp. 1997–2005. AAAI Press (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Luis Emilio Bruni
    • 1
  • Sarune Baceviciute
    • 1
  • Mohammed Arief
    • 1
  1. 1.Augmented Cognition Lab, Department of Architecture, Design and Media TechnologyAalborg UniversityCopenhagenDenmark

Personalised recommendations