Formal Aspects of Computing

, 21:513 | Cite as

Process algebraic modelling of attentional capture and human electrophysiology in interactive systems

  • Li SuEmail author
  • Howard Bowman
  • Philip Barnard
  • Brad Wyble
Open Access
Original Article


Previous research has developed a formal methods-based (cognitive-level) model of the Interacting Cognitive Subsystems central engine, with which we have simulated attentional capture in the context of Barnard’s key-distractor Attentional Blink task. This model captures core aspects of the allocation of human attention over time and as such should be applicable across a range of practical settings when human attentional limitations come into play. In addition, this model simulates human electrophysiological data, such as electroencephalogram recordings, which can be compared to real electrophysiological data recorded from human participants. We have used this model to evaluate the performance trade-offs that would arise from varying key parameters and applying either a constructive or a reactive approach to improving interactive systems in a stimulus rich environment. A strength of formal methods is that they are abstract and the resulting specifications of the operator are general purpose, ensuring that our findings are broadly applicable. Thus, we argue that new modelling techniques from computer science can also be employed in computational modelling of the mind. These would complement existing techniques, being specifically targeted at psychological level modelling, in which it is advantageous to directly represent the distribution of control.


Formal methods HCI Stimulus rich reactive interfaces Attentional blink EEG Interacting cognitive subsystems 



We are indebted to anonymous reviewers for their comments on the previous version of this manuscript. We also thank Patrick Craston, Srivas Chennu and Dell Green for their contribution to the collection and analysis of the EEG data. The UK Engineering and Physical Sciences Research Council supported this research (grant number GR/S15075/01). The participation of Philip Barnard in this project was supported by the Medical Research Council under project code U.1055.02.003.00001.01.

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.


  1. ABM93.
    Abeles M, Bergman H, Margalis E, Vaadia E (1993) Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. J Neurophysiol 70: 1629–1638Google Scholar
  2. And93.
    Anderson JR (1993) Rules of the mind. Erlbaum, HillsdaleGoogle Scholar
  3. And05.
    Anderson AK (2005) Affective influences on the attentional dynamics supporting awareness. J Exp Psychol General 134(2): 258–281CrossRefGoogle Scholar
  4. BMM02.
    Baeten JC, Middelburg CA, Middelburg K (2002) Process algebra with timing. Springer, New YorkzbMATHGoogle Scholar
  5. Bar99.
    Barnard PJ (1999) Interacting cognitive subsystems: modelling working memory phenomena within a multi-processor architecture, Models of Working Memory: Mechanisms of active maintenance and executive control, pp 298–339Google Scholar
  6. BST04.
    Barnard PJ, Scott S, Taylor J, May J, Knightley W (2004) Paying attention to meaning. Psychol Sci 15(3): 179–186CrossRefGoogle Scholar
  7. Bar03.
    Barnard PJ, Bowman H (2003) Rendering information processing models of cognition and affect computationally explicit: distributed executive control and the deployment of attention. Cogn Sci Q 3(3): 297–328Google Scholar
  8. BRB05.
    Barnard PJ, Ramponi C, Battye G, Mackintosh B (2005) Anxiety and the deployment of visual attention over time. Vis Cogn 12(1): 181–211CrossRefGoogle Scholar
  9. BPS03.
    Beverina F, Palmas G, Silvoni S, Piccione F, Giove S (2003) User adaptive BCIs: SSVEP and P300 based interfaces. Psychnology 1(4): 331–354Google Scholar
  10. Bol88.
    Bolognesi T, Brinksma E (1988) Introduction to the ISO specification language LOTOS. Comput Netw ISDN Syst 14910: 25–29Google Scholar
  11. Bon99.
    Bond AH (1999) Describing behavioural states using a system model of the primate brain. Am J Primatol 49: 315–338CrossRefGoogle Scholar
  12. BBD01.
    Bowman H, Bryans JW, Derrick J (2001) Analysis of a multimedia stream using stochastic process algebra. Comput J 44(4): 230–245zbMATHCrossRefGoogle Scholar
  13. Bow99.
    Bowman H, Faconti G (1999) Analysing cognitive behaviour using LOTOS and Mexitl. Formal Aspects Comput 11: 132–159CrossRefGoogle Scholar
  14. Bow06.
    Bowman H, Gomez RS (2006) Concurrency theory, calculi and automata for modelling untimed and timed concurrent systems. Springer, BerlinGoogle Scholar
  15. Bow07.
    Bowman H, Wyble B (2007) The simultaneous type, serial token model of temporal attention and working memory. Psychol Rev 114(1): 38–70CrossRefGoogle Scholar
  16. Car98.
    Carver CS, Scheier MF (1998) On the self-regulation of behaviour. Cambridge University Press, CambridgeGoogle Scholar
  17. Chu95.
    Chun MM, Potter MC (1995) A two-stage model for multiple target detection in rapid serial visual presentation. J Exp Psychol Hum Percept Perform 21(1): 109–127CrossRefGoogle Scholar
  18. Cil93.
    Cilliers PJ, Van Der Kouwe AJW (1993) A VEP-based computer interfcae for C2-Quadriplegics. Engineering in Medicine and Biology Society, 1993. In: Proceedings of the 15th Annual International Conference of the IEEE, vol 15(3), pp 1263–1624Google Scholar
  19. CFS96.
    Cooper R, Fox J, Farringdon J, Shallice T (1996) A systematic methodology for cognitive modelling. Artif Intell 85: 3–44CrossRefGoogle Scholar
  20. CWC08.
    Craston P, Wyble B, Chennu S, Bowman H (2008) The attentional blink reveals serial working memory encoding: evidence from virtual and human event-related potentials. J Cogn Neurosci (in press)Google Scholar
  21. Don81.
    Donchin E (1981) Presidential address, 1980. Surprise!...Surprise?. Psychophysiology 18(5): 493–513CrossRefGoogle Scholar
  22. DBD98.
    Duke DJ, Barnard PJ, Duce DA, May J (1998) Syndetic modelling. Hum Comput Interact 13(4): 337–393CrossRefGoogle Scholar
  23. EFH83.
    Ehrig H, Fey W, Hansen H (1983) ACT ONE—an algebraic specification language with two levels of semantics, ADTGoogle Scholar
  24. EBJ96.
    Elman JL, Bates EA, Johnson MH, Karmiloff-Smith A, Parisi D, Plunkett K (1996) Rethinking innateness: a connectionist perspective on development, a Bradford book. MIT Press, CambridgeGoogle Scholar
  25. Eri81.
    Erickson TD, Mattson ME (1981) From words to meaning: a semantic illusion. J Verbal Learn Verbal Behav 20: 540–551CrossRefGoogle Scholar
  26. Fod88.
    Fodor JA, Pylyshyn ZW (1988) Connectionism and cognitive architecture: a critical analysis. Cognition 28: 3–71CrossRefGoogle Scholar
  27. GLM02.
    Garavel H, Lang F, Mateescu F (2002) An overview of CADP 2001. EASST Newsl 4: 13–24Google Scholar
  28. GVZ01.
    Garavel H, Viho C, Zendri M (2001) System design of a CC-NUMA multiprocessor architecture using formal specification, model-checking, co-simulation, and test generation. Springer Int J Softw Tools Technol Transf (STTT) 3(3): 314–331zbMATHGoogle Scholar
  29. Gom98.
    Gomez ME, Santonja V (1998) Self-similiary in I/O workload: analysis and modeling. In: Workshop on Workload CharacterizationGoogle Scholar
  30. Gri05.
    Grill-Spector K, Kanwisher N (2005) Visual recognition: as soon as you know it is there, you know what it is. Psychol Sci 16(2): 152–160CrossRefGoogle Scholar
  31. Hoa85.
    Hoare CAR (1985) Communicating sequential processes. Prentice-Hall, LondonzbMATHGoogle Scholar
  32. KMM99.
    Kieras DE, Meyer DE, Mueller S, Seymour T (1999) Insights into working memory from the perspective of the EPIC architecture for modelling skilled perceptual-motor and cognitive human performance, Models of Working Memory, Mechanisms of Active Maintenance and Executive Control. Cambridge University Press, New York, pp 183–223Google Scholar
  33. Lev79.
    Leventhal H (1979) A perceptual-motor processing model of emotion. In: Pilner P, Blankstein K, Spigel IM (eds) Perception of emotion in self and others, vol 5. Plenum, New York, pp 1–46Google Scholar
  34. LHB00.
    Levine SP, Huggins JE, BeMent SL, Kushwaha RK, Schuh LA, Rohde MM (2000) A direct brain interface based on event-related potentials. IEEE Trans Rehabil Eng 8(2): 180–185CrossRefGoogle Scholar
  35. Mac63.
    Mackworth J (1963) The duration of the visual image. Can J Paychol 17(1): 62–68Google Scholar
  36. MFP97.
    Maki WS, Frigen K, Paulsen K (1997) Associative priming by targets and distractors during rapid serial presentation. J Exp Psychol Hum Percept Perform 23: 1014–1034CrossRefGoogle Scholar
  37. MKH03.
    Meinicke P, Kaper M, Hoppe F, Heumann M, Ritter H (2003) Improving transfer rates in brain computer interfacing: a case study. Adv Neural Inform Process Syst 15: 1107–1114Google Scholar
  38. Met78.
    Metz CE (1978) Basic principles of ROC analysis. Semin Nuclear Med 8: 283–298CrossRefGoogle Scholar
  39. Mey97.
    Meyer DE, Kieras DE (1997) A computational theory of executive cognitive processes and multiple task performance: Part 1. Basic mechanisms. Psychol Rev 104: 3–65Google Scholar
  40. Mil89.
    Milner R (1989) Communication and concurrency. Prentice-Hall, Hemel HempsteadzbMATHGoogle Scholar
  41. New90.
    Newell A (1990) Unified theories of cognition. Harvard University Press, CambridgeGoogle Scholar
  42. Rei00.
    O’Reilly RC, Munakata Y (2000) Computational explorations in cognitive neuroscience: understanding the mind by simulating the brain, a Bradford book. MIT Press, CambidgeGoogle Scholar
  43. RSA92.
    Raymond J, Shapiro K, Arnell KM (1992) Temporary suppression of visual processing in an RSVP Task: an attentional blink. J Exp Psychol Hum Percept Perform 18(3): 849–860CrossRefGoogle Scholar
  44. Rol01.
    Rolls ET, Stringer SM (2001) A model of the interaction between mood and memory. Netw: Comput Neural Syst 12: 89–109zbMATHCrossRefGoogle Scholar
  45. Rol98.
    Rolls ET, Treves A (1998) Neural networks and brain function. Oxford University Press, OxfordGoogle Scholar
  46. RMP86.
    Rumelhart DE, McClelland JL, thePDP Research Group (1986) Parallel distributed processing, explorations in the microstructure of cognition. Volume 1: Foundations and Volume 2: Psychological and Biological Models, a Bradford book. MIT Press, CambridgeGoogle Scholar
  47. SWH98.
    Schmolesky M, Wang Y, Hanes D, Thompson K, Leutgeb S, Schall J, Leventhal AG (1998) Signal timing across the Macaque visual system. J Neurophysiopl 79(6): 3272–3278Google Scholar
  48. SCS97.
    Shapiro KL, Caldwell JI, Sorensen RE (1997) Personal names and the attentional blink: the cocktail party revisited. J Exp Psychol Hum Percept Perform 23: 504–514CrossRefGoogle Scholar
  49. Sha99.
    Shapiro KL, Luck SJ (1999) The attentional blink: a front-end mechanism for fleeting memories. In: Fleeting memories, cognition of brief visual stimuli, A Bradford book. MIT Press, Boston, pp 95–118Google Scholar
  50. Sno88.
    Snodgrass JG, Corwin J (1988) Pragmatics of measuring recognition memory: applications to dementia and amnesia. J Exp Psychol Gen 117(1): 34–50CrossRefGoogle Scholar
  51. SSH75.
    Squires NK, Squires KC, Hillyard SA (1975) Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man. Electroencephalogr Clin Neurophysiol 38(4): 387–401CrossRefGoogle Scholar
  52. SBB07.
    Su L, Bowman H, Barnard PJ (2007) Attentional capture by meaning: a multi-level modelling study. In: Proceedings of the 29th Annual Cognitive Science Society. Cognitive Science Society, Austin, pp 1521–1526Google Scholar
  53. SBB08.
    Su L, Bowman H, Barnard PJ (2008) Performance of reactive interfaces in stimulus rich environments, applying formal methods and cognitive frameworks. In: The 2nd International Workshop on Formal Methods for Interactive Systems FMIS2007 (held in conjunction with HCI2007), Electronic Notes in Theoretical Computer Science, vol 208. Elsevier, Amsterdam, pp 95–111Google Scholar
  54. Tea93.
    Teasdale JD, Barnard PJ (1993) Affect, cognition and change: re-modelling depressive thought. Lawrence Erlbaum Associates, HoveGoogle Scholar
  55. Vid73.
    Vidal JJ (1973) Toward direct brain-computer communication. Ann Rev Biophys Bioeng 2: 157–180CrossRefGoogle Scholar
  56. VLS98.
    Vogel EK, Luck SJ, Shapiro KL (1998) Electrophysiological evidence for a postperceptual locus of suppression during the attentional blink. J Exp Psychol Hum Percept Perform 24(6): 1656–1674CrossRefGoogle Scholar
  57. WMC02.
    Wang M, Madhyastha T, Chan NH, Papadimitriou S, Faloutsos C (2002) Data mining meets performance evaluation: fast algorithms for modeling bursty traffic. In: 18th International Conference on Data EngineeringGoogle Scholar
  58. Wyb05.
    Wyble B, Bowman H (2005) Computational and experimental evaluation of the attentional blink: testing the simultaneous type serial token model. In: Bara BG, Barsalou LW, Bucciarelli M (eds) CogSci 2005, XXVII Annual Conference of the Cognitive Science Society. Cognitive Science Society, Cognitive Science Society through Lawrence Erlbaum, Austin, pp 2371–2376Google Scholar
  59. WCB06.
    Wyble B, Craston P, Bowman H (2006) Electrophysiological feedback in adaptive human–computer interfaces, Technical Report 8-06, Computing Laboratory, University of Kent, Canterbury, UKGoogle Scholar

Copyright information

© The Author(s) 2008

Authors and Affiliations

  • Li Su
    • 1
    • 4
    Email author
  • Howard Bowman
    • 1
  • Philip Barnard
    • 2
  • Brad Wyble
    • 3
  1. 1.Centre for Cognitive Neuroscience and Cognitive SystemsUniversity of KentCanterburyUK
  2. 2.MRC Cognition and Brain Sciences UnitCambridgeUK
  3. 3.Department of Brain and Cognitive SciencesMassachusetts Institute of TechnologyCambridgeUSA
  4. 4.Section of Cognitive Neuropsychiatry, Department of Psychological MedicineInstitute of Psychiatry at King’s College LondonLondonUK

Personalised recommendations