Skip to main content

The neural effect of stimulus-response modality compatibility on dual-task performance: an fMRI study

Psychological Research volume 70, pages 514–525 (2006)Cite this article

Abstract

Recent fMRI studies suggest that the inferior frontal sulcus (IFS) is involved in the coordination of interfering processes in dual-task situations. The present study aims to further specify this assumption by investigating whether the compatibility between stimulus and response modalities modulates dual-task-related activity along the IFS. It has been shown behaviorally that the degree of interference, as measured by dual-task costs, increases in modality-incompatible conditions (e.g. visual–vocal tasks combined with auditory–manual tasks) as compared to modality-compatible conditions (e.g. visual–manual tasks combined with auditory–vocal tasks). Using fMRI, we measured IFS activity when participants performed modality-compatible and modality-incompatible single and dual tasks. Behaviorally, we replicated the finding of higher dual-task costs for modality-incompatible tasks compared to modality-compatible tasks. The fMRI data revealed higher activity along the IFS in modality-incompatible dual tasks compared with modality-compatible dual tasks when inter-individual variability in functional brain organization is taken into account. We argue that in addition to temporal order coordination (Szameitat et al., 2002), the IFS is involved in the coordination of cognitive processes associated with the concurrent mapping of sensory information onto corresponding motor responses in dual-task situations.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  • Braver, T. S., Cohen, J. D., Nystrom, L. E., Jonides, J., Smith, E. E., & Noll, D.C. (1997). A parametric study of prefrontal cortex involvement in human working memory. NeuroImage, 5, 49–62.

    Article  PubMed  Google Scholar 

  • Bunge, S. A., Hazeltine, E. A., Scanlon, M. D., Rosen, A. C., & Gabrieli, J. D. E (2002). Dissociable contributions of of prefrontal and parietal cortices to response selection. NeuroImage, 17, 1562–1571.

    Article  PubMed  Google Scholar 

  • DeJong, R. (1995). The role of preparation in overlapping task performance. Quarterly Journal of Experimental Psycholgy, 48, 2–25.

    Google Scholar 

  • D’Esposito, M., Detre, J.A, Alsop, D. C., Shin, R. K., Atlas, S., & Grossmann, M. (1995). The neural basis of the central executive system of working memory. Nature, 378, 279–281.

    Article  PubMed  Google Scholar 

  • D’Esposito, M., Ballard, D., Aguirre, G. K., & Zarahn, E. (1998). Human prefrontal cortex is not specific for working memory: A functional MRI study. NeuroImage, 8, 274–282.

    Article  PubMed  Google Scholar 

  • Dove, A., Pollmann, S., Schubert, T., Wiggins, C. J., & von Cramon, D.Y. (2000). Prefrontal cortex activation in task switching: An event-related fMRI study. Cognitive Brain Research, 9, 103–109.

    Article  PubMed  Google Scholar 

  • Fan, J., Flombaum, J. I., McCandliss, B. D., Thomas, K. M., & Posner, M.I. (2003). Cognitive and brain consequences of conflict. NeuroImage, 18, 42–57.

    Article  PubMed  Google Scholar 

  • Friedman, A., & Polson, M.C. (1981). Hemispheres as independent resource systems: Limited-capacity processing and cerebral specialization. Journal of Experimental Psychology: Human Perception and Performance, 7, 1031–1058.

    Article  PubMed  Google Scholar 

  • Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J.-P., Frith, C. D., & Frackowiak, R.S.J. (1995). Statistical parametric maps in functional imaging: A general linear approach. Human Brain Mapping, 2, 189–210.

    Article  Google Scholar 

  • Genovese, C. R., Lazar, N. A., & Nichols, T. (2003). Thresholding of statistical maps in functional neuroimaging using the false discovery rate. NeuroImage, 15, 870–878

    Article  Google Scholar 

  • Greenwald, A.G. (1970). Sensory feedback mechanisms in performance control: With special reference to the ideor-motor mechanism. P sychological Review, 77, 73–99.

    PubMed  Article  Google Scholar 

  • Hazeltine, E., Ruthruff, E. & Remington, R.W. (under review). The role of input and output modality pairings in dual-task performance: Evidence for content-dependent central interference.

  • Hommel, B. (1998). Automatic stimulus-response translation in dual-task performance. Journal of Experimental Psychology: Human Perception and Performance, 24, 1368–1384.

    Article  PubMed  Google Scholar 

  • Hommel, B., Müsseler, J., Aschersleben, G., & Prinz, W. (2001). The theory of event coding (TEC): A framework for perception and action. Behavioral and Brain Sciences, 24, 849–937.

    PubMed  Article  Google Scholar 

  • Iacoboni, M., Woods, R. P., & Mazziotta, J.C. (1996). Brain-behavior relationships: Evidence from practice effects in spatial stimulus-response compatibility. Journal of Neurophysiology, 76, 321–331.

    PubMed  Google Scholar 

  • Jiang, Y., & Kanwisher, N. (2003). Common neural substrates for response selection across modalities and mapping paradigms. Journal of Cognitive Neuroscience, 15, 1080–1094.

    Article  PubMed  Google Scholar 

  • Jiang, Y., Saxe, R., & Kanwisher, N. (2004). Functional magnetic resonance imaging provides new constraints on theories of the psychological refractory period. Psychological Science, 15, 390–396.

    Article  PubMed  Google Scholar 

  • Kinsbourne, M., & Hicks, R.E. (1978). Functional cerebral space: A model for overflow, transfer and interference effects in human performance. In J. Requin (Ed.), Attention and Performance VII. Hillsdale, N.J.: Erlbaum.

  • Koch, I. & Prinz, W. (2002). Process interference and code overlap in dual-task performance. Journal of Experimental Psychology: Human Perception & Performance, 28, 192–201.

    Article  Google Scholar 

  • Koechlin, E., Basso, G., Pietrini, P., Panzer, S. & Grafman, J. (1999). The role of the anterior prefrontal cortex in human cognition. Nature, 399, 148–151.

    Article  PubMed  Google Scholar 

  • Koelsch, S., Gunter, T. C., von Cramon, D. Y., Zysset, S., Lohmann, G., & Friederici, A.D. (2002). Bach speaks: A cortical ‘language-network’ serves the processing of music. NeuroImage, 17, 956–966.

    Article  PubMed  Google Scholar 

  • Levy, J. & Pashler, H. (2001). Is dual-task slowing instruction dependent? Journal of Experimental Psychology: Human Perception and Performance, 27, 862–869.

    Article  PubMed  Google Scholar 

  • Liberman, A.M. & Mattingley, I.G. (1989). A specialization for speech perception. Science, 243, 489–494.

    PubMed  Article  Google Scholar 

  • Logan, G.D. & Schulkind, M.D. (2000). Parallel memory retrieval in dual-task situations: I. Semantic memory. Journal Experimental Psychology: Human Perception & Performance, 26, 1072–1090.

    Article  Google Scholar 

  • MacDonald, A. W., III Cohen, J. D., Stenger, V. A., Carter, C.S. (2000). Dissociating the role of the dorsolateral anterior cingulate cortex in cognitive control. Science, 288, 1835–1838.

    Article  PubMed  Google Scholar 

  • McLeod, P. (1977). A dual task response modality effect. Support for multiprocessor models of attention. Quarterly Journal of Experimental Psychology, 29, 651–667.

    Google Scholar 

  • Meyer, D. E., Kieras, D. E., Lauber, E. J., Schumacher, E. H., Glass, J. M., Zubriggen, E. L., Gmeindl, L., & Apfelblat, D. (1995). Adaptive executive control: Flexible multiple- task performance without pervasive immutable response-selection bottlenecks. Acta Psychologica, 90, 163–190.

    Article  Google Scholar 

  • Meyer, D. E., & Kieras, D.E. (1997). A computational model of human multiple task performance: The EPIC information-processing architecture and strategic response deferment model. Psychological Review, 104, 1–65.

    Google Scholar 

  • Milham, M. P., Banich, M. T., Webb, A., Barad, V., Cohen, N. J., Wszalek, T. (2001). The relative involvement of anterior cingulate and prefrontal cortex in attentional control depends on the nature of conflict. Cognitive Brain Research, 12, 467–473.

    Article  PubMed  Google Scholar 

  • Miller, M. B., van Horn, J. D., Wolford, G. L., Handy, T.C. Valsangkar-Smyth, M., Inati, S., Grafton, S., & Gazzaniga, M.S. (2002). Extensive individual differences in brain activations associated with episodic retrieval are reliable over time. Journal of Cognitive Neuroscience, 14, 1200–1214.

    Article  PubMed  Google Scholar 

  • Milner, A. D., & Goodale, M.A. (1995). The visual brain in action. New York: Oxford University Press.

    Google Scholar 

  • Pashler, H. (1994). Dual-task interference in simple tasks: Data and theory. Psychological Bulletin, 116, 220–244.

    Article  PubMed  Google Scholar 

  • Rajkowska, G., & Goldman-Rakic, P.S. (1995). Cychoarchitetonic definition of prefrontal areas in the normal human cortex: I. Remapping of areas 9 and 46 using quantitative criteria. Cerebral Cortex, 5, 307–322.

    PubMed  Google Scholar 

  • Schubert, T. (1999). Processing differences between simple and choice reaction affect bottleneck localization in overlapping tasks. Journal of Experimental Psychology: Human Perception and Performance, 25,408–425.

    Article  Google Scholar 

  • Schubert, T. & Szameitat, A.J. (2003). Functional neuroanatomy of interference in overlapping dual tasks: an fMRI study. Cognitive Brain Research, 17, 733–746.

    Article  PubMed  Google Scholar 

  • Schuch, S. & Koch, I. (2004). The costs of changing the representation of action: Response repetition and response-response compatibility in dual tasks. Journal of Experimental Psychology: Human Perception and Performance, 30, 566–582.

    Article  PubMed  Google Scholar 

  • Schumacher, E. H., Lauber, E. J., Glass, J.M. B., Zurbriggen, E. L., Gmeindl, L., Kieras, D.E., & Meyer, D.E. (1999). Concurrent response-selection processes in dual-task performance: Evidence for adaptive executive control of task scheduling. Journal of Experimental Psychology: Human Perception and Performance, 25, 791 – 814.

    Article  Google Scholar 

  • Schumacher, E. H., Seymour, T. L., Glass, J. M., Fencsik, D. E., Lauber, E. J., Kieras, D. E., & Meyer, D.E. (2001). Virtually perfect time sharing in dual-task performance: Uncorking the central cognitive bottleneck. Psychological Science, 12, 101–108.

    Article  PubMed  Google Scholar 

  • Schumacher, E.H. & D’Esposito, M. (2002). Neural implementation of response selection in humans as revealed by localized effects of stimulus-response compatibility on brain activation. Human Brain Mapping, 17, 193–201.

    Article  PubMed  Google Scholar 

  • Schumacher, E. H., Elston, P. A., & D’Esposito, M. (2003). Neural evidence for representation-specific response selection. Journal of Cognitive Neuroscience, 15,1111–1121.

    Article  PubMed  Google Scholar 

  • Scott, S. K., & Johnsrude, I.S. (2003). The neuroanatomical and functional organization of speech perception. Trends in Neurosciences, 26, 100–107.

    Article  PubMed  Google Scholar 

  • Szameitat, A. J., Schubert, T., Müller, K., & von Cramon, D.Y. (2002). Localization of executive functions in dual-task performance with fMRI. Journal of Cognitive Neuroscience, 14, 1184–1199.

    Article  PubMed  Google Scholar 

  • Szameitat, A. J., Schubert, T., Lepsien, J., von Cramon, D.Y, Sterr, A. (2005). Task- order coordination in dual-task performance and the lateral prefrontal cortex. An event-related fMRI study.

  • Talairach, P., & Tournoux, J. (1988). A stereotactic coplanar atlas of the human brain. Stuttgart: Thieme.

    Google Scholar 

  • Thompson-Schill, S. L., D’Esposito, M., Aguirre,G., & Farah, M.J. (1997). Role of left inferior prefrontal cortex in retrieval of semantic knowledge: A reeveluation. Proceedings of the National Academy Sciences, 94, 14792–14797.

    Google Scholar 

  • Van Selst, M. A., Ruthruff, E., & Johnston, J.C. (1999). Can practice eliminate the Psychological Refractory Period? Journal of Experimental Psychology: Human Perception and Performance, 25, 1268–1283.

    Article  PubMed  Google Scholar 

  • Virzi, R. A., & Egeth, H.E. (1985). Towards a translational model of Stroop interference. Memory & Cognition, 13, 304–319.

    Google Scholar 

  • Wang, H., & Proctor, R.W. (1996). Stimulus-response compatibility as of function of stimulus code and response modality. Journal of Experimental Psychology: Human Perception and Performance, 22, 1201–1217.

    Article  Google Scholar 

  • Welford, A.T. (1952). The ’psychological refractory period’ and the timing of high speed performance: a review and a theory. British Journal of Psychology, 43, 2–9.

    Google Scholar 

  • Wickens, C.D. (1980). The structure of attentional resources. In Nickerson, R.S. (Ed.), Attention & Performance VIII (pp. 239–257). Hillsdale, N.J.: Erlbaum.

    Google Scholar 

  • Zysset, S., Müller, K., Lohmann, G., & von Cramon, D.Y. (2001). Color-word matching Stroop task: Seperating interference and response conflict. NeuroImage, 13, 29–36.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This work was supported by a grant of the DFG (Deutsche Forschungsgemeinschaft) to T.S. and by a grant of the DAAD (Deutsches Akademisches Auslandsamt) to C.S. In addition, the work was further supported by the NIH, grants MH63901 and NS40813. The authors wish to thank Annabelle Singer and Ben Inglis for their help in data acquisition and Leon Deouell, Felice Sun, Lee Miller, and Christian Kaufmann for help with the data analysis. In addition, we are grateful to Eric Ruthruff, Roman Liepelt, Iring Koch, and one anonymous reviewer for helpful comments on previous versions of this article.

Author information

Affiliations

  1. Department of Psychology, Humboldt University Berlin, Rudower Chaussee 18, 12489, Berlin, Germany

    Christine Stelzel & Torsten Schubert

  2. School of Psychology, Georgia Institute of Technology, Atlanta, USA

    Eric H. Schumacher

  3. Helen Wills Neuroscience Institute and Department of Psychology, University of California, Berkeley, USA

    Mark D‘Esposito

Authors
  1. Christine Stelzel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric H. Schumacher
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Torsten Schubert
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mark D‘Esposito
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christine Stelzel.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Stelzel, C., Schumacher, E.H., Schubert, T. et al. The neural effect of stimulus-response modality compatibility on dual-task performance: an fMRI study. Psychological Research 70, 514–525 (2006). https://doi.org/10.1007/s00426-005-0013-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00426-005-0013-7

Keywords

  • Dual Task
  • Task Type
  • Vocal Response
  • Psychological Refractory Period
  • Component Task