Skip to main content
SpringerLink
Log in

In the Mind’s Eye: Eye-Tracking and Multi-modal Integration During Bilingual Spoken-Language Processing

  • Chapter
  • First Online:
Attention and Vision in Language Processing

Abstract

The human language system integrates information from multiple sources and modalities. Bilinguals, who experience increased processing demands due to competition between their two languages, may be especially likely to rely on cues from multiple modalities. The cross-modal integration involved in language processing has been frequently studied using eye-tracking, an approach that can accommodate the simultaneous presence of both auditory and visual inputs. Eye-tracking research has demonstrated that bilinguals activate both of their languages in parallel (e.g., Spivey and Marian 1999; Weber and Cutler 2004), leading to competition both within (e.g., the English word “marker” competes with the phonologically similar word “marble”) and between (e.g., the English word “marker” competes with the Russian word “marka”) their two languages (Marian and Spivey 2003a, b). Interestingly, this competition arises even when languages do not share phonology (e.g., bimodal bilinguals; Shook and Marian 2012) and in the absence of explicit linguistic input (Chabal and Marian 2015), demonstrating the highly interactive nature of bilingual language processing. This interactivity in the language system yields changes to bilingual cognitive function. For example, bilinguals’ need to control phonological competition is associated with enhanced executive control (Blumenfeld and Marian 2011), and their experience suppressing information from the non-target language improves their ability to learn novel vocabulary (Bartolotti and Marian 2012). We conclude that multi-modal investigations of language processing such as those employing eye-tracking are not only ecologically valid, as they closely resemble real-world multi-modal situations, but also can demonstrate how language interacts with other cognitive and perceptual functions in a non-modular mind.

Language is a part of our organism and no less complicated than it. … The tacit conventions on which the understanding of everyday language depends are enormously complicated.

Ludwig Wittgenstein, May 14, 1915

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The vocabulary that we use to describe functions such as language, cognition, and perception (e.g., “linguistic system”) implicitly identifies entities as distinct from one another. However, language, cognition, and perception are not distinct modules, but rather are part of a highly interactive network. In order to understand how this system operates, we use math symbols (e.g., computational models) or verbal labels (e.g., words) to describe particular functions of the network. The inclusion of these terms does not preclude interactivity, but rather gives us a way to describe functions and refer to concepts either verbally or symbolically.

References

  • Allopenna, P., Magnuson, J. S., & Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language, 38(4), 419–439. doi:10.1006/jmla.1997.2558.

    Article  Google Scholar 

  • Altmann, G. T. M. (2004). Language-mediated eye movements in the absence of a visual world: The “blank screen paradigm”. Cognition, 93(2), B79–B87. doi:10.1016/j.cognition.2004.02.005.

    Article  Google Scholar 

  • Baron, J., & Strawson, C. (1976). Use of orthographic and word-specific knowledge in reading words aloud. Journal of Experimental Psychology: Human Perception and Performance, 2(3), 386–393. doi:10.1037/0096-1523.2.3.386.

    Google Scholar 

  • Bartolotti, J., & Marian, V. (2012). Language learning and control in monolinguals and bilinguals. Cognitive Science, 36, 1129–1147. doi:10.1111/j.1551-6709.2012.01243.x.

    Article  Google Scholar 

  • Beauvillain, C., & Grainger, J. (1987). Accessing interlexical homographs: Some limitations of a language-selective access. Journal of Memory and Language, 26(6), 658–672.

    Article  Google Scholar 

  • Berry, D. C., Banbury, S., & Henry, L. (1997). Transfer across form and modality in implicit and explicit memory. The Quarterly Journal of Experimental Psychology A, 50(1), 1–24. doi:10.1080/027249897392206.

    Article  Google Scholar 

  • Bialystok, E. (2006). Effect of bilingualism and computer video game experience on the Simon task. Canadian Journal of Experimental Psychology, 60(1), 68–79. doi:10.1037/cjep2006008.

    Article  Google Scholar 

  • Bialystok, E. (2008). Bilingualism: The good, the bad, and the indifferent. Bilingualism: Language and Cognition, 12(01), 3. doi:10.1017/S1366728908003477.

  • Bialystok, E. (2011). Reshaping the mind: The benefits of bilingualism. Canadian Journal of Experimental Psychology, 65(4), 229–235. doi:10.1037/a0025406.

    Article  Google Scholar 

  • Bialystok, E., & Barac, R. (2012). Emerging bilingualism: Dissociating advantages for metalinguistic awareness and executive control. Cognition, 122(1), 67–73. doi:10.1016/j.cognition.2011.08.003.

    Article  Google Scholar 

  • Bialystok, E., Craik, F. I. M., Klein, R., & Viswanathan, M. (2004). Bilingualism, aging, and cognitive control: Evidence from the Simon task. Psychology and Aging, 19(2), 290–303. doi:10.1037/0882-7974.19.2.290.

    Article  Google Scholar 

  • Bialystok, E., Craik, F. I. M., & Ruocco, A. C. (2006). Dual-modality monitoring in a classification task: The effects of bilingualism and ageing. Quarterly Journal of Experimental Psychology, 59(11), 1968–1983. doi:10.1080/17470210500482955.

    Article  Google Scholar 

  • Blumenfeld, H. K., & Marian, V. (2011). Bilingualism influences inhibitory control in auditory comprehension. Cognition, 118(2), 245–257. doi:10.1016/j.cognition.2010.10.012.

    Article  Google Scholar 

  • Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C. R., McGuire, P. K., & David, A. S. et al. (1997). Activation of auditory cortex during silent lipreading. Science, 276(5312), 593–596. doi:10.1126/science.276.5312.593.

  • Campbell, R., MacSweeney, M., Surguladze, S., Calvert, G. A., McGuire, P. K., Suckling, J., & David, A. S. et al. (2001). Cortical substrates for the perception of face actions: An fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning). Cognitive Brain Research, 12(2), 233–243.

    Google Scholar 

  • Canseco-Gonzalez, E., Brehm, L., Brick, C., Brown-Schmidt, S., Fischer, K., & Wagner, K. (2010). Carpet or carcel: The effect of age of acquisition and language mode on bilingual lexical access. Language and Cognitive Processes, 25(5), 669–705. doi:10.1080/01690960903474912.

    Article  Google Scholar 

  • Cenoz, J. (2003). The additive effect of bilingualism on third language acquisition: A review. International Journal of Bilingualism, 7(1), 71–89. doi:10.1177/13670069030070010501.

    Article  Google Scholar 

  • Chabal, S., & Marian, V. (2015). Speakers of different languages process the visual world differently. Journal of Experimental Psychology: General, 144(3).

    Google Scholar 

  • Chen, H., & Ho, C. (1986). Development of stroop interference in Chinese-English bilinguals. Journal of Experimental Psychology: Learning, Memory, and Cognition, 12(3), 397–401. doi:10.1037/0278-7393.12.3.397.

    Google Scholar 

  • Colomé, À. (2001). Lexical activation in bilinguals’ speech production: Language-specific or language-independent? Journal of Memory and Language, 45(4), 721–736. doi:10.1006/jmla.2001.2793.

    Article  Google Scholar 

  • Cooper, R. M. (1974). The control of eye fixation by the meaning of spoken language. Cognitive Psychology, 107, 84–107.

    Article  Google Scholar 

  • Costa, A., Caramazza, A., & Sebastián-Gallés, N. (2000). The cognate facilitation effect: Implications for models of lexical access. Journal of Experimental Psychology. Learning, Memory, and Cognition, 26(5), 1283–1296. doi:10.1037/0278-7393.26.5.1283.

    Article  Google Scholar 

  • Costa, A., Hernández, M., & Sebastián-Gallés, N. (2008). Bilingualism aids conflict resolution: Evidence from the ANT task. Cognition, 106(1), 59–86. doi:10.1016/j.cognition.2006.12.013.

    Article  Google Scholar 

  • Costa, A., Hernández, M., Costa-Faidella, J., & Sebastián-Gallés, N. (2009). On the bilingual advantage in conflict processing: Now you see it, now you don’t. Cognition, 113(2), 135–149. doi:10.1016/j.cognition.2009.08.001.

    Article  Google Scholar 

  • Cutler, A., Weber, A., & Otake, T. (2006). Asymmetric mapping from phonetic to lexical representations in second-language listening. Journal of Phonetics, 34(2), 269–284. doi:10.1016/j.wocn.2005.06.002.

    Article  Google Scholar 

  • Dahan, D., Tanenhaus, M. K., & Salverda, A. P. (2007). The influence of visual processing on phonetically driven saccades in the “visual world” paradigm. In R. P. G. van Gompel, H. Fischer, Martin, S. Murray, Wayne, & L. Hill, Robin (Eds.), Eye movements: A window on mind and brain (p. 720). Amsterdam, Netherlands: Elsevier.

    Google Scholar 

  • Davis, C., & Kim, J. (2006). Audio-visual speech perception off the top of the head. Cognition, 100(3), B21–B31. doi:10.1016/j.cognition.2005.09.002.

    Article  Google Scholar 

  • Dupont, S., & Luettin, J. (2000). Audio-visual speech modeling for continuous speech recognition. IEEE Transactions on Multimedia, 2(3), 141–151. doi:10.1109/6046.865479.

    Article  Google Scholar 

  • Dupoux, E., Peperkamp, S., & Sebastián-Gallés, N. (2010). Limits on bilingualism revisited: Stress “deafness” in simultaneous French-Spanish bilinguals. Cognition, 114(2), 266–275. doi:10.1016/j.cognition.2009.10.001.

    Article  Google Scholar 

  • Durgunolu, A. Y., & Roediger, H. L. (1987). Test differences in accessing bilingual memory. Journal of Memory and Language, 26(4), 377–391. doi:10.1016/0749-596X(87)90097-0.

    Article  Google Scholar 

  • Gerard, L. D., & Scarborough, D. L. (1989). Language-specific lexical access of homographs by bilinguals. Journal of Experimental Psychology. Learning, Memory, and Cognition, 15(2), 305–315. doi:10.1037/0278-7393.15.2.305.

    Article  Google Scholar 

  • Griffin, Z. M., & Bock, K. (2000). What the eyes say about speaking. Psychological Science, 11(4), 274–279. doi:10.1111/1467-9280.00255.

    Article  Google Scholar 

  • Hardison, D. M. (2003). Acquisition of second-language speech: Effects of visual cues, context, and talker variability. Applied Psycholinguistics, 24(04), 495–522. doi:10.1017/S0142716403000250.

    Article  Google Scholar 

  • Hazan, V., Sennema, A., & Faulkner, A. (2002). Audiovisual perception in L2 learners. In Proceedings of ICSLP (pp. 1685–1688).

    Google Scholar 

  • Hoshino, N., & Kroll, J. F. (2008). Cognate effects in picture naming: Does cross-language activation survive a change of script? Cognition, 106(1), 501–511. doi:10.1016/j.cognition.2007.02.001.

    Article  Google Scholar 

  • Huettig, F., & McQueen, J. M. (2007). The tug of war between phonological, semantic and shape information in language-mediated visual search. Journal of Memory and Language, 57(4), 460–482. doi:10.1016/j.jml.2007.02.001.

    Article  Google Scholar 

  • Huettig, F., Olivers, C. N. L., & Hartsuiker, R. J. (2011). Looking, language, and memory: Bridging research from the visual world and visual search paradigms. Acta Psychologica, 137(2), 138–150. doi:10.1016/j.actpsy.2010.07.013.

    Article  Google Scholar 

  • Jones, J. A., & Munhall, K. G. (1997). The effects of separating auditory and visual sources on audiovisual integration of speech. Canadian Acoustics, 25(519), 13–20.

    Google Scholar 

  • Ju, M., & Luce, P. A. (2004). Falling on sensitive ears: Constraints on bilingual lexical activation. Psychological Science, 15(5), 314–318. doi:10.1111/j.0956-7976.2004.00675.x.

    Article  Google Scholar 

  • Kaushanskaya, M., & Marian, V. (2009). The bilingual advantage in novel word learning. Psychonomic Bulletin & Review, 16(4), 705–710. doi:10.3758/PBR.16.4.705.

    Article  Google Scholar 

  • Kelly, S. D., Kravitz, C., & Hopkins, M. (2004). Neural correlates of bimodal speech and gesture comprehension. Brain and Language, 89(1), 253–260. doi:10.1016/S0093-934X(03)00335-3.

    Article  Google Scholar 

  • Keshavarz, M. H., & Astaneh, H. (2004). The impact of bilinguality on the learning of English vocabulary as a foreign language (L3). International Journal of Bilingual Education and Bilingualism, 7(4), 295–302. doi:10.1080/13670050408667814.

    Article  Google Scholar 

  • Khare, V., Verma, A., Kar, B., Srinivasan, N., & Brysbaert, M. (2013). Bilingualism and the increased attentional blink effect: Evidence that the difference between bilinguals and monolinguals generalizes to different levels of second language proficiency. Psychological Research, 77(6), 728–737. doi:10.1007/s00426-012-0466-4.

    Article  Google Scholar 

  • Kirsner, K., Brown, H. L., Abrol, S., Chadha, N. K., & Sharma, N. K. (1980). Bilingualism and lexical representation. The Quarterly Journal of Experimental Psychology, 32(4), 585–594. doi:10.1080/14640748008401847.

    Article  Google Scholar 

  • Krauss, R. M., & Hadar, U. (1999). The role of speech-related arm/hand gestures in word retrieval. In R. Campbell & L. Messing (Eds.), Gesture, speech, and sign (pp. 93–116). Oxford: Oxford University Press.

    Chapter  Google Scholar 

  • Krizman, J., Marian, V., Shook, A., Skoe, E., & Kraus, N. (2012). Subcortical encoding of sound is enhanced in bilinguals and relates to executive function advantages. Proceedings of the National Academy of Sciences of the United States of America, 109(20), 7877–7881. doi:10.1073/pnas.1201575109.

    Article  Google Scholar 

  • Kroll, J. F. (2008). Juggling two languages in one mind. Psychological Science Agenda: American Psychological Association. 22.

    Book  Google Scholar 

  • Kroll, J. F., Bobb, S. C., & Wodniecka, Z. (2006). Language selectivity is the exception, not the rule: Arguments against a fixed locus of language selection in bilingual speech. Bilingualism, 9(02), 119. doi:10.1017/S1366728906002483.

    Article  Google Scholar 

  • Kuhl, P., & Meltzoff, A. N. (1982). The bimodal perception of speech in infancy. Science, 218, 1138–1141.

    Article  Google Scholar 

  • Lagrou, E., Hartsuiker, R. J., & Duyck, W. (2013). Interlingual lexical competition in a spoken sentence context: Evidence from the visual world paradigm. Psychonomic Bulletin & Review, 20(5), 963–972. doi:10.3758/s13423-013-0405-4.

    Google Scholar 

  • Liberman, A. M., & Mattingly, I. G. (1985). The motor theory of speech perception revised. Cognition, 21(1), 1–36.

    Article  Google Scholar 

  • Loveman, E., Van Hooff, J. C., & Gale, A. (2002). A systematic investigation of same and cross modality priming using written and spoken responses. Memory, 10(4), 267–276. doi:10.1080/09658210143000380.

    Article  Google Scholar 

  • Luk, G., De Sa, E., & Bialystok, E. (2011). Is there a relation between onset age of bilingualism and enhancement of cognitive control? Bilingualism: Language and Cognition, 14(4), 1–8. doi:10.1017/S1366728911000010.

  • MacDonald, J., Andersen, S., & Bachmann, T. (2000). Hearing by eye: How much spatial degradation can be tolerated? Perception, 29(10), 1155–1168. doi:10.1068/p3020.

    Article  Google Scholar 

  • Marian, V. (2009). Audio-visual integration during bilingual language processing. In A. Pavlenko (Ed.), The bilingual mental lexicon: Interdisciplinary approaches (pp. 52–78).

    Google Scholar 

  • Marian, V., & Spivey, M. J. (2003a). Bilingual and monolingual processing of competing lexical items. Applied Psycholinguistics, 24(2), 173–193. doi:10.1017/S0142716403000092.

    Article  Google Scholar 

  • Marian, V., & Spivey, M. J. (2003b). Competing activation in bilingual language processing: Within- and between-language. Bilingualism: Language and Cognition, 6(2), 97–115. doi:10.1017/S1366728903001068.

  • Marslen-Wilson, W. D. (1987). Functional parallelism in spoken word-recognition. Cognition, 25(1–2), 71–102.

    Article  Google Scholar 

  • Martin-Rhee, M. M., & Bialystok, E. (2008). The development of two types of inhibitory control in monolingual and bilingual children. Bilingualism: Language and Cognition, 11(01), 81–93. doi:10.1017/S1366728907003227.

  • McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.

    Article  Google Scholar 

  • Meyer, A. S., Sleiderink, A. M., & Levelt, W. J. M. (1998). Viewing and naming objects: Eye movements during noun phrase production. Cognition, 66(2), B25–33.

    Google Scholar 

  • Midgley, K. J., Holcomb, P. J., VanHeuven, W. J. B., & Grainger, J. (2008). An electrophysiological investigation of cross-language effects of orthographic neighborhood. Brain Research, 1246, 123–135. doi:10.1016/j.brainres.2008.09.078.

    Article  Google Scholar 

  • Morford, J. P., Wilkinson, E., Villwock, A., Piñar, P., & Kroll, J. F. (2011). When deaf signers read English: Do written words activate their sign translations? Cognition, 118(2), 286–292. doi:10.1016/j.cognition.2010.11.006.

    Article  Google Scholar 

  • Musacchia, G., Sams, M., Skoe, E., & Kraus, N. (2007). Musicians have enhanced subcortical auditory and audiovisual processing of speech and music. Proceedings of the National Academy of Sciences of the United States of America, 104(40), 15894–15898. doi:10.1073/pnas.0701498104.

    Article  Google Scholar 

  • Navarra, J., & Soto-Faraco, S. (2007). Hearing lips in a second language: Visual articulatory information enables the perception of second language sounds. Psychological Research, 71(1), 4–12. doi:10.1007/s00426-005-0031-5.

    Article  Google Scholar 

  • Nicoladis, E. (2007). The effect of bilingualism on the use of manual gestures. Applied Psycholinguistics, 28(03), 441–454. doi:10.1017/S0142716407070245.

    Article  Google Scholar 

  • Plass, J. L., Chun, D. M., Mayer, R. E., & Leutner, D. (1998). Supporting visual and verbal learning preferences in a second-language multimedia learning environment. Journal of Educational Psychology, 90(1), 25–36. doi:10.1037//0022-0663.90.1.25.

    Article  Google Scholar 

  • Preston, M. S., & Lambert, W. E. (1969). Interlingual interference in a bilingual version of the stroop color-word task. Journal of Verbal Learning and Verbal Behavior, 8(2), 295–301. doi:10.1016/S0022-5371(69)80079-4.

    Article  Google Scholar 

  • Prinz, J. J. (2006). Is the mind really modular? In R. Stainton (Ed.), Contemporary debates in cognitive science (pp. 22–36). Oxford: Blackwell.

    Google Scholar 

  • Prior, A., & MacWhinney, B. (2009). A bilingual advantage in task switching. Bilingualism: Language and Cognition, 13(2), 253. doi:10.1017/S1366728909990526.

  • Ransdell, S. E., & Fischler, I. (1987). Memory in a monolingual mode: When are bilinguals at a disadvantage? Journal of Memory and Language, 26(4), 392–405. doi:10.1016/0749-596X(87)90098-2.

    Article  Google Scholar 

  • Ronquest, R., & Hernandez, L. (2005). Lip-reading skills in bilinguals: Some effects of L1 on visual-only language identification. Research on Spoken Language Processing, Progress Report No. 27 (pp. 219–226). Bloominton, Indiana.

    Google Scholar 

  • Scarborough, D. L., Gerard, L. D., & Cortese, C. (1984). Independence of lexical access in bilingual word recognition. Journal of Verbal Learning and Verbal Behavior, 23(1), 84–99. doi:10.1016/S0022-5371(84)90519-X.

    Article  Google Scholar 

  • Schroeder, S. R., & Marian, V. (in preparation). Audio-visual interactions during memory encoding in monolinguals and bilinguals.

    Google Scholar 

  • Sebastián-Gallés, N., Echeverría, S., & Bosch, L. (2005). The influence of initial exposure on lexical representation: Comparing early and simultaneous bilinguals. Journal of Memory and Language, 52(2), 240–255. doi:10.1016/j.jml.2004.11.001.

    Article  Google Scholar 

  • Sebastián-Gallés, N., Albareda-Castellot, B., Weikum, W. M., & Werker, J. F. (2012). A bilingual advantage in visual language discrimination in infancy. Psychological Science, 23(9), 994–999. doi:10.1177/0956797612436817.

    Article  Google Scholar 

  • Shook, A., & Marian, V. (2012). Bimodal bilinguals co-activate both languages during spoken comprehension. Cognition, 124(3), 314–324. doi:10.1016/j.cognition.2012.05.014.

    Article  Google Scholar 

  • Shook, A., & Marian, V. (2013). The bilingual language interaction network for comprehension of speech (BLINCS). Bilingualism: Language and Cognition, 16, 304–324. doi:10.1017/S1366728912000466.

  • Singh, N., & Mishra, R. K. (2013). Second language proficiency modulates conflict-monitoring in an oculomotor stroop task: Evidence from Hindi-English bilinguals. Frontiers in Psychology, 4, 322. doi:10.3389/fpsyg.2013.00322.

    Google Scholar 

  • Soto-Faraco, S., Navarra, J., Weikum, W. M., Vouloumanos, A., Sebastián-Gallés, N., & Werker, J. F. (2007). Discriminating languages by speech-reading. Perception and Psychophysics, 69(2), 218–231. doi:10.3758/BF03193744.

    Article  Google Scholar 

  • Soveri, A., Laine, M., Hämäläinen, H., & Hugdahl, K. (2010). Bilingual advantage in attentional control: Evidence from the forced-attention dichotic listening paradigm. Bilingualism: Language and Cognition, 14(03), 371–378. doi:10.1017/S1366728910000118.

  • Spivey, M. J., & Marian, V. (1999). Cross talk between native and second languages: Partial activation of an irrelevant lexicon. Psychological Science, 10(3), 281. doi:10.1111/1467-9280.00151.

    Article  Google Scholar 

  • Tanenhaus, M. K., & Spivey-Knowlton, M. (1996). Eye-tracking. Language and Cognitive Processes, 11(6), 583–588.

    Article  Google Scholar 

  • Tanenhaus, M. K., Spivey-Knowlton, M., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217), 1632–1634. doi:10.1126/science.7777863.

    Article  Google Scholar 

  • Tanenhaus, M. K., Spivey-Knowlton, M., Eberhard, K. M., & Sedivy, J. C. (1996). Using eye movements to study spoken language comprehension: Evidence for visually mediated incremental interpretation. In Attention and performance XVI: Information integration in perception and communication (pp. 457–478). Boston: The MIT Press.

    Google Scholar 

  • Thierry, G., & Wu, Y. J. (2007). Brain potentials reveal unconscious translation during foreign-language comprehension. Proceedings of the National Academy of Sciences of the United States of America, 104(30), 12530–12535. doi:10.1073/pnas.0609927104.

    Article  Google Scholar 

  • Tzelgov, J., Henik, A., & Leiser, D. (1990). Controlling stroop interference: Evidence from a bilingual task. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16(5), 760–771. doi:10.1037/0278-7393.16.5.760.

    Google Scholar 

  • Van Heuven, W. J. B., Dijkstra, T., & Grainger, J. (1998). Orthographic neighborhood effects in bilingual word recognition. Journal of Memory and Language, 39(3), 458–483. doi:10.1006/jmla.1998.2584.

    Article  Google Scholar 

  • Van Wassenhove, V., Grant, K. W., & Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences of the United States of America, 102(4), 1181. doi:10.1073/pnas.0408949102.

    Article  Google Scholar 

  • Van Wassenhove, V., Grant, K. W., & Poeppel, D. (2007). Temporal window of integration in auditory-visual speech perception. Neuropsychologia, 45(3), 598–607. doi:10.1016/j.neuropsychologia.2006.01.001.

    Article  Google Scholar 

  • Watkins, M. J., & Peynircioglu, Z. F. (1983). On the nature of word recall: Evidence for linguistic specificity. Journal of Verbal Learning and Verbal Behavior, 22(4), 385–394. doi:10.1016/S0022-5371(83)90246-3.

    Article  Google Scholar 

  • Weber, A., & Cutler, A. (2004). Lexical competition in non-native spoken-word recognition. Journal of Memory and Language, 50(1), 1–25. doi:10.1016/S0749-596X(03)00105-0.

    Article  Google Scholar 

  • Weber, A., & Paris, G. (2004). The origin of the linguistic gender effect in spoken-word recognition: Evidence from non-native listening. In Proceedings of the 26th Annual Meeting of the Cognitive Science Society, 1446–1451.

    Google Scholar 

  • Weikum, W. M., Vouloumanos, A., Navarra, J., Soto-faraco, S., Sebastián-gallés, N., & Werker, J. F. (2007). Visual language discrimination in infancy. Science, 316, 1159.

    Article  Google Scholar 

Download references

Acknowledgments

Preparation of this chapter was supported in part by grant NICHD R01HD059858 to Viorica Marian. The authors would like to thank Anthony Shook, Scott Schoeder, James Bartolotti, Jennifer Krizman and Tuan Lam for helpful comments on this work.

Author information

Authors and Affiliations

  1. Department of Communication Sciences and Disorders, Northwestern University, 2240 N Campus Dr., Evanston, IL, 60208, USA

    Sarah Chabal & Viorica Marian

Authors
  1. Sarah Chabal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Viorica Marian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sarah Chabal .

Editor information

Editors and Affiliations

  1. Center for Neural and Cognitive Sciences, University of Hyderabad, Hyderabad, Andhra Pradesh, India

    Ramesh Kumar Mishra

  2. Centre of Behavioural and Cognitive Sciences, University of Allahabad, Allahabad, Uttar Pradesh, India

    Narayanan Srinivasan

  3. Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

    Falk Huettig

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer India

About this chapter

Cite this chapter

Chabal, S., Marian, V. (2015). In the Mind’s Eye: Eye-Tracking and Multi-modal Integration During Bilingual Spoken-Language Processing. In: Mishra, R., Srinivasan, N., Huettig, F. (eds) Attention and Vision in Language Processing. Springer, New Delhi. https://doi.org/10.1007/978-81-322-2443-3_9

Download citation

Publish with us

Policies and ethics

Search

Navigation