Abstract
In the present study, we attempted to demonstrate a synesthetic relationship between auditory frequency and visual size. In Experiment 1, participants performed a speeded visual size discrimination task in which they had to judge whether a variable-sized disk was bigger or smaller than a standard reference disk. A task-irrelevant sound that was either synesthetically congruent with the relative size of the disk (e.g., a low-frequency sound presented with a bigger disk) or synesthetically incongruent with it (e.g., a low-frequency sound presented with a smaller disk) was sometimes presented together with the variable disk. Reaction times were shorter in the synesthetically congruent condition than in the incongruent condition. Verbal labeling and semantic mediation interpretations of this interaction were explored in Experiment 2, in which high- and low-frequency sounds were presented in separate blocks of trials, and in Experiment 3, in which the tones were replaced by the spoken words “high” and “low.” Response priming/bias explanations were ruled out in Experiment 4, in which a synesthetic congruency effect was still reported even when participants made same-versus-different discrimination responses regarding the relative sizes of the two disks. Taken together, these results provide the first empirical demonstration that the relative frequency of an irrelevant sound can influence the speed with which participants judge the size of visual stimuli when the sound varies on a trial-by-trial basis along a synesthetically compatible dimension. The possible cognitive bases for this synesthetic association are also discussed.
Article PDF
References
Atchley, P., Kramer, A. F., &Hillstrom, A. P. (2000). Contingent capture for onsets and offsets: Attentional set for perceptual transients.Journal of Experimental Psychology: Human Perception & Performance,26, 594–606.
Baron-Cohen, S., Harrison, J., Goldstein, L. H., &Wyke, M. (1993). Coloured speech perception: Is synaesthesia what happens when modularity breaks down?Perception,22, 419–426.
Bee, M. A., Perrill, S. A., &Owen, P. C. (2000). Male green frogs lower the pitch of acoustic signals in defense of territories: A possible dishonest signal of size?Behavioral Ecology,11, 169–177.
Bernstein, I. H., &Edelstein, B. A. (1971). Effects of some variations in auditory input upon visual choice reaction time.Journal of Experimental Psychology,87, 241–247.
Bertelson, P., &Tisseyre, F. (1969). The time-course of preparation: Confirmatory results with visual and auditory warning signals.Acta Psychologica,30, 145–154.
Brown, R. (1958).Words and things. Glencoe, IL: Free Press.
Carello, C., Anderson, K. L., &Kunkler-Peck, A. J. (1998). Perception of object length by sound.Psychological Science,9, 211–214.
Chastain, G., &Cheal, M. (2001). Attentional capture with various distractor and target types.Perception & Psychophysics,63, 979–990.
Clark, H. H., &Brownell, H. H. (1975). Position, direction, and their variations in auditory input upon visual choice reaction time.Journal of Experimental Psychology: Human Perception & Performance,1, 339–352.
Dixon, M. J., Smilek, D., Cudahy, C., &Merikle, P. M. (2000). Five plus two equals yellow.Nature,406, 365.
Driver, J., &Spence, C. (2004). Crossmodal spatial attention: Evidence from human performance. In C. Spence & J. Driver (Eds.),Crossmodal space and crossmodal attention (pp. 179–220). Oxford: Oxford University Press.
Finney, D. J. (1964).Probit analysis: Statistical treatment of the sigmoid response curve. London: Cambridge University Press.
Fitch, W. T., &Reby, D. (2001). The descended larynx is not uniquely human.Proceedings of the Royal Society of London: Series B,268, 1669–1675.
Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement.Journal of Experimental Psychology,47, 381–391.
Garner, W. R. (1974).The processing of information and structure. Hillsdale, NJ: Erlbaum.
Garner, W. R. (1977). The effect of absolute size on the separability of the dimensions of size and brightness.Bulletin of the Psychonomic Society,9, 380–382.
Grassi, M. (2005). Do we hear size or sound? Balls dropped on plates.Perception & Psychophysics,67, 274–284.
Grossenbacher, P. G., &Lovelace, C. T. (2001). Mechanisms of synesthesia: Cognitive and physiological constraints.Trends in Cognitive Sciences,5, 36–41.
Harrington, F. H. (1987). Aggressive howling in wolves.Animal Behaviour,35, 7–12.
Howells, T. (1944). The experimental development of color-tone synesthesia.Journal of Experimental Psychology,34, 87–103.
Jonides, J., &Mack, R. (1984). On the cost and benefit of cost and benefit.Psychological Bulletin,96, 29–44.
Karwoski, T. F., & Odbert, H. S. (1938). Color-music. Psychological Monographs, 50(2, Whole No. 22).
Kelly, E. (1934). An experimental attempt to produce artificial chromaesthesia by the technique of the conditioned response.Journal of Experimental Psychology,17, 315–341.
Köhler, W. (1929).Gestalt psychology. New York: Liveright.
Long, J. (1977). Contextual assimilation and its effect on the division of attention between nonverbal signals.Quarterly Journal of Experimental Psychology,29, 397–414.
Maddox, W. T. (1992). Perceptual and decisional separability. In F. G. Ashby (Ed.),Multidimensional models of perception and cognition (pp. 147–180). Hillsdale, NJ: Erlbaum.
Marks, L. E. (1975). On colored-hearing synesthesia: Cross-modal translations of sensory dimensions.Psychological Bulletin,82, 303–331.
Marks, L. E. (1978).The unity of the senses: Interrelations among the modalities. London: Academic Press.
Marks, L. E. (1987). On cross-modal similarity: Auditory-visual interactions in speeded discrimination.Journal of Experimental Psychology: Human Perception & Performance,13, 384–394.
Marks, L. E. (2000). Synesthesia. In E. Cardeña, S. J. Lynn, & S. C. Krippner (Eds.),Varieties of anomalous experience: Examining the scientific evidence (pp. 121–149). Washington, DC: American Psychological Association.
Marks, L. E. (2004). Cross-modal interactions in speeded classification. In G. A. Calvert, C. Spence, & B. E. Stein (Eds.),Handbook of multisensory processes (pp. 85–105). Cambridge, MA: MIT Press.
Marks, L. E., Hammeal, R. J., &Bornstein, M. H. (1987). Perceiving similarity and comprehending metaphor.Monographs of the Society for Research in Child Development,52, 1–102.
Martino, G., &Marks, L. E. (1999). Perceptual and linguistic interactions in speeded classification: Test of the semantic coding hypothesis.Perception,28, 903–923.
Martino, G., &Marks, L. E. (2000). Cross-modal interaction between vision and touch: The role of synesthetic correspondence.Perception,29, 745–754.
Maurer, D. (1997). Neonatal synaesthesia: Implications for the processing of speech and faces. In S. Baron-Cohen and J. Harrison (Eds.),Synaesthesia: Classic and contemporary readings (pp. 224–242). Oxford: Blackwell.
McClain, L. (1983). Stimulus-response compatibility affects auditory Stroop interference.Perception & Psychophysics,33, 266–270.
McMahon, T. A., &Bonner, J. T. (1983).On size and life. New York: Scientific American.
Melara, R. D., &Algom, D. (2003). Driven by information: A tectonic theory of Stroop effects.Psychological Review,110, 422–471.
Melara, R. D., &O'Brien, T. P. (1987). Interaction between synesthetically corresponding dimensions.Journal of Experimental Psychology: General,116, 323–336.
Mondloch, C. J., &Maurer, D. (2004). Do small white balls squeak? Pitch-object correspondences in young children.Cognitive, Affective, & Behavioral Neuroscience,4, 133–136.
Müller, H. J., &Findlay, J. M. (1987). Sensitivity and criterion effects in the spatial cuing of visual attention.Perception & Psychophysics,42, 383–399.
Odbert, H. S., Karwoski, T. F., &Eckerson, A. B. (1942). Studies in synesthetic thinking: I. Musical and verbal associations of color and mood.Journal of General Psychology,26, 153–173.
Osgood, C. E., Suci, G. J., &Tannenbaum, P. H. (1957).The measurement of meaning. Urbana: University of Illinois Press.
Patching, G. R., &Quinlan, P. T. (2002). Garner and congruence effects in the speeded classification of bimodal signals.Journal of Experimental Psychology: Human Perception & Performance,28, 755–775.
Pomerantz, J. R. (1981). Perceptual organization in information processing. In M. Kubovy & J. R. Pomerantz (Eds.),Perceptual organization (pp. 141–180). Hillsdale, NJ: Erlbaum.
Pomerantz, J. R. (1985, November).Selective attention: Difference between Stroop and Garner interference. Paper presented at the 26th Annual Meeting of the Psychonomic Society, Boston.
Pomerantz, J. R., Pristach, E. A., &Carson, C. E. (1989). Attention and object perception. In B. E. Shepp & S. Ballesteros (Eds.),Object perception: Structure and process (pp. 53–89). Hillsdale, NJ: Erlbaum.
Posner, M. I. (1978).Chronometric explorations of mind. Hillsdale, NJ: Erlbaum.
Pratt, C. C. (1930). The spatial character of high and low tones.Journal of Experimental Psychology,13, 278–285.
Ramachandran, V. S., &Hubbard, E. M. (2001a). Psychophysical investigations into the neural basis of synaesthesia.Proceedings of the Royal Society of London: Series B,268, 979–983.
Ramachandran, V. S., &Hubbard, E. M. (2001b). Synaesthesia: A window into perception, thought and language.Journal of Consciousness Studies,8, 3–34.
Rubinstein, O., &Henik, A. (2002). Is an ant larger than a lion?Acta Psychologica,111, 141–154.
Sabri, M., Melara, R. D., &Algom, D. (2001). A confluence of contexts: Asymmetric versus global failures of selective attention to Stroop dimensions.Journal of Experimental Psychology: Human Perception & Performance,27, 515–537.
Shore, D. I., &Spence, C. (2005). Prior entry. In L. Itti, G. Rees, & J. Tsotsos (Eds.),Neurobiology of attention (pp. 89–95). Amsterdam: Elsevier, North-Holland.
Spence, C., &Driver, J. (1997). Audiovisual links in endogenous covert spatial orienting.Perception & Psychophysics,59, 1–22.
Spence, C., &McDonald, J. (2004). The crossmodal consequence of the exogenous spatial orienting of attention. In G. A. Calvert, C. Spence, & B. E. Stein (Eds.),The handbook of multisensory processing (pp. 3–25). Cambridge, MA: MIT Press.
Spence, C., Shore, D. I., &Klein, R. M. (2001). Multisensory prior entry.Journal of Experimental Psychology: General,130, 799–832.
Spence, C., &Zampini, M. (2006). Auditory contributions to multisensory product perception.Acta Acustica United With Acustica,92, 1009–1025.
Stevens, J. C., &Marks, L. E. (1965). Cross-modality matching of brightness and loudness.Proceedings of the National Academy of Sciences,54, 407–411.
Stumpf, K. (1883).Tonpsychologie I [Psychology of tone]. Leipzig: Hirzel.
Taylor, R. L., &Campbell, G. T. (1976). Sensory interaction: Vision is modulated by hearing.Perception,5, 467–477.
van den Doel, K., &Pai, D. K. (1998). The sounds of physical shapes.Presence,7, 382–395.
van Wassenhove, V., Grant, K. W., &Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech.Proceedings of the National Academy of Sciences,102, 1181–1186.
Virzi, R. A., &Egeth, H. E. (1984). Is meaning implicated in illusory conjunctions?Journal of Experimental Psychology: Human Perception & Performance,10, 573–580.
Virzi, R. A., &Egeth, H. E. (1985). Toward a translational model of Stroop interference.Memory & Cognition,13, 304–319.
Walker, P., &Smith, S. (1984). Stroop interference based on the synaesthetic qualities of auditory pitch.Perception,13, 75–81.
Walker, R. (1987). The effects of culture, environment, age, and musical training on choices of visual metaphors for sound.Perception & Psychophysics,42, 491–502.
Wang, H., &Proctor, R. W. (1996). Stimulus-response compatibility as a function of stimulus code and response modality.Journal of Experimental Psychology: Human Perception & Performance,22, 1201–1217.
Ward, J., &Simner, J. (2003). Lexical-gustatory synaesthesia: Linguistic and conceptual factors.Cognition,89, 237–261.
Wollen, K. A., &Ruggiero, F. T. (1983). Colored-letter synesthesia.Journal of Mental Imagery,7, 83–86.
Zigler, M. J. (1930). Tone shapes: A novel type of synaesthesia.Journal of General Psychology,3, 276–287.
Author information
Authors and Affiliations
Corresponding author
Additional information
A.G. was supported by a grant from the Università di Milano Bicocca.
Rights and permissions
About this article
Cite this article
Gallace, A., Spence, C. Multisensory synesthetic interactions in the speeded classification of visual size. Perception & Psychophysics 68, 1191–1203 (2006). https://doi.org/10.3758/BF03193720
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03193720