Behavior Research Methods

, Volume 50, Issue 2, pp 759–772 | Cite as

Are subjective ratings of metaphors a red herring? The big two dimensions of metaphoric sentences



What makes some metaphors easier to understand than others? Several psycholinguistic dimensions have been identified as candidate answers to this question, including appeals to familiarity and aptness. One way to operationalize these dimensions is to collect ratings of them from naive participants. In this article, we question the construct validity of this approach. Do ratings of aptness actually reflect the aptness of the metaphors? Are ratings of aptness measuring something different from ratings of familiarity? With two experiments and an analysis of existing datasets, we argue that ratings of metaphoric sentences are confounded by how easily people are able to understand the sentences (processing fluency). In the experiments, a context manipulation was designed to affect how fluently people would process the metaphors. Experiment 1 confirmed that the manipulation affected how quickly people understood the sentences in a response time task. Experiment 2 revealed that the same manipulation influenced ratings of such dimensions as familiarity and aptness. Finally, factor analyses—on the ratings data from Experiment 2 and from several existing datasets—revealed two underlying sources of variance in sentence-level ratings of metaphors (the “big two” dimensions of metaphoric sentences): processing fluency and figurativeness. We discuss the implications of these findings for theories of figurative-language processing by emphasizing more careful treatment of subjective ratings of metaphoric sentences, and by suggesting the use of alternative methods to manipulate and measure such dimensions as familiarity and aptness.


Metaphor Analogy Measurement Conventionality Language Processing fluency 

Supplementary material

13428_2017_903_MOESM1_ESM.docx (110 kb)
ESM 1 (DOCX 109 kb)
13428_2017_903_MOESM2_ESM.xlsx (63 kb)
ESM 2 (XLSX 62 kb)


  1. Alter, A. L., & Oppenheimer, D. M. (2009). Uniting the tribes of fluency to form a metacognitive nation. Personality and Social Psychology Review, 13, 219–235.CrossRefPubMedGoogle Scholar
  2. Asch, S. E. (1951). Effects of group pressure upon the modification and distortion of judgments. In H. S. Guetzkow (Ed.), Groups, leadership, and men: Research in human relations (pp. 222–236). Pittsburgh, PA: Carnegie Press.Google Scholar
  3. Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412. doi: 10.1016/j.jml.2007.12.005 CrossRefGoogle Scholar
  4. Bates, D., Maechler, M., Bolker, B., & Walker, S. (2014). lme4: Linear mixed-effects models using Eigen and S4 (R package version, 1.0-7). Retrieved from Scholar
  5. Blank, G. D. (1988). Metaphors in the lexicon. Metaphor and Symbol, 3, 21–36.CrossRefGoogle Scholar
  6. Blasko, D. G., & Connine, C. M. (1993). Effects of familiarity and aptness on metaphor processing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 295–308.PubMedGoogle Scholar
  7. Bohrn, I. C., Altmann, U., & Jacobs, A. M. (2012). Looking at the brains behind figurative language—A quantitative meta-analysis of neuroimaging studies on metaphor, idiom, and irony processing. Neuropsychologia, 50, 2669–2683.CrossRefPubMedGoogle Scholar
  8. Bowdle, B. F., & Gentner, D. (2005). The career of metaphor. Psychological Review, 112, 193.CrossRefPubMedGoogle Scholar
  9. Cardillo, E. R., Schmidt, G. L., Kranjec, A., & Chatterjee, A. (2010). Stimulus design is an obstacle course: 560 matched literal and metaphorical sentences for testing neural hypotheses about metaphor. Behavior Research Methods, 42, 651–664. doi: 10.3758/BRM.42.3.651 CrossRefPubMedPubMedCentralGoogle Scholar
  10. Cardillo, E. R., Watson, C. E., Schmidt, G. L., Kranjec, A., & Chatterjee, A. (2012). From novel to familiar: Tuning the brain for metaphors. NeuroImage, 59, 3212–3221.CrossRefPubMedGoogle Scholar
  11. Cardillo, E. R., Watson, C., & Chatterjee, A. (2016). Stimulus needs are a moving target: 240 additional matched literal and metaphorical sentences for testing neural hypotheses about metaphor. Behavior Research Methods, 49, 471–483. doi: 10.3758/s13428-016-0717-1 CrossRefGoogle Scholar
  12. Chettih, S., Durgin, F. H., & Grodner, D. J. (2012). Mixing metaphors in the cerebral hemispheres: What happens when careers collide? Journal of Experimental Psychology: Learning, Memory, and Cognition, 38, 295–311. doi: 10.1037/a0025862 PubMedGoogle Scholar
  13. Chiappe, D. L., & Kennedy, J. M. (1999). Aptness predicts preference for metaphors or similes, as well as recall bias. Psychonomic Bulletin & Review, 6, 668–676.CrossRefGoogle Scholar
  14. Chiappe, D. L., Kennedy, J. M., & Chiappe, P. (2003). Aptness is more important than comprehensibility in preference for metaphors and similes. Poetics, 31, 51–68.CrossRefGoogle Scholar
  15. Citron, F. M. M., & Goldberg, A. E. (2014). Metaphorical sentences are more emotionally engaging than their literal counterparts. Journal of Cognitive Neuroscience, 26, 2585–2595. doi: 10.1162/jocn_a_00654 CrossRefPubMedGoogle Scholar
  16. Damerall, A. W., & Kellogg, R. T. (2016). Familiarity and aptness in metaphor comprehension. American Journal of Psychology, 129, 49–64.CrossRefPubMedGoogle Scholar
  17. De Grauwe, S., Swain, A., Holcomb, P., Ditman, T., & Kuperberg, G. (2010). Electrophysiological insights into the processing of nominal metaphors. Neuropsychologia, 48, 1965–1984.CrossRefPubMedPubMedCentralGoogle Scholar
  18. Diaz, M. T., & Hogstrom, L. J. (2011). The influence of context on hemispheric recruitment during metaphor processing. Journal of Cognitive Neuroscience, 23, 3586–3597.CrossRefPubMedPubMedCentralGoogle Scholar
  19. Diaz, M. T., Barrett, K. T., & Hogstrom, L. J. (2011). The influence of sentence novelty and figurativeness on brain activity. Neuropsychologia, 49, 320–330.CrossRefPubMedGoogle Scholar
  20. Digman, J. M., & Inouye, J. (1986). Further specification of the five robust factors of personality. Journal of Personality and Social Psychology, 50, 116–123.CrossRefGoogle Scholar
  21. Dunteman, G. H. (1989). Principal components analysis (No. 69). Newbury Park, CA: Sage.CrossRefGoogle Scholar
  22. Fiske, D. W. (1949). Consistency of the factorial structures of personality ratings from different sources. Journal of Abnormal and Social Psychology, 44, 329–344.CrossRefGoogle Scholar
  23. Friedman, D., Simson, R., Ritter, W., & Rapin, I. (1975). Cortical evoked potentials elicited by real speech words and human sounds. Electroencephalography and Clinical Neurophysiology, 38, 13–19.CrossRefPubMedGoogle Scholar
  24. Gentner, D., & Markman, A. B. (1997). Structure mapping in analogy and similarity. American Psychologist, 52, 45–56. doi: 10.1037/0003-066X.52.1.45 CrossRefGoogle Scholar
  25. Gentner, D., & Wolff, P. (1997). Alignment in the processing of metaphor. Journal of Memory and Language, 37, 331–355.CrossRefGoogle Scholar
  26. Gerrig, R., & Healy, A. (1983). Dual processes in metaphor understanding: Comprehension and appreciation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 9, 667–675. doi: 10.1037/0278-7393.9.4.667 Google Scholar
  27. Gibbs, R. W. (2011). Evaluating conceptual metaphor theory. Discourse Processes, 48, 529–562.CrossRefGoogle Scholar
  28. Gibbs, R. W., Jr., & Gerrig, R. J. (1989). How context makes metaphor comprehension seem “special”. Metaphor and Symbol, 4, 145–158.CrossRefGoogle Scholar
  29. Giora, R. (1997). Understanding figurative and literal language: The graded salience hypothesis. Cognitive Linguistics, 8, 183–206.CrossRefGoogle Scholar
  30. Giora, R. (1999). On the priority of salient meanings: Studies of literal and figurative language. Journal of Pragmatics, 31, 919–929.CrossRefGoogle Scholar
  31. Giora, R. (2003). On our mind: Salience, context, and figurative language. Oxford, UK: Oxford University Press.CrossRefGoogle Scholar
  32. Giora, R. (2007). Is metaphor special? Brain and Language, 100, 111–114.CrossRefPubMedGoogle Scholar
  33. Glucksberg, S., & Haught, C. (2006). Can Florida become like the next Florida? When metaphoric comparisons fail. Psychological Science, 17, 935–938.CrossRefPubMedGoogle Scholar
  34. Glucksberg, S., & Keysar, B. (1990). Understanding metaphorical comparisons: Beyond similarity. Psychological Review, 97, 3–18. doi: 10.1037/0033-295X.97.1.3 CrossRefGoogle Scholar
  35. Henson, R. K., & Roberts, J. K. (2006). Use of exploratory factor analysis in published research common errors and some comment on improved practice. Educational and Psychological Measurement, 66, 393–416.CrossRefGoogle Scholar
  36. Jacoby, L. L., & Whitehouse, K. (1989). An illusion of memory: False recognition influenced by unconscious perception. Journal of Experimental Psychology: General, 118, 126–135. doi: 10.1037/0096-3445.118.2.126 CrossRefGoogle Scholar
  37. Jacoby, L. L., Allan, L. G., Collins, J. C., & Larwill, L. K. (1988). Memory influences subjective experience: Noise judgments. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 240–247. doi: 10.1037/0278-7393.14.2.240 Google Scholar
  38. Jones, L. L., & Estes, Z. (2005). Metaphor comprehension as attributive categorization. Journal of Memory and Language, 53, 110–124.CrossRefGoogle Scholar
  39. Jones, L. L., & Estes, Z. (2006). Roosters, robins, and alarm clocks: Aptness and conventionality in metaphor comprehension. Journal of Memory and Language, 55, 18–32. doi: 10.1016/j.jml.2006.02.004 CrossRefGoogle Scholar
  40. Kacinik, N. A., & Chiarello, C. (2007). Understanding metaphors: Is the right hemisphere uniquely involved? Brain and Language, 100, 188–207.CrossRefPubMedGoogle Scholar
  41. Kahneman, D. (2011). Thinking, fast and slow. London, UK: Macmillan.Google Scholar
  42. Katz, A. N., Paivio, A., Marschark, M., & Clark, J. M. (1988). Norms for 204 literary and 260 nonliterary metaphors on 10 psychological dimensions. Metaphor and Symbol, 3, 191–214. doi: 10.1207/s15327868ms0304_1 CrossRefGoogle Scholar
  43. Keysar, B., Shen, Y., Glucksberg, S., & Horton, W. S. (2000). Conventional language: How metaphorical is it? Journal of Memory and Language, 43, 576–593.CrossRefGoogle Scholar
  44. Kintsch, W. (2000). Metaphor comprehension: A computational theory. Psychonomic Bulletin & Review, 7, 257–266.CrossRefGoogle Scholar
  45. Kintsch, W., & Bowles, A. R. (2002). Metaphor comprehension: What makes a metaphor difficult to understand? Metaphor and Symbol, 17, 249–262.CrossRefGoogle Scholar
  46. Kuiken, D., Chudleigh, M., & Racher, D. (2010). Bilateral eye movements, attentional flexibility and metaphor comprehension: The substrate of REM dreaming? Dreaming, 20, 227–247.CrossRefGoogle Scholar
  47. Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2015). lmerTest: Tests in linear mixed effect models (R package version 2-0). Retrieved from
  48. Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104, 211–240. doi: 10.1037/0033-295X.104.2.211 CrossRefGoogle Scholar
  49. Mardia, K. V., Kent, J. T., & Bibby, J. M. (1980). Multivariate analysis. New York, NY: Academic Press.Google Scholar
  50. McCrae, R. R., & Costa, P. T., Jr. (1999). A five-factor theory of personality. In O. P. John, R. W. Robins, & L. A. Pervin (Eds.), Handbook of personality: Theory and research (2nd ed., pp. 139–153). New York, NY: Guilford Press.Google Scholar
  51. McGlone, M. S. (2011). Hyperbole, homunculi, and hindsight bias: An alternative evaluation of Conceptual Metaphor Theory. Discourse Processes, 48, 563–574.CrossRefGoogle Scholar
  52. Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67, 371–378.CrossRefGoogle Scholar
  53. Miller, G. A. (1979). Images and models, similes and metaphors. In A. Ortony (Ed.), Metaphor and thought (1st ed., pp. 202–250). Cambridge, UK: Cambridge University Press.Google Scholar
  54. Nayak, N. P., & Gibbs, R. W. (1990). Conceptual knowledge in the interpretation of idioms. Journal of Experimental Psychology: General, 119, 315–330.CrossRefGoogle Scholar
  55. Ortony, A., Schallert, D., Reynolds, R., & Antos, S. (1978). Interpreting metaphors and idioms: Some effects of context on comprehension. Journal of Verbal Learning and Verbal Behavior, 17, 465–477.CrossRefGoogle Scholar
  56. Osterhout, L. (1997). On the brain response to syntactic anomalies: Manipulations of word position and word class reveal individual differences. Brain and Language, 59, 494–522.CrossRefPubMedGoogle Scholar
  57. Roncero, C., & de Almeida, R. G. (2015). Semantic properties, aptness, familiarity, conventionality, and interpretive diversity scores for 84 metaphors and similes. Behavior Research Methods, 47, 800–812. doi: 10.3758/s13428-014-0502-y CrossRefPubMedGoogle Scholar
  58. Schmidt, G., & Seger, C. (2009). Neural correlates of metaphor processing: The roles of figurativeness, familiarity and difficulty. Brain and Cognition, 71, 375–386.CrossRefPubMedPubMedCentralGoogle Scholar
  59. Sikos, L., Thibodeau, P., Strawser, C., & Durgin, H. (2013). Advantages of extending versus mixing metaphors: An ERP study. Article presented at the CUNY Conference on Human Sentence Processing, Columbia, SC.Google Scholar
  60. Sikos, L., Thibodeau, P.H., Strawser, C., Klein, B.J., & Durgin, F.H. (2013). Advantages of extending vs. mixing metaphors: An ERP study. Poster presented at the CUNY Conference on Human Sentence Processing. Columbia, SC.Google Scholar
  61. Steen, G. J., Dorst, A. G., Herrmann, J. B., Kaal, A., Krennmayr, T., & Pasma, T. (2010). A method for linguistic metaphor identification: From MIP to MIPVU (Vol. 14). Philadelphia, PA: Benjamins.CrossRefGoogle Scholar
  62. Thibodeau, P. H., & Durgin, F. H. (2008). Productive figurative communication: Conventional metaphors facilitate the comprehension of related novel metaphors. Journal of Memory and Language, 58, 521–540.CrossRefGoogle Scholar
  63. Thibodeau, P. H., & Durgin, F. H. (2011). Metaphor aptness and conventionality: A processing fluency account. Metaphor and Symbol, 26, 206–226.CrossRefGoogle Scholar
  64. Tourangeau, R., & Sternberg, R. J. (1981). Aptness in metaphor. Cognitive Psychology, 13, 27–55.CrossRefGoogle Scholar
  65. Wolff, P., & Gentner, D. (2011). Structure-mapping in metaphor comprehension. Cognitive Science, 35, 1456–1488.CrossRefPubMedGoogle Scholar
  66. Xu, X. (2010). Interpreting metaphorical statements. Journal of Pragmatics, 42, 1622–1636.CrossRefGoogle Scholar
  67. Yang, F. G., Edens, J., Simpson, C., & Krawczyk, D. C. (2009). Differences in task demands influence the hemispheric lateralization and neural correlates of metaphor. Brain and Language, 111, 114–124.CrossRefPubMedGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  • Paul H. Thibodeau
    • 1
  • Les Sikos
    • 2
    • 3
  • Frank H. Durgin
    • 2
  1. 1.Department of PsychologyOberlin CollegeOberlinUSA
  2. 2.Swarthmore CollegeSwarthmoreUSA
  3. 3.Universität des SaarlandesSaarbrückenGermany

Personalised recommendations