Psychonomic Bulletin & Review

, Volume 22, Issue 1, pp 88–111 | Cite as

General recognition theory with individual differences: a new method for examining perceptual and decisional interactions with an application to face perception

  • Fabian A. Soto
  • Lauren Vucovich
  • Robert Musgrave
  • F. Gregory Ashby
Theoretical Review


A common question in perceptual science is to what extent different stimulus dimensions are processed independently. General recognition theory (GRT) offers a formal framework via which different notions of independence can be defined and tested rigorously, while also dissociating perceptual from decisional factors. This article presents a new GRT model that overcomes several shortcomings with previous approaches, including a clearer separation between perceptual and decisional processes and a more complete description of such processes. The model assumes that different individuals share similar perceptual representations, but vary in their attention to dimensions and in the decisional strategies they use. We apply the model to the analysis of interactions between identity and emotional expression during face recognition. The results of previous research aimed at this problem have been disparate. Participants identified four faces, which resulted from the combination of two identities and two expressions. An analysis using the new GRT model showed a complex pattern of dimensional interactions. The perception of emotional expression was not affected by changes in identity, but the perception of identity was affected by changes in emotional expression. There were violations of decisional separability of expression from identity and of identity from expression, with the former being more consistent across participants than the latter. One explanation for the disparate results in the literature is that decisional strategies may have varied across studies and influenced the results of tests of perceptual interactions, as previous studies lacked the ability to dissociate between perceptual and decisional interactions.


Mathematical models Signal detection theory Perceptual categorization and identification Face perception and recognition 


Author Note

Preparation of this article was supported in part by AFOSR grant FA9550-12-1-0355, NIH (NINDS) Grant No. P01NS044393, and by Grant No. W911NF-07-1-0072 from the U.S. Army Research Office through the Institute for Collaborative Biotechnologies. The US government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the US Government.

Supplementary material

13423_2014_661_MOESM1_ESM.pdf (36 kb)
Table S1 (PDF 35 kb)


  1. Akaike, H. (1974). A New Look at the Statistical Model Identification. IEEE Transactions on Automatic Control, 19, 716–723.CrossRefGoogle Scholar
  2. Ashby, F. G., & Lee, W. W. (1991). Predicting similarity and categorization from identification. Journal of Experimental Psychology: General, 120(2), 150.CrossRefGoogle Scholar
  3. Ashby, F. G., & Maddox, W. T. (1994). A response time theory of separability and integrality in speeded classification. Journal of Mathematical Psychology, 38(4), 423–466.CrossRefGoogle Scholar
  4. Ashby, F. G., & Soto, F. A. (2014). Multidimensional signal detection theory. In J. R. Busemeyer, J. T. Townsend, Z. Wang, & A. Eidels (Eds.), Oxford handbook of computational and mathematical psychology. New York: Oxford University Press (in press).Google Scholar
  5. Ashby, F. G., & Townsend, J. T. (1986). Varieties of perceptual independence. Psychological Review, 93(2), 154–179.PubMedCrossRefGoogle Scholar
  6. Ashby, F. G., Waldron, E. M., Lee, W. W., & Berkman, A. (2001). Suboptimality in human categorization and identification. Journal of Experimental Psychology: General, 130(1), 77.CrossRefGoogle Scholar
  7. Baudouin, J. Y., Martin, F., Tiberghien, G., Verlut, I., & Franck, N. (2002). Selective attention to facial emotion and identity in schizophrenia. Neuropsychologia, 40(5), 503–511.PubMedCrossRefGoogle Scholar
  8. Billingsley, P. (2012). Probability and Measure. Hoboken, New Jersey: John Wiley & SonsGoogle Scholar
  9. Blais, C., Arguin, M., & Marleau, I. (2009). Orientation invariance in visual shape perception. Journal of Vision, 9(2), 1–23.PubMedCrossRefGoogle Scholar
  10. Borg, I., & Groenen, P. (2005). Modern Multidimensional Scaling : Theory and Applications. New York: Springer.Google Scholar
  11. Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10(4), 433–436.PubMedCrossRefGoogle Scholar
  12. Bruce, V., & Young, A. (1986). Understanding face recognition. British Journal of Psychology, 77(3), 305–327.PubMedCrossRefGoogle Scholar
  13. Burnham, K. P., & Anderson, D. R. (2004). Multimodel inference understanding AIC and BIC in model selection. Sociological Methods and Research, 33(2), 261–304.CrossRefGoogle Scholar
  14. Carroll, J. D., & Chang, J. J. (1970). Analysis of individual differences in multidimensional scaling via an N-way generalization of “Eckart-Young” decomposition. Psychometrika, 35(3), 283–319.CrossRefGoogle Scholar
  15. Cornes, K., Donnelly, N., Godwin, H., & Wenger, M. J. (2011). Perceptual and decisional factors influencing the discrimination of inversion in the Thatcher illusion. Journal of Experimental Psychology: Human Perception and Performance, 37(3), 645.PubMedGoogle Scholar
  16. D’Errico, J. (2006). Adaptive robust numerical differentiation. MATLAB Central File Exchange. Retrieved April 19, 2014, from
  17. Dailey, M., Cottrell, G. W., & Reilly, J. (2001). California facial expressions, CAFE. Unpublished digital images, University of California, San Diego, Computer Science and Engineering Department.Google Scholar
  18. de Beeck, H. P. O., Haushofer, J., & Kanwisher, N. G. (2008). Interpreting fMRI data: maps, modules and dimensions. Nature Reviews Neuroscience, 9(2), 123–135.CrossRefGoogle Scholar
  19. Ekman, P., Friesen, W. V., & Hager, J. (1978). The Facial Action Coding System (FACS): A technique for the measurement of facial action Palo Alto. Palo Alto: Consulting Psychologists.Google Scholar
  20. Ellamil, M., Susskind, J. M., & Anderson, A. K. (2008). Examinations of identity invariance in facial expression adaptation. Cognitive, Affective, and Behavioral Neuroscience, 8(3), 273.CrossRefGoogle Scholar
  21. Ennis, D. M., & Ashby, F. G. (2003). Fitting the decision bound models to identification categorization data. Santa Barbara: University of California.Google Scholar
  22. Etcoff, N. L. (1984). Selective attention to facial identity and facial emotion. Neuropsychologia, 22(3), 281–295.PubMedCrossRefGoogle Scholar
  23. Fitousi, D., & Wenger, M. J. (2013). Variants of independence in the perception of facial identity and expression. Journal of Experimental Psychology: Human Perception and Performance, 39(1), 133–155.PubMedGoogle Scholar
  24. Fox, C. J., & Barton, J. J. S. (2007). What is adapted in face adaptation? The neural representations of expression in the human visual system. Brain Research, 1127, 80–89.PubMedCrossRefGoogle Scholar
  25. Fox, C. J., Oruç, I., & Barton, J. J. S. (2008). It doesn’t matter how you feel. The facial identity aftereffect is invariant to changes in facial expression. Journal of Vision, 8(3), 11.PubMedCrossRefGoogle Scholar
  26. Ganel, T., & Goshen-Gottstein, Y. (2004). Effects of familiarity on the perceptual integrality of the identity and expression of faces: The parallel-route hypothesis revisited. Journal of Experimental Psychology: Human Perception and Performance, 30(3), 583–596.PubMedGoogle Scholar
  27. Ganel, T., Valyear, K. F., Goshen-Gottstein, Y., & Goodale, M. A. (2005). The involvement of the “fusiform face area” in processing facial expression. Neuropsychologia, 43(11), 1645–1654.PubMedCrossRefGoogle Scholar
  28. Garner, W. R. (1974). The processing of information and structure. New York: Erlbaum.Google Scholar
  29. Hartigan, J. A., & Hartigan, P. M. (1985). The dip test of unimodality. The Annals of Statistics, 70–84.Google Scholar
  30. Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2000). The distributed human neural system for face perception. Trends in Cognitive Sciences, 4(6), 223–232.PubMedCrossRefGoogle Scholar
  31. Kadlec, H., & Townsend, J. T. (1992a). Signal detection analysis of multidimensional interactions. In F. G. Ashby (Ed.), Multidimensional Models of Perception and Cognition (pp. 181–231). Hillsdale, NJ: Erlbaum.Google Scholar
  32. Kadlec, H., & Townsend, J. T. (1992b). Implications of marginal and conditional detection parameters for the separabilities and independence of perceptual dimensions. Journal of Mathematical Psychology, 36(3), 325–374.CrossRefGoogle Scholar
  33. Kanwisher, N. (2000). Domain specificity in face perception. Nature Neuroscience, 3, 759–763.PubMedCrossRefGoogle Scholar
  34. Lee, M. D., & Wetzels, R. (2010). Individual differences in attention during category learning. In: R. Catrambone & S. Ohlsson (Eds.), Proceedings of the 32nd Annual Conference of the Cognitive Science Society (pp. 387–392). Austin, TX: Cognitive Science Society.Google Scholar
  35. Lehky, S. R. (2000). Fine discrimination of faces can be performed rapidly. Journal of Cognitive Neuroscience, 12(5), 848–855.PubMedCrossRefGoogle Scholar
  36. Mack, M. L., Richler, J. J., Gauthier, I., & Palmeri, T. J. (2011). Indecision on decisional separability. Psychonomic Bulletin & Review, 18(1), 1–9.CrossRefGoogle Scholar
  37. Maddox, W. T., & Ashby, F. G. (1996). Perceptual separability, decisional separability, and the identification- speeded classification relationship. Journal of Experimental Psychology: Human Perception & Performance, 22, 795–817Google Scholar
  38. Maddox, W. T., Ashby, F. G., & Waldron, E. M. (2002). Multiple attention systems in perceptual categorization. Memory and Cognition, 30, 325–339.PubMedCrossRefGoogle Scholar
  39. Mestry, N., Wenger, M. J., & Donnelly, N. (2012). Identifying sources of configurality in three face processing tasks. Frontiers in Perception Science, 3, 456.Google Scholar
  40. Navarro, D. J., Griffiths, T. L., Steyvers, M., & Lee, M. D. (2006). Modeling individual differences using Dirichlet processes. Journal of Mathematical Psychology, 50(2), 101–122.CrossRefGoogle Scholar
  41. Pell, P. J., & Richards, A. (2013). Overlapping facial expression representations are identity-dependent. Vision Research, 79(7), 1–7.PubMedCrossRefGoogle Scholar
  42. Preacher, K. J., & Merkle, E. C. (2012). The problem of model selection uncertainty in structural equation modeling. Psychological Methods, 17(1), 1.PubMedCrossRefGoogle Scholar
  43. Richler, J. J., Gauthier, I., Wenger, M. J., & Palmeri, T. J. (2008). Holistic Processing of Faces: Perceptual & Decisional Components. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(2), 328–342.PubMedGoogle Scholar
  44. Schweinberger, S. R., Burton, A. M., & Kelly, S. W. (1999). Asymmetric dependencies in perceiving identity and emotion: Experiments with morphed faces. Perception & Psychophysics, 61(6), 1102–1115.CrossRefGoogle Scholar
  45. Schweinberger, S. R., & Soukup, G. R. (1998). Asymmetric relationships among perceptions of facial identity, emotion, and facial speech. Journal of Experimental Psychology: Human Perception and Performance, 24(6), 1748–1765.PubMedGoogle Scholar
  46. Silbert, N. H. (2012). Syllable structure and integration of voicing and manner of articulation information in labial consonant identification. The Journal of the Acoustical Society of America, 131(5), 4076–4086.PubMedCentralPubMedCrossRefGoogle Scholar
  47. Silbert, N. H., & Thomas, R. (2013). Decisional separability, model identification, and statistical inference in the general recognition theory framework. Psychonomic Bulletin & Review, 20(1), 1–20.CrossRefGoogle Scholar
  48. Soto, F. A., & Wasserman, E. A. (2011). Asymmetrical interactions in the perception of face identity and emotional expression are not unique to the primate visual system. Journal of Vision, 11(3).Google Scholar
  49. Stankiewicz, B. J. (2002). Empirical evidence for independent dimensions in the visual representation of three-dimensional shape. Journal of Experimental Psychology: Human Perception and Performance, 28(4), 913–932.PubMedGoogle Scholar
  50. Thomas, R. (2001). Perceptual interactions of facial dimensions in speeded classification and identification. Attention, Perception, & Psychophysics, 63(4), 625–650.CrossRefGoogle Scholar
  51. Thomas, R. D., & Silbert, N. H. (2014). Technical clarification to Silbert and Thomas (2013): “Decisional separability, model identification, and statistical inference in the general recognition theory framework”. Psychonomic Bulletin & Review, 21(2), 574–575.CrossRefGoogle Scholar
  52. Ungerleider, L. G., & Haxby, J. V. (1994). “What” and “where” in the human brain. Current Opinion in Neurobiology, 4(2), 157–165.PubMedCrossRefGoogle Scholar
  53. Vogels, R., Biederman, I., Bar, M., & Lorincz, A. (2001). Inferior temporal neurons show greater sensitivity to nonaccidental than to metric shape differences. Journal of Cognitive Neuroscience, 13(4), 444–453.PubMedCrossRefGoogle Scholar
  54. Wald, A. (1943). Tests of statistical hypotheses concerning several parameters when the number of observations is large. Transactions of the American Mathematical Society, 54(3), 426–482.CrossRefGoogle Scholar
  55. Yankouskaya, A., Booth, D. A., & Humphreys, G. (2012). Interactions between facial emotion and identity in face processing: Evidence based on redundancy gains. Attention, Perception and Psychophysics, 74(8), 1692–1711.PubMedCrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2014

Authors and Affiliations

  • Fabian A. Soto
    • 1
    • 2
  • Lauren Vucovich
    • 2
  • Robert Musgrave
    • 2
  • F. Gregory Ashby
    • 2
  1. 1.Sage Center for the Study of the MindUniversity of California at Santa BarbaraSanta BarbaraUSA
  2. 2.Department of Psychological and Brain SciencesUniversity of California at Santa BarbaraSanta BarbaraUSA

Personalised recommendations