Abstract
The assessment of creativity as an individual difference has historically focused on divergent thinking, which is increasingly viewed as involving the associative processes that are also understood to be a key component of creative potential. Research on associative processes has proliferated in many sub-fields, often using Compound Remote Associates (CRA) tasks with an open response format and relatively small participant samples. In the present work, we introduce a new format that is more amenable to large-scale data collection in survey designs, and present evidence for the reliability and validity of CRA measures in general using multiple large samples. Study 1 uses a large, representative dataset (N = 1,323,480) to demonstrate strong unidimensionality and internal consistency (α = .97; ωt = .87), as well as links to individual differences in temperament, cognitive ability, occupation, and job characteristics. Study 2 uses an undergraduate sample (N = 685) to validate the use of a multiple-choice format relative to the traditional approach. Study 3 uses a crowdsourced sample (N = 357) to demonstrate high test–retest reliability of the items (r =.74). Finally, Study 4 uses a sample that overlaps with Study 1 (N = 1,502,922) to provide item response theory (IRT) parameters for a large set of high-quality CRA items that use a multiple-choice response mode, thus facilitating their use in future research on creativity, insight, and related topics.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availibility
Data, analysis code, and other supplemental materials are available at https://tinyurl.com/yuskme6n.
Notes
These last two options were included for consistency of formatting with other items in the ICAR framework. The “None of these” option was not a correct answer for any of the items described in the present studies.
References
Ackerman, P. L., & Heggestad, E. D. (1997). Intelligence, personality, and interests: evidence for overlapping traits. Psychological Bulletin, 121(2), 219. https://doi.org/10.1037/0033-2909.121.2.219
Agnoli, S., Vanucci, M., Pelagatti, C., & Corazza, G. E. (2018). Exploring the link between mind wandering, mindfulness, and creativity: A multidimensional approach. Creativity Research Journal, 30(1), 41–53. https://doi.org/10.1080/10400419.2018.1411423
Aiello, D., Jarosz, A., Cushen, P., & Wiley, J. (2012). Firing the Executive: When an Analytic Approach to Problem Solving Helps and Hurts. The Journal of Problem Solving, 4(2). https://doi.org/10.7771/1932-6246.1128
Aitken Harris, J. (2004). Measured intelligence, achievement, openness to experience, and creativity. Personality and Individual Differences, 36(4), 913–929. https://doi.org/10.1016/S0191-8869(03)00161-2
Akbari Chermahini, S., Hickendorff, M., & Hommel, B. (2012). Development and validity of a dutch version of the remote associates task: An item-response theory approach. Thinking Skills and Creativity, 7(3), 177–186. https://doi.org/10.1016/j.tsc.2012.02.003
Ansburg, P. I., & Hill, K. (2003). Creative and analytic thinkers differ in their use of attentional resources. Personality and Individual Differences, 34(7), 1141–1152. https://doi.org/10.1016/S0191-8869(02)00104-6
Arden, R., Chavez, R. S., Grazioplene, R., & Jung, R. E. (2010). Neuroimaging creativity: A psychometric view. Behavioural Brain Research, 214(2), 143–156. https://doi.org/10.1016/j.bbr.2010.05.015
Ashton, M. C., & Lee, K. (2008). The HEXACO model of personality structure and the importance of the H factor. Social and Personality Psychology Compass, 2(5), 1952–1962. https://doi.org/10.1111/j.1751-9004.2008.00134.x
Baird, B., Smallwood, J., Mrazek, M. D., Kam, J. W. Y., Franklin, M. S., & Schooler, J. W. (2012). Inspired by distraction: Mind wandering facilitates creative incubation. Psychological Science, 23(10), 1117–1122. https://doi.org/10.1177/0956797612446024
Barr, N., Pennycook, G., Stolz, J. A., & Fugelsang, J. A. (2015). Reasoned connections: A dual-process perspective on creative thought. Thinking & Reasoning, 21(1), 61–75. https://doi.org/10.1080/13546783.2014.895915
Barron, F., & Harrington, D. M. (1981). Creativity, intelligence, and personality. Annual Review of Psychology, 32(1). https://doi.org/10.1146/annurev.ps.32.020181.002255
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
Bates, D., Maechler, M., Bolker, B., Walker, S., Christensen, R. H. B., Singmann, H., & Krivitsky, P. N. (2021). lme4: Linear Mixed‐Effects Models using “Eigen” and S4 (Version 1.1‐27.1). Computer software]. https://cran.r-project.org/package=lme4
Batey, M., & Furnham, A. (2006). Creativity, intelligence, and personality: A critical review of the scattered literature. Genetic, Social, and General Psychology Monographs, 132(4). https://doi.org/10.3200/MONO.132.4.355-430
Beaty, R. E., Benedek, M., Barry Kaufman, S., & Silvia, P. J. (2015). Default and executive network coupling supports creative idea production. Scientific Reports, 5(1). https://doi.org/10.1038/srep10964
Beaty, R. E., & Johnson, D. R. (2021). Automating creativity assessment with SemDis: An open platform for computing semantic distance. Behavior Research Methods, 53(2), 757–780. https://doi.org/10.3758/s13428-020-01453-w
Beaty, R. E., & Johnson, D. R. (2021). Automating creativity assessment with SemDis: An open platform for computing semantic distance. Behavior Research Methods, 53(2), 757–780.
Beaty, R. E., Silvia, P. J., Nusbaum, E. C., Jauk, E., & Benedek, M. (2014). The roles of associative and executive processes in creative cognition. Memory & Cognition, 42(7), 1186–1197. https://doi.org/10.3758/s13421-014-0428-8
Beaussart, M. L., Andrews, C. J., & Kaufman, J. C. (2013). Creative liars: The relationship between creativity and integrity. Thinking Skills and Creativity, 9, 129–134. https://doi.org/10.1016/j.tsc.2012.10.003
Beda, Z., & Smith, S. M. (2018). Chasing red herrings: Memory of distractors causes fixation in creative problem solving. Memory & Cognition, 46(5), 671–684. https://doi.org/10.3758/s13421-018-0799-3
Behrens, J. P., & Olteţeanu, A. M. (2020). Are all remote associates tests equal? An overview of the remote associates test in different languages. Frontiers in Psychology, 11, 1125.
Benedek, M., & Fink, A. (2019). Toward a neurocognitive framework of creative cognition: The role of memory, attention, and cognitive control. Current Opinion in Behavioral Sciences, 27, 116–122.
Benedek, M., & Neubauer, A. C. (2013). Revisiting Mednick’s model on creativity-related differences in associative hierarchies. Evidence for a common path to uncommon thought. The Journal of Creative Behavior, 47(4). https://doi.org/10.1002/jocb.35
Benedek, M., Jauk, E., Sommer, M., Arendasy, M., & Neubauer, A. C. (2014). Intelligence, creativity, and cognitive control: The common and differential involvement of executive functions in intelligence and creativity. Intelligence, 46, 73–83. https://doi.org/10.1016/j.intell.2014.05.007
Benedek, M., Kenett, Y. N., Umdasch, K., Anaki, D., Faust, M., & Neubauer, A. C. (2017). How semantic memory structure and intelligence contribute to creative thought: A network science approach. Thinking & Reasoning, 23(2), 158–183.
Benedek, M., Franz, F., Heene, M., & Neubauer, A. C. (2012). Differential effects of cognitive inhibition and intelligence on creativity. Personality and Individual Differences, 53(4), 480–485. https://doi.org/10.1016/j.paid.2012.04.014
Benedek, M., Könen, T., & Neubauer, A. C. (2012). Associative abilities underlying creativity. Psychology of Aesthetics, Creativity, and the Arts, 6(3), 273–281. https://doi.org/10.1037/a0027059
Blum, D., Holling, H., Galibert, M. S., & Forthmann, B. (2016). Task difficulty prediction of figural analogies. Intelligence, 56, 72–81. https://doi.org/10.1016/j.intell.2016.03.001
Borkenau, P., & Liebler, A. (1992). Trait inferences: Sources of validity at zero acquaintance. Journal of Personality and Social Psychology, 62(4), 645–657. https://doi.org/10.1037/0022-3514.62.4.645
Bowden, E. M., & Jung-Beeman, M. (2003). Normative data for 144 compound remote associate problems. Behavior Research Methods, Instruments, & Computers, 35(4). https://doi.org/10.3758/BF03195543
Bowden, E. M., & Jung-Beeman, M. (2007). Methods for investigating the neural components of insight. Methods, 42(1), 87–99.
Bowers, P. (1978). Hypnotizability, creativity and the role of effortless experiencing. The International Journal of Clinical and Experimental Hypnosis, 26(3). https://doi.org/10.1080/00207147808409320
Carson, S. (2010). Latent inhibition and creativity. In I. Weiner & R. Lubow (Eds.), Latent Inhibition: Cognition Neuroscience and Applications to Schizophrenia (pp. 183–198). Cambridge University Press. https://doi.org/10.1017/CBO9780511730184.010
Cattell, R. B. (1943). The measurement of adult intelligence. Psychological Bulletin, 40(3), 153–193. https://doi.org/10.1037/h0059973
Chalmers, R. P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48, 1–29. https://doi.org/10.18637/jss.v048.i06
Chamorro-Premuzic, T., & Furnham, A. (2003). Personality predicts academic performance: Evidence from two longitudinal university samples. Journal of Research in Personality, 37(4). https://doi.org/10.1016/S0092-6566(02)00578-0
Chamorro-Premuzic, T., Furnham, A., & Ackerman, P. L. (2006). Ability and personality correlates of general knowledge. Personality and Individual Differences, 41(3). https://doi.org/10.1016/j.paid.2005.11.036
Chein, J. M., & Weisberg, R. W. (2014). Working memory and insight in verbal problems: Analysis of compound remote associates. Memory & Cognition, 42(1). https://doi.org/10.3758/s13421-013-0343-4
Clark, L. A., & Watson, D. (1991). Tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. Journal of Abnormal Psychology, 100(3), 316–336. https://doi.org/10.1037//0021-843x.100.3.316
Clark, R., & DeYoung, C. (2014). Creativity and the aspects of neuroticism. Personality and Individual Differences, 60, S54. https://doi.org/10.1016/j.paid.2013.07.224
Condon, D. M. (2018). The SAPA Personality Inventory: An empirically-derived, hierarchically-organized self-report personality assessment model. https://doi.org/10.31234/osf.io/sc4p9
Condon, D. M., & Revelle, W. (2014). The international cognitive ability resource: Development and initial validation of a public-domain measure. Intelligence, 43, 52–64. https://doi.org/10.1016/j.intell.2014.01.004
Corgnet, B., Espín, A. M., & Hernán-González, R. (2016). Creativity and cognitive skills among millennials: Thinking too much and creating too little. Frontiers in Psychology, 7, 1626. https://doi.org/10.3389/fpsyg.2016.01626
Corr, P. J. (2008). Reinforcement Sensitivity Theory (RST): Introduction. In P. J. Corr (Ed.), The reinforcement sensitivity theory of personality (pp. 1–43). Cambridge University Press. https://doi.org/10.1017/CBO9780511819384.002
Costa, P. T., & McCrae, R. R. (1980). Influence of extraversion and neuroticism on subjective well-being: Happy and unhappy people. Journal of Personality and Social Psychology, 38(4), 668–678. https://doi.org/10.1037//0022-3514.38.4.668
Costa, P. T., & McCrae, R. R. (1992). The five-factor model of personality and its relevance to personality disorders. Journal of Personality Disorders, 6(4), 343–359. https://doi.org/10.1521/pedi.1992.6.4.343
Costa, P. T., Jr., & McCrae, R. R. (2008). The Revised NEO Personality Inventory (NEO-PI-R). In G. J. Boyle, G. Matthews, & D. H. Saklofske (Eds.), The SAGE handbook of personality theory and assessment: Personality measurement and testing (1st ed., pp. 179–198). UK: Sage Publications Inc. https://doi.org/10.4135/9781849200479.n9
Cotter, K. N., Pretz, J. E., & Kaufman, J. C. (2016). Applicant extracurricular involvement predicts creativity better than traditional admissions factors. Psychology of Aesthetics, Creativity, and the Arts, 10(1), 2–13. https://doi.org/10.1037/a0039831
Csikszentmihalyi, M., & Beattie, O. V. (1979). Life themes: a theoretical and empirical exploration of their origin and effects. Journal of Humanistic Psychology, 19, 45–63.
Davies, M. (2010). The Corpus of Contemporary American English as the first reliable monitor corpus of English. Literary and Linguistic Computing, 25(4), 447–464. https://doi.org/10.1093/llc/fqq018
DeYoung, C. G. (2006). Higher-order factors of the Big Five in a multi-informant sample. Journal of Personality and Social Psychology, 91(6), 1138–1151. https://doi.org/10.1037/0022-3514.91.6.1138
DeYoung, C. G., Quilty, L. C., & Peterson, J. B. (2007). Between facets and domains: 10 aspects of the Big Five. Journal of Personality and Social Psychology, 93(5), 880–896. https://doi.org/10.1037/0022-3514.93.5.880
DiBello, L. V., Roussos, L. A., & Stout, W. F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. R. Rao & S. Sinharay (Eds.), Handbook of Statistics (Vol. 26, pp. 979–1030). Elsevier.
Dworak, E. M., Revelle, W., Doebler, P., & Condon, D. M. (2021). Using the International Cognitive Ability Resource as an open source tool to explore individual differences in cognitive ability. Personality and Individual Differences, 169, 109906. https://doi.org/10.1016/j.paid.2020.109906
Elleman, L. G., McDougald, S. K., Condon, D. M., & Revelle, W. (2021). That takes the biscuit: Predictive accuracy and parsimony of four statistical learning techniques in personality data, with data missingness conditions. European Journal of Psychological Assessment, 36(6), 948. https://doi.org/10.1027/1015-5759/a000590
Feist, G. J. (1998). A meta-analysis of personality in scientific and artistic creativity. Personality and Social Psychology Review, 2(4), 290–309. https://doi.org/10.1207/s15327957pspr0204_5
Feist, G. J. (2019). Creativity and the big two model of personality: plasticity and stability. Current Opinion in Behavioral Sciences, 27, 31–35. https://doi.org/10.1016/j.cobeha.2018.07.005
Fleeson, W., & Gallagher, M. P. (2009). The Implications of big-five standing for the distribution of trait manifestation in behavior: Fifteen experience-sampling studies and a meta-analysis. Journal of Personality and Social Psychology, 97(6), 1097–1114. https://doi.org/10.1037/a0016786
Forthmann, B., & Doebler, P. (2022). Fifty years later and still working: Rediscovering Paulus et al’s (1970) automated scoring of divergent thinking tests. Psychology of Aesthetics, Creativity, and the Arts. https://doi.org/10.1037/aca0000518 Advance online publication
Frederick, S. (2005). Cognitive Reflection and Decision Making. Journal of Economic Perspectives, 19(4), 25–42. https://doi.org/10.1257/089533005775196732
Furnham, A., & Bachtiar, V. (2008). Personality and intelligence as predictors of creativity. Personality and Individual Differences, 45(7). https://doi.org/10.1016/j.paid.2008.06.023
Furnham, A., Batey, M., Anand, K., & Manfield, J. (2008). Personality, hypomania, intelligence and creativity. Personality and Individual Differences, 44(5), 1060–1069. https://doi.org/10.1016/j.paid.2007.10.035
Furnham, A., Crump, J., Batey, M., & Chamorro-Premuzic, T. (2009). Personality and ability predictors of the “Consequences” Test of divergent thinking in a large non-student sample. Personality and Individual Differences, 46(4), 536–540. https://doi.org/10.1016/j.paid.2008.12.007
Gabora, L. (2018). The neural basis and evolution of divergent and convergent thought. In O. Vartanian & R. Jung (Eds.), The Cambridge Handbook of the Neuroscience of Creativity (pp. 58–70). Cambridge University Press.
Ghiselin, B. (1985). The Creative Process: Reflections on the Invention in the Arts and Sciences. University of California Press.
Gilhooly, K. J., Fioratou, E., Anthony, S. H., & Wynn, V. (2007). Divergent thinking: Strategies and executive involvement in generating novel uses for familiar objects. British Journal of Psychology, 98(4), 611–625. https://doi.org/10.1111/j.2044-8295.2007.tb00467.x
Gino, F., & Ariely, D. (2012). The dark side of creativity: Original thinkers can be more dishonest. Journal of Personality and Social Psychology, 102(3). https://doi.org/10.1037/a0026406
Grajzel, K., Acar, S., & Singer, G. (2023). The Big Five and divergent thinking: A meta-analysis. Personality and Individual Differences, 214, 112338.
Gray, J. A. (1987). The neuropsychology of emotion and personality. In S. M. Stahl, S. D. Iversen, & E. C. Goodman (Eds.), Cognitive neurochemistry (pp. 171–190). Oxford University Press.
Gühne, D., Doebler, P., Condon, D. M., Luo, F., & Sun, L. (2021). Validity and reliability of automatically generated propositional reasoning Items. European Journal of Psychological Assessment, 37(4), 325–339. https://doi.org/10.1027/1015-5759/a000616
Guilford, J. P. (1950). Creativity. American Psychologist, 5(9). https://doi.org/10.1037/h0063487
Guilford, J. P. (1967). The nature of human intelligence. McGraw-Hill.
Guilford, J. P. (1976). Aptitude for Creative Thinking: One or Many? The Journal of Creative Behavior, 10(3). https://doi.org/10.1002/j.2162-6057.1976.tb01019.x
Guilford, J. P. (1971). Some Misconceptions Regarding Measurement of Creative Talents. The Journal of Creative Behavior, 5(2), 77–87. https://doi.org/10.1002/j.2162-6057.1971.tb00877.x
Guilford, J. P., Christensen, P. R., Merrifield, P. R., & Wilson, R. C. (1978). Alternate uses: Manual of instructions and interpretations. Orange, CA: Sheridan Psychological Services.
Hadamard, J. (1945). The psychology of invention in the mathematical field (13th ed., p. 143). Princeton University Press.
Haigh, M. (2016). Has the standard Cognitive Reflection Test become a victim of its own success? Advances in Cognitive Psychology, 12(3), 145–149. https://doi.org/10.5709/acp-0193-5
Holland, J. L. (1959). A theory of vocational choice. Journal of Counseling Psychology, 6(1), 35–45. https://doi.org/10.1037/h0040767
Isen, A. M., Daubman, K. A., & Nowicki, G. P. (1987). Positive affect facilitates creative problem solving. Journal of Personality and Social Psychology, 52(6), 1122–1131. https://doi.org/10.1037/0022-3514.52.6.1122
Isen, A. M., Labroo, A. A., & Durlach, P. (2004). An Influence of product and brand name on positive affect: Implicit and explicit measures. Motivation and Emotion, 28(1), 43–63. https://doi.org/10.1023/B:MOEM.0000027277.98917.9a
Jankowska, D. M., & Karwowski, M. (2015). Measuring creative imagery abilities. Frontiers in Psychology, 6, 1591. https://doi.org/10.3389/fpsyg.2015.01591
Johnson, W., & Bouchard, T. J. (2005). The structure of human intelligence: It is verbal, perceptual, and image rotation (VPR), not fluid and crystallized. Intelligence, 33(4). https://doi.org/10.1016/j.intell.2004.12.002
Johnson, S. (2011). Where good ideas come from: The natural history of innovation (1st Riverhead trade pbk). Riverhead Books.
Jones, L. L., & Estes, Z. (2015). Convergent and divergent thinking in verbal analogy. Thinking & Reasoning, 21(4). https://doi.org/10.1080/13546783.2015.1036120
Jung, R. E., Flores, R. A., & Hunter, D. (2016). A new measure of imagination ability: Anatomical brain imaging correlates. Frontiers in Psychology, 7, 496. https://doi.org/10.3389/fpsyg.2016.00496
Jung-Beeman, M., Bowden, E. M., Haberman, J., Frymiare, J. L., Arambel-Liu, S., Greenblatt, R., Reber, P. J., & Kounios, J. (2004). Neural activity when people solve verbal problems with insight. PLOS Biology, 2(4), e97. https://doi.org/10.1371/journal.pbio.0020097
Kahneman, D. (2011). Thinking, fast and slow. (p. 499). Farrar, Straus and Giroux.
Kaufman, J. C., & Beghetto, R. A. (2009). Beyond Big and Little: The Four C Model of Creativity. Review of General Psychology, 13(1). https://doi.org/10.1037/a0013688
Kaufman, J. C., & Baer, J. (2005). The Amusement Park Theory of Creativity. In J. C. Kaufman & J. Baer (Eds.), Creativity across domains: Faces of the muse (pp. 321–328). Lawrence Erlbaum Associates Publishers.
Kaufman, J. C., Evans, M. L., & Baer, J. (2010). The American idol effect: Are students good judges of their creativity across domains? Empirical Studies of the Arts, 28(1), 3–17. https://doi.org/10.2190/EM.28.1.b
Kaufman, J. C., Pumaccahua, T. T., & Holt, R. E. (2013). Personality and creativity in realistic, investigative, artistic, social, and enterprising college majors. Personality and Individual Differences, 54(8), 913–917. https://doi.org/10.1016/j.paid.2013.01.013
Kaufman, S. B., Quilty, L. C., Grazioplene, R. G., Hirsh, J. B., Gray, J. R., Peterson, J. B., & DeYoung, C. G. (2016). Openness to Experience and Intellect Differentially Predict Creative Achievement in the Arts and Sciences. Journal of Personality, 84(2), 248–258. https://doi.org/10.1111/jopy.12156
Kenett, Y. N., Anaki, D., & Faust, M. (2014). Investigating the structure of semantic networks in low and high creative persons. Frontiers in Human Neuroscience, 8, 407. https://doi.org/10.3389/fnhum.2014.00407
Killingsworth, M. A., & Gilbert, D. T. (2010). A wandering mind is an unhappy mind. Science, 330(6006), 932–932. https://doi.org/10.1126/science.1192439
King, L. A., McKee Walker, L., & Broyles, S. J. (1996). Creativity and the five-factor model. Journal of Research in Personality, 30(2), 189–203. https://doi.org/10.1006/jrpe.1996.0013
Kounios, J., & Beeman, M. (2014). The cognitive neuroscience of insight. Annual Review of Psychology, 65(1), 71–93. https://doi.org/10.1146/annurev-psych-010213-115154
Lee, C. S., Huggins, A. C., & Therriault, D. J. (2014). A measure of creativity or intelligence? Examining internal and external structure validity evidence of the remote associates test. Psychology of Aesthetics, Creativity, and the Arts, 8(4), 446–460. https://doi.org/10.1037/a0036773
Lee, C. S., & Therriault, D. J. (2013). The cognitive underpinnings of creative thought: A latent variable analysis exploring the roles of intelligence and working memory in three creative thinking processes. Intelligence, 41(5), 306–320. https://doi.org/10.1016/j.intell.2013.04.008
van der Loo, M., van der Laan, J., Team, R. C., Logan, N., & Muir, C. (2023). Package ‘stringdist’.
Lynn, S. J., & Rhue, J. W. (1986). The fantasy-prone person: Hypnosis, imagination, and creativity. Journal of Personality and Social Psychology, 51(2), 404–408. https://doi.org/10.1037/0022-3514.51.2.404
Martinsen, Ø. L. (2011). The Creative Personality: A Synthesis and Development of the Creative Person Profile. Creativity Research Journal, 23(3). https://doi.org/10.1080/10400419.2011.595656
Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response theory models. Measurement: Interdisciplinary Research and Perspectives, 11(3), 71–101.
McCrae, R. R. (1987). Creativity, divergent thinking, and openness to experience. Journal of Personality and Social Psychology, 52(6). https://doi.org/10.1037/0022-3514.52.6.1258
McNaughton, N., & Gray, J. A. (2000). Anxiolytic action on the behavioural inhibition system implies multiple types of arousal contribute to anxiety. Journal of Affective Disorders, 61(3), 161–176. https://doi.org/10.1016/S0165-0327(00)00344-X
Mednick, S. (1962). The associative basis of the creative process. Psychological Review, 69(3). https://doi.org/10.1037/h0048850
Mednick, M. T., & Andrews, F. M. (1967). Creative Thinking and Level of Intelligence. The Journal of Creative Behavior, 1(4), 428–431. https://doi.org/10.1002/j.2162-6057.1967.tb00074.x
Mednick, S. A. (1968). The Remote Associates Test. The Journal of Creative Behavior, 2(3), 213–214. https://doi.org/10.1002/j.2162-6057.1968.tb00104.x
Mendelsohn, G. A. (1976). Associative and attentional processes in creative performance. Journal of personality, 44(2).
Meyer, G. J., & Shack, J. R. (1989). Structural convergence of mood and personality: Evidence for old and new directions. Journal of Personality and Social Psychology, 57(4), 691–706. https://doi.org/10.1037/0022-3514.57.4.691
Montoya, A. K., & Edwards, M. C. (2021). The Poor Fit of Model Fit for Selecting Number of Factors in Exploratory Factor Analysis for Scale Evaluation. Educational and Psychological Measurement, 81(3). https://doi.org/10.1177/0013164420942899
Mõttus, R., Sinick, J., Terracciano, A., Hřebíčková, M., Kandler, C., Ando, J., Mortensen, E. L., Colodro-Conde, L., & Jang, K. L. (2019). Personality characteristics below facets: A replication and meta-analysis of cross-rater agreement, rank-order stability, heritability, and utility of personality nuances. Journal of Personality and Social Psychology, 117(4), e35–e50. https://doi.org/10.1037/pspp0000202
Mõttus, R., Wood, D., Condon, D. M., Back, M. D., Baumert, A., Costantini, G., Epskamp, S., Greiff, S., Johnson, W., Lukaszewski, A., Murray, A., Revelle, W., Wright, A. G. C., Yarkoni, T., Ziegler, M., & Zimmermann, J. (2020). Descriptive, predictive and explanatory personality research: different goals, different approaches, but a shared need to move beyond the big few traits. European Journal of Personality, 34(6), 1175–1201. https://doi.org/10.1002/per.2311
Mumford, M. D., Waples, E. P., Antes, A. L., Brown, R. P., Connelly, S., Murphy, S. T., & Devenport, L. D. (2010). Creativity and Ethics: The Relationship of Creative and Ethical Problem-Solving. Creativity Research Journal, 22(1). https://doi.org/10.1080/10400410903579619
Mussel, P., McKay, A. S., Ziegler, M., Hewig, J., & Kaufman, J. C. (2015). Predicting Creativity Based on the Facets of the Theoretical Intellect Framework. European Journal of Personality, 29(4), 459–467. https://doi.org/10.1002/per.2000
National Center for O*NET Development (2023). O*NET® Career Exploration Tools. O*NET Resource Center. Retrieved January 3, 2023, from https://www.onetcenter.org/tools.html
Niepel, C., Mustafić, M., Greiff, S., & Roberts, R. D. (2015). The dark side of creativity revisited: Is students’ creativity associated with subsequent decreases in their ethical decision making? Thinking Skills and Creativity, 18, 43–52. https://doi.org/10.1016/j.tsc.2015.04.005
Nusbaum, E. C., & Silvia, P. J. (2011). Are intelligence and creativity really so different?: Fluid intelligence, executive processes, and strategy use in divergent thinking. Intelligence, 39(1). https://doi.org/10.1016/j.intell.2010.11.002
Oleynick, V. C., DeYoung, C. G., Hyde, E., Kaufman, S. B., Beaty, R. E., & Silvia, P. J. (2017). Openness/intellect: The core of the creative personality. In G. J. Feist, R. Reiter-Palmon, & J. C. Kaufman (Eds.), The Cambridge handbook of creativity and personality research (pp. 9–27). Cambridge University Press. https://doi.org/10.1017/9781316228036.002
Olteţeanu, A.-M., Taranu, M., & Ionescu, T. (2019). Normative Data for 111 Compound Remote Associates Test Problems in Romanian. Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.01859
Olteţeanu, A.-M., Schultheis, H., & Dyer, J. B. (2018). Computationally constructing a repository of compound remote associates test items in American English with comRAT-G. Behavior Research Methods, 50(5), 1971–1980. https://doi.org/10.3758/s13428-017-0965-8
Olteţeanu, A.-M., & Zunjani, F. H. (2020). A visual remote associates test and its validation. Frontiers in Psychology, 11, 26. https://doi.org/10.3389/fpsyg.2020.00026
Organisciak, P., Acar, S., Dumas, D., & Berthiaume, K. (2023). Beyond semantic distance: automated scoring of divergent thinking greatly improves with large language models. Thinking Skills and Creativity, 49, 101356. https://doi.org/10.1016/j.tsc.2023.101356
Perrine, N. E., & Brodersen, R. M. (2005). Artistic and Scientific Creative Behavior: Openness and the Mediating Role of Interests. The Journal of Creative Behavior, 39(4), 217–236. https://doi.org/10.1002/j.2162-6057.2005.tb01259.x
Petrides, K. V., Mikolajczak, M., Mavroveli, S., Sanchez-Ruiz, M.-J., Furnham, A., & Pérez-González, J.-C. (2016). Developments in Trait Emotional Intelligence Research. Emotion Review, 8(4), 335–341. https://doi.org/10.1177/1754073916650493
Poincaré, H. (1913). The foundations of science: Science and hypothesis, The value of science. The Science Press.
Primi, C., Morsanyi, K., Chiesi, F., Donati, M. A., & Hamilton, J. (2016). The Development and Testing of a New Version of the Cognitive Reflection Test Applying Item Response Theory (IRT). Journal of Behavioral Decision Making, 29(5), 453–469. https://doi.org/10.1002/bdm.1883
Puryear, J. S., Kettler, T., & Rinn, A. N. (2017). Relationships of personality to differential conceptions of creativity: A systematic review. Psychology of Aesthetics, Creativity, and the Arts, 11(1). https://doi.org/10.1037/aca0000079
R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/
Reiter-Palmon, R., Forthmann, B., & Barbot, B. (2019). Scoring divergent thinking tests: A review and systematic framework. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 144.
Revelle, W. (2022). psych: Procedures for Psychological, Psychometric, and Personality Research (2.2.5). https://CRAN.R-project.org/package=psych
Revelle, W., Condon, D. M., Wilt, J., French, J. A., Brown, A., & Elleman, L. G. (2017). Web- and Phone-based Data Collection using Planned Missing Designs. In G. Fielding, R. M. Lee, & G. Blank (Eds.), In N (pp. 578–594). Handbook of Online Research Methods, SAGE Publications Ltd. https://doi.org/10.4135/9781473957992.n33
Revelle, W., Dworak, E. M., & Condon, D. M. (2021). Exploring the persome: The power of the item in understanding personality structure. Personality and Individual Differences, 169, 109905. https://doi.org/10.1016/j.paid.2020.109905
Revelle, W., & Rocklin, T. (1979). Very simple structure: An alternative procedure for estimating the optimal number of interpretable factors. Multivariate Behavioral Research, 14(4), 403–414. https://doi.org/10.1207/s15327906mbr1404_2
Revelle, W., & Zinbarg, R. E. (2009). Coefficients Alpha, Beta, Omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. https://doi.org/10.1007/s11336-008-9102-z
Robitzsch A (2024). sirt: Supplementary Item Response Theory Models. R package version 4.0-32, https://CRAN.R-project.org/package=sirt.
Rosseel, Yves. (2012). lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48(2), 1–36. http://www.jstatsoft.org/v48/i02
Rowe, G., Hirsh, J. B., & Anderson, A. K. (2007). Positive affect increases the breadth of attentional selection. Proceedings of the National Academy of Sciences of the United States of America, 104(1), 383–388. https://doi.org/10.1073/pnas.0605198104
Runco, M. A. (2010). Divergent thinking, creativity, and ideation. The Cambridge handbook of creativity, 413, 446.
Runco, M. A., Plucker, J. A., & Lim, W. (2001). Development and Psychometric Integrity of a Measure of Ideational Behavior. Creativity Research Journal, 13(3–4), 393–400. https://doi.org/10.1207/S15326934CRJ1334_16
Salvi, C., Bricolo, E., Kounios, J., Bowden, E., & Beeman, M. (2016). Insight solutions are correct more often than analytic solutions. Thinking & Reasoning, 22(4), 443–460. https://doi.org/10.1080/13546783.2016.1141798
Salvi, C., Simoncini, C., Grafman, J., & Beeman, M. (2020). Oculometric signature of switch into awareness? Pupil size predicts sudden insight whereas microsaccades predict problem-solving via analysis. NeuroImage, 217, 116933. https://doi.org/10.1016/j.neuroimage.2020.116933
Saucier, G., Thalmayer, A. G., Payne, D. L., Carlson, R., Sanogo, L., Ole-Kotikash, L., Church, A. T., Katigbak, M. S., Somer, O., Szarota, P., Szirmák, Z., & Zhou, X. (2014). A basic bivariate structure of personality attributes evident across nine languages. Journal of Personality, 82(1), 1–14. https://doi.org/10.1111/jopy.12028
Schooler, J. W., & Melcher, J. (1995). The ineffability of insight. In S. M. Smith, T. B. Ward, & R. A. Finke (Eds.), The creative cognition approach (pp. 97–133). The MIT Press.
Ścigała, K. A., Schild, C., & Zettler, I. (2022). Dark, gray, or bright creativity? (Re) investigating the link between creativity and dishonesty. European Journal of Personality, 36(1), 108–129. https://doi.org/10.1177/08902070211010993
Silvia, P. J., Winterstein, B. P., Willse, J. T., Barona, C. M., Cram, J. T., Hess, K. I., Martinez, J. L., & Richard, C. A. (2008). Assessing creativity with divergent thinking tasks: Exploring the reliability and validity of new subjective scoring methods. Psychology of Aesthetics, Creativity, and the Arts, 2(2). https://doi.org/10.1037/1931-3896.2.2.68
Silvia, P. J., Nusbaum, E. C., Berg, C., Martin, C., & O’Connor, A. (2009). Openness to experience, plasticity, and creativity: Exploring lower-order, high-order, and interactive effects. Journal of Research in Personality, 43(6). https://doi.org/10.1016/j.jrp.2009.04.015
Silvia, P. J., Kaufman, J. C., Reiter-Palmon, R., & Wigert, B. (2011). Cantankerous creativity: Honesty-Humility, Agreeableness, and the HEXACO structure of creative achievement. Personality and Individual Differences, 51(5), 687–689. https://doi.org/10.1016/j.paid.2011.06.011
Simonton, D. K. (2021). Scientific Creativity: Discovery and Invention as Combinatorial. Frontiers in Psychology, 12, 3603. https://doi.org/10.3389/fpsyg.2021.721104
Sitton, S. C., & Pierce, E. R. (2004). Synesthesia, Creativity and Puns. Psychological Reports, 95(2), 577–580. https://doi.org/10.2466/pr0.95.2.577-580
Smeekens, B. A., & Kane, M. J. (2016). Working memory capacity, mind wandering, and creative cognition: An individual-differences investigation into the benefits of controlled versus spontaneous thought. Psychology of Aesthetics, Creativity, and the Arts, 10(4). https://doi.org/10.1037/aca0000046
Smith, K. A., Huber, D. E., & Vul, E. (2013). Multiply-constrained semantic search in the remote associates test. Cognition, 128(1), 64–75. https://doi.org/10.1016/j.cognition.2013.03.001
Sobkow, A., Traczyk, J., Kaufman, S. B., & Nosal, C. (2018). The structure of intuitive abilities and their relationships with intelligence and openness to experience. Intelligence, 67, 1–10. https://doi.org/10.1016/j.intell.2017.12.001
Sowden, P. T., Clements, L., Redlich, C., & Lewis, C. (2015). Improvisation facilitates divergent thinking and creativity: Realizing a benefit of primary school arts education. Psychology of Aesthetics, Creativity, and the Arts, 9(2), 128.
Stieger, S., & Reips, U.-D. (2016). A limitation of the cognitive reflection test: Familiarity. PeerJ, 4, e2395. https://doi.org/10.7717/peerj.2395
Storm, B. C., Angello, G., & Bjork, E. L. (2011). Thinking can cause forgetting: Memory dynamics in creative problem solving. Journal of Experimental Psychology Learning, Memory, and Cognition, 37(5), 1287–1293. https://doi.org/10.1037/a0023921
Taft, R., & Rossiter, J. R. (1966). The remote associates test: Divergent or convergent thinking? Psychological Reports, 19(3), 1313–1314. https://doi.org/10.2466/pr0.1966.19.3f.1313
Thomson, K. S., & Oppenheimer, D. M. (2016). Investigating an alternate form of the cognitive reflection test. Judgment and Decision Making, 11(1), 15.
Torrance, E. P. (1967). The minnesota studies of creative behavior: National and international extensions. The Journal of Creative Behavior, 1(2), 137–154. https://doi.org/10.1002/j.2162-6057.1967.tb00021.x
Torrance, E. P. (1972). Predictive validity of the torrance tests of creative thinking. The Journal of Creative Behavior, 6(4), 236–262. https://doi.org/10.1002/j.2162-6057.1972.tb00936.x
United States Department of Labor (2015). Employment and Training Administration: O*NET Data Collection Program. Office of Management and Budget Clearance Package Supporting Statement and Data Collection Instruments, Washington D.C. https://www.dol.gov/agencies/eta/onet/data-collection
Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41(3), 321–327. https://doi.org/10.1007/BF02293557
von Stumm, S., & Ackerman, P. L. (2013). Investment and intellect: A review and meta-analysis. Psychological Bulletin, 139(4), 841–869. https://doi.org/10.1037/a0030746
Wallach, M. A., & Kogan, N. (1965). Modes of thinking in young children. Holt, Rinehart and Winston.
Wang, Y., Luppi, A., Fawcett, J., & Anderson, M. C. (2019). Reconsidering unconscious persistence: Suppressing unwanted memories reduces their indirect expression in later thoughts. Cognition, 187, 78–94. https://doi.org/10.1016/j.cognition.2019.02.016
Warne, R. T., Golightly, S., & Black, M. (2022). Factor structure of intelligence and divergent thinking subtests: A registered report. Plos one, 17(9). https://doi.org/10.1371/journal.pone.0274921
Watson, D., & Tellegen, A. (1985). Toward a consensual structure of mood. Psychological Bulletin, 98(2), 219–235. https://doi.org/10.1037/0033-2909.98.2.219
Watson, J. D. (1968). The double helix: A personal account of the discovery of the structure of DNA. Atheneum.
Weiss, S., Steger, D., Kaur, Y., Hildebrandt, A., Schroeders, U., & Wilhelm, O. (2021). On the trail of creativity: Dimensionality of divergent thinking and its relation with cognitive abilities, personality, and insight. European Journal of Personality, 35(3), 291–314. https://doi.org/10.1002/per.2288
White, H. A., & Shah, P. (2006). Uninhibited imaginations: Creativity in adults with attention-deficit/hyperactivity disorder. Personality and Individual Differences, 40(6), 1121–1131. https://doi.org/10.1016/j.paid.2005.11.007
Wilson, S. C., & Barber, T. X. (1978). The Creative Imagination Scale as a measure of hypnotic responsiveness: Applications to experimental and clinical hypnosis. American Journal of Clinical Hypnosis, 20(4). https://doi.org/10.1080/00029157.1978.10403940
Wise, S. L. (2017). Rapid-guessing behavior: Its identification, interpretation, and implications. Educational Measurement: Issues and Practice, 36(4), 52–61.
Worthen, B. R., & Clark, P. M. (1971). Toward an improved measure of remote associational ability. Journal of Educational Measurement, 8(2), 113–123. https://doi.org/10.1111/j.1745-3984.1971.tb00914.x
Wu, C.-L., Huang, S.-Y., Chen, P.-Z., & Chen, H.-C. (2020). A Systematic Review of Creativity-Related Studies Applying the Remote Associates Test From 2000 to 2019. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.573432
Wu, C.-L., & Chen, H.-C. (2017). Normative data for Chinese compound remote associate problems. Behavior Research Methods, 49(6), 2163–2172. https://doi.org/10.3758/s13428-016-0849-3
Zabelina, D. L., & Robinson, M. D. (2010). Creativity as flexible cognitive control. Psychology of Aesthetics, Creativity, and the Arts, 4(3). https://doi.org/10.1037/a0017379
Zabelina, D. L., & Condon, D. M. (2020). The Four-Factor Imagination Scale (FFIS): A measure for assessing frequency, complexity, emotional valence, and directedness of imagination. Psychological Research, 84(8), 2287–2299. https://doi.org/10.1007/s00426-019-01227-w
Zabelina, D. L., Zaonegina, E., Revelle, W., & Condon, D. M. (2022). Creative achievement and individual differences: Associations across and within the domains of creativity. Psychology of Aesthetics, Creativity, and the Arts, 16(4), 618–636. https://doi.org/10.1037/aca0000439
Author information
Authors and Affiliations
Contributions
Kendall A. Mather – conceptualization, methodology, formal analysis, investigation, project administration, visualization, writing—original draft. Sara J. Weston – methodology, supervision, writing—review and editing. David M. Condon – conceptualization, data curation, formal analysis, funding acquisition, investigation, methodology, resources, software, supervision, validation, visualization, writing—original draft, writing—review and editing.
Corresponding author
Ethics declarations
Conflicts of interest
We have no known conflicts of interest to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open practices statement
The study design, hypotheses, and analyses for Study 1 were pre-registered (see https://tinyurl.com/3y7z9t6m), and a supplemental file with all analyses and code can be found on the Open Science Framework (OSF) at https://tinyurl.com/yuskme6n. The data for Study 1 are part of a larger ongoing project for which data are periodically published in the public domain (https://tinyurl.com/29rdnus6). All reasonable requests for immediate access to the data will be met through contact with the senior author. The design and hypotheses for Studies 2 and 3 were pre-registered on OSF at https://tinyurl.com/3y7z9t6m. All code and data used for the analyses in these studies can be found on OSF at https://tinyurl.com/yuskme6n. Data collection and procedures used in Study 4 were not pre-registered. However, all code and analyses run in Study 4 are available in the supplemental file (https://tinyurl.com/yuskme6n). When accessing the supplemental file and the folder with instructions for reproducing the analyses on the OSF page, please refer to those uploaded in September 2023.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Mather, K.A., Weston, S.J. & Condon, D.M. Scaling a common assessment of associative ability: Development and validation of a multiple-choice compound remote associates task. Behav Res 56, 1–29 (2024). https://doi.org/10.3758/s13428-024-02422-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13428-024-02422-3