Abstract
It can be argued that the Implicit Association Test (IAT) is the most popular assessment of implicit bias available. The IAT in part owes its popularity to the IAT website, launched shortly after development of the test itself, which has accumulated millions of responses from self-enrolled participants. The Implicit Relational Assessment Procedure (IRAP) is an experimental preparation grounded in relational frame theory (RFT) devised to capture specific relations among stimuli. It provides a more granular analysis of bias, a distinct advantage over the IAT, and its internal reliability is among the best of all implicit measures and certainly comparable to the IAT. The present study attempted to develop and validate an IRAP website equivalent to the IAT website by replicating a foundational IRAP study. This study tasked participants with categorizing positively and negatively valenced words as either “pleasant” or “unpleasant.” A total of 31 out of 40 participants survived performance-related criteria at similar rates to previous IRAP studies, demonstrating the feasibility of web-delivered IRAP studies. The results successfully replicated the main effect, an overall preference toward positive words, as well as a pattern of specific trial-type scores commonly observed in IRAP studies. These results were internally consistent. The successful development of the web-IRAP offers the potential to leverage advantages typical of web-delivered research, including increased power, greater ecological validity, and inclusion of underrepresented demographic groups.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
References
Barnes-Holmes, D., & Harte, C. (2022). Relational frame theory 20 years on: The Odysseus voyage and beyond. Journal of the Experimental Analysis of Behavior, 117, 240–266. https://doi.org/10.1002/jeab.733
Barnes-Holmes, D., Barnes-Holmes, Y., Power, P., Hayden, E., Milne, R., & Stewart, I. (2006). Do you really know what you believe? developing the implicit relational assessment procedure (IRAP) as a direct measure of implicit beliefs. The Irish Psychologist, 32(7), 169–177.
Barnes-Holmes, D., Barnes-Holmes, Y., Stewart, I., & Boles, S. (2010). A sketch of the Implicit Relational Assessment Procedure (IRAP) and the Relational Elaboration and Coherence (REC) Model. The Psychological Record, 60(3), 527–542. https://doi.org/10.1007/BF03395726
Barnes-Holmes, D., Barnes-Holmes, Y., & McEnteggart, C. (2020). Updating RFT (more field than frame) and its implications for process-based therapy. The Psychological Record, 70, 605–624. https://doi.org/10.1007/s40732-019-00372-3
Bortoloti, R., de Almeida, R. V., de Almeida, J. H., & de Rose, J. C. (2021). A commentary on the dynamics of arbitrarily applicable relational responding involving positive valenced stimuli and its implications for the IRAP research. The Psychological Record, 71(3), 481–486. https://doi.org/10.1007/s40732-020-00413-2
Cabrera, I., Márquez-González, M., Kishita, N., Vara-García, C., & Losada, A. (2021). Development and validation of an Implicit Relational Assessment Procedure (IRAP) to measure implicit dysfunctional beliefs about caregiving in dementia family caregivers. The Psychological Record, 71(1), 41–54. https://doi.org/10.1007/S40732-020-00445-8/TABLES/3
Campbell, C., Barnes-Holmes, Y., Barnes-Holmes, D., & Stewart, I. (2011). Exploring screen presentations in the Implicit Relational Assessment Procedure (IRAP). International Journal of Psychology & Psychological Therapy, 11(3), 377–388 https://www.researchgate.net/publication/259285958_Exploring_screen_presentations_in_the_Implicit_Relational_Assessment_Procedure_IRAP
Ellefson, M. R., & Oppenheimer, D. M. (2022). Is replication possible without fidelity? Psychological Methods. Advance online publication. https://doi.org/10.1037/met0000473
Errasti, J., Martinez, H., Rodriguez, C., Marquez, J., Maldonado, A., & Menendez, A. (2019). Social context in a collective IRAP application about gender stereotypes: Mixed versus single gender groups. The Psychological Record, 69(1), 39–48. https://doi.org/10.1007/s40732-018-0320-1
Ferroni-Bast, D., Fitzpatrick, J., Stewart, I., & Goyos, C. (2019). Using the Implicit Relational Assessment Procedure (IRAP) as a measure of reaction to perceived failure and the effects of a defusion intervention in this context. Psychological Record, 69(4), 551–563. https://doi.org/10.1007/S40732-019-00349-2
Finn, M., Barnes-Holmes, D., Hussey, I., & Graddy, J. (2016). Exploring the behavioral dynamics of the Implicit Relational Assessment Procedure: The impact of three types of introductory rules. Psychological Record, 66(2), 309–321. https://doi.org/10.1007/S40732-016-0173-4
Finn, M., Barnes-Holmes, D., & McEnteggart, C. (2018). Exploring the single-trial-type-dominance-effect in the IRAP: Developing a Differential Arbitrarily Applicable Relational Responding Effects (DAARRE) Model. Psychological Record, 68(1), 11–25. https://doi.org/10.1007/S40732-017-0262-Z
Gawronski, B., & De Houwer, J. (2014). Implicit measures in social and personality psychology. In H. T. Reiss & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (Vol. 2, pp. 283–310). Cambridge University Press. https://doi.org/10.1017/CBO9780511996481.016
Golijani-Moghaddam, N., Hart, A., & Dawson, D. L. (2013). The Implicit Relational Assessment Procedure: Emerging reliability and validity data. Journal of Contextual Behavioral Science, 2(3–4), 105–119. https://doi.org/10.1016/J.JCBS.2013.05.002
Greenwald, A. G., & Farnham, S. D. (2000). Using the implicit association test to measure self-esteem and self-concept. Journal of Personality & Social Psychology, 79(6), 1022–1038. https://doi.org/10.1037//0022-3514.79.6.1022
Greenwald, A. G., & Lai, C. K. (2020). Implicit social cognition. Annual Review of Psychology, 71(1), 419–445. https://doi.org/10.1146/annurev-psych-010419-050837
Greenwald, A. G., & Nosek, B. A. (2001). Health of the Implicit Association Test at age 3. Zeitschrift für Experimentelle Psychologie: Organ der Deutschen Gesellschaft für Psychologie, 48(2), 85–93. https://doi.org/10.1026//0949-3946.48.2.85
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality & Social Psychology, 74(6), 1464–1480. https://doi.org/10.1037/0022-3514.74.6.1464
Hughes, S., & Barnes-Holmes, D. (2013). A functional approach to the study of implicit cognition: The IRAP and the REC model. In B. Roche & S. Dymond (Eds.), Advances in relational frame theory and contextual behavioral science: Research and applications (pp. 97–126). Context Press/New Harbinger.
Hughes, S., Barnes-Holmes, D., & De Houwer, J. (2011). The dominance of associative theorizing in implicit attitude research: Propositional and behavioral alternatives. The Psychological Record, 61(3), 465–496. https://doi.org/10.1007/BF03395772
Hughes, S., Barnes-Holmes, D., & Vahey, N. A. (2012). Holding on to our functional roots when exploring new intellectual islands: A voyage through implicit cognition research. Journal of Contextual Behavioral Science, 1, 17–38. https://doi.org/10.1016/j.jcbs.2012.09.003
Hughes, S., Hussey, I., Corrigan, B., Jolie, K., Murphy, C., & Barnes-Holmes, D. (2016). Faking revisited: Exerting strategic control over performance on the Implicit Relational Assessment Procedure. European Journal of Social Psychology, 46, 632–648. https://doi.org/10.1002/ejsp.2207
Hussey, I., Thompson, M., McEnteggart, C., Barnes-Holmes, D., & Barnes-Holmes, Y. (2015). Interpreting and inverting with less cursing: A guide to interpreting IRAP data. Journal of Contextual Behavioral Science, 4(3), 157–162. https://doi.org/10.1016/J.JCBS.2015.05.001
Krantz, J. H., & Reips, U.-D. (2017). The state of web-based research: A survey and call for inclusion in curricula. Behavior Research Methods, 49(5), 1621–1629. https://doi.org/10.3758/S13428-017-0882-X
MacLeod, C. M. (1991). Half a century of research on the Stroop effect: An integrative review. Psychological Bulletin, 109(2), 163–203. https://doi.org/10.1037/0033-2909.109.2.163
Maloney, E., & Barnes-Holmes, D. (2016). Exploring the behavioral dynamics of the Implicit Relational Assessment Procedure: The role of relational contextual cues versus relational coherence indicators as response options. The Psychological Record, 66(3), 395–403. https://doi.org/10.1007/S40732-016-0180-5
Maloney, E., Foody, M., & Murphy, C. (2020). Do response options in the Implicit Relational Assessment Procedure (IRAP) matter? A comparison of contextual relations versus relational coherent indicators. Psychological Record, 70(2), 205–214. https://doi.org/10.1007/S40732-019-00360-7
McKenna, I., Barnes-Holmes, D., Barnes-Holmes, Y., & Stewart, I. (2007). Testing the fake-ability of the Implicit Relational Assessment Procedure (IRAP): The first study. International Journal of Psychology & Psychological Therapy, 7(2), 253–268.
Murphy, C., Maloney, E., & Kelly, M. (2022). The role of relational contextual cues versus relational coherence indicators as response options on the Implicit Relational Assessment Procedure. The Psychological Record, 72, 665–673. https://doi.org/10.1007/S40732-022-00512-2
Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002). Harvesting implicit group attitudes and beliefs from a demonstration web site. Group Dynamics, 6(1), 101–115. https://doi.org/10.1037/1089-2699.6.1.101
O'Shea, B., Watson, D. G., & Brown, G. D. A. (2016). Measuring implicit attitudes: A positive framing bias flaw in the Implicit Relational Assessment Procedure (IRAP). Psychological Assessment, 28(2), 158–170. https://doi.org/10.1037/pas0000172
Project Implicit. (2021). Implicit Association Test. https://implicit.harvard.edu/implicit/takeatest.html
Rosenthal, R., & Jacobson, L. (1966). Teachers’ expectancies: Determinants of pupils’ IQ gains. Psychological Reports, 19(1), 115–118. https://doi.org/10.2466/pr0.1966.19.1.115
Siegwarth, N., Larkin, K. T., & Kemmner, C. (2012). Experimenter effects on cardiovascular reactivity and task performance during mental stress testing. The Psychological Record, 62, 69–82. https://doi.org/10.1007/BF03395787
Stewart, I., Barnes-Holmes, D., & Roche, B. (2004). A functional-analytic model of analogy using the relational evaluation procedure. The Psychological Record, 54(4), 531–552. https://doi.org/10.1007/BF03395491
Vahey, N. A., Nicholson, E., & Barnes-Holmes, D. (2015). A meta-analysis of criterion effects for the Implicit Relational Assessment Procedure (IRAP) in the clinical domain. Journal of Behavior Therapy & Experimental Psychiatry, 48, 59–65. https://doi.org/10.1016/j.jbtep.2015.01.004
Yang, J., Zeng, J., Meng, X., Zhu, L., Yuan, J., Li, H., & Yusoff, N. (2013). Positive words or negative words: Whose valence strength are we more sensitive to? Brain Research, 1533, 91–104. https://doi.org/10.1016/j.brainres.2013.08.020
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of Interest
We have no known conflicts of interest to disclose.
Ethical Approval
This research involved human participants and was approved by the University of Vermont Institutional Review Board study number 00001574.
Consent to participate
All participants provided consent prior to participation and the study was conducted with adherence to all relevant ethical standards.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Geist, T., Plezia, S., Cepeda-Benito, A. et al. Online Administration of the Implicit Relational Assessment Procedure: The Web-IRAP. Psychol Rec 73, 67–74 (2023). https://doi.org/10.1007/s40732-022-00533-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40732-022-00533-x