Skip to main content
SpringerLink
Log in

Reliability and Validity of the Implicit Relational Assessment Procedure (IRAP) as a Measure of Change Agenda

  • ORIGINAL ARTICLE
  • Published:
The Psychological Record Aims and scope Submit manuscript

Abstract

This research examined the reliability and validity of the Change Agenda Implicit Relational Assessment Procedure (CA-IRAP). Change agendas that lead people to think that “reducing or removing undesirable thoughts and feelings will solve problems and lead to a more successful life” are considered unworkable in acceptance and commitment therapy (ACT). For acceptance-based interventions to succeed, relinquishing change agendas is important. However, currently available methods for measuring these types of change agendas are limited to explicit measures, which are vulnerable to distortion from social desirability and demand characteristics. The authors attempted to measure change agendas using the IRAP. A total of 131 undergraduate and graduate students (40 male, 91 female; mean age = 21.93, SD = 3.19) participated. First, we examined the CA-IRAP’s discriminant validity in relation to explicit measures such as the Acceptance and Action Questionnaire-II in 83 participants who were able to pass the CA-IRAP. The correlations between the CA-IRAP and the explicit measures were low, confirming the discriminant validity of the CA-IRAP. Second, we examined the predictive validity of the CA-IRAP. Nineteen respondents had learned ACT and 34 respondents had never learned ACT. Non-ACT learners’ CA-IRAP scores were higher than those of ACT learners, which confirmed the CA-IRAP’s predictive validity. Finally, we examined the CA-IRAP’s test–retest reliability, which was confirmed across some CA-IRAP trial types. In conclusion, the authors submit that the CA-IRAP has strong potential as a valid instrument for measuring the relative strength of relational responses around change agendas while controlling for variables such as demand characteristics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Arnold, H. J., & Feldman, D. C. (1981). Social desirability response bias in self-report choice situations. Academy of Management Journal, 24(2), 377–385.

    Google Scholar 

  • Bach, P., & Moran, D. (2008). ACT in practice: Case conceptualization in Acceptance and Commitment Therapy. Oakland, CA: New Harbinger Publications.

    Google Scholar 

  • Barnes-Holmes, D., Barnes-Holmes, Y., Stewart, I., & Boles, S. (2010a). A sketch of the Implicit Relational Assessment Procedure (IRAP) and the Relational Elaboration and Coherence (REC) Model. The Psychological Record, 60(3), 527–542. https://doi.org/10.1007/BF03395726.

    Article  Google Scholar 

  • Barnes-Holmes, D., Murtagh, L., Barnes-Holmes, Y., & Stewart, I. (2010b). Using the Implicit Association Test and the Implicit Relational Assessment Procedure to measure attitudes toward meat and vegetables in vegetarians and meat-eaters. The Psychological Record, 60(2), 287–305. https://doi.org/10.1007/BF03395708.

    Article  Google Scholar 

  • Barnes-Holmes, D., Barnes-Holmes, Y., Luciano, C., & McEnteggart, C. (2017). From the IRAP and REC model to a multi-dimensional multi-level framework for analyzing the dynamics of arbitrarily applicable relational responding. Journal of Contextual Behavioral Science, 6(4), 434–445. https://doi.org/10.1016/j.jcbs.2017.08.001.

    Article  Google Scholar 

  • Bond, F. W., Hayes, S. C., Baer, R. A., Carpenter, K. M., Guenole, N., Orcutt, H. K., & Zettle, R. D. (2011). Preliminary Psychometric Properties of the Acceptance and Action Questionnaire-II: A revised measure of psychological inflexibility and experiential avoidance. Behavior Therapy, 42(4), 676–688. https://doi.org/10.1016/j.beth.2011.03.007.

    Article  Google Scholar 

  • Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. https://doi.org/10.1037/1040-3590.6.4.284.

    Article  Google Scholar 

  • Dalrymple, K. L., & Herbert, J. D. (2007). Acceptance and commitment therapy for generalized social anxiety disorder: A pilot study. Behavior Modification, 31(5), 543–568. https://doi.org/10.1177/0145445507302037.

    Article  PubMed  Google Scholar 

  • Drake, C. E., Kramer, S., Sain, T., Swiatek, R., Kohn, K., & Murphy, M. (2015). Exploring the reliability and convergent validity of implicit racial evaluations. Behavior & Social Issues, 87, 68–87. https://doi.org/10.5210/bsi.v.24i0.5496.

    Article  Google Scholar 

  • Eifert, G. H., & Forsyth, J. P. (2005). Acceptance and commitment therapy for anxiety disorders: A practitioner's treatment guide to using mindfulness, acceptance, and values-based behavior change strategies. Oakland, CA: New Harbinger Publications.

    Google Scholar 

  • Finn, M., Barnes-Holmes, D., & McEnteggart, C. (2018). Exploring the single-trial-type-dominance-effect in the IRAP: Developing a differential arbitrarily applicable relational responding effects (DAARRE) model. The Psychological Record, 68, 11–25. https://doi.org/10.1007/s40732-017-0262-z.

    Article  Google Scholar 

  • Flaxman, P. E., Blackledge, J. T., & Bond, F. W. (2011). Acceptance and commitment therapy. New York, NY: Routledge.

    Google Scholar 

  • Fledderus, M., Bohlmeijer, E. T., & Pieterse, M. E. (2010). Does experiential avoidance mediate the effects of maladaptive coping styles on psychopathology and mental health? Behavior Modification, 34(6), 503–519. https://doi.org/10.1177/0145445510378379.

    Article  PubMed  Google Scholar 

  • Forsyth, L., & Hayes, L. L. (2014). The effects of acceptance of thoughts, mindful awareness of breathing, and spontaneous coping on an experimentally induced pain task. The Psychological Record, 64(3), 447–455. https://doi.org/10.1007/s40732-014-0010-6.

    Article  Google Scholar 

  • Golijani-Moghaddam, N., Hart, A., & Dawson, D. L. (2013). The Implicit Relational Assessment Procedure: Emerging reliability and validity data. Journal of Contextual Behavioral Science, 2(3–4), 105–119. https://doi.org/10.1016/J.JCBS.2013.05.002.

    Article  Google Scholar 

  • Hayes, S. C., & Smith, S. (2005). Get out of your mind and into your life: The new acceptance and commitment therapy. Oakland, CA: New Harbinger Publications.

    Google Scholar 

  • Hayes, S. C., Wilson, K. G., Gifford, E. V., Follette, V. M., & Strosahl, K. (1996). Experiential avoidance and behavioral disorders: A functional dimensional approach to diagnosis and treatment. Journal of Consulting & Clinical Psychology, 64(6), 1152–1168. https://doi.org/10.1037/0022-006X.64.6.1152.

    Article  Google Scholar 

  • Hayes, S. C., Strosahl, K., & Wilson, K. G. (1999a). Acceptance and commitment therapy: An experiential approach to behavior change. New York, NY: Guilford Press.

    Google Scholar 

  • Hayes, S. C., Bissett, R. T., Korn, Z., Zettle, R. D., Rosenfarb, I. S., Cooper, L. D., & Grundt, A. M. (1999b). The impact of acceptance versus control rationales on pain tolerance. The Psychological Record, 49(1), 33–47. https://doi.org/10.1007/BF03395305.

    Article  Google Scholar 

  • Hayes, S. C., Barnes-Holmes, D., & Roche, B. (2001). Relational frame theory: A post-Skinnerian Account of human language and cognition. New York, NY: Kluwer Academic/Plenum Publishers.

    Book  Google Scholar 

  • Hayes, S. C., Strosahl, K. D., & Wilson, K. G. (2012). Acceptance and commitment therapy: The process and practice of mindful change (2nd ed.). New York, NY: Guilford Press.

    Google Scholar 

  • Hughes, S., & Barnes-Holmes, D. (2016). Relational frame theory: Implications for the study of human language and cognition. In R. D. Zettle, S. C. Hayes, D. Barnes-Holmes, & A. Biglan (Eds.), The Wiley handbook of contextual behavioral science (pp. 179–226). Wiley Blackwell: Chichester, UK.

    Google Scholar 

  • Hughes, S., Barnes-Holmes, D., & De Houwer, J. (2011). The dominance of associative theorizing in implicit attitude research: Propositional and behavioral alternatives. The Psychological Record, 61, 465–496. https://doi.org/10.1007/BF03395772.

    Article  Google Scholar 

  • Hughes, S., Barnes-Holmes, D., & Vahey, N. (2012). Holding on to our functional roots when exploring new intellectual islands: A voyage through implicit cognition research. Journal of Contextual Behavioral Science, 1(1), 17–38. https://doi.org/10.1016/j.jcbs.2012.09.003.

    Article  Google Scholar 

  • Hughes, S., De Houwer, J., & Barnes-Holmes, D. (2016). The moderating impact of distal regularities on the effects of stimulus pairings. Experimental Psychology, 63(1), 20–44. https://doi.org/10.1027/1618-3169/a000310.

    Article  PubMed  Google Scholar 

  • Hussey, I., Thompson, M., McEnteggart, C., Barnes-Holmes, D., & Barnes-Holmes, Y. (2015). Interpreting and inverting with less cursing: A guide to interpreting IRAP data. Journal of Contextual Behavioral Science, 4(3), 157–162. https://doi.org/10.1016/j.jcbs.2015.05.001.

    Article  Google Scholar 

  • Kishita, N., Ohtsuki, T., Sakai, M., & Muto, T. (2012). Behavioral assessment methodology for cognitive defusion: Testing the utility of the implicit relational assessment procedure. Japanese Association of Behavioral & Cognitive Therapies, 39(2), 99–110.

    Google Scholar 

  • Marx, B. P., & Sloan, D. M. (2005). Peritraumatic dissociation and experiential avoidance as predictors of posttraumatic stress symptomatology. Behaviour Research & Therapy, 43, 569–583. https://doi.org/10.1016/j.brat.2004.04.004.

    Article  Google Scholar 

  • McCracken, L. M., Vowles, K. E., & Eccleston, C. (2004). Acceptance of chronic pain: Component analysis and a revised assessment method. Pain, 107(1–2), 159–166. https://doi.org/10.1016/J.PAIN.2003.10.012.

    Article  PubMed  Google Scholar 

  • McKenna, I. M., Barnes-Holmes, D., Barnes-Holmes, Y., & Stewart, I. (2007). Testing the fake-ability of the Implicit Relational Assessment Procedure (IRAP): The first study. International Journal of Psychology & Psychological Therapy, 7(2), 253–268.

    Google Scholar 

  • Muto, T., Hayes, S. C., & Jeffcoat, T. (2011). The effectiveness of acceptance and commitment therapy bibliotherapy for enhancing the psychological health of Japanese college students living abroad. Behavior Therapy, 42(2), 323–335. https://doi.org/10.1016/j.beth.2010.08.009.

    Article  PubMed  Google Scholar 

  • Nicholson, E., & Barnes-Holmes, D. (2012). The Implicit Relational Assessment Procedure (IRAP) as a measure of spider fear. The Psychological Record, 62(2), 263–277. https://doi.org/10.1007/BF03395801.

    Article  Google Scholar 

  • Orne, M. T. (1962). On the social psychology of the psychological experiment. American Psychologist, 17(11), 776–783. https://doi.org/10.1037/h0043424.

    Article  Google Scholar 

  • Ramnerö, J., & Törneke, N. (2008). The ABCs of human behavior: Behavioral principles for the practicing clinician. Oakland, CA: New Harbinger Publications.

    Google Scholar 

  • Sakai, M., Ito, Y., Koda, M., & Muto, T. (2013). Effects of the acquisition of creative hopelessness: Correspondence between saying and doing. Japanese Association of Behavioral & Cognitive Therapies, 39(1), 1–11.

    Google Scholar 

  • Sakai, M., Masuda, A., Kishita, N., & Muto, T. (2014). The effects of Creative Hopelessness on the avoidant behaviors of undergraduate students with high social avoidance tendency: Focused on subjective evaluations of “Change Agenda”. Japanese Journal of Research on Emotions, 21(2), 58–64.

    Article  Google Scholar 

  • Sakai, M., Muto, T., & Ohtsuki, T. (2016). Effects of rules acquired through Creative Hopelessness on behavioral change: An experimental study of motivative augmentals. Japanese Association of Behavioral & Cognitive Therapies, 42(1), 51–62.

    Google Scholar 

  • Shima, T., Yanagihara, M., Kawai, T., & Kumano, H. (2013). The Acceptance and Action Questionnaire-II Japanese version". 77th Annual Convention of the Japanese Psychological Association. September 19, 2013, Hokkaido.

  • Shima, T., Tomita, N., Takahashi, M., & Kumano, H. (2018). Development of the Change Agenda Questionnaire and evaluation of its reliability and validity. Japanese Journal of Behavioral Medicine, 23(2), 103–110.

    Google Scholar 

  • Shimizu, H., & Imae, K. (1981). Nihongoban State-Trait anxiety inventory no sakusei. Japanese Journal of Educational Psychology, 29, 348–353.

    Article  Google Scholar 

  • Society of Clinical Psychology. (2016). Psychological Treatments. Retrieved September 27, 2019, from https://www.div12.org/treatments/

  • Spielberger, C. D., Gorsuch, R. L., & Lushene, R. E. (1970). Manual for the State-Trait Anxiety Inventory (Self-Evaluation Questionnaire). Palo Alto, CA: Consulting Psychologist Press.

    Google Scholar 

  • Strosahl, K., Robinson, P., & Gustavsson, T. (2012). Brief intervention for radical change: Principles and practice of focused acceptance and commitment therapy. Oakland, CA: New Harbinger Publications.

    Google Scholar 

  • Tartakovsky, M. (2014). How to sit with painful emotions. Retrieved April 19, 2020, from https://psychcentral.com/blog/how-to-sit-with-painful-emotions/

  • Vahey, N., Boles, S., & Barnes-Holmes, D. (2010). Measuring adolescents’ smoking-related social identity preferences with the implicit relational assessment procedure (IRAP) for the first time: A starting point that explains later IRAP evolutions. International Journal of Psychology & Psychological Therapy, 10(3), 453–474.

    Google Scholar 

  • Yavuz, F., Ulusoy, S., Iskin, M., Esen, F. B., Burhan, H. S., Karadere, M. E., & Yavuz, N. (2016). Turkish Version of Acceptance and Action Questionnaire-II (AAQ-II): A reliability and validity analysis in clinical and non-clinical samples. Klinik Psikofarmakoloji Bulteni/Bulletin of Clinical Psychopharmacology, 26(4), 397–408.

    Article  Google Scholar 

Download references

Funding

There are no funders to report for this submission.

Author information

Authors and Affiliations

  1. Faculty of Human Sciences, Waseda University, 2-579-15, Mikajima, Tokorozawa City, Saitama, 359-1192, Japan

    Kazuya Inoue, Steve K. Lee, Tomu Ohtsuki & Hiroaki Kumano

  2. Doshisha University, Kyotanabe City, Japan

    Taiki Shima

  3. Suzunoki Hospital, Kuki City, Japan

    Madoka Takahashi

Authors
  1. Kazuya Inoue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Taiki Shima
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Madoka Takahashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Steve K. Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tomu Ohtsuki
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hiroaki Kumano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kazuya Inoue.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflicts of interest.

Consent to Participate

Informed consent was obtained from all individual participants.

Ethical Approval

All procedures involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Availability of Data and Materials

K. Inoue, Data set: Reliability and validity of the Implicit Relational Assessment Procedure (IRAP) as a measure of Change Agenda. Figshare, https://figshare.com/s/176006a3665ed285be88

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Inoue, K., Shima, T., Takahashi, M. et al. Reliability and Validity of the Implicit Relational Assessment Procedure (IRAP) as a Measure of Change Agenda. Psychol Rec 70, 499–513 (2020). https://doi.org/10.1007/s40732-020-00416-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40732-020-00416-z

Keywords

Search

Navigation