Skip to main content
SpringerLink
Log in

Reliability of Identification of Behavior Change Techniques in Intervention Descriptions

  • Original Article
  • Published:
Annals of Behavioral Medicine

Abstract

Purpose

The aim of this paper is to assess the frequency of identification as well as the inter-coder and test–retest reliability of identification of behavior change techniques (BCTs) in written intervention descriptions.

Methods

Forty trained coders applied the “Behavior Change Technique Taxonomy version 1” (BCTTv1) to 40 intervention descriptions published in protocols and repeated this 1 month later.

Results

Eighty of 93 defined BCTs were identified by at least one trained coder, and 22 BCTs were identified in 16 (40 %) or more of 40 descriptions. Good inter-coder reliability was observed across 80 BCTs identified in the protocols: 66 (80 %) achieved mean prevalence and bias-adjusted kappa (PABAK) scores of 0.70 or greater, and 59 (74 %) achieved mean scores of 0.80 or greater. There was good within-coder agreement between baseline and 1 month, demonstrating good test–retest reliability.

Conclusions

BCTTv1 can be used by trained coders to identify BCTs in intervention descriptions reliably. However, some frequently occurring BCT definitions require further clarification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Whitlock G, Lewington S, Sherliker P, et al. Body-mass index and cause-specific mortality in 900 000 adults: Collaborative analyses of 57 prospective studies. Lancet. 2009; 373: 1083-1096.

    Article  PubMed  Google Scholar 

  2. Wing R, Jeffery R. Benefits of recruiting participants with friends and increasing social support for weight loss and maintenance. J Consult Clin Psychol. 1999; 67: 132-138.

    Article  CAS  PubMed  Google Scholar 

  3. Fisher J, Fisher W. Changing AIDS-risk behavior. Psychol Bull. 1992; 111: 455-474.

    Article  CAS  PubMed  Google Scholar 

  4. Denford S, Abraham C, Smith J, et al. Designing and evaluating behavior change interventions to promote health. In: Reynolds KJ, Branscombe NR, eds. The Psychology of Change: Life Contexts, Experiences, and Identities. New York: Psychology Press; 2015.

    Google Scholar 

  5. Davidson KW, Goldstein M, Kaplan RM, et al. Evidence-based behavioral medicine: What is it and how do we achieve it? Ann Behav Med. 2003; 26: 161-171.

    Article  PubMed  Google Scholar 

  6. Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol. 2008; 27: 379-387.

    Article  PubMed  Google Scholar 

  7. Albarracín D, Gillette JC, Earl AN, et al. A test of major assumptions about behavior change: A comprehensive look at the effects of passive and active HIV-prevention interventions since the beginning of the epidemic. Psychol Bull. 2005; 131: 856-897.

    Article  PubMed Central  PubMed  Google Scholar 

  8. Moher D, Schulz K, Altman D. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomised trials. Clin Oral Investig. 2003; 7: 2-7.

    PubMed  Google Scholar 

  9. CONSORT Group, Boutron I, Moher D, et al. Statement to randomized trials of non-pharmacologic treatment: Explanation and elaboration. Ann Intern Med. 2008; 148: 295-309.

    Article  Google Scholar 

  10. Hoffmann T, Glasziou P, Boutron I, et al. Better reporting of interventions: Template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014; 348.

  11. Abraham C. Designing and evaluating interventions to change health-related behaviour patterns. In: Boutron I, Ravaud P, Moher D, eds. Randomized Clinical Trials of Non-Pharmacologic Treatments. London: Chapman and Hall; 2012: 357-368.

    Google Scholar 

  12. Abraham C, Johnson BT, de Bruin M, et al. Enhancing reporting of behavior change intervention evaluations. JAIDS. 2014; 15: S293-9.

    Google Scholar 

  13. Albrecht L, Archibald M, Arseneau D, et al. Development of a checklist to assess the quality of reporting of knowledge translation interventions using the Workgroup for Intervention Development and Evaluation Research (WIDER) recommendations. Implement Sci. 2013; 8: 52.

    Article  PubMed Central  PubMed  Google Scholar 

  14. Hoffmann T, English T, Glasziou P. Reporting of interventions in randomised trials: An audit of journal instructions to authors. Trials. 2014; 15: 20.

    Article  PubMed Central  PubMed  Google Scholar 

  15. Glasgow R, Bull S, Gillette C, et al. Behavior change intervention research in healthcare settings: A review of recent reports with emphasis on external validity. Am J Prev Med. 2002; 23: 62-69.

    Article  PubMed  Google Scholar 

  16. Bartholomew L, Parcel G, Kok G, et al., eds. Planning Health Promotion Programs: An Intervention Mapping Approach. 3rd ed. San Francisco: Jossey-Bass; 2011.

    Google Scholar 

  17. Michie S, van Stralen MM, West R. The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implement Sci. 2011; 6: 42.

    Article  PubMed Central  PubMed  Google Scholar 

  18. Michie S. Behaviour change techniques. In: Gellman MT, Rick J, eds. Encyclopedia of Behavioural Medicine. London: Springer; 2013: 182-187.

    Google Scholar 

  19. Michie S, Abraham C, Whittington C, et al. Effective techniques in healthy eating and physical activity interventions: A meta-regression. Health Psychol. 2009; 28: 690-701.

    Article  PubMed  Google Scholar 

  20. Carver CS, Scheier MF. Control theory: A useful conceptual framework for personality—social, clinical, and health psychology. Psychol Bull. 1982; 92: 111.

    Article  CAS  PubMed  Google Scholar 

  21. Michie S, Ashford S, Sniehotta FF, et al. A refined taxonomy of behaviour change techniques to help people change their physical activity and healthy eating behaviours: The CALO-RE taxonomy. Psychol Health. 2011; 26: 1479-1498.

    Article  PubMed  Google Scholar 

  22. Michie S, Hyder N, Walia A, et al. Development of a taxonomy of behaviour change techniques used in individual behavioural support for smoking cessation. Addict Behav. 2011; 36: 315-319.

    Article  PubMed  Google Scholar 

  23. Michie S, Churchill S, West R. Identifying evidence-based competences required to deliver behavioural support for smoking cessation. Ann Behav Med. 2011; 41: 59-70.

    Article  PubMed  Google Scholar 

  24. Michie S, Whittington C, Hamoudi Z, et al. Identification of behaviour change techniques to reduce excessive alcohol consumption. Addiction. 2012; 107: 1431-1440.

    Article  PubMed  Google Scholar 

  25. Abraham C, Good A, Huedo-Medina TB, et al. Reliability and utility of the SHARP taxonomy of behaviour change techniques. Psychol Health. 2012; 27: 1-2.

    Google Scholar 

  26. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012; 6: Cd000259.

    PubMed  Google Scholar 

  27. Schulz R, Czaja SJ, McKay JR, et al. Intervention taxonomy (ITAX): Describing essential features of interventions (HMC). Am J Health Behav. 2010; 34: 811-821.

    Article  PubMed Central  PubMed  Google Scholar 

  28. Michie S, Richardson M, Johnston M, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013; 46: 81-95.

    Article  PubMed  Google Scholar 

  29. Michie S, Abraham C, Eccles MP, et al. Strengthening evaluation and implementation by specifying components of behaviour change interventions: A study protocol. Implement Sci. 2011; 6: 10.

    Article  PubMed Central  PubMed  Google Scholar 

  30. Wood C, Richardson M, Johnston M, et al.: Applying the Behaviour Change Technique (BCT) Taxonomy v1: A study of coder training. Tran Behav Med. 2014:1–15.

  31. Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993; 46: 423-429.

    Article  CAS  PubMed  Google Scholar 

  32. Cohen J. Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol Bull. 1968; 70: 213.

    Article  CAS  PubMed  Google Scholar 

  33. Gwet KL: Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Multiple Raters: Advanced Analytics Press, 2012.

  34. Gwet K: Inter-rater reliability: dependency on trait prevalence and marginal homogeneity. Statistical Methods for Inter-Rater Reliability Assessment Series, 2002.

  35. van Beurden, S. B., Greaves, C. J., Smith, J. R. & Abraham, C. (under review). Techniques for managing impulsive processes associated with unhealthy eating: A systematic review. Health Psychology.

  36. Hovland CI, Janis IL, Kelley HH: Communication and persuasion; psychological studies of opinion change. 1953.

  37. Durantini MR, Albarracin D, Mitchell AL, et al. Conceptualizing the influence of social agents of behavior change: A meta-analysis of the effectiveness of HIV-prevention interventionists for different groups. Psychol Bull. 2006; 132: 212-248.

    Article  PubMed  Google Scholar 

  38. Dombrowski SU, Sniehotta FF, Avenell A, et al. Identifying active ingredients in complex behavioural interventions for obese adults with obesity-related co-morbidities or additional risk factors for co-morbidities: A systematic review. Health Psychol Rev. 2010; 6: 7-32.

    Article  Google Scholar 

Download references

Acknowledgements

The work was carried out as part of the BCT Taxonomy project funded by the Medical Research Council [G0901474/1]. We are grateful to Kate Sheals (KS) for help with coding and to Professor Tania Huedo-Medina of the Department of Statistics, University of Connecticut for sharing her AC1 calculator with us. We acknowledge funding from the UK National Institute for Health Research (NIHR) and the Peninsula Collaboration for Leadership in Applied Health Research and Care. The views expressed in this paper are those of the authors and not necessarily those of NIHR or the UK Department of Health. Author order reflects contribution to the paper.

Author Statement of Conflict of Interest and Adherence to Ethical Standards

The authors, Charles Abraham, Caroline E Wood, Marie Johnston, Jill Francis, Michelle Richardson, and Susan Michie, have no conflicts of interest to disclose. Wendy Hardeman has provided consultancy for AbbVie Ltd. All procedures, including the informed consent process, were conducted in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000.

Author information

Authors and Affiliations

  1. Psychology Applied to Health Group, University of Exeter Medical School, University of Exeter, St Luke’s Campus, Exeter, EX1 2 LU, UK

    Charles Abraham PhD

  2. Research Department of Clinical, Educational and Health Psychology, University College London, 1-19 Torrington Place, London, WC1E 7HB, UK

    Caroline E Wood PhD & Marie Johnston PhD

  3. Institute of Applied Health Sciences, College of Life Science and Medicine, University of Aberdeen, 2nd Floor, Health Sciences Building, Aberdeen, AB25 2ZD, UK

    Marie Johnston PhD

  4. School of Health Sciences, City University London, Myddelton Street Building, Northampton Square, London, EC1V 0HB, UK

    Jill Francis PhD

  5. Primary Care Unit, Cambridge Institute of Public Health, University of Cambridge, Forvie Site, Robinson Way, Cambridge, CB2 0SR, UK

    Wendy Hardeman PhD

  6. Evidence for Policy and Practice Information and Coordinating Centre, Social Science Research Unit, UCL Institute of Education, London, UK

    Michelle Richardson PhD & Susan Michie PhD

Authors
  1. Charles Abraham PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Caroline E Wood PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marie Johnston PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jill Francis PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wendy Hardeman PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michelle Richardson PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Susan Michie PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles Abraham PhD.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

ESM 1

(DOC 575 kb)

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abraham, C., Wood, C.E., Johnston, M. et al. Reliability of Identification of Behavior Change Techniques in Intervention Descriptions. ann. behav. med. 49, 885–900 (2015). https://doi.org/10.1007/s12160-015-9727-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12160-015-9727-y

Keywords

Search

Navigation