Abstract
Purpose
The aim of this paper is to assess the frequency of identification as well as the inter-coder and test–retest reliability of identification of behavior change techniques (BCTs) in written intervention descriptions.
Methods
Forty trained coders applied the “Behavior Change Technique Taxonomy version 1” (BCTTv1) to 40 intervention descriptions published in protocols and repeated this 1 month later.
Results
Eighty of 93 defined BCTs were identified by at least one trained coder, and 22 BCTs were identified in 16 (40 %) or more of 40 descriptions. Good inter-coder reliability was observed across 80 BCTs identified in the protocols: 66 (80 %) achieved mean prevalence and bias-adjusted kappa (PABAK) scores of 0.70 or greater, and 59 (74 %) achieved mean scores of 0.80 or greater. There was good within-coder agreement between baseline and 1 month, demonstrating good test–retest reliability.
Conclusions
BCTTv1 can be used by trained coders to identify BCTs in intervention descriptions reliably. However, some frequently occurring BCT definitions require further clarification.
Similar content being viewed by others
References
Whitlock G, Lewington S, Sherliker P, et al. Body-mass index and cause-specific mortality in 900 000 adults: Collaborative analyses of 57 prospective studies. Lancet. 2009; 373: 1083-1096.
Wing R, Jeffery R. Benefits of recruiting participants with friends and increasing social support for weight loss and maintenance. J Consult Clin Psychol. 1999; 67: 132-138.
Fisher J, Fisher W. Changing AIDS-risk behavior. Psychol Bull. 1992; 111: 455-474.
Denford S, Abraham C, Smith J, et al. Designing and evaluating behavior change interventions to promote health. In: Reynolds KJ, Branscombe NR, eds. The Psychology of Change: Life Contexts, Experiences, and Identities. New York: Psychology Press; 2015.
Davidson KW, Goldstein M, Kaplan RM, et al. Evidence-based behavioral medicine: What is it and how do we achieve it? Ann Behav Med. 2003; 26: 161-171.
Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol. 2008; 27: 379-387.
Albarracín D, Gillette JC, Earl AN, et al. A test of major assumptions about behavior change: A comprehensive look at the effects of passive and active HIV-prevention interventions since the beginning of the epidemic. Psychol Bull. 2005; 131: 856-897.
Moher D, Schulz K, Altman D. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomised trials. Clin Oral Investig. 2003; 7: 2-7.
CONSORT Group, Boutron I, Moher D, et al. Statement to randomized trials of non-pharmacologic treatment: Explanation and elaboration. Ann Intern Med. 2008; 148: 295-309.
Hoffmann T, Glasziou P, Boutron I, et al. Better reporting of interventions: Template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014; 348.
Abraham C. Designing and evaluating interventions to change health-related behaviour patterns. In: Boutron I, Ravaud P, Moher D, eds. Randomized Clinical Trials of Non-Pharmacologic Treatments. London: Chapman and Hall; 2012: 357-368.
Abraham C, Johnson BT, de Bruin M, et al. Enhancing reporting of behavior change intervention evaluations. JAIDS. 2014; 15: S293-9.
Albrecht L, Archibald M, Arseneau D, et al. Development of a checklist to assess the quality of reporting of knowledge translation interventions using the Workgroup for Intervention Development and Evaluation Research (WIDER) recommendations. Implement Sci. 2013; 8: 52.
Hoffmann T, English T, Glasziou P. Reporting of interventions in randomised trials: An audit of journal instructions to authors. Trials. 2014; 15: 20.
Glasgow R, Bull S, Gillette C, et al. Behavior change intervention research in healthcare settings: A review of recent reports with emphasis on external validity. Am J Prev Med. 2002; 23: 62-69.
Bartholomew L, Parcel G, Kok G, et al., eds. Planning Health Promotion Programs: An Intervention Mapping Approach. 3rd ed. San Francisco: Jossey-Bass; 2011.
Michie S, van Stralen MM, West R. The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implement Sci. 2011; 6: 42.
Michie S. Behaviour change techniques. In: Gellman MT, Rick J, eds. Encyclopedia of Behavioural Medicine. London: Springer; 2013: 182-187.
Michie S, Abraham C, Whittington C, et al. Effective techniques in healthy eating and physical activity interventions: A meta-regression. Health Psychol. 2009; 28: 690-701.
Carver CS, Scheier MF. Control theory: A useful conceptual framework for personality—social, clinical, and health psychology. Psychol Bull. 1982; 92: 111.
Michie S, Ashford S, Sniehotta FF, et al. A refined taxonomy of behaviour change techniques to help people change their physical activity and healthy eating behaviours: The CALO-RE taxonomy. Psychol Health. 2011; 26: 1479-1498.
Michie S, Hyder N, Walia A, et al. Development of a taxonomy of behaviour change techniques used in individual behavioural support for smoking cessation. Addict Behav. 2011; 36: 315-319.
Michie S, Churchill S, West R. Identifying evidence-based competences required to deliver behavioural support for smoking cessation. Ann Behav Med. 2011; 41: 59-70.
Michie S, Whittington C, Hamoudi Z, et al. Identification of behaviour change techniques to reduce excessive alcohol consumption. Addiction. 2012; 107: 1431-1440.
Abraham C, Good A, Huedo-Medina TB, et al. Reliability and utility of the SHARP taxonomy of behaviour change techniques. Psychol Health. 2012; 27: 1-2.
Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012; 6: Cd000259.
Schulz R, Czaja SJ, McKay JR, et al. Intervention taxonomy (ITAX): Describing essential features of interventions (HMC). Am J Health Behav. 2010; 34: 811-821.
Michie S, Richardson M, Johnston M, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013; 46: 81-95.
Michie S, Abraham C, Eccles MP, et al. Strengthening evaluation and implementation by specifying components of behaviour change interventions: A study protocol. Implement Sci. 2011; 6: 10.
Wood C, Richardson M, Johnston M, et al.: Applying the Behaviour Change Technique (BCT) Taxonomy v1: A study of coder training. Tran Behav Med. 2014:1–15.
Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993; 46: 423-429.
Cohen J. Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol Bull. 1968; 70: 213.
Gwet KL: Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Multiple Raters: Advanced Analytics Press, 2012.
Gwet K: Inter-rater reliability: dependency on trait prevalence and marginal homogeneity. Statistical Methods for Inter-Rater Reliability Assessment Series, 2002.
van Beurden, S. B., Greaves, C. J., Smith, J. R. & Abraham, C. (under review). Techniques for managing impulsive processes associated with unhealthy eating: A systematic review. Health Psychology.
Hovland CI, Janis IL, Kelley HH: Communication and persuasion; psychological studies of opinion change. 1953.
Durantini MR, Albarracin D, Mitchell AL, et al. Conceptualizing the influence of social agents of behavior change: A meta-analysis of the effectiveness of HIV-prevention interventionists for different groups. Psychol Bull. 2006; 132: 212-248.
Dombrowski SU, Sniehotta FF, Avenell A, et al. Identifying active ingredients in complex behavioural interventions for obese adults with obesity-related co-morbidities or additional risk factors for co-morbidities: A systematic review. Health Psychol Rev. 2010; 6: 7-32.
Acknowledgements
The work was carried out as part of the BCT Taxonomy project funded by the Medical Research Council [G0901474/1]. We are grateful to Kate Sheals (KS) for help with coding and to Professor Tania Huedo-Medina of the Department of Statistics, University of Connecticut for sharing her AC1 calculator with us. We acknowledge funding from the UK National Institute for Health Research (NIHR) and the Peninsula Collaboration for Leadership in Applied Health Research and Care. The views expressed in this paper are those of the authors and not necessarily those of NIHR or the UK Department of Health. Author order reflects contribution to the paper.
Author Statement of Conflict of Interest and Adherence to Ethical Standards
The authors, Charles Abraham, Caroline E Wood, Marie Johnston, Jill Francis, Michelle Richardson, and Susan Michie, have no conflicts of interest to disclose. Wendy Hardeman has provided consultancy for AbbVie Ltd. All procedures, including the informed consent process, were conducted in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000.
Author information
Authors and Affiliations
Corresponding author
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
ESM 1
(DOC 575 kb)
About this article
Cite this article
Abraham, C., Wood, C.E., Johnston, M. et al. Reliability of Identification of Behavior Change Techniques in Intervention Descriptions. ann. behav. med. 49, 885–900 (2015). https://doi.org/10.1007/s12160-015-9727-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12160-015-9727-y