Skip to main content
SpringerLink
Log in

Effect Sizes and Primary Outcomes in Large-Budget, Cardiovascular-Related Behavioral Randomized Controlled Trials Funded by NIH Since 1980

  • Original Article
  • Published:
Annals of Behavioral Medicine

Abstract

Purpose

We reviewed large-budget, National Institutes of Health (NIH)-supported randomized controlled trials (RCTs) with behavioral interventions to assess (1) publication rates, (2) trial registration, (3) use of objective measures, (4) significant behavior and physiological change, and (5) effect sizes.

Methods

We identified large-budget grants (>$500,000/year) funded by NIH (National Heart Lung and Blood Institute (NHLBI) or National Institute of Diabetes & Digestive and Kidney Diseases (NIDDK)) for cardiovascular disease (dates January 1, 1980 to December 31, 2012). Among 106 grants that potentially met inclusion criteria, 20 studies were not published and 48 publications were excluded, leaving 38 publications for analysis. ClinicalTrials.gov abstracts were used to determine whether outcome measures had been pre-specified.

Results

Three fourths of trials were registered in ClinicalTrials.gov and all published pre-specified outcomes. Twenty-six trials reported a behavioral outcome with 81 % reporting significant improvements for the target behavior. Thirty-two trials reported a physiological outcome. All were objectively measured, and 81 % reported significant benefit. Seventeen trials reported morbidity outcomes, and seven reported a significant benefit. Nine trials assessed mortality, and all were null for this outcome.

Conclusions

Behavioral trials complied with trial registration standards. Most reported a physiological benefit, but few documented morbidity or mortality benefits.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Fisher EB, Fitzgibbon ML, Glasgow RE, et al. Behavior matters. Am J Prev Med. 2011; 40: e15-30.

    Article  PubMed Central  PubMed  Google Scholar 

  2. Neiberg RH, Wing RR, Bray GA, et al. Patterns of weight change associated with long-term weight change and cardiovascular disease risk factors in the Look AHEAD Study. Obesity (Silver Spring). 2012; 20: 2048-2056.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  3. Look ARG, Wing RR, Bolin P, et al. Cardiovascular effects of intensive lifestyle intervention in type 2 diabetes. N Engl J Med. 2013; 369: 145-154.

    Article  Google Scholar 

  4. World Health Organization. Non-communicable diseases. Available at http://www.who.int/mediacentre/factsheets/fs355/en/. Accessibility verified July 3, 2015.

  5. Goff DC Jr, Lloyd-Jones DM, Bennett G, et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014; 129(25 Suppl 2): S49-73.

    Article  PubMed  Google Scholar 

  6. Stone NJ, Robinson J, Lichtenstein AH, et al. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014; 129(25 Suppl 2): S1-45.

    Article  PubMed  Google Scholar 

  7. Phillips SM, Glasgov RE, Bello G, et al. Frequency and prioritization of patient health risks from a structured health risk assessment. Ann Fam Med. 2014; 12: 505-513.

    Article  PubMed Central  PubMed  Google Scholar 

  8. Kaplan RM, Morgan G. Providers advice concerning smoking cessation: Evidence from the medical expenditures panel survey. San Antonio: Presented at the annual meeting of the Society of Behavioral Medicine; 2015.

    Google Scholar 

  9. Kaplan RM, Irvin VL. Likelihood of null effects of large NHLBI clinical trials has increased over time. PLOS One. 2015; 10(8): e0132382.

    Article  PubMed Central  PubMed  Google Scholar 

  10. Goldacre B. Preventing bad reporting on health research. BMJ. 2014; 349: g7465.

    Article  PubMed  Google Scholar 

  11. Sumner P, Vivian-Griffiths S, Boivin J, et al. The association between exaggeration in health related science news and academic press releases: retrospective observational study. BMJ. 2014; 349: g7015.

    Article  PubMed Central  PubMed  Google Scholar 

  12. Pharmaceutical Research and Manufacturers of American. PhRMA 2012 Industry Profile. Available at http://www.phrma.org/sites/default/files/159/phrma_industry_profile.pdf. Accessibility verified July 3, 2015.

  13. Gordon D, Taddei-Peters W, Mascette A, et al. Publication of trials funded by the National Heart, Lung, and Blood Institute. N Engl J Med. 2013; 369: 1926-1934.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  14. Easterbrook PJ, Gopalan R, Berlin J, Matthews DR. Publication bias in clinical research. The Lancet. 1991; 337: 867-872.

    Article  CAS  Google Scholar 

  15. Knowler WC, Barrett-Connor E, Fowler SE, et al. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med. 2002; 346: 393-403.

    Article  CAS  PubMed  Google Scholar 

  16. Cambell Collaboration. David B. Wilson. Practical meta-analysis effect size calculator. Available at http://www.campbellcollaboration.org/escalc/html/EffectSizeCalculator-SMD8.php . Accessibility verified July 6, 2015.

  17. Cohen J. A power primer. Psychol Bull. 1992; 112: 155-159.

    Article  CAS  PubMed  Google Scholar 

  18. Zarin DA, Tse T. Moving toward transparency of clinical trials. Science. 2008; 319: 1340-1342.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  19. Lovshin JA, Drucker DJ. Metabolic disease puts up a fight: Are diet and exercise helpful for the heart? Nat Med. 2013; 19: 1216-1217.

    Article  CAS  PubMed  Google Scholar 

  20. Lin JS, O'Connor E, Evans CV, et al. Behavioral counseling to promote a healthy lifestyle in persons with cardiovascular risk factors: A systematic review for the U.S. Preventive Services Task Force. Ann Intern Med. 2014; 161(8): 568-578.

    Article  PubMed  Google Scholar 

  21. National Cancer Institute. Dietary assessment primer. Available at http://dietassessmentprimer.cancer.gov/learn/. Accessibility verified July 3, 2015.

  22. Writing Group for the Activity Counseling Trial Research G. Effects of physical activity counseling in primary care: The Activity Counseling Trial: A randomized controlled trial. JAMA. 2001; 286: 677-687.

    Article  Google Scholar 

  23. Harvey-Berino J, West D, Krukowski R, et al. Internet delivered behavioral obesity treatment. Prev Med. 2010; 51: 123-128.

    Article  PubMed Central  PubMed  Google Scholar 

  24. No authors. The lipid research clinics coronary primary prevention trial results. 1. Reduction in incidence of coronary heart-disease. JAMA. 1984; 251:351-364.

  25. No authors. Hennekens CH: Preliminary report: Findings from the aspirin component of the ongoing physicians’ health study. N Engl J Med. 1988; 318:262-264.

  26. Nigg CR, Long CR. A systematic review of single health behavior change interventions vs multiple health behavior change interventions among older adults. TBM. 2012; 2: 166-179.

    Google Scholar 

  27. Milette K, Roseman M, Thombs BD. Transparency of outcome reporting and trial registration of randomized controlled trials in top psychosomatic and behavioral health journals: A systematic review. J Psychosom Res. 2011; 70: 205-217.

    Article  PubMed  Google Scholar 

  28. National Institutes of Health. HHS and NIH take steps to enhance transparency of clinical trial results. Available at http://www.nih.gov/news/health/nov2014/od-19.htm. Accessibility verified July 6, 2015.

  29. Linden W, Satin JR. Avoidable pitfalls in behavioral medicine outcome research. Ann Behav Med. 2007; 33: 143-147.

    Article  PubMed  Google Scholar 

  30. Schneider S, Moyer A, Knapp-Oliver S, et al. Pre-intervention distress moderates the efficacy of psychosocial treatment for cancer patients: A meta-analysis. J Behav Med. 2010; 33: 1-14.

    Article  PubMed Central  PubMed  Google Scholar 

  31. McDermott MM, Reed G, Greenland P, et al. Activating peripheral arterial disease patients to reduce cholesterol: A randomized trial. Am J Med. 2011; 124: 557-565.

    Article  PubMed Central  PubMed  Google Scholar 

  32. Bennett GG, Warner ET, Glasgow RE, et al. Obesity treatment for socioeconomically disadvantaged patients in primary care practice. Arch Intern Med. 2012; 172: 565-574.

    Article  PubMed Central  PubMed  Google Scholar 

  33. Johnson W, Shaya FT, Khanna N, et al. The Baltimore Partnership to Educate and Achieve Control of Hypertension (The BPTEACH Trial): A randomized trial of the effect of education on improving blood pressure control in a largely African American population. J Clin Hypertens (Greenwich). 2011; 13: 563-570.

    Article  Google Scholar 

  34. Stefanick ML, Mackey S, Sheehan M, et al. Effects of diet and exercise in men and postmenopausal women with low levels of HDL cholesterol and high levels of LDL cholesterol. N Engl J Med. 1998; 339: 12-20.

    Article  CAS  PubMed  Google Scholar 

  35. Langford HG, Blaufox MD, Oberman A, et al. Dietary therapy slows the return of hypertension after stopping prolonged medication. JAMA. 1985; 253: 657-664.

    Article  CAS  PubMed  Google Scholar 

  36. Delahanty LM, Pan Q, Jablonski KA, et al. Genetic predictors of weight loss and weight regain after intensive lifestyle modification, metformin treatment, or standard care in the Diabetes Prevention Program. Diabetes Care. 2012; 35: 363-366.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  37. Berkman LF, Blumenthal J, Burg M, et al. Effects of treating depression and low perceived social support on clinical events after myocardial infarction: The Enhancing Recovery in Coronary Heart Disease Patients (ENRICHD) Randomized Trial. JAMA. 2003; 289: 3106-3116.

    Article  PubMed  Google Scholar 

  38. Martin MY, Kim YI, Kratt P, et al. Medication adherence among rural, low-income hypertensive adults: A randomized trial of a multimedia community-based intervention. Am J Health Promot. 2011; 25: 372-378.

    Article  PubMed Central  PubMed  Google Scholar 

  39. Powell LH, Calvin JE Jr, Richardson D, et al. Self-management counseling in patients with heart failure: The heart failure adherence and retention randomized behavioral trial. JAMA. 2010; 304: 1331-1338.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  40. Stamler R, Stamler J, Grimm R, et al. Nutritional therapy for high blood pressure. Final report of a four-year randomized controlled trial—The Hypertension Control Program. JAMA. 1987; 257: 1484-1491.

    Article  CAS  PubMed  Google Scholar 

  41. DeWalt DA, Schillinger D, Ruo B, et al. Multisite randomized trial of a single-session versus multisession literacy-sensitive self-care intervention for patients with heart failure. Circulation. 2012; 125: 2854-2862.

    Article  PubMed Central  PubMed  Google Scholar 

  42. Baker DW, Dewalt DA, Schillinger D, et al. The effect of progressive, reinforcing telephone education and counseling versus brief educational intervention on knowledge, self-care behaviors, and heart failure symptoms. Journal of Cardiac Failure. 2011; 17(1): 789-796.

    Article  PubMed Central  PubMed  Google Scholar 

  43. Katula JA, Vitolins MZ, Morgan TM, et al. The Healthy Living Partnerships to Prevent Diabetes study: 2-year outcomes of a randomized controlled trial. Am J Prev Med. 2013; 44: S324-332.

    Article  PubMed Central  PubMed  Google Scholar 

  44. O'Connor CM, Whellan DJ, Lee KL, et al. Efficacy and safety of exercise training in patients with chronic heart failure: HF-ACTION randomized controlled trial. JAMA. 2009; 301: 1439-1450.

    Article  PubMed Central  PubMed  Google Scholar 

  45. Pezzin LE, Feldman PH, Mongoven JM, et al. Improving blood pressure control: Results of home-based post-acute care interventions. J Gen Intern Med. 2011; 26: 280-286.

    Article  PubMed Central  PubMed  Google Scholar 

  46. McBride CM, Curry SJ, Lando HA, et al. Prevention of relapse in women who quit smoking during pregnancy. Am J Public Health. 1999; 89: 706-711.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  47. Hypertension Prevention Trial Research Group. The Hypertension Prevention Trial: Three-year effects of dietary changes on blood pressure. Arch Intern Med. 1990; 150: 153-162.

    Article  Google Scholar 

  48. Stamler R, Stamler J, Gosch FC, et al. Primary prevention of hypertension by nutritional-hygienic means. Final report of a randomized, controlled trial. JAMA. 1989; 262: 1801-1807.

    Article  CAS  PubMed  Google Scholar 

  49. Wolf AM, Conaway MR, Crowther JQ, et al. Translating lifestyle intervention to practice in obese patients with type 2 diabetes: Improving Control with Activity and Nutrition (ICAN) study. Diabetes Care. 2004; 27: 1570-1576.

    Article  PubMed  Google Scholar 

  50. Krieger J, Collier C, Song L, Martin D. Linking community-based blood pressure measurement to clinical care: A randomized controlled trial of outreach and tracking by community health workers. Am J Public Health. 1999; 89: 856-861.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  51. Pi-Sunyer X, Blackburn G, Brancati FL, et al. Reduction in weight and cardiovascular disease risk factors in individuals with type 2 diabetes: One-year results of the LOOK AHEAD trial. Diabetes Care. 2007; 30: 1374-1383.

    Article  PubMed  Google Scholar 

  52. Eight-year weight losses with an intensive lifestyle intervention: The LOOK AHEAD study. Obesity (Silver Spring). 2014; 22:5-13.

  53. Toobert DJ, Glasgow RE, Strycker LA, et al. Biologic and quality-of-life outcomes from the Mediterranean Lifestyle Program: A randomized clinical trial. Diabetes Care. 2003; 26: 2288-2293.

    Article  PubMed  Google Scholar 

  54. Toobert DJ, Glasgow RE, Strycker LA, et al. Long-term effects of the Mediterranean lifestyle program: A randomized clinical trial for postmenopausal women with type 2 diabetes. Int J Behav Nutr Phys Act. 2007; 4: 1.

    Article  PubMed Central  PubMed  Google Scholar 

  55. Toobert DJ, Strycker LA, Glasgow RE, Barrera M Jr, Angell K. Effects of the mediterranean lifestyle program on multiple risk behaviors and psychosocial outcomes among women at risk for heart disease. Ann Behav Med. 2005; 29: 128-137.

    Article  PubMed Central  PubMed  Google Scholar 

  56. Multiple Risk Factor Intervention Trial Research Group. Multiple risk factor intervention trial. Risk factor changes and mortality results. JAMA. 1982; 248: 1465-1477.

    Article  Google Scholar 

  57. Multiple Risk Factor Intervention Trial Research Group. Coronary heart disease death, nonfatal acute myocardial infarction and other clinical outcomes in the Multiple Risk Factor Intervention Trial. Am J Cardiol. 1986; 58: 1-13.

    Article  Google Scholar 

  58. Benfari RC. The multiple risk factor intervention trial (MRFIT). III. The model for intervention. Prev Med. 1981; 10: 426-442.

    Article  CAS  PubMed  Google Scholar 

  59. Neaton JD, Broste S, Cohen L, et al. The multiple risk factor intervention trial (MRFIT). VII. A comparison of risk factor changes between the two study groups. Prev Med. 1981; 10: 519-543.

    Article  CAS  PubMed  Google Scholar 

  60. King AC, Haskell WL, Taylor CB, Kraemer HC, DeBusk RF. Group- vs home-based exercise training in healthy older men and women. A community-based clinical trial. JAMA. 1991; 266: 1535-1542.

    Article  CAS  PubMed  Google Scholar 

  61. McDermott MM, Ades P, Guralnik JM, et al. Treadmill exercise and resistance training in patients with peripheral arterial disease with and without intermittent claudication: A randomized controlled trial. JAMA. 2009; 301: 165-174.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  62. Appel LJ, Clark JM, Yeh HC, et al. Comparative effectiveness of weight-loss interventions in clinical practice. N Engl J Med. 2011; 365: 1959-1968.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  63. Wadden TA, Volger S, Sarwer DB, et al. A two-year randomized trial of obesity treatment in primary care practice. N Engl J Med. 2011; 365: 1969-1979.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  64. Appel LJ, Champagne CM, Harsha DW, et al. Effects of comprehensive lifestyle modification on blood pressure control: Main results of the PREMIER clinical trial. JAMA. 2003; 289: 2083-2093.

    PubMed  Google Scholar 

  65. Haskell WL, Alderman EL, Fair JM, et al. Effects of intensive multiple risk factor reduction on coronary atherosclerosis and clinical cardiac events in men and women with coronary artery disease. The Stanford Coronary Risk Intervention Project (SCRIP). Circulation. 1994; 89: 975-990.

    Article  CAS  PubMed  Google Scholar 

  66. Wood PD, Stefanick ML, Williams PT, Haskell WL. The effects on plasma lipoproteins of a prudent weight-reducing diet, with or without exercise, in overweight men and women. N Engl J Med. 1991; 325: 461-466.

    Article  CAS  PubMed  Google Scholar 

  67. Bosworth HB, Olsen MK, Grubber JM, et al. Two self-management interventions to improve hypertension control: A randomized trial. Ann Intern Med. 2009; 151: 687-695.

    Article  PubMed Central  PubMed  Google Scholar 

  68. Chaudhry SI, Mattera JA, Curtis JP, et al. Telemonitoring in patients with heart failure. N Engl J Med. 2010; 363: 2301-2309.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  69. Stevens VJ, Obarzanek E, Cook NR, et al. Long-term weight loss and changes in blood pressure: Results of the Trials of Hypertension Prevention, phase II. Ann Intern Med. 2001; 134: 1-11.

    Article  CAS  PubMed  Google Scholar 

  70. Perri MG, Limacher MC, Durning PE, et al. Extended-care programs for weight management in rural communities: The treatment of obesity in underserved rural settings (TOURS) randomized trial. Arch Intern Med. 2008; 168: 2347-2354.

    Article  PubMed Central  PubMed  Google Scholar 

  71. Kim JR, Oberman A, Fletcher GF, Lee JY. Effect of exercise intensity and frequency on lipid levels in men with coronary heart disease: Training Level Comparison Trial. Am J Cardiol. 2001; 87: 942-946. a943.

    Article  CAS  PubMed  Google Scholar 

  72. Howard BV, Van Horn L, Hsia J, et al. Low-fat dietary pattern and risk of cardiovascular disease: The Women's Health Initiative Randomized Controlled Dietary Modification Trial. JAMA. 2006; 295: 655-666.

    Article  CAS  PubMed  Google Scholar 

  73. Svetkey LP, Stevens VJ, Brantley PJ, et al. Comparison of strategies for sustaining weight loss: The weight loss maintenance randomized controlled trial. JAMA. 2008; 299: 1139-1148.

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Public Health and Human Sciences, School of Social & Behavioral Health Sciences, Oregon State University, 457 Waldo Hall, Corvallis, OR, 97331, USA

    Veronica L. Irvin PhD, MPH

  2. Agency for Healthcare Research and Quality, US Department of Health and Human Services, Rockville, MD, USA

    Robert M. Kaplan PhD

Authors
  1. Veronica L. Irvin PhD, MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert M. Kaplan PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Veronica L. Irvin PhD, MPH.

Ethics declarations

Manuscript submitted to Annals of Behavioral Medicine

Study authors: Veronica L. Irvin & Robert M. Kaplan

Study title: Effect Sizes and Primary Outcomes in Large Behavioral Randomized-Controlled Trials Funded by NIH Since 1980

Funding Source

This research was conducted while both authors were affiliated with the Intramural Research Program of the National Institutes of Health, Clinical Center, Department of Rehabilitation Medicine. Dr. Irvin was a post-doctoral fellow with the National Institutes of Health. The views expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the National Institutes of Health or the US government.

Conflict of Interest

Both authors were with National Institutes of Health at the time of data analysis—Dr. Kaplan as an employee and Dr. Irvin as a post-doctoral fellow. There was no official grant funded by NIH to conduct these analyses. Dr. Irvin is now Assistant Professor at Oregon State University and Dr. Kaplan is now Chief Scientific Officer at the Agency for Healthcare Research and Quality. There are no other conflicts of interests to report.

Ethical Adherence

This study was determined exempt from review by the National Institute of Health, Office of Human Subjects Research (NIH OHSRP no. 11648 PI: Kaplan).

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1

(DOC 88 kb)

ESM 2

(DOC 235 kb)

ESM 3

(DOC 260 kb)

ESM 4

(DOC 237 kb)

ESM 5

(DOC 229 kb)

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Irvin, V.L., Kaplan, R.M. Effect Sizes and Primary Outcomes in Large-Budget, Cardiovascular-Related Behavioral Randomized Controlled Trials Funded by NIH Since 1980. ann. behav. med. 50, 130–146 (2016). https://doi.org/10.1007/s12160-015-9739-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12160-015-9739-7

Keywords

Search

Navigation