Annals of Behavioral Medicine

, Volume 50, Issue 1, pp 130–146 | Cite as

Effect Sizes and Primary Outcomes in Large-Budget, Cardiovascular-Related Behavioral Randomized Controlled Trials Funded by NIH Since 1980

  • Veronica L. IrvinEmail author
  • Robert M. Kaplan
Original Article



We reviewed large-budget, National Institutes of Health (NIH)-supported randomized controlled trials (RCTs) with behavioral interventions to assess (1) publication rates, (2) trial registration, (3) use of objective measures, (4) significant behavior and physiological change, and (5) effect sizes.


We identified large-budget grants (>$500,000/year) funded by NIH (National Heart Lung and Blood Institute (NHLBI) or National Institute of Diabetes & Digestive and Kidney Diseases (NIDDK)) for cardiovascular disease (dates January 1, 1980 to December 31, 2012). Among 106 grants that potentially met inclusion criteria, 20 studies were not published and 48 publications were excluded, leaving 38 publications for analysis. abstracts were used to determine whether outcome measures had been pre-specified.


Three fourths of trials were registered in and all published pre-specified outcomes. Twenty-six trials reported a behavioral outcome with 81 % reporting significant improvements for the target behavior. Thirty-two trials reported a physiological outcome. All were objectively measured, and 81 % reported significant benefit. Seventeen trials reported morbidity outcomes, and seven reported a significant benefit. Nine trials assessed mortality, and all were null for this outcome.


Behavioral trials complied with trial registration standards. Most reported a physiological benefit, but few documented morbidity or mortality benefits.


Behavior Intervention Cardiovascular Trial registration Publication statistics Effect sizes Periodicals as topics/statistics  Randomized controlled trials as topics/statistics 


Conflict of Interest and Ethical Adherence

Manuscript submitted to Annals of Behavioral Medicine

Study authors: Veronica L. Irvin & Robert M. Kaplan

Study title: Effect Sizes and Primary Outcomes in Large Behavioral Randomized-Controlled Trials Funded by NIH Since 1980

Funding Source

This research was conducted while both authors were affiliated with the Intramural Research Program of the National Institutes of Health, Clinical Center, Department of Rehabilitation Medicine. Dr. Irvin was a post-doctoral fellow with the National Institutes of Health. The views expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the National Institutes of Health or the US government.

Conflict of Interest

Both authors were with National Institutes of Health at the time of data analysis—Dr. Kaplan as an employee and Dr. Irvin as a post-doctoral fellow. There was no official grant funded by NIH to conduct these analyses. Dr. Irvin is now Assistant Professor at Oregon State University and Dr. Kaplan is now Chief Scientific Officer at the Agency for Healthcare Research and Quality. There are no other conflicts of interests to report.

Ethical Adherence

This study was determined exempt from review by the National Institute of Health, Office of Human Subjects Research (NIH OHSRP no. 11648 PI: Kaplan).

Supplementary material

12160_2015_9739_MOESM1_ESM.doc (88 kb)
ESM 1 (DOC 88 kb)
12160_2015_9739_MOESM2_ESM.doc (235 kb)
ESM 2 (DOC 235 kb)
12160_2015_9739_MOESM3_ESM.doc (260 kb)
ESM 3 (DOC 260 kb)
12160_2015_9739_MOESM4_ESM.doc (237 kb)
ESM 4 (DOC 237 kb)
12160_2015_9739_MOESM5_ESM.doc (230 kb)
ESM 5 (DOC 229 kb)


  1. 1.
    Fisher EB, Fitzgibbon ML, Glasgow RE, et al. Behavior matters. Am J Prev Med. 2011; 40: e15-30.PubMedCentralCrossRefPubMedGoogle Scholar
  2. 2.
    Neiberg RH, Wing RR, Bray GA, et al. Patterns of weight change associated with long-term weight change and cardiovascular disease risk factors in the Look AHEAD Study. Obesity (Silver Spring). 2012; 20: 2048-2056.PubMedCentralCrossRefPubMedGoogle Scholar
  3. 3.
    Look ARG, Wing RR, Bolin P, et al. Cardiovascular effects of intensive lifestyle intervention in type 2 diabetes. N Engl J Med. 2013; 369: 145-154.CrossRefGoogle Scholar
  4. 4.
    World Health Organization. Non-communicable diseases. Available at Accessibility verified July 3, 2015.
  5. 5.
    Goff DC Jr, Lloyd-Jones DM, Bennett G, et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014; 129(25 Suppl 2): S49-73.CrossRefPubMedGoogle Scholar
  6. 6.
    Stone NJ, Robinson J, Lichtenstein AH, et al. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014; 129(25 Suppl 2): S1-45.CrossRefPubMedGoogle Scholar
  7. 7.
    Phillips SM, Glasgov RE, Bello G, et al. Frequency and prioritization of patient health risks from a structured health risk assessment. Ann Fam Med. 2014; 12: 505-513.PubMedCentralCrossRefPubMedGoogle Scholar
  8. 8.
    Kaplan RM, Morgan G. Providers advice concerning smoking cessation: Evidence from the medical expenditures panel survey. San Antonio: Presented at the annual meeting of the Society of Behavioral Medicine; 2015.Google Scholar
  9. 9.
    Kaplan RM, Irvin VL. Likelihood of null effects of large NHLBI clinical trials has increased over time. PLOS One. 2015; 10(8): e0132382.PubMedCentralCrossRefPubMedGoogle Scholar
  10. 10.
    Goldacre B. Preventing bad reporting on health research. BMJ. 2014; 349: g7465.CrossRefPubMedGoogle Scholar
  11. 11.
    Sumner P, Vivian-Griffiths S, Boivin J, et al. The association between exaggeration in health related science news and academic press releases: retrospective observational study. BMJ. 2014; 349: g7015.PubMedCentralCrossRefPubMedGoogle Scholar
  12. 12.
    Pharmaceutical Research and Manufacturers of American. PhRMA 2012 Industry Profile. Available at Accessibility verified July 3, 2015.
  13. 13.
    Gordon D, Taddei-Peters W, Mascette A, et al. Publication of trials funded by the National Heart, Lung, and Blood Institute. N Engl J Med. 2013; 369: 1926-1934.PubMedCentralCrossRefPubMedGoogle Scholar
  14. 14.
    Easterbrook PJ, Gopalan R, Berlin J, Matthews DR. Publication bias in clinical research. The Lancet. 1991; 337: 867-872.CrossRefGoogle Scholar
  15. 15.
    Knowler WC, Barrett-Connor E, Fowler SE, et al. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med. 2002; 346: 393-403.CrossRefPubMedGoogle Scholar
  16. 16.
    Cambell Collaboration. David B. Wilson. Practical meta-analysis effect size calculator. Available at . Accessibility verified July 6, 2015.
  17. 17.
    Cohen J. A power primer. Psychol Bull. 1992; 112: 155-159.CrossRefPubMedGoogle Scholar
  18. 18.
    Zarin DA, Tse T. Moving toward transparency of clinical trials. Science. 2008; 319: 1340-1342.PubMedCentralCrossRefPubMedGoogle Scholar
  19. 19.
    Lovshin JA, Drucker DJ. Metabolic disease puts up a fight: Are diet and exercise helpful for the heart? Nat Med. 2013; 19: 1216-1217.CrossRefPubMedGoogle Scholar
  20. 20.
    Lin JS, O'Connor E, Evans CV, et al. Behavioral counseling to promote a healthy lifestyle in persons with cardiovascular risk factors: A systematic review for the U.S. Preventive Services Task Force. Ann Intern Med. 2014; 161(8): 568-578.CrossRefPubMedGoogle Scholar
  21. 21.
    National Cancer Institute. Dietary assessment primer. Available at Accessibility verified July 3, 2015.
  22. 22.
    Writing Group for the Activity Counseling Trial Research G. Effects of physical activity counseling in primary care: The Activity Counseling Trial: A randomized controlled trial. JAMA. 2001; 286: 677-687.CrossRefGoogle Scholar
  23. 23.
    Harvey-Berino J, West D, Krukowski R, et al. Internet delivered behavioral obesity treatment. Prev Med. 2010; 51: 123-128.PubMedCentralCrossRefPubMedGoogle Scholar
  24. 24.
    No authors. The lipid research clinics coronary primary prevention trial results. 1. Reduction in incidence of coronary heart-disease. JAMA. 1984; 251:351-364.Google Scholar
  25. 25.
    No authors. Hennekens CH: Preliminary report: Findings from the aspirin component of the ongoing physicians’ health study. N Engl J Med. 1988; 318:262-264.Google Scholar
  26. 26.
    Nigg CR, Long CR. A systematic review of single health behavior change interventions vs multiple health behavior change interventions among older adults. TBM. 2012; 2: 166-179.Google Scholar
  27. 27.
    Milette K, Roseman M, Thombs BD. Transparency of outcome reporting and trial registration of randomized controlled trials in top psychosomatic and behavioral health journals: A systematic review. J Psychosom Res. 2011; 70: 205-217.CrossRefPubMedGoogle Scholar
  28. 28.
    National Institutes of Health. HHS and NIH take steps to enhance transparency of clinical trial results. Available at Accessibility verified July 6, 2015.
  29. 29.
    Linden W, Satin JR. Avoidable pitfalls in behavioral medicine outcome research. Ann Behav Med. 2007; 33: 143-147.CrossRefPubMedGoogle Scholar
  30. 30.
    Schneider S, Moyer A, Knapp-Oliver S, et al. Pre-intervention distress moderates the efficacy of psychosocial treatment for cancer patients: A meta-analysis. J Behav Med. 2010; 33: 1-14.PubMedCentralCrossRefPubMedGoogle Scholar
  31. 31.
    McDermott MM, Reed G, Greenland P, et al. Activating peripheral arterial disease patients to reduce cholesterol: A randomized trial. Am J Med. 2011; 124: 557-565.PubMedCentralCrossRefPubMedGoogle Scholar
  32. 32.
    Bennett GG, Warner ET, Glasgow RE, et al. Obesity treatment for socioeconomically disadvantaged patients in primary care practice. Arch Intern Med. 2012; 172: 565-574.PubMedCentralCrossRefPubMedGoogle Scholar
  33. 33.
    Johnson W, Shaya FT, Khanna N, et al. The Baltimore Partnership to Educate and Achieve Control of Hypertension (The BPTEACH Trial): A randomized trial of the effect of education on improving blood pressure control in a largely African American population. J Clin Hypertens (Greenwich). 2011; 13: 563-570.CrossRefGoogle Scholar
  34. 34.
    Stefanick ML, Mackey S, Sheehan M, et al. Effects of diet and exercise in men and postmenopausal women with low levels of HDL cholesterol and high levels of LDL cholesterol. N Engl J Med. 1998; 339: 12-20.CrossRefPubMedGoogle Scholar
  35. 35.
    Langford HG, Blaufox MD, Oberman A, et al. Dietary therapy slows the return of hypertension after stopping prolonged medication. JAMA. 1985; 253: 657-664.CrossRefPubMedGoogle Scholar
  36. 36.
    Delahanty LM, Pan Q, Jablonski KA, et al. Genetic predictors of weight loss and weight regain after intensive lifestyle modification, metformin treatment, or standard care in the Diabetes Prevention Program. Diabetes Care. 2012; 35: 363-366.PubMedCentralCrossRefPubMedGoogle Scholar
  37. 37.
    Berkman LF, Blumenthal J, Burg M, et al. Effects of treating depression and low perceived social support on clinical events after myocardial infarction: The Enhancing Recovery in Coronary Heart Disease Patients (ENRICHD) Randomized Trial. JAMA. 2003; 289: 3106-3116.CrossRefPubMedGoogle Scholar
  38. 38.
    Martin MY, Kim YI, Kratt P, et al. Medication adherence among rural, low-income hypertensive adults: A randomized trial of a multimedia community-based intervention. Am J Health Promot. 2011; 25: 372-378.PubMedCentralCrossRefPubMedGoogle Scholar
  39. 39.
    Powell LH, Calvin JE Jr, Richardson D, et al. Self-management counseling in patients with heart failure: The heart failure adherence and retention randomized behavioral trial. JAMA. 2010; 304: 1331-1338.PubMedCentralCrossRefPubMedGoogle Scholar
  40. 40.
    Stamler R, Stamler J, Grimm R, et al. Nutritional therapy for high blood pressure. Final report of a four-year randomized controlled trial—The Hypertension Control Program. JAMA. 1987; 257: 1484-1491.CrossRefPubMedGoogle Scholar
  41. 41.
    DeWalt DA, Schillinger D, Ruo B, et al. Multisite randomized trial of a single-session versus multisession literacy-sensitive self-care intervention for patients with heart failure. Circulation. 2012; 125: 2854-2862.PubMedCentralCrossRefPubMedGoogle Scholar
  42. 42.
    Baker DW, Dewalt DA, Schillinger D, et al. The effect of progressive, reinforcing telephone education and counseling versus brief educational intervention on knowledge, self-care behaviors, and heart failure symptoms. Journal of Cardiac Failure. 2011; 17(1): 789-796.PubMedCentralCrossRefPubMedGoogle Scholar
  43. 43.
    Katula JA, Vitolins MZ, Morgan TM, et al. The Healthy Living Partnerships to Prevent Diabetes study: 2-year outcomes of a randomized controlled trial. Am J Prev Med. 2013; 44: S324-332.PubMedCentralCrossRefPubMedGoogle Scholar
  44. 44.
    O'Connor CM, Whellan DJ, Lee KL, et al. Efficacy and safety of exercise training in patients with chronic heart failure: HF-ACTION randomized controlled trial. JAMA. 2009; 301: 1439-1450.PubMedCentralCrossRefPubMedGoogle Scholar
  45. 45.
    Pezzin LE, Feldman PH, Mongoven JM, et al. Improving blood pressure control: Results of home-based post-acute care interventions. J Gen Intern Med. 2011; 26: 280-286.PubMedCentralCrossRefPubMedGoogle Scholar
  46. 46.
    McBride CM, Curry SJ, Lando HA, et al. Prevention of relapse in women who quit smoking during pregnancy. Am J Public Health. 1999; 89: 706-711.PubMedCentralCrossRefPubMedGoogle Scholar
  47. 47.
    Hypertension Prevention Trial Research Group. The Hypertension Prevention Trial: Three-year effects of dietary changes on blood pressure. Arch Intern Med. 1990; 150: 153-162.CrossRefGoogle Scholar
  48. 48.
    Stamler R, Stamler J, Gosch FC, et al. Primary prevention of hypertension by nutritional-hygienic means. Final report of a randomized, controlled trial. JAMA. 1989; 262: 1801-1807.CrossRefPubMedGoogle Scholar
  49. 49.
    Wolf AM, Conaway MR, Crowther JQ, et al. Translating lifestyle intervention to practice in obese patients with type 2 diabetes: Improving Control with Activity and Nutrition (ICAN) study. Diabetes Care. 2004; 27: 1570-1576.CrossRefPubMedGoogle Scholar
  50. 50.
    Krieger J, Collier C, Song L, Martin D. Linking community-based blood pressure measurement to clinical care: A randomized controlled trial of outreach and tracking by community health workers. Am J Public Health. 1999; 89: 856-861.PubMedCentralCrossRefPubMedGoogle Scholar
  51. 51.
    Pi-Sunyer X, Blackburn G, Brancati FL, et al. Reduction in weight and cardiovascular disease risk factors in individuals with type 2 diabetes: One-year results of the LOOK AHEAD trial. Diabetes Care. 2007; 30: 1374-1383.CrossRefPubMedGoogle Scholar
  52. 52.
    Eight-year weight losses with an intensive lifestyle intervention: The LOOK AHEAD study. Obesity (Silver Spring). 2014; 22:5-13.Google Scholar
  53. 53.
    Toobert DJ, Glasgow RE, Strycker LA, et al. Biologic and quality-of-life outcomes from the Mediterranean Lifestyle Program: A randomized clinical trial. Diabetes Care. 2003; 26: 2288-2293.CrossRefPubMedGoogle Scholar
  54. 54.
    Toobert DJ, Glasgow RE, Strycker LA, et al. Long-term effects of the Mediterranean lifestyle program: A randomized clinical trial for postmenopausal women with type 2 diabetes. Int J Behav Nutr Phys Act. 2007; 4: 1.PubMedCentralCrossRefPubMedGoogle Scholar
  55. 55.
    Toobert DJ, Strycker LA, Glasgow RE, Barrera M Jr, Angell K. Effects of the mediterranean lifestyle program on multiple risk behaviors and psychosocial outcomes among women at risk for heart disease. Ann Behav Med. 2005; 29: 128-137.PubMedCentralCrossRefPubMedGoogle Scholar
  56. 56.
    Multiple Risk Factor Intervention Trial Research Group. Multiple risk factor intervention trial. Risk factor changes and mortality results. JAMA. 1982; 248: 1465-1477.CrossRefGoogle Scholar
  57. 57.
    Multiple Risk Factor Intervention Trial Research Group. Coronary heart disease death, nonfatal acute myocardial infarction and other clinical outcomes in the Multiple Risk Factor Intervention Trial. Am J Cardiol. 1986; 58: 1-13.CrossRefGoogle Scholar
  58. 58.
    Benfari RC. The multiple risk factor intervention trial (MRFIT). III. The model for intervention. Prev Med. 1981; 10: 426-442.CrossRefPubMedGoogle Scholar
  59. 59.
    Neaton JD, Broste S, Cohen L, et al. The multiple risk factor intervention trial (MRFIT). VII. A comparison of risk factor changes between the two study groups. Prev Med. 1981; 10: 519-543.CrossRefPubMedGoogle Scholar
  60. 60.
    King AC, Haskell WL, Taylor CB, Kraemer HC, DeBusk RF. Group- vs home-based exercise training in healthy older men and women. A community-based clinical trial. JAMA. 1991; 266: 1535-1542.CrossRefPubMedGoogle Scholar
  61. 61.
    McDermott MM, Ades P, Guralnik JM, et al. Treadmill exercise and resistance training in patients with peripheral arterial disease with and without intermittent claudication: A randomized controlled trial. JAMA. 2009; 301: 165-174.PubMedCentralCrossRefPubMedGoogle Scholar
  62. 62.
    Appel LJ, Clark JM, Yeh HC, et al. Comparative effectiveness of weight-loss interventions in clinical practice. N Engl J Med. 2011; 365: 1959-1968.PubMedCentralCrossRefPubMedGoogle Scholar
  63. 63.
    Wadden TA, Volger S, Sarwer DB, et al. A two-year randomized trial of obesity treatment in primary care practice. N Engl J Med. 2011; 365: 1969-1979.PubMedCentralCrossRefPubMedGoogle Scholar
  64. 64.
    Appel LJ, Champagne CM, Harsha DW, et al. Effects of comprehensive lifestyle modification on blood pressure control: Main results of the PREMIER clinical trial. JAMA. 2003; 289: 2083-2093.PubMedGoogle Scholar
  65. 65.
    Haskell WL, Alderman EL, Fair JM, et al. Effects of intensive multiple risk factor reduction on coronary atherosclerosis and clinical cardiac events in men and women with coronary artery disease. The Stanford Coronary Risk Intervention Project (SCRIP). Circulation. 1994; 89: 975-990.CrossRefPubMedGoogle Scholar
  66. 66.
    Wood PD, Stefanick ML, Williams PT, Haskell WL. The effects on plasma lipoproteins of a prudent weight-reducing diet, with or without exercise, in overweight men and women. N Engl J Med. 1991; 325: 461-466.CrossRefPubMedGoogle Scholar
  67. 67.
    Bosworth HB, Olsen MK, Grubber JM, et al. Two self-management interventions to improve hypertension control: A randomized trial. Ann Intern Med. 2009; 151: 687-695.PubMedCentralCrossRefPubMedGoogle Scholar
  68. 68.
    Chaudhry SI, Mattera JA, Curtis JP, et al. Telemonitoring in patients with heart failure. N Engl J Med. 2010; 363: 2301-2309.PubMedCentralCrossRefPubMedGoogle Scholar
  69. 69.
    Stevens VJ, Obarzanek E, Cook NR, et al. Long-term weight loss and changes in blood pressure: Results of the Trials of Hypertension Prevention, phase II. Ann Intern Med. 2001; 134: 1-11.CrossRefPubMedGoogle Scholar
  70. 70.
    Perri MG, Limacher MC, Durning PE, et al. Extended-care programs for weight management in rural communities: The treatment of obesity in underserved rural settings (TOURS) randomized trial. Arch Intern Med. 2008; 168: 2347-2354.PubMedCentralCrossRefPubMedGoogle Scholar
  71. 71.
    Kim JR, Oberman A, Fletcher GF, Lee JY. Effect of exercise intensity and frequency on lipid levels in men with coronary heart disease: Training Level Comparison Trial. Am J Cardiol. 2001; 87: 942-946. a943.CrossRefPubMedGoogle Scholar
  72. 72.
    Howard BV, Van Horn L, Hsia J, et al. Low-fat dietary pattern and risk of cardiovascular disease: The Women's Health Initiative Randomized Controlled Dietary Modification Trial. JAMA. 2006; 295: 655-666.CrossRefPubMedGoogle Scholar
  73. 73.
    Svetkey LP, Stevens VJ, Brantley PJ, et al. Comparison of strategies for sustaining weight loss: The weight loss maintenance randomized controlled trial. JAMA. 2008; 299: 1139-1148.CrossRefPubMedGoogle Scholar

Copyright information

© The Society of Behavioral Medicine 2015

Authors and Affiliations

  1. 1.College of Public Health and Human Sciences, School of Social & Behavioral Health SciencesOregon State UniversityCorvallisUSA
  2. 2.Agency for Healthcare Research and QualityUS Department of Health and Human ServicesRockvilleUSA

Personalised recommendations