Skip to main content

Advertisement

SpringerLink
Log in

Comparisons between protocols and publications of case-control studies: analysis of potential causes of non-reproducibility and recommendations for enhancing the quality of personalization in healthcare

  • Research
  • Published:
EPMA Journal Aims and scope Submit manuscript

Abstract

Background

Selective reporting of results in published case-control studies has been widely suspected, but little comprehensive information on selective reporting is available with regard to case-control studies. We aimed to evaluate the concordance of findings between publications and the protocols of case-control studies and to assess the level of selective reporting of results in case-control studies.

Methods

The databases of Embase, Medline, Scopus, and Web of Science were searched to identify case-control study protocols published between January 1, 1990 and December 31, 2017. The numbers and characteristics of predefined exposures (or factors) were extracted from the protocols. The reported and unreported factors were both collected from the published studies and protocols. The frequency of selective reporting of results were estimated by identifying the discrepancies of factors between the protocols and the published studies. Study sample size and the extent of selective reporting of factors were measured by a Spearman correlation analysis.

Results

Fourteen protocols with 24 published studies and 159 factors were identified, of which eight protocols (57.1%) had discrepancies between the publications and protocols. The prevalence of incomplete reporting in published case-control studies was 42.9% (6/14), with participant characteristics, anthropometric and laboratory measurement variables more likely to be unreported. A total of 16,835 cases and 56,049 controls were recruited in the 14 protocols of case-control studies (sample size ranges from 428 to 52,596 per study). Sample size had no statistical significance with selective reporting of results (P > 0.05).

Conclusion

The study protocols should be publicly available prior to the completion of case-control studies so that the potential bias can be assessed by the readers. Our findings highlight the need for investigators, peer reviewers, and readers to exercise increased awareness and scrutiny due to the undesirable practice of selective reporting of results in medical sciences causing the loss of potentially important information, thus impacting quality of personalized attitude in healthcare in the context of the predictive, preventive, and personalized medicine.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

CCS:

Case-control study

CONSORT:

The Consolidated Standards of Reporting Trials

FDA:

Food and Drug Administration

ICMJE:

The International Committee of Medical Journal Editors

NIH:

National Institutes of Health

NOS :

Newcastle-Ottawa Scale

PPPM:

Predictive, preventive, and personalized medicine

References

  1. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–65.

    Article  CAS  PubMed  Google Scholar 

  2. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3):252–60.

    Article  CAS  PubMed  Google Scholar 

  3. Al-Marzouki S, Roberts I, Evans S, Marshall T. Selective reporting in clinical trials: analysis of trial protocols accepted by The Lancet. Lancet. 2008;372(9634):201.

    Article  PubMed  Google Scholar 

  4. Zhang S, Liang F, Li W. Comparison between publicly accessible publications, registries, and protocols of phase III trials indicated persistence of selective outcome reporting. J Clin Epidemiol. 2017;91:87–94.

    Article  PubMed  Google Scholar 

  5. Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361:1963–71.

    Article  CAS  PubMed  Google Scholar 

  6. You B, Gan HK, Pond G, Chen EX. Consistency in the analysis and reporting of primary end points in oncology randomized controlled trials from registration to publication: a systematic review. J Clin Oncol. 2012;30(2):210–6.

    Article  PubMed  Google Scholar 

  7. DeAngelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. JAMA. 2004;292(11):1363–4.

    Article  CAS  PubMed  Google Scholar 

  8. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276(8):637–9.

    Article  CAS  PubMed  Google Scholar 

  9. Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, Platts-Mills TF. Non-publication of large randomized clinical trials: cross sectional analysis. BMJ. 2013;347:f6104.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The Clinical Trials.gov results database update and key issues. N Engl J Med. 2011;364:852–60.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Zarin DA, Tse T, Sheehan J. The proposed rule for U.S. clinical trial registration and results submission. N Engl J Med. 2015;372:174–80.

    Article  CAS  PubMed  Google Scholar 

  12. Weerasinghe M, Konradsen F, Eddleston M, Pearson M, Gunnell D, Hawton K, et al. Risk factors associated with purchasing pesticide from shops for self-poisoning: a protocol for a population-based case-control study. BMJ Open. 2015;5(5):e007822.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Estcourt LJ, Stanworth SJ, Collett D, Murphy MF. Intracranial haemorrhage in thrombocytopenic haematology patients--a nested case-control study: the InCiTe study protocol. BMJ Open. 2014;4(2):e004199.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Cramond F, Irvine C, Liao J, Howells D, Sena E, Currie G, et al. Protocol for a retrospective, controlled cohort study of the impact of a change in Nature journals’ editorial policy for life sciences research on the completeness of reporting study design and execution. Scientometrics. 2016;108:315–28.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Stosic M, Vukovic D, Babic D, Antonijevic G, Foley KL, Vujcic I, et al. Risk factors for multidrug-resistant tuberculosis among tuberculosis patients in Serbia: a case-control study. BMC Public Health. 2018;18(1):1114.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Soares AM, Szejnfeld VL, Enokihara MY, Michalany N, Castro CH. High serum 25-hydroxyvitamin D concentration in patients with a recent diagnosis of non-melanoma skin cancer: a case-control study. Eur J Dermatol. 2018;28(5):649–53.

    PubMed  Google Scholar 

  17. Canova C, Anello P, Barbiellini Amidei C, Parolin V, Zanier L, Simonato L. Use of healthcare services at the end of life in decedents compared to their surviving counterparts: a case-control study among adults born before 1946 in Friuli Venezia Giulia. PLoS One. 2019;14(2):e0212086.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Golubnitschaja O, Costigliola V, EPMA. General report & recommendations in predictive, preventive and personalised medicine 2012: white paper of the European Association for Predictive, Preventive and Personalised Medicine. EPMA J. 2012;3(1):14.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Lu M, Zhan X. The crucial role of multiomic approach in cancer research and clinically relevant outcomes. EPMA J. 2018;9(1):77–102.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Hou H, Zhao Y, Yu W, Dong H, Xue X, Ding J, et al. Association of obstructive sleep apnea with hypertension: a systematic review and meta-analysis. J Glob Health. 2018;8(1):010405.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Leone M, Bottacchi E, Beghi E, Morgando E, Mutani R, Amedeo G, et al. A case-control study on alcohol and seizures: study design, protocol, and data collection. Ital J Neurol Sci. 1997;18(2):119–24.

    Article  CAS  PubMed  Google Scholar 

  22. Heinemann LA, Garbe E, Lewis M, van der Woude F, Graf H. Case-control study on analgesics and nephropathy (SAN): protocol. BMC Nephrol. 2005;6:9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Boneberger A, Radon K, Baer J, Kausel L, Kabesch M, Haider D, et al. Asthma in changing environments--chances and challenges of international research collaborations between South America and Europe--study protocol and description of the data acquisition of a case-control-study. BMC Pulm Med. 2010;10:43.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Shariful Islam SM, Lechner A, Ferrari U, Froeschl G, Niessen LW, Seissler J, et al. Social and economic impact of diabetics in Bangladesh: protocol for a case-control study. BMC Public Health. 2013;13:1217.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Grigg MJ, William T, Drakeley CJ, Jelip J, von Seidlein L, Barber BE, et al. Factors that are associated with the risk of acquiring Plasmodium knowlesi malaria in Sabah, Malaysia: a case-control study protocol. BMJ Open. 2014;4(8):e006004.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Tovilla-Zárate CA, González-Castro TB, Juárez-Rojop I, Pool García S, Velázquez-Sánchez MP, Villar-Soto M. Study on genes of the serotonergic system and suicidal behavior: protocol for a case-control study in Mexican population. BMC Psychiatry. 2014;14:29.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Lewis DM, Assmann A, Heinemann L, Spitzer WO. Interim review of the transnational case-control study of oral contraceptives and health: approved protocol revisions through September 1995. Pharmacoepidemiol Drug Saf. 1996;5(1):43–51.

    Article  CAS  PubMed  Google Scholar 

  28. Wynn P, Stewart J, Kumar A, Clacy R, Coffey F, Cooper N, et al. Keeping children safe at home: protocol for a case-control study of modifiable risk factors for scalds. Inj Prev. 2014;20(5):e11.

    Article  CAS  PubMed  Google Scholar 

  29. Heinemann LA, Thomas DB, Möhner M. Multicentre international liver tumour study protocol of the case-control study on hepatocellular cancer. Pharmacoepidemiol Drug Saf. 1996;5(3):173–86.

    Article  CAS  PubMed  Google Scholar 

  30. Klussmann A, Gebhardt H, Liebers F, von Engelhardt LV, Dávid A, Bouillon B, et al. Individual and occupational risk factors for knee osteoarthritis - study protocol of a case control study. BMC Musculoskelet Disord. 2008;9:26.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Picot VS, Bénet T, Messaoudi M, Telles JN, Chou M, Eap T, et al. Multicenter case-control study protocol of pneumonia etiology in children: global approach to biological research, infectious diseases and epidemics in low-income countries (GABRIEL network). BMC Infect Dis. 2014;14:635.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Gomez-Marcos MA, Gonzalez-Sarmiento R, Recio-Rodríguez JI, Agudo-Conde C, Gamella-Pozuelo L, Perretta-Tejedor N, et al. Relationship between target organ damage and blood pressure, retinal vessel calibre, oxidative stress and polymorphisms in VAV-2 and VAV-3 genes in patients with hypertension: a case-control study protocol (LOD-Hipertension). BMJ Open. 2014;4(4):e005112.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Vinogradova Y, Coupland C, Hippisley-Cox J. Exposure to combined oral contraceptives and risk of venous thromboembolism: a protocol for nested case-control studies using the QResearch and the CPRD databases. BMJ Open. 2014;4(4):e004499.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Buckley CM, Ali F, Roberts G, Kearney PM, Perry IJ, Bradley CP. Timing of access to secondary healthcare services for diabetes management and lower extremity amputation in people with diabetes: a protocol of a case-control study. BMJ Open. 2013;3(10):e003871.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Summerskill W, Collingridge D, Frankish H. Protocols, probity, and publication. Lancet. 2009;373(9668):992.

    Article  PubMed  Google Scholar 

  36. Schulz KF, Altman DG, Moher D. Protocols, probity, and publication. Lancet. 2009;373(9674):1524.

    Article  PubMed  Google Scholar 

  37. Cheng T, Zhan X. Pattern recognition for predictive, preventive, and personalized medicine in cancer. EPMA J. 2017;8(1):51–60.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Chan AW, Krleza-Jerić K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ. 2004;171(7):735–40.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302(9):977–84.

    Article  CAS  PubMed  Google Scholar 

  40. Wayant C, Scheckel C, Hicks C, Nissen T, Leduc L, Som M, et al. Evidence of selective reporting bias in hematology journals: a systematic review. PLoS One. 2017;12(6):e0178379.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Calméjane L, Dechartres A, Tran VT, Ravaud P. Making protocols available with the article improved evaluation of selective outcome reporting. J Clin Epidemiol. 2018;104:95–102.

    Article  PubMed  Google Scholar 

  42. Swaen GMH, Urlings MJE, Zeegers MP. Outcome reporting bias in observational epidemiology studies on phthalates. Ann Epidemiol. 2016;26(8):597–9.

    Article  PubMed  Google Scholar 

  43. Golubnitschaja O, Baban B, Boniolo G, Wang W, Bubnov R, Kapalla M, et al. Medicine in the early twenty-first century: paradigm and anticipation - EPMA position paper 2016. EPMA J. 2016;7:23.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

This work was supported by the Natural Science Foundation of Shandong Province, China (No. ZR2017MH100). The sponsor did not participate in the study design, data collection, data analysis, or manuscript preparation.

Author information

Author notes

  1. Haifeng Hou, Guoyong Ding and Xuan Zhao contributed equally to this work.

Authors and Affiliations

  1. School of Public Health, Taishan Medical University, Taian, China

    Haifeng Hou, Guoyong Ding, Xuan Zhao, Zixiu Meng, Dong Li & Wei Wang

  2. School of Medical and Health Sciences, Edith Cowan University, Perth, WA, 6027, Australia

    Haifeng Hou, Zheng Guo, Yulu Zheng & Wei Wang

  3. School of Health Sciences, Winston-Salem State University, Winston-Salem, USA

    Jiangmin Xu

Authors
  1. Haifeng Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guoyong Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zixiu Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiangmin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zheng Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yulu Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Dong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Wei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HH and WW designed this study. GD, XZ, ZM, YZ, and ZG collected the data. HH, and DL performed the statistical analysis and interpretation of the data. JX provided statistical expertise. HH, WW, and JX wrote the manuscript. All authors have read and approved the manuscript.

Corresponding author

Correspondence to Wei Wang.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 29 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hou, H., Ding, G., Zhao, X. et al. Comparisons between protocols and publications of case-control studies: analysis of potential causes of non-reproducibility and recommendations for enhancing the quality of personalization in healthcare. EPMA Journal 10, 101–108 (2019). https://doi.org/10.1007/s13167-019-00165-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13167-019-00165-2

Keywords

Search

Navigation