EPMA Journal

, Volume 10, Issue 2, pp 101–108 | Cite as

Comparisons between protocols and publications of case-control studies: analysis of potential causes of non-reproducibility and recommendations for enhancing the quality of personalization in healthcare

  • Haifeng Hou
  • Guoyong Ding
  • Xuan Zhao
  • Zixiu Meng
  • Jiangmin Xu
  • Zheng Guo
  • Yulu Zheng
  • Dong Li
  • Wei WangEmail author



Selective reporting of results in published case-control studies has been widely suspected, but little comprehensive information on selective reporting is available with regard to case-control studies. We aimed to evaluate the concordance of findings between publications and the protocols of case-control studies and to assess the level of selective reporting of results in case-control studies.


The databases of Embase, Medline, Scopus, and Web of Science were searched to identify case-control study protocols published between January 1, 1990 and December 31, 2017. The numbers and characteristics of predefined exposures (or factors) were extracted from the protocols. The reported and unreported factors were both collected from the published studies and protocols. The frequency of selective reporting of results were estimated by identifying the discrepancies of factors between the protocols and the published studies. Study sample size and the extent of selective reporting of factors were measured by a Spearman correlation analysis.


Fourteen protocols with 24 published studies and 159 factors were identified, of which eight protocols (57.1%) had discrepancies between the publications and protocols. The prevalence of incomplete reporting in published case-control studies was 42.9% (6/14), with participant characteristics, anthropometric and laboratory measurement variables more likely to be unreported. A total of 16,835 cases and 56,049 controls were recruited in the 14 protocols of case-control studies (sample size ranges from 428 to 52,596 per study). Sample size had no statistical significance with selective reporting of results (P > 0.05).


The study protocols should be publicly available prior to the completion of case-control studies so that the potential bias can be assessed by the readers. Our findings highlight the need for investigators, peer reviewers, and readers to exercise increased awareness and scrutiny due to the undesirable practice of selective reporting of results in medical sciences causing the loss of potentially important information, thus impacting quality of personalized attitude in healthcare in the context of the predictive, preventive, and personalized medicine.


Case-control study Protocol Selective reporting of results Publication bias Predictive preventive personalized medicine Quality of healthcare 



Case-control study


The Consolidated Standards of Reporting Trials


Food and Drug Administration


The International Committee of Medical Journal Editors


National Institutes of Health


Newcastle-Ottawa Scale


Predictive, preventive, and personalized medicine


Author contributions

HH and WW designed this study. GD, XZ, ZM, YZ, and ZG collected the data. HH, and DL performed the statistical analysis and interpretation of the data. JX provided statistical expertise. HH, WW, and JX wrote the manuscript. All authors have read and approved the manuscript.

Funding information

This work was supported by the Natural Science Foundation of Shandong Province, China (No. ZR2017MH100). The sponsor did not participate in the study design, data collection, data analysis, or manuscript preparation.

Compliance with ethical standards

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Supplementary material

13167_2019_165_MOESM1_ESM.docx (30 kb)
ESM 1 (DOCX 29 kb)


  1. 1.
    Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–65.CrossRefGoogle Scholar
  2. 2.
    Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3):252–60.CrossRefGoogle Scholar
  3. 3.
    Al-Marzouki S, Roberts I, Evans S, Marshall T. Selective reporting in clinical trials: analysis of trial protocols accepted by The Lancet. Lancet. 2008;372(9634):201.CrossRefGoogle Scholar
  4. 4.
    Zhang S, Liang F, Li W. Comparison between publicly accessible publications, registries, and protocols of phase III trials indicated persistence of selective outcome reporting. J Clin Epidemiol. 2017;91:87–94.CrossRefGoogle Scholar
  5. 5.
    Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361:1963–71.CrossRefGoogle Scholar
  6. 6.
    You B, Gan HK, Pond G, Chen EX. Consistency in the analysis and reporting of primary end points in oncology randomized controlled trials from registration to publication: a systematic review. J Clin Oncol. 2012;30(2):210–6.CrossRefGoogle Scholar
  7. 7.
    DeAngelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. JAMA. 2004;292(11):1363–4.CrossRefGoogle Scholar
  8. 8.
    Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276(8):637–9.CrossRefGoogle Scholar
  9. 9.
    Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, Platts-Mills TF. Non-publication of large randomized clinical trials: cross sectional analysis. BMJ. 2013;347:f6104.CrossRefGoogle Scholar
  10. 10.
    Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The Clinical results database update and key issues. N Engl J Med. 2011;364:852–60.CrossRefGoogle Scholar
  11. 11.
    Zarin DA, Tse T, Sheehan J. The proposed rule for U.S. clinical trial registration and results submission. N Engl J Med. 2015;372:174–80.CrossRefGoogle Scholar
  12. 12.
    Weerasinghe M, Konradsen F, Eddleston M, Pearson M, Gunnell D, Hawton K, et al. Risk factors associated with purchasing pesticide from shops for self-poisoning: a protocol for a population-based case-control study. BMJ Open. 2015;5(5):e007822.CrossRefGoogle Scholar
  13. 13.
    Estcourt LJ, Stanworth SJ, Collett D, Murphy MF. Intracranial haemorrhage in thrombocytopenic haematology patients--a nested case-control study: the InCiTe study protocol. BMJ Open. 2014;4(2):e004199.CrossRefGoogle Scholar
  14. 14.
    Cramond F, Irvine C, Liao J, Howells D, Sena E, Currie G, et al. Protocol for a retrospective, controlled cohort study of the impact of a change in Nature journals’ editorial policy for life sciences research on the completeness of reporting study design and execution. Scientometrics. 2016;108:315–28.CrossRefGoogle Scholar
  15. 15.
    Stosic M, Vukovic D, Babic D, Antonijevic G, Foley KL, Vujcic I, et al. Risk factors for multidrug-resistant tuberculosis among tuberculosis patients in Serbia: a case-control study. BMC Public Health. 2018;18(1):1114.CrossRefGoogle Scholar
  16. 16.
    Soares AM, Szejnfeld VL, Enokihara MY, Michalany N, Castro CH. High serum 25-hydroxyvitamin D concentration in patients with a recent diagnosis of non-melanoma skin cancer: a case-control study. Eur J Dermatol. 2018;28(5):649–53.Google Scholar
  17. 17.
    Canova C, Anello P, Barbiellini Amidei C, Parolin V, Zanier L, Simonato L. Use of healthcare services at the end of life in decedents compared to their surviving counterparts: a case-control study among adults born before 1946 in Friuli Venezia Giulia. PLoS One. 2019;14(2):e0212086.CrossRefGoogle Scholar
  18. 18.
    Golubnitschaja O, Costigliola V, EPMA. General report & recommendations in predictive, preventive and personalised medicine 2012: white paper of the European Association for Predictive, Preventive and Personalised Medicine. EPMA J. 2012;3(1):14.CrossRefGoogle Scholar
  19. 19.
    Lu M, Zhan X. The crucial role of multiomic approach in cancer research and clinically relevant outcomes. EPMA J. 2018;9(1):77–102.CrossRefGoogle Scholar
  20. 20.
    Hou H, Zhao Y, Yu W, Dong H, Xue X, Ding J, et al. Association of obstructive sleep apnea with hypertension: a systematic review and meta-analysis. J Glob Health. 2018;8(1):010405.CrossRefGoogle Scholar
  21. 21.
    Leone M, Bottacchi E, Beghi E, Morgando E, Mutani R, Amedeo G, et al. A case-control study on alcohol and seizures: study design, protocol, and data collection. Ital J Neurol Sci. 1997;18(2):119–24.CrossRefGoogle Scholar
  22. 22.
    Heinemann LA, Garbe E, Lewis M, van der Woude F, Graf H. Case-control study on analgesics and nephropathy (SAN): protocol. BMC Nephrol. 2005;6:9.CrossRefGoogle Scholar
  23. 23.
    Boneberger A, Radon K, Baer J, Kausel L, Kabesch M, Haider D, et al. Asthma in changing environments--chances and challenges of international research collaborations between South America and Europe--study protocol and description of the data acquisition of a case-control-study. BMC Pulm Med. 2010;10:43.CrossRefGoogle Scholar
  24. 24.
    Shariful Islam SM, Lechner A, Ferrari U, Froeschl G, Niessen LW, Seissler J, et al. Social and economic impact of diabetics in Bangladesh: protocol for a case-control study. BMC Public Health. 2013;13:1217.CrossRefGoogle Scholar
  25. 25.
    Grigg MJ, William T, Drakeley CJ, Jelip J, von Seidlein L, Barber BE, et al. Factors that are associated with the risk of acquiring Plasmodium knowlesi malaria in Sabah, Malaysia: a case-control study protocol. BMJ Open. 2014;4(8):e006004.CrossRefGoogle Scholar
  26. 26.
    Tovilla-Zárate CA, González-Castro TB, Juárez-Rojop I, Pool García S, Velázquez-Sánchez MP, Villar-Soto M. Study on genes of the serotonergic system and suicidal behavior: protocol for a case-control study in Mexican population. BMC Psychiatry. 2014;14:29.CrossRefGoogle Scholar
  27. 27.
    Lewis DM, Assmann A, Heinemann L, Spitzer WO. Interim review of the transnational case-control study of oral contraceptives and health: approved protocol revisions through September 1995. Pharmacoepidemiol Drug Saf. 1996;5(1):43–51.CrossRefGoogle Scholar
  28. 28.
    Wynn P, Stewart J, Kumar A, Clacy R, Coffey F, Cooper N, et al. Keeping children safe at home: protocol for a case-control study of modifiable risk factors for scalds. Inj Prev. 2014;20(5):e11.CrossRefGoogle Scholar
  29. 29.
    Heinemann LA, Thomas DB, Möhner M. Multicentre international liver tumour study protocol of the case-control study on hepatocellular cancer. Pharmacoepidemiol Drug Saf. 1996;5(3):173–86.CrossRefGoogle Scholar
  30. 30.
    Klussmann A, Gebhardt H, Liebers F, von Engelhardt LV, Dávid A, Bouillon B, et al. Individual and occupational risk factors for knee osteoarthritis - study protocol of a case control study. BMC Musculoskelet Disord. 2008;9:26.CrossRefGoogle Scholar
  31. 31.
    Picot VS, Bénet T, Messaoudi M, Telles JN, Chou M, Eap T, et al. Multicenter case-control study protocol of pneumonia etiology in children: global approach to biological research, infectious diseases and epidemics in low-income countries (GABRIEL network). BMC Infect Dis. 2014;14:635.CrossRefGoogle Scholar
  32. 32.
    Gomez-Marcos MA, Gonzalez-Sarmiento R, Recio-Rodríguez JI, Agudo-Conde C, Gamella-Pozuelo L, Perretta-Tejedor N, et al. Relationship between target organ damage and blood pressure, retinal vessel calibre, oxidative stress and polymorphisms in VAV-2 and VAV-3 genes in patients with hypertension: a case-control study protocol (LOD-Hipertension). BMJ Open. 2014;4(4):e005112.CrossRefGoogle Scholar
  33. 33.
    Vinogradova Y, Coupland C, Hippisley-Cox J. Exposure to combined oral contraceptives and risk of venous thromboembolism: a protocol for nested case-control studies using the QResearch and the CPRD databases. BMJ Open. 2014;4(4):e004499.CrossRefGoogle Scholar
  34. 34.
    Buckley CM, Ali F, Roberts G, Kearney PM, Perry IJ, Bradley CP. Timing of access to secondary healthcare services for diabetes management and lower extremity amputation in people with diabetes: a protocol of a case-control study. BMJ Open. 2013;3(10):e003871.CrossRefGoogle Scholar
  35. 35.
    Summerskill W, Collingridge D, Frankish H. Protocols, probity, and publication. Lancet. 2009;373(9668):992.CrossRefGoogle Scholar
  36. 36.
    Schulz KF, Altman DG, Moher D. Protocols, probity, and publication. Lancet. 2009;373(9674):1524.CrossRefGoogle Scholar
  37. 37.
    Cheng T, Zhan X. Pattern recognition for predictive, preventive, and personalized medicine in cancer. EPMA J. 2017;8(1):51–60.CrossRefGoogle Scholar
  38. 38.
    Chan AW, Krleza-Jerić K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ. 2004;171(7):735–40.CrossRefGoogle Scholar
  39. 39.
    Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302(9):977–84.CrossRefGoogle Scholar
  40. 40.
    Wayant C, Scheckel C, Hicks C, Nissen T, Leduc L, Som M, et al. Evidence of selective reporting bias in hematology journals: a systematic review. PLoS One. 2017;12(6):e0178379.CrossRefGoogle Scholar
  41. 41.
    Calméjane L, Dechartres A, Tran VT, Ravaud P. Making protocols available with the article improved evaluation of selective outcome reporting. J Clin Epidemiol. 2018;104:95–102.CrossRefGoogle Scholar
  42. 42.
    Swaen GMH, Urlings MJE, Zeegers MP. Outcome reporting bias in observational epidemiology studies on phthalates. Ann Epidemiol. 2016;26(8):597–9.CrossRefGoogle Scholar
  43. 43.
    Golubnitschaja O, Baban B, Boniolo G, Wang W, Bubnov R, Kapalla M, et al. Medicine in the early twenty-first century: paradigm and anticipation - EPMA position paper 2016. EPMA J. 2016;7:23.CrossRefGoogle Scholar

Copyright information

© European Association for Predictive, Preventive and Personalised Medicine (EPMA) 2019

Authors and Affiliations

  • Haifeng Hou
    • 1
    • 2
  • Guoyong Ding
    • 1
  • Xuan Zhao
    • 1
  • Zixiu Meng
    • 1
  • Jiangmin Xu
    • 3
  • Zheng Guo
    • 2
  • Yulu Zheng
    • 2
  • Dong Li
    • 1
  • Wei Wang
    • 1
    • 2
    Email author
  1. 1.School of Public HealthTaishan Medical UniversityTaianChina
  2. 2.School of Medical and Health SciencesEdith Cowan UniversityPerthAustralia
  3. 3.School of Health SciencesWinston-Salem State UniversityWinston-SalemUSA

Personalised recommendations