Clarifying the distinction between case series and cohort studies in systematic reviews of comparative studies: potential impact on body of evidence and workload
- 18k Downloads
Distinguishing cohort studies from case series is difficult.
We propose a conceptualization of cohort studies in systematic reviews of comparative studies. The main aim of this conceptualization is to clarify the distinction between cohort studies and case series. We discuss the potential impact of the proposed conceptualization on the body of evidence and workload.
All studies with exposure-based sampling gather multiple exposures (with at least two different exposures or levels of exposure) and enable calculation of relative risks that should be considered cohort studies in systematic reviews, including non-randomized studies. The term “enables/can” means that a predefined analytic comparison is not a prerequisite (i.e., the absolute risks per group and/or a risk ratio are provided). Instead, all studies for which sufficient data are available for reanalysis to compare different exposures (e.g., sufficient data in the publication) are classified as cohort studies.
There are possibly large numbers of studies without a comparison for the exposure of interest but that do provide the necessary data to calculate effect measures for a comparison. Consequently, more studies could be included in a systematic review. Therefore, on the one hand, the outlined approach can increase the confidence in effect estimates and the strengths of conclusions. On the other hand, the workload would increase (e.g., additional data extraction and risk of bias assessment, as well as reanalyses).
Example definitions of cohort studies and case series 
A study in which a defined group of people (the cohort) is followed over time, to examine associations between different interventions received and subsequent outcomes.
Observations are made on a series of individuals, usually all receiving the same intervention, before and after an intervention but with no control group.
Possible presentation of a study with a preexisting exposure based comparison (cohort study not requiring a reanalysis)
Absolute risk for outcome (e.g. mortality rate) in all exposed (e.g. treated) patients
Absolute risk for outcome (e.g. mortality rate) in all unexposed (e.g. not treated) patients
Risk ratio with 95% confidence interval (e.g. relative risk of mortality for treated vs. untreated)
Possible presentation of study without a preexisting exposure based comparison (cohort study requiring a reanalysis)
Yes (e.g. treated)
Yes (e.g. died)
No (e.g. not treated)
No (e.g. alive)
Yes (e.g. treated)
Yes (e.g. alive)
These examples illustrate that distinguishing between cohort studies and case series is difficult. Vague definitions are probably the reason for the common confusion between study designs. A recent study found that approximately 72% of cohort studies are mislabelled as case series . Many systematic reviews of non-randomized studies included cohort studies but excluded case series (see examples in [23, 24, 25, 26, 27, 28]). Therefore, the unclear distinction between case series and cohort studies can result in inconsistent study selection and unjustified exclusions from a systematic review. The risk of misclassification is particularly high because study authors also often mislabel their study or studies are not classified by their authors at all (see examples in [6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21]).
We propose a conceptualization of cohort studies in systematic reviews of comparative studies. The main objective of this conceptualization is to clarify the distinction between cohort studies and case series in systematic reviews, including non-randomized comparative studies. We discuss the potential impact of the proposed conceptualization on the body of evidence and workload.
Clarifying the distinction between case series and cohort studies (the solution)
In the following report, we propose a conceptualization for cohort studies and case series (e.g., sampling) for systematic reviews, including comparative non-randomized studies. Our proposal is based on a recent conceptualization of cohort studies and case series by Dekkers et al. . The main feature of this conceptualization is that it is exclusively based on inherent design features and is not affected by the analysis.
Cohort studies of one exposure/one group
Summary of the distinction proposed by Dekkers et al. 
Cohort study: Patients are sampled on the basis of exposure. The occurrence of outcomes is assessed during a specified follow-up period.
Case series: Patients with a particular disease or disease-related outcome are sampled. Case series exist in 2 types:
1. Sampling is based on a specific outcome and presence of a specific exposure.
2. Selection is based only on a specific outcome, and data are collected on previous exposures. Cases are reported regardless of whether they have specific exposures. This type of case series can be seen as the case group from a case–control study .
Cohort studies of multiple exposures/more than one group
In short, all studies that enable calculation of a relative risk to quantify a difference in outcomes between different groups should be considered cohort studies.
In short, for a case series, there are no absolute risks, and also, no risk ratios can be calculated. Consequently, a case series cannot be comparative. The definition of a case series by Dekkers et al.  is summarized in Table 4.
It is noteworthy that the conceptualization also ensures a clear distinction of case series from other study designs that apply outcome-based sampling. Case series, case-control studies (including case-time-control), and self-controlled case-control designs (e.g., case-crossover) all have outcome-based sampling in common .
Case series have no control at all because only patients with a certain manifestation of outcomes are sampled (e.g., individuals with a disease or deceased individuals). In contrast, all case-control designs as well as self-controlled case-control designs have a control group. In case-control studies, the control group constitutes individuals with another manifestation of the outcome (e.g., healthy individuals or survivors). This outcome can be considered as two case series (i.e., case group and no case group).
Self-controlled case-control studies are characterized by an intra-individual comparison (each individual is their own control) . Information is also sampled when patients are not exposed. Therefore, case-control designs as well as self-controlled case-control studies enable the calculation of risk ratios. This approach is not possible for a case series.
Above, we illustrated that by using a vague definition, the classification of a study design might be influenced by the preparation and analysis of the study data. The proposed conceptualization is exclusively based on the inherent design features (e.g., sampling, exposure). After considering the example studies again using the proposed conceptualization, all studies would be classified as cohort studies because the relative risk can be calculated. This outcome becomes clear looking at Table 2 and Table 3. If the patients in Table 3 are rearranged according the exposure and the data are reanalysed (i.e., calculation of absolute risk per group and relative risks to compare groups), Table 3 can be converted into Table 2 (and also, Fig. 2 can be converted to Fig. 3). In the study by Wong et al. , the mean blood loss in the group with placental invasion and in the group without placental invasion can be calculated and compared (e.g., relative risk with 95% confidence limits). In this study, the data on gestational age are also provided in the table. Therefore, it is even possible to adjust the results for gestational age (e.g., using a logistic regression).
Discussion (the impact)
Influence on the body of evidence
The proposed conceptualization is exclusively based on inherent study design features; therefore, there is less room for misinterpretation compared to existing conceptualizations because analysis features, presentation of data and labelling of the study are not determined. Thus, the conceptualization ensures consistent study selection for systematic reviews.
The prerequisite of an analytical comparison in the publication can lead to the unjustified exclusion of relevant studies from a systematic review. Study 1 would likely be included, and Study 2 would be excluded from the systematic review. The only differences between Study 1 and Study 2 are the analysis and preparation of data. If the data source (e.g., chart review) and the reanalysis (calculation of effect measures and statistical tests) to compare the intervention and control group in Study 2 are performed exactly with the same approach as the existing analysis in Study 1, there can be no difference in the effect estimates between studies, and the studies are at the same risk of bias. Thus, the inclusion of Study 1 and the exclusion of Study 2 are contradictory to the requirement that systematic reviews identify all available evidence .
Considering that more studies would be eligible for inclusion and that the hierarchical paradigm of the levels of evidence is not valid per se, the proposed conceptualization can potentially enrich bodies of evidence and increase confidence in effect estimates.
Influence on workload
The additional inclusion of all studies that enable calculating relative risk for the comparison of interest might impact the workload of systematic reviews. There might be a considerable number of studies not performing a comparison already but that provide sufficient data for reanalysis. Usually the electronic search strategy for systematic reviews of non-randomized studies is not limited to certain study types because there are no sensitive search filters available yet . Therefore, the search results do not usually include cohort studies as discussed above. However, in many abstracts it would be not directly clear if sufficient data for re-calculations are reported in the full text article (e.g., a table like Table 3). Consequently, many additional potentially relevant full-text studies have to be screened. Additionally, studies often assess various exposures (e.g., different baseline characteristics), and it might thus be difficult to identify relevant exposures. Considering the large amount of wrongly labelled studies, this approach can lead to additional screening effort .
As a result, more studies would be included in systematic reviews. All articles that provide potentially relevant data would have to be assessed in detail to decide whether reanalysis is feasible. For these data extractions, a risk of bias assessment would have to be performed. Challenges in the risk of bias assessment would arise because most assessment tools are constructed to assess a predefined control group . For example, items regarding the adequacy of analysis (e.g., adjustment for confounders) cannot be assessed anymore. Effect measures must be calculated (e.g., risks by group and relative risk with a 95% confidence limit), and eventually further analyses (e.g., adjustments for confounders) might be necessary for studies that provide sufficient data. Moreover, advanced biometrical expertise would be necessary to judge the feasibility (i.e., determining the possibility to calculate relative risks and whether there are sufficient data to adjust for confounders) of a re-analysis and to conduct the reanalysis.
Promising areas of application
In the medical literature, it is likely that more retrospective mislabelled cohort studies (comparison planned after data collection) based on routinely collected data (e.g., chart review, review of radiology databases) than prospectively planned (i.e., comparisons planned before data collection) and wrongly labelled cohort studies can be found. Thus, it can be assumed that the wrongly labelled studies tend to have lower methodological quality than studies that already include a comparison. This aspect should be considered in decisions about including studies that must be reanalysed. In research areas in which randomized controlled trials or large planned prospective and well-conducted cohort studies can be expected (e.g., risk factors for widespread diseases), the approach is less promising for enriching the body of evidence. Consequently, in these areas, the additional effort might not be worthwhile.
Again, the conceptualization is particularly promising in research areas in which evidence is sparse because studies are difficult to conduct or populations are small or the event rates are low. These areas include rare diseases, adverse events/complications, sensitive groups (e.g., children or individuals with cognitive deficiencies) or rarely used interventions (e.g., costly innovations). In these areas, there might be no well-conducted studies at all [34, 35]. Therefore, the proposed conceptualization in this report has great potential to increase confidence in effect estimates.
We proposed a conceptualization for cohort studies with multiple exposures that ensures a clear distinction from case series. In this conceptualization, all studies that contain sufficient data to conduct a reanalysis and not only studies with a pre-existing analytic comparison are classified as cohort studies and are considered appropriate for inclusion in systematic reviews. To the best of our knowledge, no systematic reviews exist that reanalyse (mislabelled) case series to create cohort studies. The outlined approach is a method that can potentially enrich the body of evidence and subsequently enhance confidence in effect estimates and the strengths of conclusions. However, the enrichment of the body of evidence should be balanced against the additional workload.
There was no external funding for the research or publication of this article.
Availability of data and materials
All authors have made substantial contributions to the work. Both authors read and approved the final manuscript.
Ethics approval and consent to participate
Not applicable. No human data involved.
Consent for publication
Not applicable. The manuscript contains no individual person’s data.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
- 3.Reeves BC, Deeks JJ, Higgins JP. 13 including non-randomized studies. Cochrane Handbook Syst Rev Interventions. 2008;1:391.Google Scholar
- 5.EPOC-specific resources for review authors: What study designs should be included in an EPOC review and what should they be called? [http://epoc.cochrane.org/resources/epoc-resources-review-authors]. Accessed 12 July 2017.
- 7.Hernot S, Wadhera R, Kaintura M, Bhukar S, Pillai DS, Sehrawat U, George JS. Tracheocutaneous fistula closure: comparison of rhomboid flap repair with Z Plasty repair in a case series of 40 patients. Aesthet Plast Surg. 2016.Google Scholar
- 8.Stacchiotti S, Provenzano S, Dagrada G, Negri T, Brich S, Basso U, Brunello A, Grosso F, Galli L, Palassini E, et al. Sirolimus in advanced Epithelioid Hemangioendothelioma: a retrospective case-series analysis from the Italian rare cancer network database. Ann Surg Oncol. 2016;23(9):2735–44.CrossRefPubMedGoogle Scholar
- 9.Sofiah S, Fung LYC. Placenta accreta: clinical risk factors, accuracy of antenatal diagnosis and effect on pregnancy outcome. Med J Malays. 2009;64(4):298–302.Google Scholar
- 11.Mayorandan S, Meyer U, Gokcay G, Segarra NG, de Baulny HO, van Spronsen F, Zeman J, de Laet C, Spiekerkoetter U, Thimm E, et al. Cross-sectional study of 168 patients with hepatorenal tyrosinaemia and implications for clinical practice. Orphanet J Rare Dis. 2014;9(1):107.CrossRefPubMedPubMedCentralGoogle Scholar
- 17.de Mauroy JC, Journe A, Gagaliano F, Lecante C, Barral F, Pourret S. The new Lyon ARTbrace versus the historical Lyon brace: a prospective case series of 148 consecutive scoliosis with short time results after 1 year compared with a historical retrospective case series of 100 consecutive scoliosis; SOSORT award 2015 winner. Scoliosis. 2015;10:26.CrossRefPubMedPubMedCentralGoogle Scholar
- 18.Forner D, Phillips T, Rigby M, Hart R, Taylor M, Trites J. Submental island flap reconstruction reduces cost in oral cancer reconstruction compared to radial forearm free flap reconstruction: a case series and cost analysis. J Otolaryngol Head Neck Surg. 2016;45:11.CrossRefPubMedPubMedCentralGoogle Scholar
- 19.Kuhnt D, Bauer MHA, Sommer J, Merhof D, Nimsky C. Optic radiation fiber Tractography in Glioma patients based on high angular resolution diffusion imaging with compressed sensing compared with diffusion tensor imaging - initial experience. PLoS One. 2013;8(7):e70973.CrossRefPubMedPubMedCentralGoogle Scholar
- 20.Naesens R, Vlieghe E, Verbrugghe W, Jorens P, Ieven M. A retrospective observational study on the efficacy of colistin by inhalation as compared to parenteral administration for the treatment of nosocomial pneumonia associated with multidrug-resistant Pseudomonas Aeruginosa. BMC Infect Dis. 2011;11:317.CrossRefPubMedPubMedCentralGoogle Scholar
- 21.Toktas ZO, Konakci M, Yilmaz B, Eksi MS, Aksoy T, Yener Y, Koban O, Kilic T, Konya D. Pain control following posterior spine fusion: patient-controlled continuous epidural catheter infusion method yields better post-operative analgesia control compared to intravenous patient controlled analgesia method. A retrospective case series. Eur Spine J. 2016;25(5):1608–13.CrossRefPubMedGoogle Scholar
- 27.Kellesarian SV, Abduljabbar T, Vohra F, Gholamiazizi E, Malmstrom H, Romanos GE, Javed F. Does local Ibandronate and/or Pamidronate delivery enhance Osseointegration? A systematic review. J Prosthodont. 2016.Google Scholar
- 28.Crandall M, Eastman A, Violano P, Greene W, Allen S, Block E, Christmas AB, Dennis A, Duncan T, Foster S, et al. Prevention of firearm-related injuries with restrictive licensing and concealed carry laws: an eastern Association for the Surgery of trauma systematic review. J Trauma Acute Care Surg. 2016;81(5):952–60.CrossRefPubMedGoogle Scholar
- 30.Petersen I, Douglas I, Whitaker H. Self controlled case series methods: an alternative to standard epidemiological study designs. BMJ. 2016;354.Google Scholar
- 31.Higgins JP, Green S. Cochrane handbook for systematic reviews of interventions, vol. 5: Wiley Online Library; 2008.Google Scholar
- 32.Marcano Belisario JS, Tudor Car L, Reeves TJA, Gunn LH, Car J. Search strategies to identify observational studies in MEDLINE and EMBASE. Cochrane Database Syst Rev. 2013;12.Google Scholar
- 34.Institute for Quality and Efficiency in Health Care (IQWIG): Newborn screening for severe combined immunodeficiency (S15–02). In.; 2017.Google Scholar
- 35.Institute for Quality and Efficiency in Health Care (IQWIG): Newborn screening for tyrosinaemia type 1 (S15–01). In.; 2017.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.