Background

Health care service research aims to evaluate care strategies by obtaining an understanding of the causal factors in improving dementia-related outcomes for individuals living in nursing homes [1]. It is self-evident that a valid method to ascertain a case is a prerequisite for any study [2]. Case ascertainment strategies must rely on an established definition, and diagnostic criteria must be both valid and feasible to conduct.

Dementia may be defined as a clinical syndrome of mental capacity characterized by a substantial global decline in cognitive function that is not attributable to altered consciousness; it consists of a combination of symptoms attributable to various causes or pathological events [3]. Following the Diagnostic and Statistical Manual (DSM) 5th edition [4], dementia is subsumed under the entity of neurocognitive disorders (NCD). The diagnostic criteria for NCD are as follows:

  1. i.

    Evidence of a significant cognitive decline from a previous level of performance in one or more cognitive domains (complex attention, executive function, learning and memory, language, perceptual-motor or social cognition)

  2. ii.

    The cognitive deficits interfere with independence in everyday activities (present in major NCD but not in mild NCD)

  3. iii.

    The cognitive deficits do not occur exclusively in the context of a delirium

  4. iv.

    The cognitive deficits are not better explained by another mental disorder

Evidence for the cognitive decline should be based on concerns from the individual patient, a knowledgeable informant, or the clinician regarding the presence of a significant decline in cognitive function and a substantial impairment in cognitive performance; ideally, this evidence will be documented by standardized neuropsychological testing. The DSM-V manual notes that both concern and objective evidence are required to establish a diagnosis because they are complementary. If alterations in cognition are only tested utilizing objective measures, a disorder may be underdiagnosed in individuals who demonstrate high-functioning performance and score in the “normal” range but are still substantially impaired relative to their baseline performance. On the other hand, an exclusive reliance on subjective symptoms may result in underdiagnosing individuals who deny or fail to express their impairments. In either case, it is essential to interpret results in comparison with the individual patients’ prior performance.

The International Classification of Diseases (ICD) defines dementia as a decline in both memory and other cognitive abilities such as deterioration in judgment and thinking or the general processing of information. The decline in memory and cognitive abilities should be objectively verified by obtaining a reliable history from an informant and supplemented by neuropsychological tests or quantified cognitive assessments. The diagnosis of mild impairment is based on the degree of memory loss that interferes with everyday activities. A diagnosis of dementia requires, in addition to the previously mentioned symptoms for mild impairment, a decline in emotional control or motivation, changes in social behavior, and the absence of delirium. Furthermore, the described symptoms must be present for at least 6 months [5].

The diagnostic process follows two steps, starting with the initial recognition of the dementia syndrome and ending with the specification of an etiological subtype. The diagnostic criteria for each of the subtypes are specified in the DSM-V, ICD-10 [4, 5] and publications from different disease-related societies or national institutes (e.g., the National Institute of Neurological and Communicative Disorders and Stroke, NINCDS) [6–10]. The subtype differentiation may require biomarker diagnostics such as genetic, blood and liquor testing or diverse imaging procedures.

For more than 3 decades, the history of classification systems for mental disorders has reflected both many changes and a relative emphasis on phenomenology, etiology, and course as defining features. When the DSM-III’s (1980) explicit diagnostic criteria were introduced (Feighner-criteria), several revisions followed: the DSM-III-R (revision) in 1987, the DSM-IV in 1994, and the DSM-V in 2013. The ICD introduced diagnostic criteria in its revised version 9, ICD-9-CM (for clinical modification). Every revision aimed both to reduce inconsistencies and to improve clarity and precision. Different versions included changes in the classification, diagnostic criteria sets, and descriptive texts [11].

Dementia diagnoses pose several challenges. First, the lack of valid biomarkers forces healthcare providers to diagnose dementia based on the presence of specific symptoms and the elimination of other conditions with similar symptoms. To arrive at a clinical diagnosis of dementia, a combination of basic assessments,—including physical, psychopathological and basic neuropsychological examinations—are recommended [12, 13]. For the basic neuropsychological diagnosis, different short screening tests such as the Mini Mental Status Examination (MMSE) [14, 15], the DemTect [16] and the Clock Drawing Test [17] are recommended. In clinical practice, these tests should be administered to every patient with dementia or suspected dementia on a regular basis both to quantify cognitive impairments and to enable providers to supervise the progression of the disease. Comprehensive neuropsychological diagnostics are necessary either if diagnostic findings are not congruent in the early stage of dementia or if an etiological classification is required. A dementia diagnosis should not be based solely on one neuropsychological test but instead must be based on both behavioral symptoms and impairment assessments related to daily function. Additionally, blood tests and imaging diagnostics should be conducted to assure both the diagnosis and its etiology [18].

The reliability, validity and utility of the new diagnostic criteria have discussed by clinicians and researchers since the date of their release; furthermore, a consensus about how to best diagnose dementia has still not been found [19, 20]. One result of this controversy is the methodological differences in epidemiological studies that lead to discrepancies in case ascertainment and varying incidence and prevalence rates [3, 21]. Another limitation of the generalizability of study results on dementia in nursing homes originates from the methodological difficulties of assessing the disease in the oldest old population. The normative data necessary to determine “impairment” for this population are lacking and therefore, aging is constitutionally associated with a decline in both cognition and function. Consequently, pathological processes are difficult to distinguish from normal aging; in addition, alterations in function and sensory impairments may both affect subjects’ cognitive performance and limit an accurate assessment [20].

The quality of a dementia diagnosis in German nursing homes is suboptimal: many studies of nursing home residents show both vast inaccuracies and diagnostics that do not conform to either the ICD or the DSM. Studies from Germany show that between 30 and 40 % of people with dementia living in nursing homes are not accurately diagnosed; for these residents, either an etiological differentiation is missing or there is an inappropriate diagnosis [22–24]. Inaccurate diagnosis has also been found in studies from the United States, Norway, Israel and Ireland [25–29].

Nursing home research is challenged by inaccurate recorded diagnoses. Ignoring inconsistencies in a recorded dementia diagnosis may provoke misclassifications, bias selection and confound study results [30]. However, the evaluation of a diagnosis within a study is resource-intensive and may be ethically questionable because the diagnosis process is reported as burdensome and stigmatizing [31, 32].

Researchers must develop a sampling strategy that eliminates inconsistencies while remaining both reliable and practicable. We assume that in the past, researchers defined dementia differently and applied different methods to identify residents with dementia in nursing homes. Therefore, we investigated the methods used to define dementia and to identify people with dementia in health services research. Although we assume that the delineated problem is obvious in various countries, we limited the scope of our study to research undertaken in a single country (Germany). In Germany, physicians are not constantly present in nursing homes, and nurses are not allowed to assist in diagnostic procedures; resources for dementia diagnostics in primary care are scarce [33]; hence, diagnostics are often superficial, performed rapidly and lacking recommended measures [34, 35]. Due to the free choice of medical practitioners that is guaranteed by law in Germany, residents within one nursing home may be cared for by different practitioners, making it difficult for researchers to reconstruct the diagnostic procedures that were applied by various medical doctors. These conditions require a carefully considered sampling strategy. The narrowed focus on one country ensures the comparability of the studies because the nursing-home conditions are identical. This allows a synthesis and comparison of the various methods used in the included studies.

Because cognitive impairment (CI) is the leading symptom in dementia and the conditions “dementia” and “CI” are sometimes used interchangeably in scientific publications, the scope of our review encompasses studies that do not exclusively focus on residents with dementia but instead include residents with CI. Provided we found differences, we discuss the implications of different methods on the generalizability and comparability of the study findings.

The following research questions were addressed:

  • How is dementia defined and measured in health services research studies in German nursing homes?

  • Which implications can be derived for health services research in nursing homes?

Based on the results, we discuss the implications of the different methods and suggest principles to guide future studies.

Methodology of the review

Compared to systematic reviews that aim to answer questions about the effectiveness of clinical procedures, there are few well-defined tools and processes for methodological reviews; not all items of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines are applicable (see PRISMA checklist in the Additional file 1: Table S1) [36]. Unlike a traditional Cochrane-style review, Lilford and colleagues [36] recommend methodological reviewers not to obtain a thorough collection of studies, but to ensure a wide search that provides an overview of the topic. They also recommend various safeguards to reduce potential bias, such as multi-disciplinary teams and peer reviewing of the final report. The methods applied in this review consider these recommendations.

Literature search strategy

We conducted a systematic literature search to identify relevant publications in the MEDLINE [PubMed] and CINAHL [EBSCO] databases in January 2014 (MND) and updated it in December 2015 (RP).These databases were chosen because we considered them as the most relevant to the field of health services research. We detail the search strategy in the Additional file 2: Table S2 and Table S3. The search was not limited to a specific time and study design, but it was limited to the German and English languages. After the initial title and abstract screening, publications were excluded if they were considered irrelevant to the topic or setting (RP, CGGS). The remaining publications were divided to be read in full-text and to be excluded according to predefined criteria by one research team member (RP, SR, MND, CGGS, BH, MH). Publications fulfilling the following criteria were included for review: the research aim or question focused on residents with dementia or CI in German nursing homes and was answered based on empirical data. Each decision to exclude a publication was discussed and consented by the research team.

Data extraction and method of analysis

Two researchers (RP, SJ) read the included articles in full-text and extracted the data: SJ conducted an initial data extraction, and RP checked all of the information for accuracy. In the first step of the data extraction, an overview of the data on the study aim, design and sample size were compiled. In the second step, the publications were sorted according to sample determination procedure. Publications from one study using the same method to identify residents with dementia/CI were summarized; this was the case with publications from large projects with various research aims and questions. To answer the research questions, the following information was extracted: information on the method of sample determination, the definition and criteria used for dementia diagnostics, the screening instrument(s) used, and the qualification and training of the professionals involved. The extracted findings and key characteristics of each method were summarized in an overview, and differing approaches were compared.

Results

The literature search yielded 650 articles. Sixty-four articles were identified to answer our research questions. The results of the literature search can be seen in the flow chart in Fig. 1.

Fig. 1
figure 1

Flow chart of literature search

Full size image

The characteristics of the included studies are presented in Table 1.

Table 1 Characteristics of included studies
Full size table

Methods to identify the study population residents with dementia

All 64 articles focused on the population of nursing home residents with CI. In 60 articles, dementia was specified as the etiology of CI. In these articles, the term “dementia” was used consistently in the title, abstract, key words and research questions. Four articles focused their research question on residents with CI without any specification of etiology, which means they did not clearly define dementia residents as their study population [37–40]. However, in these articles the terminology was not consistent throughout the article: the term dementia and CI were used interchangeably.

The articles focusing on residents with dementia used different methods to determine their study sample. We identified 4 methods that were clearly described.

Nursing home residents with dementia were identified in one of the following ways:

  1. 1.

    A study diagnosis that was evaluated during the study.

  2. 2.

    A recorded medical diagnosis.

  3. 3.

    A recorded medical diagnosis and an additional cognitive screening performed during the study.

  4. 4.

    A cognitive or accordant screening.

  1. 1.

    Study diagnosis

    • In 17 articles, a study diagnosis of dementia was used to determine the participant sample (Table 2). The diagnosis was newly assessed either in all residents or in residents with a diagnosis that had previously been documented.

      Table 2 Overview of studies that determine study participants with dementia based on a study diagnosis
      Full size table
  2. 2.

    Recorded medical diagnosis

    • In 6 articles, the recorded diagnosis was the criterion for determining the sample participants (Table 3). In the study by Graessel et al. [41], attending physicians confirmed the recorded diagnosis; in the study by Weyerer et al. [42, 43], the diagnosis was used as an indirect inclusion criterion. In this study, the dementia diagnosis was an admission criterion for the care unit that participated in the study (Dementia Special Care Unit). This study does not report whether the diagnosis was confirmed. In 3 articles, the diagnosis was obtained from the residents’ records [44–46]; 1 article reported that the dementia type was evaluated according to the ICD classification (as recorded), and the dementia severity evaluation was guideline-based and performed according to the recommended MMSE cutoff values [46].

      Table 3 Overview of studies that defined study participants with dementia based on a recorded diagnosis
      Full size table
  3. 3.

    Recorded medical diagnosis and screening

    • In 20 articles, the recorded medical diagnosis of dementia was used as a criterion to determine the sample participants, but it was combined with the results of a cognitive screening measure (Table 4). In 19 studies, a resident was included if the result of the MMSE also indicated CI; 1 study included a resident if a diagnosis was recorded and the result of a screening using the Functional Assessment Staging Test (FAST) indicated dementia [47]. One study used a stepwise approach [48–51]: in the case of an incongruity between the diagnosis and the MMSE result, additional diagnostics were performed to decide whether a resident fulfilled the inclusion criteria.

      Table 4 Overview of studies that defined study participants with dementia based on a recorded diagnosis and additional cognitive screenings
      Full size table
  4. 4.

    Screening

    • In 10 articles, residents with dementia were identified by 1 screening or a combination of 2 screenings (Table 5). In these studies, an existing diagnosis was recorded, but it was not used as the inclusion criterion. Palm et al. [52] used the place of residence in a Dementia Special Care Unit as an inclusion criteria as well as the results of a cognitive screening to define their sample. One article reported exclusion criteria that were used to differentiate dementia from other psychiatric disorders [53].

      Table 5 Overview of studies that defined study participants with dementia based on cognitive screenings
      Full size table

    • The 3 studies (4 articles) that investigated residents with CI exclusively used screenings to identify their study participants (Table 6).

      Table 6 Overview of studies that investigated study participants with cognitive impairment
      Full size table

    • Seven publications could not be classified according to the 4 groups explained above (Table 7). In 4 of these publications, all of the residents from a participating nursing home or living unit were included in the study and investigated with respect to various indicators for dementia (dementia diagnosis in the records and results of cognitive screenings) [24, 54–56]. In these studies, the sample was described, but participants were not selected or assigned based on these indicators. One article is a study protocol; the method to define residents with dementia was not determined, only the measurements that were intended to assess dementia [57]. In 2 publications [58, 59], the method to identify residents with dementia was not reported at all.

      Table 7 Overview of studies that did not clearly define participants with dementia
      Full size table

Reporting about dementia diagnostics within the study

Fifteen articles reporting on studies with a newly assessed diagnosis detailed diagnostic criteria and gave information on the diagnostic process or referred to related articles that described the diagnostic process [22, 23, 48–51, 60–68]. As shown in Table 2, these studies referred to the definition and diagnostic criteria for Alzheimer’s disease and vascular dementia from the ICD, DSM, or NINCDS. Studies that referred to the diagnostic criteria for dementia subtypes reported a more comprehensive diagnostic procedure [22, 60–68] than those reported in studies that did not refer to dementia subtypes. These diagnostic procedures were composed of clinical examinations, interviews, neuropsychological testing and a review of existing diagnostic findings from technical investigations. The articles examined did not specify the number or types of findings available from technical investigations or how missing findings were accommodated.

Three studies did not refer to any definition or diagnostic criteria of dementia, but solely reported the instrument on which the diagnosis was based [69–71]. Two studies conducted prior to 2000 [72, 73], assessed a diagnosis on the basis of the Feighner criteria [74]. Neither of those studies provided any detailed information about the diagnostic process or about which instruments were used to assess the criteria. One pre-1990 study did not report the definition, the diagnostic criteria, or any instrument that was used [75].

The majority of studies in which the diagnosis was newly assessed within the study indicated that a battery of neuropsychological tests was used [22, 23, 48–51, 60–70, 72]. Two of these studies indicated that the final diagnosis was based on the Clinical Dementia Rating (CDR) [69, 70]; in another study, this instrument was used for staging [68].

The compilation of psychological tests differed in the studies with regard to the assessment of cognition, behavior and function. For assessing cognition, the MMSE was the only instrument that was used in all of these studies. Studies using a battery of neuropsychological tests included different tests for cognitive function: the MMSE was combined with the Global Deterioration Scale (GDS), the CDR, the Clock Drawing Test, the Brief Assessment Schedule (BAS-Dem) and the Parkinson Neuropsychometric Dementia Assessment (PANDA). With exception of the PANDA (used for a subsample of residents with Parkinson’s disease) [67] and the CDR (used in cases of severe physical impairment) [66], the studies did not specify whether all of the instruments were administered to each resident or whether any particular instrument was used for residents with certain characteristics. For assessing behavior and function, the researchers also used different instruments, which can be viewed in Tables 2, 3, 4, 5 and 6.

The scope and the rationale of each instrument’s use in the diagnostic process were not clearly documented in most studies. In 10 studies, it was not clear which instruments were essential to the decision about the diagnosis because no cutoff values were reported [22, 23, 60–67]. Moreover, it was not clearly stated which instruments were used for diagnostics and which were used for study purposes only (including the description of the study sample), as an outcome measure, or for validity testing. Five studies reported cutoff values [69–73]; 2 of these studies used the CDR with a cutoff ≥ 1 to assess the dementia diagnosis [69, 70]; 3 studies reported cutoff values for the MMSE and the BAI to stage dementia severity [71–73].

The assessment process for the diagnosis was described briefly in 5 articles: 3 articles reported that the administration of instruments was carried out with the help of interviewers [22, 66, 67], whereas 3 studies reported that the diagnosis was agreed to in a multidisciplinary conference [66, 68, 70]. In all of the studies that assessed a new diagnosis of dementia medical staff with geriatric experience, primarily physicians, performed the diagnostics.

Reporting about dementia diagnostics prior to the study

Altogether, in 26 articles, the recorded diagnosis was used as an inclusion criterion with or without additional screenings (Tables 3 and 4). Twelve articles reported that existing diagnoses used to determine the sample were performed or confirmed according to ICD [41, 46, 48–51, 76–81]. None of these articles gave more explicit information about the pre-study diagnostic procedure or about whether an existing diagnosis was confirmed by the attending physician. In 14 articles, neither the definition of dementia and/or its diagnostic criteria were reported; in addition, no information was provided about the diagnostic procedure that had been undertaken prior to the study [42–45, 47, 82–90].

In 17 articles, it was reported that in addition to the recorded diagnosis or performed screenings, exclusion criteria were also used in the study [48–51, 53, 76–80, 82, 85–90]. These comprised other neurological or psychiatric diseases that could cause CI.

Reporting about screening of cognitive impairment

CI was measured in all studies, but with different instruments and cutoff values.

The use of the MMSE was reported in 47 articles and represents the instrument most often used to identify residents with dementia and the stage of its severity. A broad range of cutoff scores for dementia severity was reported for the MMSE. Seven studies referred to an MMSE of < 25 [37, 40, 48–51], 6 studies used a cutoff of ≤ 24 [85–90], 8 articles referred to a cutoff of < 24 to detect mild dementia [76–80, 82, 91, 92], and 4 articles referred to a more conservative cutoff. [39, 41, 83, 93] One article specified a higher cutoff value of ≤ 27 but conducted confirmatory testing with the Geriatric Depression Scale (GDS) for residents with an MMSE between 24 and 27 [84]. One article use a cutoff value of < 27 without additional testing [46]. The studies also used different cutoff values for staging dementia (Table 8).

Table 8 MMSE cutoff values for dementia staginga
Full size table

Thirteen articles did not report a cutoff value for the MMSE but instead reported mean scores for the study sample [22, 23, 60–70].

For the CDR, a consistent cutoff value was reported (≥ 1) in 3 studies [68–70]; 3 articles did not report any cutoff values for the CDR [22, 66, 82]. For the GDS, consistent cutoff values were most often reported. Schumacher et al. and Bär et al. [39, 54, 55] specified mild CI as corresponding to a GDS of 2-3, moderate CI as corresponding to a GDS of 4-5, and severe CI as corresponding to a GDS of 6-7. Pickel et al. [80]. set a cutoff value of > 3 for mild, moderate or severe dementia.

In addition to differences concerning the use of instruments, we also identified differences regarding the professionals who administered the instruments to assess CI. Studies using the assessments to state a dementia diagnosis primarily used physicians. Studies assessing CI without making a statement on the dementia diagnosis used physicians, psychologists, nurses, medical students and students from other health care programs (see Tables 2, 3, 4, 5 and 6). Some studies stated that the assessors were experienced in the field and when nurses were assessors, it was stated that they were familiar with the residents. Training for raters who performed the cognitive assessments was conducted in numerous studies (see Tables 2, 3, 4, 5 and 6); however, the duration of training, training concepts and training evaluation were not specified.

Discussion

This review reveals a sampling challenge for dementia studies in German nursing homes in health services research: uncertainty related to existing dementia diagnoses, constrained resources and ethical dilemmas concerning accurate diagnostic procedures and the achievement of study validity. When designing a sampling plan, the advantages of convenient and efficient access and ethical considerations related to the residents must be weighed against participant-related validity concerns.

In our review, we found 4 sampling methods to identify residents with dementia:

  1. 1.

    A study diagnosis;

  2. 2.

    A recorded medical diagnosis;

  3. 3.

    A recorded medical diagnosis and additional cognitive screening; or

  4. 4.

    Cognitive or accordant screening.

Each method has to be discussed with regard to its practicability and validity.

A guideline- and criteria-based diagnosis of dementia is the most valid criterion to select study participants when the aim of the study is to target people with dementia. The clear benefits of this method are a thorough assessment of dementia-related conditions and a sound differential diagnosis to ensure that the CI is not caused by conditions other than dementia. Challenges related to this approach are the resources required in terms of time, costs and availability of physicians.

The performance of diagnostics by a member of the research team enhances study validity by ensuring conformity to the criteria. As demonstrated in several studies [22, 23, 48–51, 60–68], criteria conformity can be achieved in the evaluation of a dementia syndrome. In these studies, both subjective and objective information on the patient’s cognitive decline were thoroughly assessed. However, the validity of the dementia subtype diagnosis, as performed in a few of these studies [22, 23, 60–65], was questionable. In particular, during the late stage of dementia, a differentiation between dementia subtypes is difficult and requires the use of disease progression measures such as repetitive cognitive tests. It can be assumed that this information was not readily available for a large number of residents because the implementation of diagnostic measures, other than the collection of subjective information, is infrequent in the German primary care sector. A survey of 23 German primary care physicians regarding the use of diagnostic measures showed that only 3 physicians used neuropsychological tests as a component of their diagnosis [34]. There is also evidence showing that in primary care, imaging techniques and other diagnostic features are seldom used; misclassifications that lead to an overestimation of vascular dementia were frequently discovered in several studies [34, 94, 95].

With respect to the reporting of the diagnostic performance, the studies lacked detailed information. Several studies did not report which instrument was used to assess which criterion, whether normative data were used to determine CI, which cutoffs were used and how missing data were addressed. It is recommended that this information be provided when a dementia diagnosis in the oldest-old population is evaluated [20]. Additionally, information on training for the raters who conducted the cognitive assessments was missing in all of the studies. In our opinion, this is a crucial issue because 1 study showed that even physicians who underwent training in cognitive assessments may still fail to use these diagnostic instruments correctly [34].

However, evaluating a study diagnosis also lacks feasibility when resources are constrained. Imaging diagnostics are critical because nursing home residents are often too frail to transport, and nursing homes are often not affiliated with a clinic. Consequently, appropriate technical devices may not be available. As we can see in the studies that evaluated dementia subtypes, only existing findings were used to conduct a subtype diagnosis. Thus, the diagnosis must be made based on neuropsychological and functional tests, clinical examinations and interviews that are conducted by either geriatric psychiatrists or clinical psychologists; this process requires time, experienced assessors and willing participants. The study with the largest sample size involved 589 residents from twenty nursing homes in a German city [69]. Thus, performing dementia diagnostics within a study seems feasible for a sample size up to 600 people who live in a single city. However, we assume that the evaluation of a dementia diagnosis may have a negative impact on the willingness to participate in a study if the assessment is overly burdening or involves invasive procedures [96]. For example, in 1 study, 20 of 113 eligible residents were unwilling to submit to an MMSE [93]. For the majority of the nursing home population, legal guardians decide whether the resident will participate. In our experience from studies in the field, legal guardians are hesitant to permit participation because studies involve measures that will burden the resident.

The use of a recorded medical diagnosis as an inclusion criteria may be appealing, especially in studies with large samples and with a geographically widely dispersed population. However, the well-documented inaccuracy of recorded dementia diagnoses indicates that in a research situation, this approach must be questioned [6, 30]. If researchers cannot be sure which criteria have been used and how they were measured, we must assume that the diagnosis may not correspond to the established ICD- or DSM-criteria. Consequently, misclassification threatens the internal validity of study results. In experimental trials, diagnostic discrepancies may lead to heterogeneity in the intervention and control group that may have an impact on treatment effects. The inclusion of false-positive participants may explain the failure to show treatment effects, and the exclusion of false-negative participants both prolongs the recruitment process and may cause a selection bias.

The third sampling method that combines a recorded diagnosis with a cognitive screening is more feasible than evaluating a diagnosis following defined criteria and recommended procedures. Cognitive tests are comparatively easy and quick to administer and therefore, their administration can be delegated to nursing-home staff members who are more easily accessible than geriatricians or psychologists, such as nurses or medical students. This method may ensure a more homogenous sample with respect to cognition, but it misses other information about functioning and decline in functionality and cognition. Without this information, it is impossible to verify the diagnosis according to established diagnostic criteria [30, 97]. The risk of including false-positives can be reduced, but not eliminated. In this respect, exclusion criteria are relevant. Several studies reported that residents were excluded when diseases other than dementia explained their cognitive decline. However, the combination of a recorded diagnosis, a cognitive screening and exclusion criteria still does not reduce potential selection bias resulting from systematically missing false negatives. Furthermore, it is necessary to decide what to do when combined inclusion criteria contradict each other. In 1 study, these concerns were addressed. Majic and colleagues report that participants were included with a recorded diagnosis of dementia, an MMSE ≤ 24 and a duration of CI of > 6 months, but diagnostics were performed in case of incongruity [48]. They report elsewhere that additional diagnostics were performed in residents if the nursing home staff suspected dementia, but there was no recorded diagnosis [49]. Other studies did not sufficiently report how they addressed incongruity in the combined inclusion criteria and whether they performed any measures to rule out the outlined problems.

The fourth method that was identified in this review is the use of a single test result. The instruments FAST, DSS and the MMSE were used for this purpose.

The FAST staging procedure is used as a diagnostic measure for dementia and relies on the assessment of functional impairments that are attributable to dementia [98]. This instrument was tested in its original language on a sample that was composed of 16 participants who had different levels of cognitive impairment and numerous possible underlying diseases. The testing was conducted by physicians who were in their postdoctoral training phase; their results revealed excellent values for inter- and intra-rater reliability and concurrent validity [99]. Unfortunately, data on internal consistency and discriminant validity are absent [100] as well as the results from the German version of the FAST. To our knowledge, this instrument has not been evaluated for its sensitivity or specificity when being used to detect dementia. In the reviewed studies, it was administered by health personnel, not physicians. Additionally, there are no studies that compare the accuracy of this instrument depending on if the instrument is administered by physicians versus other health personnel.

The DSS is a 7-item scale that assesses both memory and orientation at the time of assessment [69]. The instrument was originally developed in the German language, tested in 589 nursing home residents, and administered by nurses [69]. It was validated against the MMSE, BAS-DEM and the CDR. Utilizing the CDR as the gold standard, the DSS was able to correctly classify more than 80 % of residents; however, it showed small inaccuracies when compared to the MMSE and BAS-DEM. Because the scale is fast and easy to administer, this approach can be considered to be feasible in health services research, even though its validity must be critically discussed. The scientific literature concordantly states that the assessment of single domains of cognition, especially a single cognitive test, cannot accurately diagnose dementia and should not be used to substitute systematic evaluations, examinations and laboratory tests [101]. Because the DSS does not assess changes in cognitive functions, it is not possible to determine if the present cognitive deficits had declined over the past 6 months. Hence, the DSS will misclassify residents who have acute cognitive deficits that are due to infection or dehydration. The same misclassification can occur in residents who have chronic cognitive impairments due to other diseases.

In our review, the MMSE was the instrument that was most often chosen to define the inclusion criteria. The rationale for using the MMSE in the studies was primarily its degree of popularity and widespread utilization. A meta-analysis on the accuracy of the MMSE confirms this choice: in high prevalence settings, the MMSE shows a sensitivity and specificity of 77 and 90 %, respectively [15]. The notable shortcomings of the MMSE in the context of nursing home research were rarely discussed in the studies, although those shortcomings threatened the internal validity of the study results. The MMSE shows a floor effect in very severe score ranges, people with little formal education, and people with severe language problems. [101] Several items are strongly influenced by age and education, which implies a need to adjust MMSE scores when establishing thresholds. We found a variety of MMSE thresholds for staging dementia in the reviewed studies - a result that was also reported in other reviews [12, 20]. This result may be a consequence of missing normative data for the age group of the oldest-old and standardized age-adjusted cutoff values. The authors of a meta-analysis on dementia screening and case-finding tool validation studies suggest alternatives to the MMSE such as the Mini-Cog [102]. However, the suitability of these screening tools for nursing home research needs to be investigated.

In this respect, an associated question must be discussed in health services research in nursing homes: is a valid diagnosis of dementia necessary to define the study population? If the aim of the study is to prove the benefit of an intervention that should be provided exclusively to residents with dementia, a valid diagnosis is essential to prove the benefit for this population. If the aim of the study is to prove a benefit for residents with cognitive and functional impairments independent of their etiology, a dementia diagnosis is dispensable. The same question must be considered by clinicians because in the absence of disease-modifying treatments, the primary advantage of a diagnosis for the oldest-old population has to be determined to justify resource-intensive diagnostics [20]. The need to perform diagnostics in the nursing home population must also be ethically justified because psychological tests can be significantly burdensome to people with CIs.

We also found 4 articles that aimed to investigate residents who had CI but did not explicitly have dementia. To us, it was unclear why some studies focused on the population of people with CI but not on people with dementia because the authors did not elaborate on the question whether the etiology of CI played a role with respect to their research focus. In 2 publications, the terms “dementia” and “CI” were used interchangeably [39, 40]. If the authors did not explain why they defined their population based on a disease or a symptom and later failed to distinguish between the 2 conditions, one may ask whether the question was sufficiently addressed.

In addition to the variability of case ascertainment strategies in dementia studies, this review mirrors the developments in knowledge, concepts, differentiations and diagnostic criteria related to “dementia” that have occurred during the last 30 years. With each decade of dementia studies in German nursing home research, the diagnostic procedures were refined and reflect methodological innovations. Therefore, the definition and assessment of dementia and CI must be considered against the respective state of knowledge at the time a particular study was conducted. For example, in 1 study from 1993 [72], the Feighner criteria were applied; these criteria had been introduced in the DSM-III and became available in the German language with the German translation of the DSM-III in 1984. On the contrary, another study [75] was published the same year; it is obvious that those researchers could not apply today’s diagnostic criteria because they were not available at the time the study was designed and conducted.

An overview of the advantages and disadvantages of the different case ascertainment strategies is summarized in Table 9.

Table 9 Case ascertainment strategies in comparison (summary)
Full size table

Strengths and limitations

The problem of inconsistent management of methodological challenges in dementia research has already been recognized and has led to an initiative on how to improve the reporting of these challenges in clinical studies [103]. However, studies of the outlined problem of inconsistent case ascertainment strategies in nursing home research have been lacking to date. We consider this as a prerequisite for improving dementia research in the nursing home sector and outline this as the major strength of this study.

The generalizability of the displayed results is constrained to one country (Germany). However, we assume that the outlined problem can also be demonstrated in other countries, but this must be proven in a later review. Perhaps in countries in which physicians are employed in nursing homes and adherence to diagnostic standards can be guaranteed, the reliance on a recorded diagnosis can be received less critically.

As discussed, the reporting of diagnostic procedures was partially insufficient; in particular, different publications from a single study provided varied information about their case ascertainment. In some studies, it was not clear whether the assessments were used exclusively for study purposes, for diagnosis, or both. In particular, the scope of assessments with respect to behavior and function was not described clearly, making it unclear whether these instruments were also used for diagnostic procedures. In an effort to rule out this lack of clarity, we unsuccessfully attempted to contact the authors. The short reporting made it also impossible to assess the quality of diagnostic procedures or the associated risk of bias.

Because of the descriptive objective of the review, it has not been registered in a review register.

Conclusions

Considering the findings of our review, we suggest the following principles to improve the validity and comparability of study results on dementia in nursing homes.

Investigators must clearly distinguish whether their research addresses residents with CIs or residents with dementia with an appropriate etiology. If residents with dementia are addressed, it should be in accordance with the study aim. Studies that address the population of nursing home residents with dementia should report in detail which method of case ascertainment was used and should discuss the limitations associated with each method. In our opinion, a criterion-based diagnosis is essential when a study addresses explicitly residents with dementia. If residents are included in studies based on diagnoses that do not conform to established criteria, the results from different studies are not comparable. To ensure that a diagnosis conforms to criteria, clinicians who are members of the research team should evaluate a diagnosis during the research project. When financial or human capacities are constrained, one should consider alternative methods to a face-to-face consultation by a geriatric physician, psychologist or psychiatrist. Several methods utilizing an expert panel are described for the diagnosis of dementia [104]. The method by Magaziner et al. [30] appears promising for research purposes: 2 clinicians decide whether dementia is present based on the resident’s history and neuropsychological tests performed by lay evaluators. Compared to direct clinical evaluation, agreement was satisfactory (76 %). Another promising approach is a nurse-administered diagnosis. Studies showed a high agreement between a nurse-administered diagnosis and a multidisciplinary team diagnosis in determining mild CI in primary care [105]; in addition, there is agreement between a Memory Clinic diagnosis [106] and a moderate agreement with the ICD-10 diagnosis (ibd.). To our knowledge, a nurse-administered diagnostic procedure for dementia in nursing home residents has not yet been developed and tested.

Regardless of which method is used to evaluate a diagnosis of dementia within a study, reporting requires details regarding the domains that are tested, the rationale behind instrument use for each domain examined, the cutoff values set for each domain and the management of missing data. Transparency in reporting missing data is needed because approaches for adequately managing these gaps can differ and produce varying effect estimates [103].

If a dementia diagnosis is considered as dispensable or a criteria-based diagnostic procedure cannot be applied due to constrained resources, the study should not target the dementia population, but instead clearly state which population they are addressing.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and materials

All data are contained within the manuscript and its supplementary files.