Primary care physicians (PCPs) often daily address diagnoses and treatment of mental disorders in their practices. The current study examined the Connected Mind Fast Check (CMFC), a two-tiered electronic screen, assessing six common mental disorders. The eight-item Initial Screen assesses possible symptoms, whereas SAM modules establish provisional diagnoses and areas of clinical concern. With 234 patients from five independent PCP offices, diagnostic accuracy was tested with the SCID-5-RV as the external criterion. Concerningly, many patients were unaware of their current mental disorders and comorbidities. The CMFC Initial Screen evidenced strong sensitivity, identifying with very few missing diagnoses. About two-thirds of provisional SAM diagnoses were confirmed with high specificities. Bipolar Disorder posed the most challenges at both tiers. Importantly, the suicide screen identified all patients with suicide plans and three-fourths with ideation. In general, the CMFC effectively identified provisional diagnoses, impairment, and potential suicidality.
Globally, mental disorders continue to increase with an estimated 264 million people functionally impaired by depression alone (James et al., 2018). According to World Health Organization (WHO), most persons with mental disorders, including those in the USA, remain untreated in medical and psychiatric settings (Wang et al., 2007). The National Institute of Mental Health (NIMH, 2020) identified over 46 million adults that were diagnosed with at least one serious mental disorder. Diagnostic severity with its functional impairment must also be considered. For example, despite its low prevalence, 82.9% of persons with bipolar disorders were classified at the highest level of severity (Kessler et al., 2005a). Moreover, lifetime prevalences for four diagnostic categories are troubling: 28.8% for anxiety disorders, 24.8% for impulse control disorders, 20.8% for mood disorders, and 14.6% for substance use disorders (Kessler et al., 2005b).
The effects of mental disorders are far-reaching and marked with life-changing events, such as at greater risk for employment termination (Nelson & Kim, 2011; Olesen et al., 2013), victimization (Swartz & Bhattacharya, 2017), and suicide attempts (Brådvik, 2018). Financially, treatment costs associated with inpatient hospitalizations for serious mental disorders can exceed $5000 per week (Stensland et al., 2012). For patients, both access and cost barriers can substantially undermine the treatment process, that is, if they can afford to seek treatment at all (Andrade et al., 2014; Rowan et al., 2013; Russell, 2010).
Clearly, the prevalence, severity, and effects of mental disorders (e.g., Kroenke et al., 2007) underscore the compelling need to improve clinical methods for screening and assessment. From a multidisciplinary perspective, one valuable approach involves primary care physicians (PCPs), who may integrate mental health screens into their practices (Shedler et al., 2000).
Mental Disorders in Primary Care Settings
The majority of primary care practices specialize in family medicine, followed by general internal medicine and pediatrics (Petterson et al., 2018). In 2016, more than half (54.5%) of 883 million physician office visits in the USA involved PCPs (Rui & Okeyode, 2016) with many visits including mental health issues, such as depression and anxiety (Ashman et al., 2019; Olfson, 2016; Olfson et al., 1995). Primary care providers must often address a wide range of mental health concerns in their practices. More than three decades ago, Regler et al. (1978) insightfully recognized that primary care operated as the de facto mental health service for most patients (see also Kessler, 2005, 2005a, b, c). The need for mental health screening has become even more urgent in the last decade (Faghri et al., 2010; Kroenke & Unutzer, 2017; Weisberg et al., 2007).
This urgency is undoubtedly associated with patient access to care and the complex relationship between mental and physical health. Patients often experience a host of barriers, for example low income or limited insurance coverage, that prevent them from obtaining proper mental health treatment (Kohn et al., 2004; Rowan et al., 2013). Furthermore, research has generated ever-increasing evidence that poor psychological health is inextricably linked to physical illness (Pomerantz et al., 2009; Weiss et al., 2009). Regarding access, primary care settings often serve as the only point of contact for individuals experiencing mental health problems (Ross et al., 2015). Thus, PCPs—regardless of their specialty—play a crucial role in the diagnosis, treatment, and management of mental disorders (Phillips et al., 2011; WHO, 2018; Wittchen et al., 2003).
For physicians, assessing and treating behavioral health disorders may vary by type of diagnosis (Terry & Terry, 2019). For instance, a survey of primary care practices in the USA revealed most physicians felt generally more comfortable screening for depression (78%) and anxiety (70%; Beck et al., 2019). Conversely, more than half described little confidence treating patients with severe mental disorders, with similar concerns for bipolar and substance use disorders (Beck et al., 2019).
Diagnostic Challenges in Primary Care Settings
Primary care settings attempt to be responsive to patients’ expressed needs, but typically lack systematic methods for evaluating these needs, thus resulting in diagnostic errors. Two types of inaccuracies involve missed diagnoses (i.e., undetected diagnoses) and misdiagnoses (i.e., incorrect diagnoses) which remain as enduring problems in primary care settings. Classic US and European studies of major depression (Christensen et al., 2003; Löwe et al., 2004; Tiemens et al., 1999) illustrate this point; diagnosis of depression approximated a coin toss with missed diagnoses (54–60%) and misdiagnoses (48–54%; see Rogers & Shuman, 2005). In the last decade, undetected diagnoses have reached as high as 80% (Fernández et al., 2010). As a further complication, depression is sometimes correctly identified, but manic or hypomanic episodes are not recognized (Ghaemi et al., 1999; Zimmerman et al., 2008), leading to misdiagnoses of depression rather than bipolar disorders (Cerimele et al., 2019). Troublingly, Wittchen et al. (2003) described “diminishing halves” (p. 115) with about 50% being correctly diagnosed and just half of those receiving effective treatment.
Several factors substantially contribute to diagnostic inaccuracies in primary care settings. PCPs are understandably more focused on medical care and may overlook psychological distress, especially when faced with challenging medical conditions such as HIV (Asch et al., 2003), or for patients with multiple physical ailments (Grant et al., 2013; Loeb et al., 2012). Relatedly, time constraints during office visits may necessitate an almost exclusive focus on physical symptoms for patients with serious medical illnesses (Kroenke & Unutzer, 2017).
The intersectionality of mental and medical disorders can further contribute to overlooked diagnoses. For example, some medical conditions (e.g., dementia; Allan et al., 2014) mask depression and other mental disorders. Extensive comorbidity of mental disorders is more the rule than the exception (e.g., Pietrzak et al., 2012), further complicating diagnoses. Moreover, accurate diagnoses are also particularly difficult when patients lack insight and plausibly misattribute the physiological symptoms of mental disorders (e.g., muscle tension in GAD; Combs & Markman, 2014) to solely medical causes. As a final consideration, PCPs may understandably focus on the stated purpose of the visit without the time and resources to delve into other diagnostic matters (Russell, 2010). As discussed in the next section, time-efficient mental health screens may be used to assist PCPs by systematically checking for common mental disorders.
Approaches to Mental Health Assessment in Primary Care
By its very nature, primary care is characterized by effectiveness and efficiency. Toward this end, Spitzer et al. (1994) developed one of the first mental health screens for medical settings: The Primary Care Evaluation of Mental Disorders (PRIME-MD). Although the initial step was efficient (Spitzer et al., 1999), the follow-up interviews imposed substantial time demands on primary care staff. Moreover, its effectiveness varies with the level of severity for the designated mental disorders (Tamburrino et al., 2009).
Very brief screens may also help capture mental disorders and pressing clinical issues, such as suicidal thinking. Several self-report questionnaires, like the Patient Health Questionnaire (PHQ-9; Spitzer et al., 1999), were adapted from the PRIME-MD to balance the importance of standardization with the very real time restrictions in primary care (Kahn et al., 2004). Brief screens can also aid PCPs in monitoring the course and severity of a particular disorder (Kocalevent et al., 2013). Importantly, their implementation has become increasingly more common (e.g., Beck et al., 2019).
A review by Mulvaney-Day et al. (2018) identified 24 mental health screens that primarily focused on anxiety, depression, and substance use. The most popular measures were variants of the PRIME-MD with benefits of being brief and easily accessible (Lakkis & Mahmassani, 2015). However, paper-based screens require individual scoring by primary care staff and are often confined (e.g., the PHQ for depression) to a single diagnosis.
A Two-Tiered Electronic Approach to Mental Health Screening
The Connected Mind Fast Check (CMFC) is an electronic mental health screen developed in 2017 by Lehinger (see https://connectedmind.me/) and his multidisciplinary team including psychologists. Of practical importance, the CMFC is widely eligible for insurance re-imbursement for PCPs. The CMFC assesses six common mental disorders (Beck et al., 2019; Cerimele & Kern, 2017; Wittchen et al., 2003) frequently found in primary care settings: (a) Major Depressive Disorders (MDD), (b) Anxiety Disorders with a primary focus on Generalized Anxiety Disorder (GAD), (c) Bipolar Disorders, (d) Attention-Deficit/Hyperactivity Disorders (ADHD), (e) Somatic Symptom Disorders, and (f) Substance and Alcohol Use Disorders.
Similar to the PRIME-MD, the CMFC has adopted a two-tiered assessment process, which is administered entirely on a tablet. The Initial Screen constitutes the first tier with eight yes/no questions for the six disorders; patients with no affirmative responses do not warrant further assessment. Each disorder has at least on screening item; for the two most prevalent disorders (anxiety and depression), a second item was added to improve its screening accuracy. Any positive response on the Initial Screen immediately triggers the second tier with disorder-specific Standardized Assessment Modules (SAMs). They employ more comprehensive questions for identifying provisional DSM-5 diagnoses. The CMFC utilizes a continuous scoring algorithm for SAM modules. This prevents patients from being asked redundant questions. If responses warrant further consideration, then additional questions are provided. Thus, everything is standardized and consistently applied via the scoring algorithm. As a result, the CMFC has twin goals of accuracy and efficiency. Once completed, the CMFC also generates total scores and an interpretative report directly to the PCP’s assessment portal.
The CMFC has yet to be subjected to any peer-reviewed published research. Specifically, we recently (accessed on 7-9-21) searched for “Connected Mind Fast Check” and “CMFC” in the following data bases: Academic Search Complete, MEDLINE, PsychINFO, and Google Scholar. Although “CMFC” did occur, it was completely unrelated to mental health screens (e.g., Couples, Marriage, and Family Counseling). Moreover, a personal communication with its developer (Christian Lehinger, 7-9-21) failed to produce any unpublished reports. As subsequently described in the Current Study, the next step clearly involves independent research on the effectiveness of the CMFC with primary care patients.
The primary objective was to evaluate the classificatory accuracy of the Initial Screen and SAMs with the Structured Clinical Interview for DSM-5 Research Version (SCID-5-RV; First et al., 2018) as the external criteria. Both sensitivity and specificity are importantFootnote 1 and are applied differently to Initial Screen (step 1) and SAMs (step 2). For step 1, sensitivity is highlighted for the Initial Screen to make sure most patients with a specific disorder are administered the appropriate SAM. For step 2, specificity is emphasized to ensure that few patients are provisionally diagnosed with a specific disorder that they do not warrant. The application of these steps serves to improve both sensitivities and specificities. In addition to the provisional disorders, the CMFC suicide screen was also tested for its classificatory accuracy.
As the second objective, the current study evaluated an ANOVA-based discriminant validity of the CMFC provisional disorders. Provisional diagnoses (i.e., present or absent) constituted the independent variables with symptoms of the disorder being the dependent variables. For evidence of discriminant validity, patients with specific CMFC provisional diagnoses were expected to exhibit much greater symptom severity (i.e., large effect sizes) than their non-diagnosed counterparts. Two methodological considerations were implemented to improve the rigor of this research component. First, regarding effect sizes, the common convention is to utilize Cohen’s benchmarks (e.g., 0.50 for medium effect size) for group differences in social science research. However, these benchmarks are far too lax clinical practice; for example a d of 0.50 has a 67% overlap between criterion groups (Cohen, 1988, p. 22). Instead, rigorous clinical standards were applied to Cohen’s ds (see Rogers, 2008, 2018): for moderate (d ≥ 0.75), large (d ≥ 1.25), and very large (d ≥ 1.50) effect sizes. Second, although it is customary to use a single source for external criteria, we strengthened the design by adding a second measure. The SCID-5-RV was complemented with a comprehensive mental health screen, the Psychiatric Diagnostic Screening Questionnaire (PDSQ; Zimmerman & Mattia, 2001), which has 21 items for depression and suicidality alone.
The current study complied with the Standards for Reporting Diagnostic Accuracy Studies 2015 (STARD, 2015). The STARD 2015 consists of a 30-item checklist to address methodological rigor and to improve transparency and completeness of reporting data in studies of diagnostic accuracy (Bossuyt et al., 2015). Occasionally, STARD 2015 requires specific wording which is carefully denoted in the text (e.g., “index test” for the measure being validated). The full checklist is included in the supplementary material.
The cross-sectional, prospective design evaluated diagnostic accuracy and comorbidity at five PCP settings. As the index test, the effectiveness of the CMFC Initial Screen and SAMs in identifying common mental disorders was tested using SCID-RV diagnoses as the diagnostic reference standard. As later described, diagnostic efficiency statistics (e.g., sensitivity) were used to assess diagnostic accuracy.
From an initial group of 258 participants, the final sample was comprised of 234 consecutive patients recruited from five independent PCP offices in the Dallas-Ft. Worth Metroplex. Data from 24 participants were excluded because of missing electronic data due to poor internet connectivity (n = 23) or active psychosis (n = 1; see Fig. 1). With the one noted exception, all other adult patients were able to participate in the study irrespective of medical or psychiatric conditions. There were no other inclusion or exclusion criteria. Scheduled to end in May 2020, in-person data collection were administratively discontinued six weeks early by the university due to Covid-19 restrictions. As a result, the projected number decreased from 300 to 258 participants in total and 234 in final sample This number is generally consistent with other published studies on the diagnostic efficiency of mental health screens (see Vilagut et al., 2016).
Connected Mind Fast Check (CMFC)
The CMFC (personal communication with Christian Lehinger on 2/23/2021) is an electronic mental health screen for six common DSM-5 disorders. As previously described, it is composed of an Initial Screen and SAMs that automatically generate provisional diagnoses and clinical concerns (i.e., “red flags”), such as suicidal thinking. The Initial Screen consists of eight “yes/no” questions. The number of items on SAM modules range from 11 for GAD to 27 for MDD. The total number of SAM items administered are determined by the continuous scoring algorithm. Finally, relevant details (e.g., bereavement for depression) are provided for provisional diagnoses as well as severity scores for each administered SAM. The CMFC can usually be completed within 15 min and is automatically scored at the PCP’s secure assessment portal.
Structured Clinical Interview for DSM-5 Disorders–Research Version (SCID-5-RV)
The SCID-5-RV (First et al., 2018) is a semi-structured interview administered by trained mental health professionals. It is widely considered the most rigorous diagnostic assessment approach (Hersen et al., 2007) with the most recent version (SCID-5-Clinical Version or SCID-5-CV) representing the best external standard for research and publication (Brodey et al., 2018). The SCID-5-RV implemented additional modules and refined the SCID-CV scoring by providing an intermediary rating (i.e., 0 = false/absent, 1 = subthreshold, 2 = threshold/true). The SCID-5-CV, and by extension the SCID-5-RV, has excellent internal consistencies (alphas > 0.80; Shankman et al., 2018), interrater reliability, and validity (Osorio et al., 2019).
Psychiatric Diagnostic Screening Questionnaire (PDSQ)
The PDSQ (Zimmerman & Mattia, 2001) is a 125-item self-report inventory designed to initially evaluate 13 DSM-IV disorders. Responses in a “Yes/No” format evaluate symptoms in the last 2 weeks and 6 months. The PDSQ scales have evidenced moderate to high internal consistency (mean α = 0.82; Zimmerman & Mattia, 1999, 2001), and good concurrent validity with the SCID (Zimmerman & Chelminski, 2006).
Patient Health Questionnaire-9 (PHQ-9)
The PHQ-9 (Spitzer et al., 1999) is a 9-item screen of depressive symptoms experienced in the last two weeks. Severity is rated on a four-point scale (0 = not at all, 1 = several days, 2 = more than half the days, 3 = nearly every day). The PHQ-9 has demonstrated excellent internal consistency (α = 0.86; Kroenke & Spitzer, 2002), and criterion validity for depressed patients (Kroenke et al., 2001).
Patient Health Questionnaire-15 (PHQ-15)
The PHQ-15 (Kroenke et al., 1998) is a 15-item screen for the presence and impact of common physical symptoms. The negative effects of symptoms are rated on a three-point scale (0 = not at all, 1 = bothered a little, 2 = bothered a lot). The PHQ-15 has demonstrated adequate reliability in the general population (Kocalevent et al., 2013). However, it evidenced only modest sensitivity and specificity with somatic disorders (Ravesteijn et al. 2009).
Generalized Anxiety Disorder-7 (GAD-7)
The GAD-7 (Spitzer et al., 2006) is a seven-item screen for generalized anxiety with the same severity ratings as the PHQ-9. The GAD-7 evidenced good reliability and validity for anxiety in both general (Löwe et al., 2004) and clinical (Johnson et al., 2019) settings.
CAGE Adapted to Include Drugs (CAGE-AID)
The CAGE-AID (Brown & Rounds, 1995) is a 4-item screen related to alcohol and drug consumption, answered in a “Yes/No” format. It has moderate sensitivity and specificity (Brown & Rounds, 1995), and excellent concurrent validity (Basu et al., 2016).
Adult ADHD Self-Report Scale (ASRS-v1.1) Part A
The ASRS-v1.1 (Kessler et al., 2005a) Part A consists of a six-item screen of ADHD symptoms. Responses are rated for frequency during the last six months (0 = never, 1 = rarely, 2 = sometimes, 3 = often, 4 = very often). It has demonstrated good internal consistency and concurrent validity (Adler et al., 2012). It also has acceptable sensitivity for identifying possible ADHD cases (Hines et al., 2012).
Mood Disorder Questionnaire (MDQ)
The MDQ (Hirschfeld et al., 2000) is a “Yes/No” 15-item screen designed to assess symptoms consistent with Bipolar Disorder. “Yes” responses to #2 and six or more additional questions represents a “positive” screen, if at least moderate impairment is reported. The MDQ is more effective with Bipolar I Disorder (i.e., depression and mania) than Bipolar II Disorder (i.e., depression and hypomania; Dodd et al., 2009).
To facilitate data collection, researchers visited each PCP office to administer the measures over the span of close to one year (April 2019 to March 2020). Later in data collection (July 2019), six traditional single-diagnosis screens were implemented (see Sect. Ancillary Measures) with the potential goal of prescreening patients for single diagnoses. Given the extensive comorbidity, however, this approach was not feasible, and no patients were excluded from the study.
Outpatients attending medical appointments learned about the study via flyers. Initial patient questions were answered by a designated staff person with IRB-approved training. Next, interested patients met with a researcher, typically in the following week. After written informed consent, participants were then individually administered the SCID-5-RV diagnostic sections, followed by the CMFC and PDSQ in a counterbalanced order. As mentioned, single-diagnosis screens were implemented for eight months of data collection. Following data collection, patients were compensated with $60 for their travel, wait, and test time, which was estimated at about 2 h.
STARD 2015 requires the following information be reported. To avoid any criterion contamination, researchers administering the SCID-5-RV interviews (diagnostic reference standard) did not have access to CMFC results (index test) until after data collection were completed. For the timing of data collection, SCID-5-RV and CMFC administrations occurred with no time delay or clinical interventions. For participants, no adverse reactions were experienced that might alter their results during test administration. For data entry, two research assistants were utilized who played no role in data collection. For diagnostic efficiency statistics, data for all participants were classified (i.e., no indeterminate data were excluded).
Data Analytic Plan
For diagnostic efficiency statistics, the minimum number was set at 10 for each corresponding disorder. This benchmark is consistent with past SCID research (Osório et al., 2019). Diagnostic efficiency statistics (Riley et al., 2015; Wong and Lim, 2011) included sensitivity, specificity, positive predictive power (PPP), and negative predictive power (NPP). As noted, sensitivity measured the proportion of patients with a particular disorder correctly classified by the CMFC; conversely, specificity identified the proportion of patients without a particular disorder correctly classified. In addition, PPP measured the likelihood of patients with a positive CMFC finding (Initial Screen or SAM) warranting the corresponding SCID-5-RV diagnosis, whereas NPP estimated the likelihood of patients with a negative CMFC finding not having the diagnosis which is sometimes referred to as “non-cases.”
The final sample consisted of 149 female and 85 male patients with none self-identifying as gender non-binary or transgender. Patients averaged 47.10 years (SD = 16.43), ranging widely from 18 to 82. For self-identified ethnicity, most described themselves as European American (74.7%), followed by African American (11.0%), Hispanic American (9.4%), and Multiracial (4.9%). For marital status, the breakdown was 48.1% married, 30.9% never married, 16.3% divorced, and 4.7% other/unreported. Educational attainment varied considerably: 14.0% had less than high school, 58.8% graduated from high school, and 35.2% went beyond high school to earn at least an associate degree. Concerning past treatment for psychological issues, 42.3% had been treated for a specific mental disorder with a much smaller but appreciable percent (15.8%) being psychiatrically hospitalized.
Diagnoses and Comorbidity
In total, 171 patients (73.1%) warranted one or more of the SCID-5-RV diagnoses with considerable comorbidity being observed. As reported in Table 1, current diagnoses most frequently included GADs (25.6%) and MDDs (20.9%). For those diagnosed, comorbidity averaged 1.82 disorders. Because the study focused on 11 SCID-5-RV disorders, this average should be considered partial comorbidities, likely underestimating actual comorbidities. Interestingly, lifetime diagnoses showed a reverse pattern from current diagnoses, with MDDs (52.1%) outstripping GADs (34.2%).
Patient-reported psychiatric histories revealed striking differences when compared to current SCID-5-RV diagnoses. For example, 42.1% of the currently diagnosed patients did not report ever having a mental disorder. Moreover, 48.5% lacked insight into co-occurring disorders (i.e., reported only one diagnosis). If confirmed in PCP medical records, missed diagnoses appeared to be very common in the treatment of primary care patients.
CMFC Diagnostic Accuracy
Six SCID-5-RV diagnoses were categorized as corresponding disorders (i.e., closely aligned with the CMFC SAMs): MDDs, GADs, Alcohol Use Disorders, Substance Use Disorders, Bipolar Disorders, Somatic Symptom Disorders, and ADHD. The remaining diagnoses were characterized as related disorders.
The CMFC Initial Screen excluded from further consideration 61 participants who did not respond positively to any of its eight questions. As strong evidence of its effectiveness, 95.1% of those excluded did not warrant any of the six corresponding mental disorders.
As noted, the Initial Screen was intended to retain a high proportion of patients with possible disorders, which was largely achieved via sensitivities (weighted M = 0.89; see Table 2). For Bipolar Disorders, the sensitivity was only in the low moderate range. As a practical alternative, however, a positive screen for MDD could also be used clinically to trigger the Bipolar SAM because the MDD Initial Screen showed excellent sensitivity (0.94). With this simple step, the weighted M for sensitivities increases to 0.92.
The strong sensitivities are directly linked to high NPPs so that patients scoring below the cut score are almost always correctly classified as not having the corresponding disorders. With NPP ranging from 0.89 to 1.00 (see Table 2), weighted M of 0.97 is very high. In other words, the CMFC Initial Screen correctly classified patients as being without specific mental disorders (i.e., “non-cases”) in 97% of cases. Thus, most without current mental health concerns were quickly screened out easily in less than one minute.
For related disorders, Agoraphobia was intentionally omitted from Table 2 because of its very low prevalence (1.7%). For the remaining disorders, the NPPs yielded high estimates ranging from 0.86 to 0.96 with excellent sensitivities for two disorders (Persistent Depressive Disorders and Social Anxiety Disorders). Panic Disorder only evidenced moderate sensitivity, and the CMFC Initial Screen was least sensitive to specific phobias (0.68).
As an ancillary consideration, diagnostic accuracy of single-disorder screens was also assessed (see Supplementary Table S1). Regarding sensitivity, none of the single screen measures were considered optimal (i.e., ≥ 0.90). The CAGE-AID demonstrated the greatest sensitivity (0.86) and specificity (0.88). In contrast, the MDQ performed the worst with its sensitivity being no greater than chance (0.50), although its specificity was excellent (0.94). With the exception of the ASRS-v1.1 and MDQ, most screens illustrated moderate to strong specificity, and predominantly moderate to moderately strong sensitivity.
As noted, SAMs are intended to maintain higher specificities with at least moderate sensitivities for CMFC provisional diagnoses. See Fig. 1 for the diagnostic outcomes for the index test (i.e., SAMs) and the reference standard (i.e., SCID-5-RV diagnosis). This goal was mostly achieved; specificities for corresponding CMFC disorders averaged 0.912 (see Table 3). Sensitivities ranged from low (0.45 for MDD) to high (0.93 for Somatic Symptom Disorders). Overall, approximately two-thirds (66.2%) of CMFC provisional diagnoses were confirmed with SCID-5-RV corresponding diagnoses. Somatic Symptom Disorder and GAD evidenced the strongest CMFC performance with excellent sensitivities (0.93 and 0.90, respectively) and moderately strong specificities (0.86 for both). ADHD and Substance Use Disorder exhibited low-moderate sensitivities; however, their specificities were moderately strong (0.86) and excellent (0.96), respectively. The Bipolar SAM again illustrated lowest sensitivity (0.50), though its specificity was excellent (0.97).
As a preliminary (i.e., not aprioristic) analysis, CMFC decision rules for SAMs were reviewed for the MDD and GAD. As a preliminary result, sensitivities were substantially increased for MDD (from 0.45 to 0.73) and GAD (from 0.73 to 0.90) while only minimally affecting specificity. Obviously, further validation would be required before implementing any changes.
As a strength, the SAM’s diagnostic accuracy appeared to maintain excellent NPPs from the Initial Screen while the PPPs were higher than the Initial Screen. Provisional diagnoses on SAMs classifying the majority of patients ranged from 54% (ADHD) to 81% (Bipolar Disorders in depressed patients). Successful exclusion of outpatients without a corresponding SCID-5-RV disorder was excellent, ranging from 89% (Bipolar Disorders) to 98% (Somatic Symptom Disorder).
In final consideration, SAMs for related disorders were also evaluated via diagnostic efficiency statistics (Table 3). All sensitivities were very low to low (0.18 to 0.42), which was expected because the CMFC does not specifically assess for these disorders, and thus, would not be expected to identify them as provisional diagnoses.
Clinical indicators of suicide on the CMFC span from thoughts about death to intended suicide. Fortunately, no patients intended suicide, and very few had a specific plan (see Table 2). Sensitivities ranged from moderate (0.75) for suicidal thoughts to perfect (1.00) for suicidal plans. Of critical importance, no patients with suicide plans were missed.
Differences in Symptom Severity with and without CMFC Provisional Diagnoses
PCPs are interested in whether those provisionally diagnosed have markedly greater symptom severity—thus becoming a treatment priority—than others. Where feasible, differences in symptom severity were tested with both the SCID-5-RV and PDSQ with large and comparable effect sizes providing very strong evidence of discriminant validity. The two most common disorders (see Table 4) clearly provided this evidence: GADs (ds of 1.66 and 1.79) and MDDs (ds of 1.64 and 1.58). ADHD also exhibited an extremely large difference (d = 2.16) in symptom severity (i.e., 66.9% more intense symptoms for those with than without ADHD). More generally, large effect sizes were observed for Somatic Symptom Disorders, Alcohol Use Disorders, and Substance Use Disorders. Despite challenges in diagnostic accuracy, Bipolar Disorders yielded a moderately large (d = 1.02) effect size. Thus, patients provisionally diagnosed with Bipolar Disorders by the CMFC are likely to be in considerable distress.
Overlooked Mental Disorders and Comorbidity in Primary Care
Primary care patients appeared remarkably unaware of their diagnoses related to mental health. Regarding psychiatric histories, more than 40% were unmindful of any mental disorders at any time, despite warranting at least one current SCID-5-RV diagnosis. Even those aware almost always lacked knowledge of their own comorbidity. This lack of patient insight sharply constrains the ability to communicate effectively with physicians (Combs & Markman, 2014), and thus receive needed treatment. Based on past and current results, PCPs are cautioned against relying solely on reported mental disorders by primary care patients. Over-reliance on patient reports may grossly underestimate the presence of many treatable conditions, which remain undiagnosed, and thus, untreated (e.g., see Wittchen et al., 2003).
The current study also found high rates of co-occurring disorders in primary care patients. Kessler (2005, 2005a, b, c) reviewed the highly consequential effects of psychological comorbidity in medical settings, including decreased physical health (Jansen et al., 2018), more complex case management (Loeb et al., 2012), and more costly medical services (Lerner et al., 2004). As previously noted, untreated mental disorders are also linked to additional medical illnesses (Bijl et al., 2003; De Hert et al., 2011) and exacerbation of existing medical disorders. Thus, undiagnosed mental disorders and overlooked comorbidity may create serious ripple effects on the overall health of primary care patients. Given the bidirectional relationship between mental and physical health (Glew & Chapman, 2016), further integration of mental health screening and treatment into primary care settings is needed (De Hert et al., 2011; Phillips et al., 2011).
PCPs have multiple options on how they implement mental health screens into their medical practices. There are clear differences in the comparative effectiveness of such screens (e.g., Henkel et al., 2004). Nonetheless, the take-home message is compelling: Any systematic screens for mental health issues are far superior to the absence of such methods. This point appears equally persuasive in light of missed diagnoses and misdiagnoses.
Effectiveness of the CMFC in Primary Care Patients
The current study evaluated the diagnostic accuracy of the CMFC Initial Screen and SAMs in identifying common mental health disorders for primary care patients. As noted, diagnostic efficiency statistics, strong sensitivities are recommended for preliminary screens so that potential cases are not missed, and higher specificities are indicated for more comprehensive measures to decrease misdiagnoses (Ren et al., 2015).
Utility of the CMFC Initial Screen and Suicide Screen
The CMFC Initial Screen worked exceptionally well in accurately identifying patients with specific mental disorders while missing very few “false negatives” (i.e., patients with the disorder not identified by the screen). In other words, the CMFC Initial Screen mostly achieved its intended goal to accurately screen-in patients and markedly reducing missed diagnoses.
In the current study, the CMFC Initial Screen performed comparably to and sometimes stronger than traditional screens (e.g., PHQ-9) used in primary care settings (Mulvaney-Day et al., 2018). For example, the Initial Screen for Somatic Symptom Disorder displayed perfect sensitivity. In contrast, the longer PHQ-15 for somatic symptoms revealed comparatively lower diagnostic efficiency statistics (Ravesteijn et al. 2009; Toussaint et al., 2020; Vroege et al., 2012). Similarly, the CMFC Initial Screen appeared to surpass the PHQ for depression (see meta-analysis by Manea et al., 2015), the GAD-7 for anxiety (Plummer et al., 2016; Zhong et al., 2015), and the ASRS–v1.1 for ADHD (Dunlop et al., 2018; Kessler et al., 2005c). The notable exception on the CMFC Initial Screen is for Bipolar Disorder, although its limitations are generally shared by all screening tools for this disorder (e.g., Miller et al., 2004, 2011). It evidenced more accuracy than the MDQ, the most widely used assessment for Bipolar Disorder in primary care (see Kilbourne et al., 2012).
Beyond sensitivity and specificity, the Initial Screen’s impressive NPPs also have important practical implications. The CMFC was able to successfully screen-out patients without a mental health disorder. For physicians, time is a very real concern during office visits (Kroenke & Unutzer, 2017). In less than one minute, the CMFC can aid PCPs in quickly ruling-out common mental disorders.
Finally, the CMFC suicide screen proved effective at identifying patients with varying degrees of suicidal ideation. The inclusion of suicide risk assessment can be critically important to early and effective management. Regrettably, some very brief screens have omitted suicide items (e.g., PHQ-2; Duewke et al., 2018) with the potential for tragic outcomes. On this point, a review of medical records in the USA revealed that nearly half of suicide decedents sought healthcare services within the month prior to their deaths (Ahmedani et al., 2014). As a related matter, screens may allow primary care patients to acknowledge suicidal thinking, perhaps easing the discomfort of directly disclosing suicidal ideation to healthcare professionals (King et al., 2017).
Provisional Diagnoses with the CMFC SAMs
As previously discussed, the diagnostic efficiency of the SAMs focused on increased specificity (Ren et al., 2015). As expected, the majority of the SAMs performed strongly regarding specificity with substantially improved PPP, meaning the diagnostic accuracy of identifying patients with a particular disorder increased. Moreover, excellent specificities and NPPs indicate the SAMs correctly ruled-out most patients without these mental disorders.
Beyond diagnoses per se, PCPs would benefit from clinical data that documents increased symptom severity frequently associated with psychological distress and impairment. For the CMFC SAMs, comparing data from two different evaluative methods (i.e., structured interview and written questionnaire) produced strong and generally consistent evidence of discriminant validity. On average, effect sizes are very large for both the SCID-5-RV (M d = 1.46) and PDSQ (M d = 1.53). Promisingly, the Bipolar SAM—the most challenging module—still demonstrated moderately large differences in symptom severity between patients with and without this provisional diagnosis. Generally, preliminary diagnoses on the SAMs effectively differentiated between patients experiencing varying level of distress.
As a methodological strength, the current procedures preserved patient confidentiality and provided independent administrations to avoid any criterion contamination. With the sole exception—clearly noted on informed consent—the SCID-5-RV interviewer could break confidentiality for actively suicidal patients. Fortunately, this psychiatric emergency did not occur, and confidentiality was never broken.
A second strength of the current research is its reliance on aprioristic decision rules for CMFC provisional diagnoses. Thus, the resulting classifications are not influenced by current clinical findings. That said, two preliminary scoring rules for MDD and GAD should be rigorously evaluated in future research.
A basic limitation of the current cross-sectional design is that changes in clinical status could not be evaluated. It would be useful to know whether the CMFC would be effective at assessing improvements as well as deteriorations in common mental disorders. This research would allow current and past episodes to be examined separately. Moreover, in patients with stable disorders, a short-term follow-up could assess the stability of CMFC findings via test–retest reliability.
Methodologically, the design could be modified so that all the CMFC SAMs were administered to every participant. This modification would allow comorbidity to be examined more thoroughly on the CMFC. While test items are typically scored in a positive direction (i.e., the presence of a symptom), the absence of certain items may be instructive, which could be tested if all modules were administered. For example, a negative response to an item such as “I wish my life was more interesting and stimulating,” might help to rule-out Bipolar Disorders.
Future CMFC research could also consider its usefulness in other health care and professional settings, such hospitals. When faced with public health crises, such as Covid-19, CMFC online applications may be appropriate and timely. On this particular point, a free online CMFC version was created in response to Covid-19. Additionally, adapted versions could be empirically tested for school settings and institutions for higher education.
PCPs need efficient and effective tools, such as mental health screens, to accurately inform their busy and often time-constrained practices. The CMFC may play an indispensable role in addressing undiagnosed mental disorders with its automatic scoring and electronic generation of reports. With re-administrations, it may help track changes in symptom severity over the course of treatment. In addition, the CMFC suicide screen is effective at identifying patients with suicide plans. In closing, PCPs need resources such as mental health screens to treat the whole person addressing physical and mental health needs and the often-dynamic interactions between the two.
Sensitivity is the proportion with a specific disorder that are correctly identified, whereas specificity is the proportion without the specific disorder that are correctly identified.
Adler, L. A., Shaw, D. M., Spencer, T. J., Newcorn, J. H., Hammerness, P., Sitt, D. J., … Faraone, S. V. (2012). Preliminary examination of the reliability and concurrent validity of the attention deficit/hyperactivity disorder self-report scale v1.1 symptom checklist to rate symptoms of attention deficit/hyperactivity disorder in adolescents. Journal of Child and Adolescent Psychopharmacology, 22(3), 238–244. https://doi.org/10.1089/cap.2011.0062
Ahmedani, B. K., Simon, G. E., Stewart, C., Beck, A., Waitzfelder, B. E., Rossom, R., … Solberg, L. I. (2014). Health care contacts in the year before suicide death. Journal of General Internal Medicine, 29(6), 870–877. https://doi.org/10.1007/s11606-014-2767-3
Allan, C. E., Valkanova, V., & Ebmeier, K. P. (2014). Depression in older people is underdiagnosed. The Practitioner, 258(1771), 19–22.
Andrade, L. H., Alonso, J., Mneimneh, Z., Wells, J. E., Al-Hamzawi, A., Borges, G., … Kessler, R. C. (2014). Barriers to mental health treatment: Results from the WHO world mental health surveys. Psychological Medicine, 44(6), 1303–1317. https://doi.org/10.1017/S0033291713001943
Asch, S. M., Kilbourne, A. M., Gifford, A. L., Burnam, M. A., Turner, B., Shapiro, M. F., & Bozzette, S. A. (2003). Underdiagnosis of depression in HIV. Journal of General Internal Medicine, 18(6), 450–460.
Ashman, J. J., Rui, P., & Okeyode, T. (2019). Characteristics of office-based physician visits, 2016. NCHS Data Brief, 331, 1–8.
Basu, D., Ghosh, A., Hazari, N., & Parakh, P. (2016). Use of family CAGE-AID questionnaire to screen the family members for diagnosis of substance dependence. The Indian Journal of Medical Research, 143(6), 722–730. https://doi.org/10.4103/0971-5916.191931
Beck, A. J., Page, C., Buche, J., Schoebel, V., & Wayment, C. (2019). Behavioral health service provision by primary care physicians. University of Michigan Behavioral Health Workforce Research Center. Retrieved February 1, 2021 from https://behavioralhealthworkforce.org/wp-content/uploads/2019/12/Y4-P10-BH-Capacityof-PC-Phys_Full.pdf
Bijl, R. V., de Graaf, R., Hiripi, E., Kessler, R. C., Kohn, R., Offord, D. R., … Wittchen, H. U. (2003). The prevalence of treated and untreated mental disorders in five countries. Health Affairs, 22(3), 122–133. https://doi.org/10.1377/hlthaff.22.3.122
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L., … Cohen, J. F. (2015). STARD 2015: An updated list of essential items for reporting diagnostic accuracy studies. BMJ (clinical Research Ed.), 351, h5527. https://doi.org/10.1136/bmj.h5527
Brådvik, L. (2018). Suicide risk and mental disorders. International Journal of Environmental Research and Public Health, 15(9), 2028. https://doi.org/10.3390/ijerph15092028
Brodey, B., Purcell, S. E., Rhea, K., Maier, P., First, M., Zweede, L., … Brodey, I. S. (2018). Rapid and accurate behavioral health diagnostic screening: Initial validation study of a web-based, self-report tool (the SAGE-SR). Journal of Medical Internet Research, 20(3), e108. https://doi.org/10.2196/jmir.9428
Brown, R. L., & Rounds, L. A. (1995). Conjoint screening questionnaires for alcohol and other drug abuse: Criterion validity in a primary care practice. Wisconsin Medical Journal, 94(3), 135–140.
Cerimele, J. M., & Kern, J. S. (2017). Bipolar disorder in primary care: Integrated care experiences. Focus, 15(3), 244–248. https://doi.org/10.1176/appi.focus.20170005
Cerimele, J. M., Fortney, J. C., Pyne, J. M., & Curran, G. M. (2019). Bipolar disorder in primary care: A qualitative study of clinician and patient experiences with diagnosis and treatment. Family Practice, 36(1), 32–37. https://doi.org/10.1093/fampra/cmy019
Christensen, K. S., Toft, T., Frostholm, L., Ørnbol, E., Fink, P., & Olesen, F. (2003). The FIP study: A randomized, controlled trial of screening and recognition of psychiatric disorders. British Journal of General Practice, 53(495), 758–763.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
Combs, H., & Markman, J. (2014). Anxiety disorders in primary care. Medical Clinics, 98(5), 1007–1023. https://doi.org/10.1016/j.mcna.2014.06.003
De Hert, M., Correll, C. U., Bobes, J., Cetkovich-Bakmas, M., Cohen, D. A. N., Asai, I., … Newcomer, J. W. (2011). Physical illness in patients with severe mental disorders. I. Prevalence, impact of medications and disparities in health care. World Psychiatry, 10(1), 52–77.
Dodd, S., Williams, L. J., Jacka, F. N., Pasco, J. A., Bjerkeset, O., & Berk, M. (2009). Reliability of the mood disorder questionnaire: Comparison with the structured clinical interview for the DSM-IV-TR in a population sample. Australian & New Zealand Journal of Psychiatry, 43(6), 526–530. https://doi.org/10.1080/00048670902873706
Dueweke, A. R., Marin, M. S., Sparkman, D. J., & Bridges, A. J. (2018). Inadequacy of the PHQ-2 depression screener for identifying suicidal primary care patients. Families, Systems, and Health, 36(3), 281–288. https://doi.org/10.1037/fsh0000350
Dunlop, B. W., Wu, R., & Helms, K. (2018). Performance of the adult ADHD self-report scale-v1. 1 in adults with major depressive disorder. Behavioral Sciences, 8(4), 37–51. https://doi.org/10.3390/bs8040037
Faghri, N. M. A., Boisvert, C. M., & Faghri, S. (2010). Understanding the expanding role of primary care physicians (PCPs) to primary psychiatric care physicians (PPCPs): Enhancing the assessment and treatment of psychiatric conditions. Mental Health in Family Medicine, 7(1), 17–25.
Fernández, A., Pinto-Meza, A., Bellón, J. A., Roura-Poch, P., Haro, J. M., Autonell, J., … Serrano-Blanco, A. (2010). Is major depression adequately diagnosed and treated by general practitioners? Results from an epidemiological study. General Hospital Psychiatry, 32(2), 201–209. https://doi.org/10.1016/j.genhosppsych.2009.11.015
First, M. B., Williams, J. B. W., Karg, R. S., Spitzer, R. L., Meyer, E. C., Walser, R., … Schnurr, P. P. (2018). Structured clinical interview for DSM-5-research version. Journal of Traumatic Stress, 31, 781–789.
Ghaemi, S. N., Sachs, G. S., Chiou, A. M., Pandurangi, A. K., & Goodwin, F. K. (1999). Is bipolar disorder still underdiagnosed? Are antidepressants overutilized? Journal of Affective Disorders, 52(1–3), 135–144. https://doi.org/10.1016/S0165-0327(98)00076-7
Glew, S., & Chapman, B. (2016). Closing the gap between physical and mental health training. The British Journal of General Practice: The Journal of the Royal College of General Practitioners, 66(651), 506–507. https://doi.org/10.3399/bjgp16X687157
Grant, R. W., Adams, A. S., Bayliss, E. A., & Heisler, M. (2013). Establishing visit priorities for complex patients: A summary of the literature and conceptual model to guide innovative interventions. Healthcare, 1(3–4), 117–122.
Henkel, V., Mergl, R., Kohnen, R., Allgaier, A. K., Möller, H. J., & Hegerl, U. (2004). Use of brief depression screening tools in primary care: Consideration of heterogeneity in performance in different patient groups. General Hospital Psychiatry, 26(3), 190–198. https://doi.org/10.1016/j.genhosppsych.2004.02.003
Hersen, M., Turner, S. M., & Beidel, D. C. (Eds.). (2007). Adult psychopathology and diagnosis (5th ed.). Wiley.
Hines, J. L., King, T. S., & Curry, W. J. (2012). The adult ADHD self-report scale for screening for adult attention deficit-hyperactivity disorder (ADHD). The Journal of the American Board of Family Medicine, 25(6), 847–853. https://doi.org/10.3122/jabfm.2012.06.120065
Hirschfeld, R. M., Williams, J. B., Spitzer, R. L., Calabrese, J. R., Flynn, L., Keck, P. E., Jr., … Russell, J. M. (2000). Development and validation of a screening instrument for bipolar spectrum disorder: The mood disorder questionnaire. American Journal of Psychiatry, 157(11), 1873–1875. https://doi.org/10.1176/appi.ajp.157.11.1873
James, S. L., Abate, D., Abate, K. H., Abay, S. L., Abbafati, C., Abbasi, N., … Murray, C. J. L. (2018). Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: A systematic analysis for the global burden of disease study 2017. Lancet, 392(10159), 1789–1858. https://doi.org/10.1016/S0140-6736(18)32279-7
Jansen, L., van Schijndel, M., van Waarde, J., & van Busschbach, J. (2018). Health-economic outcomes in hospital patients with medical-psychiatric comorbidity: A systematic review and meta-analysis. PLoS ONE, 13(3), e0194029. https://doi.org/10.1371/journal.pone.0194029
Johnson, S. U., Ulvenes, P. G., Øktedalen, T., & Hoffart, A. (2019). Psychometric properties of the general anxiety disorder 7-item (GAD-7) scale in a heterogeneous psychiatric sample. Frontiers in Psychology, 10(1713), 1–8. https://doi.org/10.3389/fpsyg.2019.01713
Kahn, L. S., Halbreich, U., Bloom, M. S., Bidani, R., Rich, E., & Hershey, C. O. (2004). Screening for mental illness in primary care clinics. International Journal of Psychiatry in Medicine, 34(4), 345–362.
Kessler, R. (2005). Treating psychological problems in medical settings: Primary care as the de facto mental health system and the role of hypnosis. The International Journal of Clinical and Experimental Hypnosis, 53(3), 290–305.
Kessler, R. C., Adler, L., Ames, M., Demler, O., Faraone, S., Hiripi, E. V. A., … Walters, E. E. (2005a). The world health organization adult ADHD self-report scale (ASRS): A short screening scale for use in the general population. Psychological Medicine, 35(2), 245–256. https://doi.org/10.1017/S0033291704002892
Kessler, R. C., Berglund, P., Demler, O., Jin, R., Merikangas, K. R., & Walters, E. E. (2005b). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the national comorbidity survey replication. Archives of General Psychiatry, 62(6), 593–602. https://doi.org/10.1001/archpsyc.62.6.593
Kessler, R. C., Chiu, W. T., Demler, O., & Walters, E. E. (2005c). Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the national comorbidity survey replication. Archives of General Psychiatry, 62(6), 617–627. https://doi.org/10.1001/archpsyc.62.6.617
Kilbourne, A. M., Goodrich, D. E., O’Donnell, A. N., & Miller, C. J. (2012). Integrating bipolar disorder management in primary care. Current Psychiatry Reports, 14(6), 687–695. https://doi.org/10.1007/s11920-012-0325-4
King, C. A., Horwitz, A., Czyz, E., & Lindsay, R. (2017). Suicide risk screening in healthcare settings: Identifying males and females at risk. Journal of Clinical Psychology in Medical Settings, 24(1), 8–20. https://doi.org/10.1007/s10880-017-9486-y
Kocalevent, R. D., Hinz, A., & Brähler, E. (2013). Standardization of a screening instrument (PHQ-15) for somatization syndromes in the general population. BMC Psychiatry, 13(1), 91. https://doi.org/10.1186/1471-244X-13-91
Kohn, R., Saxena, S., Levav, I., & Saraceno, B. (2004). The treatment gap in mental health care. Bulletin World Health Organization, 82(11), 858–866.
Kroenke, K., & Spitzer, R. L. (2002). The PHQ-9: A new depression diagnostic and severity measure. Psychiatric Annals, 32(9), 509–515. https://doi.org/10.3928/0048-5713-20020901-06
Kroenke, K., Spitzer, R. L., & deGruySwindle, F. V. R. (1998). A symptom checklist to screen for somatoform disorders in primary care. Psychosomatics, 39(3), 263–272. https://doi.org/10.1016/S0033-3182(98)71343-X
Kroenke, K., Spitzer, R. L., & Williams, J. B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x
Kroenke, K., Spitzer, R. L., Williams, J. B., Monahan, P. O., & Löwe, B. (2007). Anxiety disorders in primary care: Prevalence, impairment, comorbidity, and detection. Annals of Internal Medicine, 146(5), 317–325.
Kroenke, K., & Unutzer, J. (2017). Closing the false divide: Sustainable approaches to integrating mental health services into primary care. Journal General Internal Medicine, 32(4), 404–410. https://doi.org/10.1007/s11606-016-3967-9
Lakkis, N. A., & Mahmassani, D. M. (2015). Screening instruments for depression in primary care: A concise review for clinicians. Postgraduate Medicine, 127(1), 99–106.
Lerner, D., Adler, D. A., Chang, H., Lapitsky, L., Hood, M. Y., Perissinotto, C., … Rogers, W. H. (2004). Unemployment, job retention, and productivity loss among employees with depression. Psychiatric Services, 55(12), 1371–1378. https://doi.org/10.1176/appi.ps.55.12.1371
Loeb, D. F., Bayliss, E. A., Binswanger, I. A., Candrian, C., & de Gruy, F. V. (2012). Primary care physician perceptions on caring for complex patients with medical and mental illness. Journal of General Internal Medicine, 27(8), 945–952. https://doi.org/10.1007/s11606-012-2005-9
Löwe, B., Spitzer, R. L., Gräfe, K., Kroenke, K., Quenter, A., Zipfel, S., … Herzog, W. (2004). Comparative validity of three screening questionnaires for DSM-IV depressive disorders and physicians’ diagnoses. Journal of Affective Disorders, 78(2), 131–140. https://doi.org/10.1016/s0165-0327(02)00237-9
Manea, L., Gilbody, S., & McMillan, D. (2015). A diagnostic meta-analysis of the Patient Health Questionnaire-9 (PHQ-9) algorithm scoring method as a screen for depression. General Hospital Psychiatry, 37(1), 67–75. https://doi.org/10.1016/j.genhosppsych.2014.09.009
Miller, C. J., Johnson, S. L., Kwapil, T. R., & Carver, C. S. (2011). Three studies on self-report scales to detect bipolar disorder. Journal of Affective Disorders, 128(3), 199–210. https://doi.org/10.1016/j.jad.2010.07.012
Miller, C. J., Klugman, J., Berv, D. A., Rosenquist, K. J., & Ghaemi, S. N. (2004). Sensitivity and specificity of the Mood Disorder Questionnaire for detecting bipolar disorder. Journal of Affective Disorders, 81(2), 167–171. https://doi.org/10.1016/S0165-0327(03)00156-3
Mulvaney-Day, N., Marshall, T., Downey Piscopo, K., Korsen, N., Lynch, S., Karnell, L. H., … Ghose, S. S. (2018). Screening for behavioral health conditions in primary care settings: A systematic review of the literature. Journal of General Internal Medicine, 33(3), 335–346. https://doi.org/10.1007/s11606-017-4181-0
National Institute of Mental Health (2020). Mental health information. Retrieved February 1, 2021 from https://www.nimh.nih.gov/health/statistics/mental-illness
Nelson, R. E., & Kim, J. (2011). The impact of mental illness on the risk of employment termination. The Journal of Mental Health Policy and Economics, 14(1), 39–52. Retrieved August 2, 2020 from http://www.icmpe.org/test1/journal/journal.htm
Olesen, S. C., Butterworth, P., Leach, L. S., Kelaher, M., & Pirkis, J. (2013). Mental health affects future employment as job loss affects mental health: Findings from a longitudinal population study. BMC Psychiatry, 13(1), 144. https://doi.org/10.1186/1471-244X-13-144
Olfson, M. (2016). The rise of primary care physicians in the provision of US mental health care. Journal of Health Politics, Policy and Law, 41(4), 559–583. https://doi.org/10.1215/03616878-3620821
Olfson, M., Gilbert, T., Weissman, M., Blacklow, R. S., & Broadhead, W. E. (1995). Recognition of emotional distress in physically healthy primary care patients who perceive poor physical health. General Hospital Psychiatry, 17(3), 173–180. https://doi.org/10.1016/0163-8343(95)00023-K
Osório, F. L., Loureiro, S. R., Hallak, J. E. C., Machado-de-Sousa, J. P., Ushirohira, J. M., Baes, C. V. W., & Crippa, J. A. S. (2019). Clinical validity and intrarater and test-retest reliability of the Structured Clinical Interview for DSM-5—Clinician Version (SCID-5-CV). Psychiatry and Clinical Neurosciences, 73(12), 754–760. https://doi.org/10.1111/pcn.12931
Petterson, S., McNellis, R., Klink, K., Meyers, D., & Bazemore, A. (2018). The state of primary care in the United States: A chartbook of facts and statistics. Robert Graham Center.
Phillips, R. L., Jr., Miller, B. F., Petterson, S., & Teevan, B. (2011). Better integration of mental health care improves depression screening and treatment in primary care. American Family Physician, 84(9), 980.
Pietrzak, R. H., Goldstein, R. B., Southwick, S. M., & Grant, B. F. (2012). Psychiatric comorbidity of full and partial posttraumatic stress disorder among older adults in the United States: Results from wave 2 of the national epidemiologic survey on alcohol and related conditions. The American Journal of Geriatric Psychiatry, 20(5), 380–390. https://doi.org/10.1097/JGP.0b013e31820d92e7
Plummer, F., Manea, L., Trepel, D., & McMillan, D. (2016). Screening for anxiety disorders with the GAD-7 and GAD-2: A systematic review and diagnostic meta-analysis. General Hospital Psychiatry, 39, 24–31. https://doi.org/10.1016/j.genhosppsych.2015.11.005
Pomerantz, A. S., Corson, J. A., & Detzer, M. J. (2009). The challenge of integrated care for mental health: Leaving the 50 minute hour and other sacred things. Journal of Clinical Psychology in Medical Settings, 16(1), 40–46. https://doi.org/10.1007/s10880-009-9147-x
Ravesteijn, H., Wittkampf, K., Lucassen, P., van de Lisdonk, E., van den Hoogen, H., van Weert, H., … Speckens, A. (2009). Detecting somatoform disorders in primary care with the PHQ-15. The Annals of Family Medicine, 7(3), 232–238. https://doi.org/10.1370/afm.985
Regler, D., Goldberg, I., & Taube, C. (1978). The de facto mental health services system. Archives of General Psychiatry, 35, 685–693. https://doi.org/10.1001/archpsyc.1978.01770300027002
Ren, Y., Yang, H., Browning, C., Thomas, S., & Liu, M. (2015). Performance of screening tools in detecting major depressive disorder among patients with coronary heart disease: A systematic review. Medical Science Monitor: International Medical Journal of Experimental and Clinical Research, 21, 646–653. https://doi.org/10.12659/MSM.892537
Riley, R. D., Ahmed, I., Debray, T. P., Willis, B. H., Noordzij, J. P., Higgins, J. P., & Deeks, J. J. (2015). Summarizing and validating test accuracy results across multiple studies for use in clinical practice. Statistics in Medicine, 34(13), 2081–2103. https://doi.org/10.1002/sim.6471
Rogers, R. (2008). Detection strategies for malingering and defensiveness. In R. Rogers (Ed.), Clinical assessment of malingering and deception (3rd ed., pp. 14–35). Guilford Press.
Rogers, R. (2018). Detection strategies for malingering and defensiveness. In R. Rogers & S. D. Bender (Eds.), Clinical assessment of malingering and deception (4th ed., pp. 18–41). Guilford Press.
Rogers, R., & Shuman, D. W. (2005). Fundamentals of forensic practice: Mental health and criminal law. Springer Science + Business Media.
Ross, L. E., Vigod, S., Wishart, J., Waese, M., Spence, J. D., Oliver, J., … Shields, R. (2015). Barriers and facilitators to primary care for people with mental health and/or substance use issues: A qualitative study. BMC Family Practice, 16, 135–135. https://doi.org/10.1186/s12875-015-0353-3
Rowan, K., McAlpine, D. D., & Blewett, L. A. (2013). Access and cost barriers to mental health care, by insurance status, 1999–2010. Health Affairs, 32(10), 1723–1730. https://doi.org/10.1377/hlthaff.2013.0133
Rui, P., & Okeyode, T. (2016). National ambulatory medical care survey: 2016 national summary tables. Centers for Disease Control and Prevention. Retrieved August 2, 2020 from https://www.cdc.gov/nchs/data/ahcd/namcs_summary/2016_namcs_web_tables.pdf
Russell, L. (2010). Mental health care services in primary care: Tackling the issues in the context of health care reform. Center for American Progress.
Shankman, S. A., Funkhouser, C. J., Klein, D. N., Davila, J., Lerner, D., & Hee, D. (2018). Reliability and validity of severity dimensions of psychopathology assessed using the structured clinical interview for DSM-5 (SCID-5). International Journal of Methods in Psychiatric Research, 27(1), e1590. https://doi.org/10.1002/mpr.1590
Shedler, J., Beck, A., & Bensen, S. (2000). Practical mental health assessment in primary care. Journal of Family Practice, 49(7), 614–622.
Spitzer, R. L., Kroenke, K., & Williams, J. B. (1999). Validation and utility of a self-report version of PRIME-MD: The PHQ primary care study. Primary care evaluation of mental disorders. Patient health questionnaire. JAMA, 282(18), 1737–1744. https://doi.org/10.1001/jama.282.18.1737
Spitzer, R. L., Kroenke, K., Williams, J. B., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097. https://doi.org/10.1001/archinte.166.10.1092
Spitzer, R. L., Williams, J. B., Kroenke, K., Linzer, M., deGruy, F. V., Hahn, S. R., … Johnson, J. G. (1994). Utility of a new procedure for diagnosing mental disorders in primary care. The PRIME-MD 1000 study. JAMA, 272(22), 1749–1756. https://doi.org/10.1001/jama.272.22.1749
Stensland, M., Watson, P. R., & Grazier, K. L. (2012). An examination of costs, charges, and payments for inpatient psychiatric treatment in community hospitals. Psychiatric Services, 63(7), 666–671. https://doi.org/10.1176/appi.ps.201100402
Swartz, M. S., & Bhattacharya, S. (2017). Victimization of persons with severe mental illness: A pressing global health problem. World Psychiatry, 16(1), 26–27. https://doi.org/10.1002/wps.20393
Tamburrino, M. B., Lynch, D. J., Nagel, R. W., & Smith, M. K. (2009). Primary care evaluation of mental disorders (PRIME-MD) screening for minor depressive disorder in primary care. Primary Care Companion to the Journal of Clinical Psychiatry, 11(6), 339–343. https://doi.org/10.4088/PCC.08.m00711
Terry, D. L., & Terry, C. P. (2019). Addressing mental health concerns in primary care: Practices among medical residents in a rural setting. Journal of Clinical Psychology in Medical Settings, 26(4), 395–401. https://doi.org/10.1007/s10880-019-09609-3
Tiemens, B. G., Vonkorff, M., & Lin, E. H. (1999). Diagnosis of depression by primary care physicians versus a structured diagnostic interview. General Hospital Psychiatry, 21(2), 87–96. https://doi.org/10.1016/s0163-8343(98)00077-2
Toussaint, A., Hüsing, P., Kohlmann, S., & Löwe, B. (2020). Detecting DSM-5 somatic symptom disorder: Criterion validity of the patient health questionnaire-15 (PHQ-15) and the Somatic Symptom Scale-8 (SSS-8) in combination with the somatic symptom disorder–B criteria scale (SSD-12). Psychological Medicine, 50(2), 324–333. https://doi.org/10.1017/S003329171900014X
Vilagut, G., Forero, C. G., Barbaglia, G., & Alonso, J. (2016). Screening for depression in general population with the Center for Epidemiologic Studies Depression (CES-D): A systematic review with meta-analysis. PLoS ONE, 11(5), e0155431. https://doi.org/10.1371/journal.pone.0155431
Vroege, L., Hoedeman, R., Nuyen, J., Sijtsma, K., & van der Feltz-Cornelis, C. M. (2012). Validation of the PHQ-15 for somatoform disorder in the occupational health care setting. Journal of Occupational Rehabilitation, 22(1), 51–58. https://doi.org/10.1007/s10926-011-9320-6
Wang, P. S., Aguilar-Gaxiola, S., Alonso, J., Angermeyer, M. C., Borges, G., Bromet, E. J., … Elisabeth Wells, J. (2007). Use of mental health services for anxiety, mood, and substance disorders in 17 countries in the WHO world mental health surveys. Lancet, 370(9590), 841–850. https://doi.org/10.1016/S0140-6736(07)61414-7
Weisberg, R. B., Dyck, I., Culpepper, L., & Keller, M. B. (2007). Psychiatric treatment in primary care patients with anxiety disorders: A comparison of care received from primary care providers and psychiatrists. The American Journal of Psychiatry, 164(2), 276–282. https://doi.org/10.1176/ajp.2007.164.2.276
Weiss, S. J., Haber, J., Horowitz, J. A., Stuart, G. W., & Wolfe, B. (2009). The inextricable nature of mental and physical health: Implications for integrative care. Journal of the American Psychiatric Nurses Association, 15(6), 371–382. https://doi.org/10.1177/1078390309352513
Wittchen, H. U., Mühlig, S., & Beesdo, K. (2003). Mental disorders in primary care. Dialogues in Clinical Neuroscience, 5(2), 115–128.
Wong, H. B., & Lim, G. H. (2011). Measures of diagnostic accuracy: Sensitivity, specificity, PPV and NPV. Proceedings of Singapore Healthcare, 20(4), 316–318. https://doi.org/10.1177/201010581102000411
World Health Organization. (2018). Mental health in primary care: Illusion or inclusion? (No. WHO/HIS/SDS/2018.38). World Health Organization.
Zhong, Q. Y., Gelaye, B., Zaslavsky, A. M., Fann, J. R., Rondon, M. B., Sánchez, S. E., & Williams, M. A. (2015). Diagnostic validity of the generalized anxiety disorder-7 (GAD-7) among pregnant women. PLoS ONE, 10(4), e0125096. https://doi.org/10.1371/journal.pone.0125096
Zimmerman, M., & Chelminski, I. (2006). A scale to screen for DSM-IV Axis I disorders in psychiatric out-patients: Performance of the psychiatric diagnostic screening questionnaire. Psychological Medicine, 36(11), 1601–1611. https://doi.org/10.1017/S0033291706008257
Zimmerman, M., & Mattia, J. I. (1999). The reliability and validity of a screening questionnaire for 13 DSM-IV axis I disorders (the Psychiatric Diagnostic Screening Questionnaire) in psychiatric outpatients. The Journal of Clinical Psychiatry, 60(10), 677–683. https://doi.org/10.4088/jcp.v60n1006
Zimmerman, M., & Mattia, J. I. (2001). A self-report scale to help make psychiatric diagnoses: The Psychiatric Diagnostic Screening Questionnaire. Archives of General Psychiatry, 58(8), 787–794. https://doi.org/10.1001/archpsyc.58.8.787
Zimmerman, M., Ruggero, C. J., Chelminski, I., & Young, D. (2008). Is bipolar disorder overdiagnosed? The Journal of Clinical Psychiatry, 69(6), 935–940. https://doi.org/10.4088/jcp.v69n0608
This study was supported by Connected Mind defrayed costs related to data collection.
Conflict of interest
Richard Rogers, Sara E. Hartigan and Courtney E. Sanders declares that they have no financial or proprietary interests in any material discussed in this article.
Human and Animal Rights and Informed Consent
This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of the University of North Texas (UNT) Internal Review Board (IRB) on April 1st, 2019 (# 19-21). Written informed consent was obtained by all study participants. The authors also affirm that human research participants provided informed consent for publication.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Below is the link to the electronic supplementary material.
About this article
Cite this article
Rogers, R., Hartigan, S.E. & Sanders, C.E. Identifying Mental Disorders in Primary Care: Diagnostic Accuracy of the Connected Mind Fast Check (CMFC) Electronic Screen. J Clin Psychol Med Settings (2021). https://doi.org/10.1007/s10880-021-09820-1
- Mental health screens
- Missed diagnoses
- Connected Mind Fast Check
- Primary health care