Background

Chronic obstructive pulmonary disease (COPD) places a significant burden on healthcare systems worldwide and is a leading cause of mortality and health care expenditure [1, 2]. COPD is also a leading cause of potentially preventable hospitalisations in Australia [3] in which admissions could be prevented by timely and optimal health care in the community [4]. Preventative health interventions, such as smoking cessation and promotion of physical activity, have been shown to have positive outcomes for people with COPD, in terms of reducing the risk of all-cause mortality and hospital admissions, as well as improving health-related quality of life [5,6,7,8,9]. If COPD can be detected early, then implementation of these preventative health strategies could help to reduce the burden on the healthcare system by keeping people healthy for longer, thereby preventing exacerbations and hospitalisations.

The Global Initiative for Chronic Obstructive Lung Disease (GOLD) guidelines advocate for early detection of COPD [2]. Many people with chronic or persistent respiratory symptoms present to a general practitioner (GP) before a respiratory physician, which places primary care in an ideal position for the early diagnosis of COPD [10]. Despite this, there is evidence that COPD is currently underdiagnosed in primary care [11,12,13,14]. Poor utilisation of spirometry in general practice could contribute to this, with some studies reporting that up to 50% of patients with respiratory symptoms do not undergo spirometry testing [15,16,17]. Reported challenges to performance of spirometry include lack of time, lack of access to resources such as working spirometers and also training in performance of spirometry [18,19,20]. An additional challenge for the GP is that patients can often adapt to the insidious onset of COPD symptoms and may only seek medical advice when symptoms impair their quality of life, potentially delaying diagnosis and optimisation of treatment [21,22,23].

Several studies have also demonstrated high rates of misdiagnosis of COPD in general practice, with a high proportion of patients with an existing COPD diagnosis not meeting the spirometric definition of COPD [13, 19, 24,25,26]. This could relate to diagnosis often being based on clinical symptoms rather than gold standard spirometry testing or problems with quality or interpretation of spirometry in primary care [18, 26,27,28]. For example, one Australian study completed in 31 general practices found that spirometric confirmation was not present in 31% of cases of COPD [29]. Strategies are needed in primary care to improve the diagnosis of COPD, including not only improved detection, but also the accuracy and confirmation of an existing diagnosis.

Active case finding of ‘at risk’ patients is one way to improve the early diagnosis of COPD [2]. Widespread spirometric testing could be useful as pre and post-bronchodilator spirometry with a post-bronchodilator forced expiratory volume in 1 second (FEV1)/forced vital capacity (FVC) of < 0.7 remains the gold standard for a diagnosis of COPD [2, 10]. However, pre-selection of patients who present with symptoms or have risk factors may result in more economical utilisation of healthcare resources, by reducing unwarranted spirometry testing. Microspirometry has been shown to be a sensitive and effective tool to use in primary care to detect cases of undiagnosed airflow obstruction [30, 31]. Yet the use of microspirometry in primary care may present with similar barriers to spirometry including access to equipment and training in the use of the device.

Symptom-based questionnaires, which identify factors such as smoking history and symptoms commonly associated with COPD, may be helpful as an initial screening tool to identify those patients who have an increased likelihood of having airflow obstruction and would likely benefit from spirometry testing [32]. The COPD Diagnostic Questionnaire (CDQ) is one such screening tool developed for use in primary care [33, 34]. This 8-item tool comprises questions regarding demographics, smoking history and symptoms, and was designed for use in patients aged 40 years and over with a history of smoking, but no prior respiratory diagnosis or differential diagnosis. Participants receive a score out of 38 and subjects are then classified as being at decreased likelihood (0 < 16.5 points), intermediate likelihood (≥16.5–< 19.5 points) or increased likelihood (≥19.5 points) of airflow obstruction. Price and colleagues (2006) proposed that those at increased likelihood of obstruction should undergo spirometry, and those at decreased likelihood would not require spirometry in most cases, with a reasonable degree of certainty [34].

In the initial development of the CDQ by Price et al. (2006), accuracy was reported to be high according to receiver operating characteristic (ROC) analysis, with an area under the curve (AUC) of 0.82 [33]. In a following paper examining the scoring system and clinical application of the CDQ, this group reported a sensitivity of 58.7% and specificity of 77.0% when applying a cut-off score of 16.5 [34]. Since then, external validation studies of the CDQ have reported mixed results on the accuracy of the CDQ at detecting patients at risk of airflow obstruction. Reported sensitivities have ranged from 79.7 to 93.9% for 16.5 as a cut-off score with corresponding low specificities, in a population of people at risk of airflow obstruction [35,36,37,38,39]. There are currently no studies examining the accuracy of the CDQ in detecting the presence of airflow obstruction in people with an existing COPD diagnosis, to assist clinicians in confirming a diagnosis of COPD. Therefore, the primary aim of this study was to determine the sensitivity, specificity and accuracy of using a cut-off score of 16.5 on the CDQ as an indicator of airflow obstruction, in a mixed population of people ‘at risk’ of COPD or with ‘existing’ COPD. Secondary aims were to a) determine the sensitivity, specificity and accuracy of a cut-off score of 16.5 on the CDQ as an indicator of airflow obstruction in the ‘existing’ COPD group alone, and b) determine the optimal cut-off score using ROC curve analysis in a population of people ‘at risk’ of COPD and ‘existing’ COPD.

Methods

This cross-sectional study was embedded in a larger pilot study evaluating a GP-Physiotherapist partnership for COPD in primary care. The detailed methods have been published previously [40]. In brief, four general practices were selected from a metropolitan Sydney Primary Health Network and consented to participate. Potentially eligible participants at risk of COPD or with an existing diagnosis of COPD were identified from a search of the general practice databases and invited to take part in the study via telephone or letter. Two groups of participants were included: those ‘at risk’ of COPD and those with ‘existing’ COPD. Participants were considered eligible if they: (i) were adults aged 40 years and over; (ii) had attended the practice at least twice with one visit in the preceding 12 months; and (iii) had a documented history of smoking (current or former smoker) in their medical notes or (iv) had a recorded diagnosis of COPD or were taking medications prescribed for COPD (i.e. short acting inhaled β2 agonists (SABA), short acting muscarinic antagonists (SAMA), long acting inhaled β2 agonists (LABA), long acting muscarinic antagonists (LAMA), combination of LABA/LAMA and inhaled corticosteroids). The list of potential participants was screened by GPs or practice nurses and participants were excluded if they had terminal cancer, a cognitive impairment, required home oxygen, were unable to understand sufficient English or were pregnant.

After obtaining written informed consent, participants were invited to attend an assessment with a senior respiratory physiotherapist at the general practice. At the assessment, participants completed the CDQ and were classified as having decreased likelihood (0 < 16.5 points), intermediate likelihood (≥16.5–< 19.5 points) or increased likelihood (≥19.5 points) of airflow obstruction. Pre and post-bronchodilator spirometry according to the American Thoracic Society/European Respiratory Society (ATS/ERS) lung function guidelines [41] was completed by the physiotherapist. A diagnosis of COPD based on GOLD guidelines was assigned to all those participants with a post-bronchodilator FEV1/FVC of < 0.7 [2].

The study protocol was approved by the Northern Sydney Local Health District Human Research Ethics committee (HREC reference; HREC/15/HAWKE/434) and was conducted in accordance with the WMA Declaration of Helsinki. The study protocol was registered at the Australia New Zealand Clinical Trials Registry (ACTRN12619001127190).

Statistical analysis

The sum score for the CDQ was calculated based on the original scoring protocol outlined by Price et al. (2006) [34]. A 2 × 2 contingency table was used for the entire mixed population to calculate the sensitivity, specificity, accuracy, positive likelihood ratio (+LR) and negative likelihood ratio (−LR) for the cut-off score of 16.5 on the CDQ at detecting those at intermediate to increased risk of COPD. A second 2 × 2 contingency table was used for the ‘existing’ COPD cohort alone to also calculate the sensitivity, specificity and accuracy of a score of 16.5 on the CDQ at detecting airflow obstruction. The participants were classified according to their “predicted” likelihood of obstruction from the CDQ results and their “actual” obstruction from the post-bronchodilator spirometry results.

The discriminatory ability and accuracy of the CDQ was determined using ROC curve analysis. For calculation of the ROC curve, the raw CDQ scores were used as the screening test variable and a diagnosis of COPD based on post-bronchodilator spirometry results, as the classification variable. The AUC value was analysed to determine the discriminative ability of the CDQ across the full range of cut-off scores: high accuracy (AUC > 0.9), moderate accuracy (AUC 0.7 to 0.9), low accuracy (AUC 0.5 to 0.7) or a chance result (AUC < 0.5) [42, 43]. The sensitivity and specificity of different cut-points were also analysed using ROC curve data to determine the score with optimal balance of sensitivity and specificity. Data was analysed using IBM SPSS Statistics for Windows, version 24.0. (IBM Corp., Armonk, N.Y., USA).

Results

Data on participant flow is shown in Fig. 1. A total of 748 eligible participants were invited to participate. Of these, 658 were ‘at risk’ of COPD and 90 had an ‘existing’ COPD diagnosis. Of those invited to participate, 146 declined to participate and 447 did not respond. A total of 148 participants provided written informed consent and attended a baseline assessment with the physiotherapist. Of these, 141 (95%) participants had complete CDQ data recorded and spirometry meeting criteria for analysis [41] and so these participants were included in the data analysis.

Fig. 1
figure 1

Study Enrolment. Abbreviations: COPD: Chronic Obstructive Pulmonary Disease; DNA: did not attend

Full size image

Demographic characteristics of the participants is presented in Table 1. There were 111 participants classified as ‘at risk’ of COPD and 30 participants had an ‘existing’ COPD diagnosis. Based on post-bronchodilator spirometry results, 30% (n = 42) of participants demonstrated airflow obstruction and were diagnosed as having COPD (n = 18 in the ‘at risk’ group, n = 24 in the ‘existing’ COPD group). According to GOLD criteria for classification of COPD severity [2], the majority of participants with airflow obstruction were classified as GOLD Stage II (n = 26). The mean CDQ score for the total cohort was 19.1 (SD 5.6).

Table 1 Population characteristics of subjects
Full size table

Sensitivity and specificity analyses for the CDQ cut-off score of 16.5 for the entire mixed population (‘at risk’ of COPD and ‘existing’ COPD) is presented in Table 2. Ninety-seven (69%) participants scored ≥16.5 and were predicted to have an intermediate to increased likelihood of airflow obstruction. The remaining 44 (31%) participants scored < 16.5 and were predicted to have a reduced likelihood of obstruction on spirometry. Sensitivity and specificity for the 16.5 cut-off point were 81.0 and 36.4% respectively, whilst accuracy of this cut-off score was determined to be 50.0%. The +LR was 1.27, indicating that a score of ≥16.5 on the CDQ is 1.27 times more likely among patients with obstruction than those without obstruction. The -LR was 0.52, indicating that someone with airway obstruction is about half as likely to score ≥ 16.5 on the CDQ than someone without obstruction. The ROCAUC was 0.59 ± 0.50 (shown in Fig. 2), indicating overall poor diagnostic accuracy of the CDQ as a screening tool.

Table 2 2 × 2 contingency table detailing the sensitivity and specificity analyses for the CDQ cut off score of 16.5 for subjects in ‘at risk’ and ‘existing’ COPD groups
Full size table
Fig. 2
figure 2

Receiver operating characteristic (ROC) curve comparing CDQ score to COPD diagnosis using GOLD spirometry criteria for obstruction (post-bronchodilator FEV1/FVC < 0.70) as the reference standard. Area under the ROC curve (ROCAUC) = 0.59 ± 0.50. A ROCAUC of 0.5 is indicated by the solid diagonal line. Cut point 16.5 is indicated by the black star>

Full size image

Sensitivity and specificity analyses for the CDQ cut off score of 16.5 in our cohort of ‘existing’ COPD participants is presented in Table 3. Sensitivity for the 16.5 cut-off point in this cohort was determined to be 83.3%, whilst specificity was 33.3% and accuracy was 73.3%.

Table 3 2 × 2 contingency table detailing the sensitivity and specificity analyses for the CDQ cut off score of 16.5 for ‘existing’ cases of COPD
Full size table

Discussion

In this paper we examined the usefulness of the CDQ as a screening tool in primary care to identify within a mixed population of patients, those at increased likelihood of COPD. When applying the cut-off point of 16.5 to categorise patients in terms of their likelihood of obstruction [34], the sensitivity was 81.0%, indicating that this score could be used to identify whom would benefit from spirometry. However, the corresponding specificity was low at only 36.4%, with accuracy at 50.0%. This means that in our mixed cohort, whilst the CDQ was sensitive and identified those who truly had obstruction, there were a high number of false positives with approximately 45% of the population having undergone testing, with no evidence of obstruction on spirometry. In addition, accuracy of the CDQ for people with previous doctor diagnosed COPD was only 73%, highlighting potential issues with detecting airflow obstruction, as well as misdiagnosis.

In the initial development of the CDQ in COPD, Price and colleagues (2006) reported overall high accuracy of the CDQ (ROCAUC 0.82) [33] and when applying a cut-off score of 16.5, moderate sensitivity and high specificity [34]. In contrast, in our mixed population of ‘at risk’ of COPD and ‘existing’ COPD when using the same cut-off point, we found high sensitivity and low specificity, as well as poor overall diagnostic accuracy (ROCAUC 0.59 ± 0.50). The resulting likelihood ratios further indicate limited utility of this cut-off score at detecting those at risk of obstruction. According to our results, when utilising the +LR, the pre-test probability of obstruction of 30% would be associated with a very small increase in post-test probability to 35%. Similarly, when looking at the –LR, the same pre-test probability of 30%, would be associated with a small reduction in post-test probability to 18%. This reflects the low specificity and high rate of false positives when using 16.5 as a cut-off score in our mixed cohort.

Other studies examining the CDQ have also reported similar findings of moderate to high sensitivity (79.7 to 93.9%) but low specificity (24.4 to 46.8%) and accuracy (ROCAUC 0.65 to 0.79) of the cut-off score of 16.5 [35,36,37,38,39]. The reasons for these differences in results from the original sample [33] used to develop and validate the CDQ are not readily apparent. Discrepancies could be due to different populations utilised by each study. When examining our cohort, there were some differences compared to the original sample [33]. Our cohort was older (68 ± 11 vs 58 ± 11 years), with lower cigarette consumption (21 ± 19 vs 26 ± 24 pack years) and comprised a lower number of current smokers compared to former smokers. Considering that higher scores are allocated to higher pack years and higher age, this may have impacted results. For example, approximately 45% of our population reported a pack-year history of 14 years or less of which only 25% had airflow obstruction on spirometry. This is in comparison to 34% of confirmed cases that reported a 25 to 49 pack-year history. Although our study population was also mixed (i.e. ‘at risk’ of COPD and ‘existing’ COPD), when comparing baseline characteristics of the two groups (Table 1), the groups were comparable except for worse lung function in the ‘existing’ COPD group (FEV1/FVC 0.62 vs 0.78). If the CDQ was truly accurate at discriminating between those with and without obstruction, it would be feasible to postulate that our sample should have produced results with higher overall accuracy similar to the original sample used for the development and validation of the CDQ [33, 34], given that our sample contained participants with known obstruction.

A secondary aim of this study was to review the sensitivity, specificity and accuracy of a cut-score of 16.5 in the ‘existing’ COPD cohort alone. In this population, the results were similar to our mixed cohort with a sensitivity and specificity of 83.3 and 33.3% respectively. Whilst accuracy was higher compared to the ‘mixed’ group, four existing cases of COPD were not detected. One would have anticipated that the ‘existing’ COPD population would have produced higher CDQ scores over the 16.5 cut-off, however, only 80% of this cohort had a score ≥ 16.5. Furthermore, six participants with a previous diagnosis of COPD did not show obstruction on spirometry. This is not surprising as there is evidence of high rates of misdiagnosis of COPD in primary care [13, 19, 24,25,26]. Yet, the CDQ was only able to correctly identify two of these cases as true negatives, whilst the remaining four cases would still have undergone spirometry testing. Despite the small sample size of this group, these results add further weight to the concerns we have with the CDQ as a screening tool in identifying those who may have obstruction.

There is debate in the literature about whether a higher or lower cut-off score on the CDQ is more useful in screening for COPD [36, 37, 44]. Literature suggests that a diagnostic test that is most useful at discriminating between people with and without a disease should combine high levels of sensitivity and specificity [45, 46]. The original study [34] proposed that those who score less than 16.5 on the CDQ are less likely to have obstruction, and those who score ≥ 19.5 are at high risk of obstruction and should undergo spirometry. Of those that fall in the intermediate zone of scoring ≥16.5 to < 19.5, the decision to undergo spirometry is left to the clinician. This decision is due to the different healthcare contexts clinicians may be faced with in terms of budget, resources and policy. Our ROC curve, as seen in Fig. 2, is fairly linear indicating there is some evidence that multiple scores between 13.5 and 19.5 could be considered for use as a cut-off score. Yet the implications of using a higher or lower cut-off score must be considered. A lower cut-off score will have higher sensitivity yet, would also require a substantial increase in the number of spirometry tests being performed and subsequent false positives. For example, according to our ROC curve, a score of 15.5 when compared to a score of 16.5, would lead to a decrease in specificity by approximately 5%, with only a 2% gain in sensitivity. This means that seven additional spirometry tests would have to be performed in order to detect one additional case of COPD. Alternatively, whilst a higher cut-off score may lower the number of false positives, COPD cases may be missed.

The choice of optimal cut-off score on the CDQ may come down to clinician preference and patient presentation. For example, clinicians who are concerned with missing potential diagnoses and those that have the resources and time available may prefer to use a lower cut-off score. Yet, literature suggests that the use of spirometry in primary care is low [15,16,17] with GPs citing time constraints and availability of resources as barriers to completing spirometry [18, 47, 48]. As such, GPs may potentially be more inclined to use a higher cut-off score of 19.5 for screening to ensure that COPD is more likely to be identified. However, when you examine our ROC curve, the sensitivity and specificity when using 19.5 as the cut-off score was 54.8 and 63.6% respectively and this is similar to other studies [35,36,37,38,39]. Whilst this would result in less spirometries required, sensitivity would decrease by 26%, such that 19 cases of COPD would have been missed compared with only 8 cases when 16.5 is used as the cut-off score. Furthermore, it is important to consider that a screening tool that utilises a lower cut-off point with higher sensitivity may be more desirable when the consequence of a false negative exceeds the consequence of a false positive. For example, screening more people at risk of COPD may still be useful as it can provide the clinician with the opportunity to provide preventative lifestyle interventions, which are likely to benefit the patient if adopted, such as physical activity and smoking cessation advice.

An additional consideration is the need to address the underdiagnosis and misdiagnosis of COPD in primary care, by providing strategies for clinicians to screen for symptoms and refer those at increased risk of fixed obstruction for spirometry testing. The use of handheld flow meters under the supervision of trained health professionals was found to be significantly more accurate than the CDQ for discriminating between smokers with and without airway obstruction [30, 38, 39]. Yet, due to the high sensitivity of the CDQ as well as the practical advantages of being low cost, readily available and easy to use, the CDQ could still assist clinicians to target the use of spirometry on patients at greater risk of COPD to improve the efficiency and accuracy of COPD diagnosis. A patient can complete the CDQ with minimal assistance prior to attending an appointment at low cost versus a handheld flow meter which would require a trained clinician to be present.

There were some limitations to this study. Firstly, only a small number of general practices were involved with a small sample size of patients which may limit the generalisability of findings to the broader COPD population. Recruitment of patients was ceased due to COVID-19 in January 2020 which also affected sample size. Secondly, identifying potentially eligible patients from practice records based on smoking status may have affected recruitment as smoking status is not always accurately recorded.

Conclusion

The results of this study suggest that a score of 16.5 on the CDQ was sensitive to identify those who would benefit from diagnostic spirometry. Therefore, this cut-off score could be used by clinicians to screen for those at high risk of obstruction, without missing too many cases of COPD. However, clinicians should be aware that there will be a high number of false positives due to the low specificity of this cut-off score. In addition, whilst our results suggest overall low diagnostic accuracy of the CDQ, the CDQ may have a place in clinical practice to assist clinicians in determining who would be most appropriate for spirometry and negating the need to perform spirometry on all patients but may not have a place in confirming an existing COPD diagnosis.