Background

Various visual examination methods (VE) have been developed for detecting and differentiating carious lesions in the enamel or dentin [1, 2]. One of the systems used with the VE, that has been studied and compared extensively with other caries detection methods, is the International Caries Detection and Assessment System (ICDAS II) [3,4,5,6,7,8].

Another method used in the detection of dental caries is the radiographic method (RM) [9, 10]. Although bitewing radiographs are better for detecting interproximal carious lesions, they are also capable of detecting occlusal caries as demonstrated in the study by Hopcraft and Morgan, who concluded that the prevalence of interproximal and occlusal caries was underestimated when only the visual and tactile examination were used [11].

The validity and reliability of caries detection systems are based on their sensitivity and specificity in detecting a tooth with caries (true positive-sensitivity of the test) or a caries-free tooth (true negative-specificity of the test) [12, 13].

To determine the sensitivity and specificity of a diagnostic test, it is known that a gold standard test must be used. The ideal gold standard should have 100% sensitivity and 100% specificity. We do not know of any method, system or technique used in the detection of occlusal caries that has these two characteristics. Regarding the last point, it has been suggested that any caries diagnostic method used to detect caries must have sensitivity greater than or equal to 0.75 and specificity of 0.85 or greater [14].

Because it is less expensive and time consuming, most studies comparing the sensitivity, specificity, accuracy and reliability of the systems, used for detecting dental caries, have been performed in vitro, using the histological method as the reference test [15, 16]. Although the histological test gives a more controlled scenario and is a feasible option in an in vitro study, in an in vivo study, the use of the histological test requires the removal of the tooth which in many cases cannot be implemented for ethical reasons. In addition to this ethical conflict, the histological test has also been questioned as a gold standard due to the manipulation, sectioning and grinding of the tissue which might create changes in the area of the tooth that must be examined [17]. Therefore, the dilemma that an in vivo study design has in comparing the sensitivity and specificity of a caries detection method is to select a gold standard with a similar or better diagnostic ability than the histological test. Then the question is: what system can be used to compare the sensitivity and specificity of a caries detection method in an in vivo study?

Technological advances in the field of optics have fostered the development of new caries detection instruments such as the Quantitative Light-Induced Fluorescence, the DIAGNOdent™, and light fluorescent cameras such as the Spectra™ Caries Detection System (SCDS) (Air Techniques, Inc. Melville, NY, USA), which facilitates the detection of caries using high energy light on the tooth’s surface making cariogenic bacteria fluoresce red and healthy enamel fluoresce green. These systems have made it possible to detect changes in the structure of the enamel that could not be seen with VE or RM. Moreover, conventional techniques have been shown to have poor sensitivity and are only capable of detecting dental caries in the most advanced stages [18, 19].

Various studies comparing fluorescent cameras such as the SCDS, and a similar caries detection system named Vista Proof with conventional and histological tests have shown that these systems have similar or better sensitivity and specificity, suggesting that they can be used as a gold standard in an in vivo study instead of the histological test [1, 20,21,22,23,24,25,26,27,28].

In clinical and epidemiological studies, identifying a test with high sensitivity and specificity not only helps detect groups with a high prevalence of the disease or at risk of developing the disease, but also helps to implement strategies to prevent the disease, distribute financial resources efficiently, and improve access to the health system [29, 30].

In developed countries, the use of the light fluorescence system (LFS) has become an available method to detect, diagnose and monitor incipient carious lesions in a clinical setting [15, 31]. In these countries, the LFS may be an easy option to be implemented, whereas in underserved communities of developing countries, the implementation and use of these systems are almost impossible due to their high cost and lack of basic services such as electricity and running water.

The aims of this study were to compare the sensitivity and specificity of The International Caries Detection and Assessment System (ICDAS II) and radiographic method (RM) in detecting occlusal carious lesions while using the Spectra™ Caries Detection System (SCDS) as the reference test, and to determine if the sensitivity and specificity varies according to the type of tooth examined, and whether the carious lesion is compromising the enamel only or the dentin.

Methods

This descriptive and comparative study was approved and performed following the regulations of the Institutional Ethics Committee of the Universidad Peruana Cayetano Heredia (UPCH) (code number: 60917). The sample size of 140 children was estimated based on the prevalence of dental caries (90%) in the population with an error and precision of 5%. To avoid a reduction in the sample size during the time of the study, an additional 15% of children for a total of 161 children ages 7–12 years from an underserved community located in Ventanilla-Callao-Peru, were included in the study. The term underserved was based on the following criteria: (1) the community does not have access to basic public services, such as running water, electricity, and a health care system, and (2) is in a peri-urban area with poor transportation access [32, 33].

An informed consent and assent including the purpose, benefits and risks involved in the study were obtained from the parents and participants in the study.

Detection of occlusal caries

All VE using the ICDAS-II criteria and SCDS were performed by one of the authors (JC). Calibration in the use of the ICDAS-II codification and criteria to detect occlusal caries was performed through the learning program sponsored by the ICDAS foundation [34]. The intra-examiner reliability at the end of the learning process was 90%. Children with any chronic medical conditions and those who could not tolerate the non-invasive VE, RM, and the SCDS examinations were excluded from the study. The detection of occlusal caries using ICDAS II and SCDS was performed on the same day. The teeth of all children were cleaned with a disposable toothbrush and water, and then dried with a piece of gauze to avoid any distortion due to the reflection, absorption or scattering of the light. To avoid a bias associated with the visualization of the image on the occlusal surface when SCDS was used, all VE were performed first. A #4 mouth mirror and a combination of natural and LED light were used to examine the occlusal surfaces of all primary first and second molars, and permanent first molars. Caries detection using ICDAS II was performed in five zones of the occlusal surface following a geometric format previously designed and following the codification and criteria established by ICDAS.org and modified by the principal investigator to fulfill the codification used in the study (Fig. 1a, b, Table 1).

Fig. 1
figure 1

Occlusal zones evaluated in the detection of caries (a) ICDAS II System, (b) SpectraTM Caries Detection System, and (c) Radiographic Method

Full size image
Table 1 Criteria and scores used for data analysis using the ICDAS II, the radiographic method (RM) and the Spectra™ Caries Detection System (SCDS)
Full size table

The detection of occlusal caries with SCDS was performed using a 10 mm black plastic separator (Fig. 2). This separator avoided the penetration of external light and allowed for the same distance between the lens of the camera and the occlusal surface, preventing distortions of the image. All images were transferred to the Visix® (Air Technique, Inc., Melville, NY) dental imaging processor and stored in a file generated by the program for further interpretation and analysis. The criteria established for the detection of occlusal carious lesions using SCDS was based on the manufacturer’s recommendations.

Fig. 2
figure 2

Spectra™ light fluorescent system showing the 10 mm black plastic separator

Full size image

Conventional radiograph Kodak films (Ultra-speed DF-58 and DF-54) and a calibrated Kodak radiographic equipment model CS2100 attached to a mobile dental unit (kilovoltage: 60, milliamp: 7, exposure time: 0.25–0.40 s, focal point: 7 mm and a cone of 20 cm) from the Department of Community Dentistry-UPCH were used to take two bitewing radiographs in two consecutive days on all children that participated in the caries detection examination with ICDAS II and SCDS. All radiographs were taken by an experienced dental radiologist using the parallel technique with zero-degree angulation and processed manually the same day in a dark room with a safety light and a room temperature ranging between 750—800 F. Two control films were used to determine the time of the radiographic film in the developer solution. The developer and fixer solutions were new and the water to wash the films between solutions was constantly drained during the process. After the radiographs were processed, they were scanned and digitalized using a Hewlett Packard model hp Scanjet 5470c. All radiographic films were transferred to Corel Office 2000 (Corel Print Office 5®), coded, de-identified and then stored in a Microsoft PowerPoint file (Microsoft® 2010) for their interpretation. This process not only allowed to maintain the quality of the imagen but also to obtain a standardized method for the interpretation of the radiographs including the use of the same computer screen, light intensity, contrast, and size of the image.

The values used to score caries free teeth and caries in the enamel and dentin were based on the diagnostic criteria used by Otis and Sherman [35]. The scores and criteria to determine healthy teeth, as well as caries in enamel and dentin for the three systems are depicted in Table 1.

After the occlusal caries detection examinations were performed, a copy of the oral examination findings, indicating the teeth that needed dental treatment was delivered to each classroom’s teacher. The teachers were asked to give a copy of the oral examination findings to the parents making sure that they were informed of the need for dental treatment of their child. At the time of this study, The Department of Community Dentistry at UPCH, School of Dentistry provided community dental service in the educational institution where the study was performed and all children that participated in the study were part of this community dental service.

Data processing

The data of each occlusal zone obtained with ICDAS II was entered into a hard copy format (Fig. 1a). For SCDS, the data was obtained from each zone of the occlusal surface’s image using a transparent template created for each type of tooth and superimposed on the computer screen so that the zones of the occlusal surface examined were consistently the same (Fig. 1b). Because the bitewing radiographs have two dimensions, and do not allow for direct caries detection on the occlusal surface, the data obtained from zone 2 (Fig. 1c) which includes the central, buccal, and lingual zones were compared with the values obtained from the central, buccal, and lingual zones of ICDAS II and SCDS. This modification reduced the bias and error in the comparative analysis among zones. The mesial and distal occlusal zones from the radiographic images were compared with the values obtained from mesial and distal occlusal zones of ICDAS II and SCDS. The values from the interproximal surfaces were not considered in the statistical analysis. To avoid bias and duplication of the information, the data entering process was performed independently for each method over several consecutive days. In all cases, the value entered for each zone on the occlusal surface was always the highest one.

The sensitivity and specificity of ICDAS II and RM were determined using SCDS as the reference test. The decision to have SCDS as the reference test was based on the high accuracy and reliability of LFS previously reported [15, 24, 36,37,38,39]. The sensitivity and specificity of ICDAS II were determined based on the detection of dental caries in the five zones of the occlusal surface. For RM, the sensitivity and specificity were determined using the modified format explained above.

Statistical analysis

The data obtained from ICDAS II, SCDS and RM was transferred to an electronic version similar to the original hard copy format which was created using the EpiInfo program (CDC-EpiInfo™ 7). The data stored in the EpiInfo program was transferred to Excel 2010 (Microsoft® Inc. 2010) and then to SAS 9.4 program for statistical analysis. The initial caries detection information obtained with the three methods was recategorized considering the criteria shown in Table 1. All identifiers were deleted, and a numerical code was assigned to each participant. Descriptive statistical analysis included the mean, standard deviation, and confidence interval at 95%. The inferential analysis to determine differences in the sensitivity and specificity between ICDAS II and RM was performed using the paired sample t-test for repeated measures with a significance level set at 5%.

Results

Out of one hundred sixty-one participants, 160 children, 77 males and 83 females with a mean age of 9.42 and 9.45 years, respectively, completed all three caries detection exams. Six hundred thirty-two permanent first molars (maxilla: 315, mandible: 317), 364 primary first molars (maxilla: 179, mandible: 185), and 495 primary second molars (maxilla: 258, mandible: 237) were examined. Taking into consideration the three systems used in this study, a total of 22,365 occlusal zones were included in the analysis. Descriptive statistical analysis showed that the sensitivity and specificity values of ICDAS II and RM vary depending on whether the analysis is performed considering: (1) permanent and primary molars together, (2) primary molars together, (3) permanent or primary molars independently, and (4) whether the carious lesion is compromising the enamel or dentin (Table 2).

Table 2 Comparison of the sensitivity and specificity values according to the type of tooth and caries detection method
Full size table

The sensitivity and specificity were analyzed for all primary and permanent first molars together, and independently. For each category, the mean of the sensitivity and specificity of each occlusal zone was calculated and then averaged for total caries (caries in enamel and/or in dentin) and whether the caries were in enamel only or in dentin.

Sensitivity

For total caries the mean and standard deviation of sensitivity with ICDAS II for all molars was 0.57 ± 0.19. The higher mean value was found in the primary first molars (0.64 ± 0.22) when compared to the permanent first molars (0.47 ± 0.16), and primary second molar (0.60 ± 0.15). When the carious lesions were stratified into caries in enamel only or in dentin the mean values were different. For caries in the enamel only, the mean for all molars was 0.33 ± 0.20. The permanent first molars showed the highest mean value (0.42 ± 0.15) in comparison to the primary molars together (0.29 ± 0.21), primary first molars (0.19 ± 0.16), and primary second molars (0.39 ± 0.21). The mean sensitivity for the primary and permanent first molars together with caries in dentin was 0.68 ± 0.29, with the primary first molars showing the highest value (0.90 ± 0.19) in comparison with all primary molars together (0.77 ± 0.22), the primary second molars (0.65 ± 0.18), and the permanent first molars (0.50 ± 0.32) (Table 2).

Overall, the sensitivity values with RM for total caries and caries in enamel were lower than those found with ICDAS II, however, the sensitivity of RM for caries in dentin for permanent and primary molars together (0.60 ± 0.30), primary molars together (0.70 ± 0.22), permanent molars only (0.41 ± 0.35), primary first molars (0.75 ± 0.27), and primary second molars (0.64 ± 0.16), showed a high to moderate sensitivity according to the color scale used in this study (Table 2). Except for caries in dentin for permanent first molars (p = 0.214), and primary second molars (p = 0.761), the sensitivity of ICDAS II for all categories was significantly higher than the one of RM (p ≤ 0.05).

Specificity

When analyzing the specificity with ICDAS II, it was found that all values were very high. The specificity value for total caries was 0.85 ± 0.12. When the specificity was calculated for the primary and permanent first molars independently, the specificity was 0.87 ± 0.10 and 0.81 ± 0.13, respectively. The mean values of the specificity were also high when the carious lesions were stratified into caries in enamel only and those in dentin. The mean value for caries in the enamel for all molars was 0.84 ± 0.14. The mean value of the specificity for all primary molars together was (0.88 ± 0.11). Interestingly, when the specificity was estimated for the primary first and second molars independently, the mean values changed. Therefore, the first primary molars had the highest mean values (0.94 ± 0.05) when compared to primary second molars (0.82 ± 0.12) and permanent first molars (0.76 ± 0.15). The specificity values for caries in dentin were 0.95 ± 0.05 for all molars, 0.94 ± 0.05 for all primary molars together, 0.92 ± 0.06 and 0.95 ± 0.03 for primary first molars and second molars, respectively, and 0.98 ± 0.02 for permanent first molars. With RM, the mean value of the specificity for all categories was between 0.88 ± 0.07 and 1.00 ± 0.00 which is considered very high. The values of the sensitivity and specificity for the primary and permanent first molars, according to whether the carious lesion was compromising the enamel or dentin are shown in Table 2. The Specificity of RM was higher for the total caries when all molars were combined, and for permanent first molars only (p < 0.001). Although the mean specificity of RM was higher for the total caries in the primary molars together, there was no statistical difference when compared to ICDAS II (p = 0.156), however, the mean was higher for primary second molars alone (p = 0.017). The specificity of RM for caries in enamel was higher than ICDAS II in all categories (p < 0.001). The specificity of ICDAS II for caries in dentin was significantly higher when all molars were combined (p < 0.001), for all primary molars together (p < 0.001), and for primary first molars (p = 0.004), and second primary molars (p < 0.001). Even though the mean specificity for caries in dentin of ICDAS II was higher in the permanent molars, there was no statistical difference when compared to RM (p = 0.181).

Discussion

To our knowledge, this is the first in vivo study designed to compare the sensitivity and specificity of ICDAS II and RM in detecting dental caries based on five zones of the occlusal surface of primary first and second molars, and permanent first molars having SCDS as the reference test. The decision to use the SCDS as the reference test was based on the results of in-vitro studies which have demonstrated that the performance of SCDS is similar to the clinical and radiographic methods with a receiver operating characteristic curve of 0.82 [26, 40]. Most recently, these findings have been supported by the diagnostic accuracy in detecting incipient enamel and dentin occlusal carious lesions [27].

In the present study, the results of the sensitivity and specificity values showed that the RM and ICDAS II varied according to both the type of carious lesion and tooth examined. An important component of this study was to perform an accurate interpretation of the radiographic images. For this reason, all bitewing radiographs taken using a conventional method were scanned and digitalized. Although this step helped avoid errors and be more precise in the radiographic interpretation, the results of this study would not be different if a conventional radiographic method was used instead [41]. Thus, RM showed low sensitivity, but a high specificity in teeth with carious lesions that were compromising the enamel. An explanation for this would be the superimposition of the image obtained from the bitewing radiographs. If a carious lesion is affecting the central-occlusal zone, it is possible that due to the size and depth of the carious lesion it can be hidden by healthy tooth tissue from the occlusal-lingual and occlusal-buccal zones and be interpreted as a caries-free zone. The low values of sensitivity and high values of specificity of RM found in this study are similar to those reported by in vivo and in vitro studies [20, 26, 42].

It has been noted in previous studies that the sensitivity of RM to detect caries in the enamel is quite low, but very good for detecting caries in dentin and caries in the inter-proximal surfaces [43, 44]. These findings suggest that bitewing radiographs are not the best system to detect incipient occlusal caries and should not be used as an indicator to establish preventive treatment. For instance, the prevalence of carious lesions in enamel may be underestimated and not considered as part of the problem.

The sensitivity of ICDAS II for all teeth examined in this study showed values lower than those reported by in vitro and in vivo studies [20, 37, 39, 42, 45]. The discrepancy may be because in this study, all comparisons were made in the same occlusal zone examined, which differs from other studies that do not indicate with certainty whether the comparison was from the same area where the carious lesion was detected, suggesting that the results of those studies could have had a bias related to the location of the examined zone, and therefore may not be accurate. This study not only demonstrated that the sensitivity and specificity values vary according to the type of tooth examined, but also whether the carious lesions were interpreted as a total caries (enamel and dentin together) or independently as caries in enamel only or in dentin.

According to the color scale, the mean values of specificity for the enamel and dentin were very high for ICDAS II and RM. In contrast, the sensitivity of ICDAS II and RM showed different mean values for total caries, as well as for caries that compromise the enamel or dentin of primary first and second molars, and permanent first molars. These differences suggest that the sensitivity must be estimated based on the type of tooth and whether the detection of caries is compromising the enamel or dentin. The differences might also be related to the age of the children included in the study, and the fact that primary molars had a higher prevalence of caries, particularly in cases of caries that compromise the dentin. On the latter point, it has been suggested that the prevalence of the disease may have a direct influence on the sensitivity and specificity values of a diagnostic test [46]. The prevalence of caries found in this study varied depending on the system used. Therefore, using the ICDASII, RM, and SCDS, the prevalence of caries was 96.9%, 92.5% and 100%, respectively. Whether these differences have influenced the results found in this study is dependent on an analysis that was out of the scope of this study.

Another interpretation of these differences may be associated with (1) the irregular anatomy of the occlusal surface making the visual examination more difficult, (2) the presence of non-cleansable deep occlusal pits and fissures, (3) the estimation of the sensitivity which was based on combining all primary molars and first permanent molars, and independently for each type of molars, (4) the highest capacity of the SCDS to detect caries in enamel and dentin, (5) the estimation of the sensitivity in vivo and in real time which differ substantially from in vitro studies, and (6) the calibration of the examiner to detect occlusal caries.

This study shows the direct results of the sensitivity and specificity in vivo, which are based on the natural and biological status of occlusal caries in children. Compared with other in vitro studies, this study controlled the confounders associated with the methods used to store and prepare teeth which have been shown to change the optical properties of the tooth, the concentration of fluorophores, and the fluorescence values of LFS [47, 48].

Although the determination of the sensitivity and specificity of a test or system are important epidemiological indicators to differentiate between the true positive and true negative cases, it may also allow for the establishment of preventive programs and prioritizing resources specifically oriented for people from underserved communities who need preventive treatment, thus avoiding reparative treatments which are invasive and associated with an increase in the cost of healthcare services.

This study has shown that sensitivity and specificity of the ICDAS-II and RM for detecting occlusal caries having the SCDS as the reference test varies according to the type of molars examined and whether the caries lesion is compromising the enamel or dentin.

Conclusions

Using the SCDS as the gold standard, the values of sensitivity of the ICDAS II and RM for detecting caries in dentin ranged from moderate to very high, whereas their sensitivity for detecting caries in enamel was very low to low. Regarding the specificity, both methods show values considered high and very high for detecting caries in the enamel or in dentin of the primary first and second molars, and permanent first molars. This finding may be related to the subjectivity in detecting caries when the ICDAS II and RM are used, and to the optical properties of the SCDS system which gives an objective result in the detection of caries. In any case, both the subjective and objective components of the method used, which are inherent in any study must be controlled throughout a good calibration process.

Recommendations

The values of sensitivity and specificity of the ICDAS II and RM in detecting occlusal caries should be estimated according to the type of molars examined, and whether the carious lesion is compromising the enamel or dentin of the primary first and second molars or first permanent molars.