Skip to main content
SpringerLink
Log in

Inter-and intra-observer agreement on the judgment of toluidine blue staining for screening of oral potentially malignant disorders and oral cancer

  • Original Article
  • Published:
Clinical Oral Investigations Aims and scope Submit manuscript

Abstract

Objective

Toluidine blue staining (TBS) is used for screening of oral potentially malignant disorders (OPMDs) and oral cancer. Nevertheless, the judgment of TBS is based on the observer’s personal experience and largely considered subjective. The aim of this study was to evaluate the inter- and intra-observer agreement on the judgment of TBS for screening of OPMDs and oral cancer.

Materials and methods

Two hundred randomly ordered TBS images of OPMDs and oral cancer were judged twice independently by four clinicians and four graduated students majored in oral medicine, with an interval of 3 months. Inter- and intra-observer agreement was evaluated by the Cohen’s kappa (κ) coefficient. The correlation between κ-value and individual experience was assessed by Spearman’s correlation coefficient.

Results

For the two sets of observation, good overall inter-observer agreement was found in the study (κ = 0.620 and 0.667). Sub-group analysis yielded good inter-observer agreement both for clinicians (κ = 0.613 and 0.611) and students (κ = 0.610 and 0.665). Pairwise inter-observer agreement was presented to be moderate to excellent between clinician and clinician (κ = 0.41~0.87, 0.43~0.81), moderate to excellent between student and student (κ = 0.43~0.71, 0.53~0.82), and poor to excellent between clinician and student (κ = 0.25~0.81, 0.35~0.87). Good to excellent intra-observer agreement (κ = 0.69~0.85) was presented for eight observers. However, no correlation was illustrated between years of experience and inter- or intra-observer agreement.

Conclusion

Judgment of TBS has a good overall inter-observer agreement and good to excellent intra-observer agreement, which guarantees its utilization and popularity in detecting OPMDs and oral cancer.

Clinical relevance

To our knowledge, this is the first study to investigate the inter- and intra-observer agreement of TBS judgment. The present results revealed a good inter-observer agreement and good to excellent intra-observer agreement, which provide a beneficial supplement to the knowledge of TBS technique.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ye X, Zhang J, Tan Y, Chen G, Zhou G (2015) Meta-analysis of two computer-assisted screening methods for diagnosing oral precancer and cancer. Oral Oncol 51(11):966–975. https://doi.org/10.1016/j.oraloncology.2015.09.002

    Article  PubMed  Google Scholar 

  2. Siegel RL, Miller KD, Jemal A (2017) Cancer statistics, 2017. CA Cancer J Clin 67(1):7–30. https://doi.org/10.3322/caac.21387

    Article  PubMed  Google Scholar 

  3. Warnakulasuriya S (2010) Living with oral cancer: epidemiology with particular reference to prevalence and life-style changes that influence survival. Oral Oncol 46(6):407–410. https://doi.org/10.1016/j.oraloncology.2010.02.015

    Article  PubMed  Google Scholar 

  4. Seoane J, Alvarez-Novoa P, Gomez I, Takkouche B, Diz P, Warnakulasiruya S, Seoane-Romero JM, Varela-Centelles P (2016) Early oral cancer diagnosis: the Aarhus statement perspective. A systematic review and meta-analysis. Head Neck 38(S1):E2182–E2189. https://doi.org/10.1002/hed.24050

    Article  PubMed  Google Scholar 

  5. Wang Y-Y, Tail Y-H, Wang W-C, Chen C-Y, Kao Y-H, Chen Y-K, Chen C-H (2014) Malignant transformation in 5071 southern Taiwanese patients with potentially malignant oral mucosal disorders. BMC Oral Health 14:99. https://doi.org/10.1186/1472-6831-14-99

    Article  PubMed  PubMed Central  Google Scholar 

  6. Zhang S-K, Zheng R, Chen Q, Zhang S, Sun X, Chen W (2015) Oral cancer incidence and mortality in China, 2011. Chin J Cancer Res 27(1):44–51. https://doi.org/10.3978/j.issn.1000-9604.2015.01.03

    Article  PubMed  PubMed Central  Google Scholar 

  7. Rivera C (2015) Essentials of oral cancer. Int J Clin Exp Pathol 8(9):11884–11894

    PubMed  PubMed Central  Google Scholar 

  8. Chaudhry A, Manjunath M, Ashwatappa D, Krishna S, Krishna AG (2016) Comparison of chemiluminescence and toluidine blue in the diagnosis of dysplasia in leukoplakia: a cross-sectional study. J Investig Clin Dent 7(2):132–140. https://doi.org/10.1111/jicd.12141

    Article  PubMed  Google Scholar 

  9. Chainani-Wu N, Madden E, Cox D, Sroussi H, Epstein J, Silverman S (2015) Toluidine blue aids in detection of dysplasia and carcinoma in suspicious oral lesions. Oral Dis 21(7):879–885. https://doi.org/10.1111/odi.12358

    Article  PubMed  Google Scholar 

  10. Burket LW, Greenberg MS, Glick M, Ship JA (2003) Burket's oral medicine, 10th edn. BC Decker, Hamilton, pp 85–194

    Google Scholar 

  11. Omar E (2015) Current concepts and future of noninvasive procedures for diagnosing oral squamous cell carcinoma - a systematic review. Head Face Med 11:6. https://doi.org/10.1186/s13005-015-0063-z

    Article  PubMed  PubMed Central  Google Scholar 

  12. Awan KH, Morgan PR, Warnakulasuriya S (2015) Assessing the accuracy of autofluorescence, chemiluminescence and toluidine blue as diagnostic tools for oral potentially malignant disorders—a clinicopathological evaluation. Clin Oral Investig 19(9):2267–2272. https://doi.org/10.1007/s00784-015-1457-9

    Article  PubMed  Google Scholar 

  13. Sridharan G, Shankar AA (2012) Toluidine blue: a review of its chemistry and clinical utility. J Oral Maxillofac Pathol 16(2):251–255. https://doi.org/10.4103/0973-029X.99081

    Article  PubMed  PubMed Central  Google Scholar 

  14. Martin IC, Kerawala CJ, Reed M (1998) The application of toluidine blue as a diagnostic adjunct in the detection of epithelial dysplasia. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 85(4):444–446

    Article  PubMed  Google Scholar 

  15. Chhabra N, Chhabra S, Sapra N (2015) Diagnostic modalities for squamous cell carcinoma: an extensive review of literature-considering toluidine blue as a useful adjunct. J Maxillofac Oral Surg 14(2):188–200. https://doi.org/10.1007/s12663-014-0660-6

    Article  PubMed  Google Scholar 

  16. Zhang L, Williams M, Poh CF, Laronde D, Epstein JB, Durham S, Nakamura H, Berean K, Hovan A, Le ND, Hislop G, Priddy R, Hay J, Lam WL, Rosin MP (2005) Toluidine blue staining identifies high-risk primary oral premalignant lesions with poor outcome. Cancer Res 65(17):8017–8021. https://doi.org/10.1158/0008-5472.can-04-3153

    Article  PubMed  Google Scholar 

  17. Rosenberg D, Cretin S (1989) Use of meta-analysis to evaluate tolonium chloride in oral cancer screening. Oral Surg Oral Med Oral Pathol 67(5):621–627

    Article  PubMed  Google Scholar 

  18. Allegra E, Lombardo N, Puzzo L, Garozzo A (2009) The usefulness of toluidine staining as a diagnostic tool for precancerous and cancerous oropharyngeal and oral cavity lesions. Acta Otorhinolaryngol Ital 29(4):187–190

    PubMed  PubMed Central  Google Scholar 

  19. Guneri P, Epstein JB, Kaya A, Veral A, Kazandi A, Boyacioglu H (2011) The utility of toluidine blue staining and brush cytology as adjuncts in clinical examination of suspicious oral mucosal lesions. Int J Oral Maxillofac Surg 40(2):155–161. https://doi.org/10.1016/j.ijom.2010.10.022

    Article  PubMed  Google Scholar 

  20. Onofre MA, Sposto MR, Navarro CM, Paulo S (2001) Reliability of toluidine blue application in the detection of oral epithelial dysplasia and in situ and invasive squamous cell carcinomas. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 91(5):535–540. https://doi.org/10.1067/moe.2001.112949

    Article  PubMed  Google Scholar 

  21. Macey R, Walsh T, Brocklehurst P, Kerr AR, Liu JLY, Lingen MW, Ogden GR, Warnakulasuriya S, Scully C (2015) Diagnostic tests for oral cancer and potentially malignant disorders in patients presenting with clinically evident lesions. Cochrane Database Syst Rev 5:CD010276. https://doi.org/10.1002/14651858.cd010276.pub2

    Article  Google Scholar 

  22. Pallagatti S, Sheikh S, Aggarwal A, Gupta D, Singh R, Handa R, Kaur S, Mago J (2013) Toluidine blue staining as an adjunctive tool for early diagnosis of dysplastic changes in the oral mucosa. J Clin Exp Dent 5(4):e187–e191. https://doi.org/10.4317/jced.51121

    Article  PubMed  PubMed Central  Google Scholar 

  23. Vashisht N, Ravikiran A, Samatha Y, Rao PC, Naik R, Vashisht D (2014) Chemiluminescence and toluidine blue as diagnostic tools for detecting early stages of oral Cancer: an invivo study. J Clin Diagn Res 8(4):ZC35–ZC38. https://doi.org/10.7860/JCDR/2014/7746.4259

    Article  PubMed  PubMed Central  Google Scholar 

  24. Krishnan L, Karpagaselvi K, Kumarswamy J, Sudheendra US, Santosh KV, Patil A (2016) Inter- and intra-observer variability in three grading systems for oral epithelial dysplasia. J Oral Maxillofac Pathol 20(2):261–268. https://doi.org/10.4103/0973-029X.185928

    Article  PubMed  PubMed Central  Google Scholar 

  25. Gandolfo S, Pentenero M, Broccoletti R, Pagano M, Carrozzo M, Scully C (2006) Toluidine blue uptake in potentially malignant oral lesions in vivo: clinical and histological assessment. Oral Oncol 42(1):88–94. https://doi.org/10.1016/j.oraloncology.2005.06.016

    Article  Google Scholar 

  26. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159–174

    Article  PubMed  Google Scholar 

  27. Lingen MW, Kalmar JR, Karrison T, Speight PM (2008) Critical evaluation of diagnostic aids for the detection of oral cancer. Oral Oncol 44(1):10–22. https://doi.org/10.1016/j.oraloncology.2007.06.011

    Article  PubMed  Google Scholar 

  28. Guneri P, Epstein JB, Ergun S, Boyacioglu H (2011) Toluidine blue color perception in identification of oral mucosal lesions. Clin Oral Investig 15(3):337–345. https://doi.org/10.1007/s00784-010-0398-6

    Article  PubMed  Google Scholar 

  29. Patton LL, Epstein JB, Kerr AR (2008) Adjunctive techniques for oral cancer examination and lesion diagnosis: a systematic review of the literature. J Am Dent Assoc 139(7):896–905. https://doi.org/10.14219/jada.archive.2008.0276

    Article  PubMed  Google Scholar 

  30. Epstein JB, Silverman S,J, Epstein JD, Lonky SA, Bride MA (2008) Analysis of oral lesion biopsies identified and evaluated by visual examination, chemiluminescence and toluidine blue. Oral Oncol 44(6):538–544. https://doi.org/10.1016/j.oraloncology.2007.08.011

    Article  PubMed  Google Scholar 

  31. Mashberg A (1980) Reevaluation of toluidine blue application as a diagnostic adjunct in the detection of asymptomatic oral squamous carcinoma: a continuing prospective study of oral cancer III. Cancer 46(758–763):2408–2410. https://doi.org/10.1001/jama.1981.03310480024019

    Article  Google Scholar 

  32. Paternot G, Devroe J, Debrock S, D'Hooghe TM, Spiessens C (2009) Intra- and inter-observer analysis in the morphological assessment of early-stage embryos. Reprod Biol Endocrinol 7:105. https://doi.org/10.1186/1477-7827-7-105

    Article  PubMed  PubMed Central  Google Scholar 

  33. Pegard C, Gallazzini-Crépin C, Giai J, Dubreuil J, Caoduro C, Desruet M-D, Roux J, Calizzano A, Fagret D, Lamesa C, Boulahdour H, Vuillez J-P (2014) Study of inter- and intra-observer reproducibility in the interpretation of [18F]choline PET/CT examinations in patients suffering from biochemically recurrent prostate cancer following curative treatment. EJNMMI Res 4:25. https://doi.org/10.1186/s13550-014-0025-7

    Article  PubMed  PubMed Central  Google Scholar 

  34. Álvaro L, Moreira H, Lillo J, Franklin A (2015) Color preference in red–green dichromats. Proc Natl Acad Sci U S A 112(30):9316–9321. https://doi.org/10.1073/pnas.1502104112

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

This work was supported by grants from the National Natural Science Foundation of China (No.81371147 and No. 81771080) to Prof. Zhou Gang.

Author information

Authors and Affiliations

  1. The State Key Laboratory Breeding Base of Basic Science of Stomatology (Hubei-MOST) and Key Laboratory of Oral Biomedicine Ministry of Education, School and Hospital of Stomatology, Wuhan University, Wuhan, Hubei, People’s Republic of China

    Ya-Nan Li, Rui Lu, Jing Zhang & Gang Zhou

  2. Department of Oral Medicine, School and Hospital of Stomatology, Wuhan University, Luoyu Road 237, Wuhan, China

    Rui Lu, Jing Zhang & Gang Zhou

Authors
  1. Ya-Nan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gang Zhou.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

The study was approved by the Ethnical Committee Board of School and Hospital of Stomatology, Wuhan University.

Informed consent

Informed consent is not required for this study.

Electronic supplementary material

ESM 1

(DOCX 19.3 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, YN., Lu, R., Zhang, J. et al. Inter-and intra-observer agreement on the judgment of toluidine blue staining for screening of oral potentially malignant disorders and oral cancer. Clin Oral Invest 23, 1709–1714 (2019). https://doi.org/10.1007/s00784-018-2595-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00784-018-2595-7

Keywords

Search

Navigation