Clinical Oral Investigations

, Volume 23, Issue 1, pp 99–105 | Cite as

Detection accuracy of maxillary sinus floor septa in panoramic radiographs using CBCT as gold standard: a multi-observer receiver operating characteristic (ROC) study

  • AC Lang
  • RK SchulzeEmail author
Original Article



To investigate diagnostic accuracy of panoramic radiography in detecting maxillary sinus floor septa by means of a multi-observer receiver operating characteristic (ROC) analysis and a standardized protocol for reporting (STARD protocol; Clin Chem 49(1):1–6, 2003).

Material and methods

From our database, 25 cone beam computed tomographies (CBCTs) were selected with one maxillary sinus floor septum (height ≥ 2.5 mm). For the same patient, a recent panoramic radiograph (PAN) had to be available in the database. As controls, 28 CBCTs plus corresponding PANs without evidence of a sinus septum were selected. Using the CBCTs as ground truth, 17 observers from our dental school on a five-point confidence scale rated both sinuses in all 53 PANs with respect to presence/absence of a sinus septum. Areas beneath ROC curves (Az-values), sensitivity/specificity (SNT/SPF), positive/negative predictive values (PPV, NPV), and positive/negative likelihood ratios (LR+, LR−) were computed for each observer and pooled over all observers. Inter-rater reproducibility was assessed by means of the intraclass coefficient (ICC) using a two-way random effects model.


A pooled Az-value of 0.839 was observed (SNT 84.6%, SPF 73.5%). PPV ranged between 0.492 and 0.824 (median 0.627) and NPV between 0.838 and 0.976 (median 0.917). A median LR+ of 3.567 was computed (LR− median 0.193). Inter-rater reliability revealed an ICC of 0.55 (95% confidence interval 0.48 to 0.62).


Our results indicate that PAN is a moderately accurate method for sinus elevation planning for the purpose of septum detection. Ruling out a septum by PAN seems to work more accurately than ruling in.

Clinical relevance

For the purpose of maxillary sinus floor septa detection, panoramic radiography can be relatively safely advocated, particularly for judgment of a septum-free sinus.


Maxillary sinus septa CBCT Diagnostic accuracy Panoramic radiography ROC analysis 



The manuscript bases on the thesis of Alexandra Carina Lang entitled “Comparison of accuracy of panoramic versus Cone Beam Computed Tomography radiographs regarding maxillary sinus-floor septa: ROC-analysis.”


The entire study has been solely financed by the University Medical Center of the Johannes Gutenberg-University of Mainz.

Compliance with ethical standards

Conflict of interest

For the submitted work: Both authors declare there were no potential conflicts of interest involved with this research.

Outside the submitted work: Author Ralf Schulze has received a research grant from Sirona Dental Systems GmbH for a different study. Author Ralf Schulze is non-honorarium-based member of several committees of the German Institut of Standardization (DIN) and also of a national technical committee on radiation protection (Arbeitskreis Röntenverordnung, AKRöV). He is also representative for dental radiology and radiation protection for the World Dental Federation (FDI).

Ethical approval

As retrospective study using radiographs from an existing database without publication of any patient-related information, no ethical approval is required in our University Medical Center.

Informed consent

Since all the data (radiographic images) used for the study were taken from the existing database and were used in an anonymized fashion, no extra informed consent was required for this type of study. In our center, patients give a general consent that their data in anonymized fashion may be used for scientific study purposes, if their personal data rights and medical confidentiality is adhered.


  1. 1.
    Kim M-J, Jung U-W, Kim C-S, Kim KD, Choi SH, Kim CK, Cho KS (2006) Maxillary sinus septa: prevalence, height, location, and morphology. A reformatted computed tomography scan analysis. J Periodontol 7(5):903–908. CrossRefGoogle Scholar
  2. 2.
    Krennmair G, Ulm C, Lugmayr H (1997) Maxillary sinus septa: incidence, morphology and clinical implications. J Craniomaxillofac Surg 25(5):261–265CrossRefGoogle Scholar
  3. 3.
    Krennmair G, Ulm CW, Lugmayr H, Solar P (1999) The incidence, location, and height of maxillary sinus septa in the edentulous and dentate maxilla. J Oral Maxillofac Surg 57(6):667–671 discussion 671-2CrossRefGoogle Scholar
  4. 4.
    Ulm CW, Solar P, Krennmair G, Matejka M, Watzek G (1995) Incidence and suggested surgical management of septa in sinus-lift procedures. Int J Oral Maxillofac Implants 10(4):462–465Google Scholar
  5. 5.
    Bornstein MM, Seiffert C, Maestre-Ferrín L, Fodich I, Jacobs R, Buser D, von Arx T (2016) An analysis of frequency, morphology, and locations of maxillary sinus septa using cone beam computed tomography. Int J Oral Maxillofac Implants 31(2):280–287. CrossRefGoogle Scholar
  6. 6.
    Pommer B, Ulm C, Lorenzoni M, Palmer R, Watzek G, Zechner W (2012) Prevalence, location and morphology of maxillary sinus septa: systematic review and meta-analysis. J Clin Periodontol 39(8):769–773. CrossRefGoogle Scholar
  7. 7.
    Schwarz L, Schiebel V, Hof M, Ulm C, Watzek G, Pommer B (2015) Risk factors of membrane perforation and postoperative complications in sinus floor elevation surgery: review of 407 augmentation procedures. J Oral Maxillofac Surg 73(7):1275–1282. CrossRefGoogle Scholar
  8. 8.
    von Arx T, Fodich I, Bornstein MM et al (2014) Perforation of the sinus membrane during sinus floor elevation: a retrospective study of frequency and possible risk factors. Int J Oral Maxillofac Implants 29(3):718–726. CrossRefGoogle Scholar
  9. 9.
    González-Santana H, Peñarrocha-Diago M, Guarinos-Carbó J, Sorní-Bröker M (2007) A study of the septa in the maxillary sinuses and the subantral alveolar processes in 30 patients. J Oral Implantol 33(6):340–343.[340:ASOTSI]2.0.CO;2Google Scholar
  10. 10.
    Orhan K, Kusakci Seker B, Aksoy S, Bayindir H, Berberoglu A, Seker E (2013) Cone beam CT evaluation of maxillary sinus septa prevalence, height, location and morphology in children and an adult population. Med Princ Pract 22(1):47–53. CrossRefGoogle Scholar
  11. 11.
    Šimundić A-M (2009) Measures of diagnostic accuracy: basic definitions. Electron J Int Fed Clin Chem Lab Med 19(4):203–211Google Scholar
  12. 12.
    Hanley JA (1989) Receiver operating characteristic (ROC) methodology: the state of the art. Crit Rev Diagn Imaging 29:307–335Google Scholar
  13. 13.
    Metz CE (2006) Receiver operating characteristic analysis: a tool for the quantitative evaluation of observer performance and imaging systems. J Am Coll Radiol 3(6):413–422. CrossRefGoogle Scholar
  14. 14.
    Ludlow JB, Timothy R, Walker C, Hunter R, Benavides E, Samuelson DB (2015) Correction to effective dose of dental CBCT—a meta analysis of published data and additional data for nine CBCT units. Dentomaxillofac Radiol 44(7):20159003. CrossRefGoogle Scholar
  15. 15.
    Pauwels R (2015) Cone beam CT for dental and maxillofacial imaging: dose matters. RadiatProt Dosim 165(1–4):156–161. CrossRefGoogle Scholar
  16. 16.
    Maestre-Ferrín L, Carrillo-García C, Galán-Gil S et al (2011) Prevalence, location, and size of maxillary sinus septa: panoramic radiograph versus computed tomography scan. J Oral Maxillofac Surg 69(2):507–511. CrossRefGoogle Scholar
  17. 17.
    Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Lijmer JG, Moher D, Rennie D, de Vet HC, Standards for Reporting of Diagnostic Accuracy (2003) Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for reporting of diagnostic accuracy. Clin Chem 49(1):1–6CrossRefGoogle Scholar
  18. 18.
    Alpert HR, Hillman BJ (2004) Quality and variability in diagnostic radiology. J Am Coll Radiol 1(2):127–132. CrossRefGoogle Scholar
  19. 19.
    Robinson PJ (1997) Radiology’s Achilles’ heel: error and variation in the interpretation of the Röntgen image. Br J Radiol 70(839):1085–1098. CrossRefGoogle Scholar
  20. 20.
    See J, Howe S, Warm J et al (1995) Meta-analysis of the sensitivity decrement in vigilance. Psychol Bull 117:230–249CrossRefGoogle Scholar
  21. 21.
    Bundesregierung BRD (2002) Verordnung zur Änderung der Röntgenverordnung und anderer atomrechtlicher Verordnungen: Röntgenverordnung, BGBl G5702, Nr.36Google Scholar
  22. 22.
    Deutsches Institut für Normung DIN (2001) Sicherung der Bildqualität in röntgendiagnostischen Betrieben – Teil 57: Abnahmeprüfung an Bildwiedergabegeräten(6868–57)Google Scholar
  23. 23.
    Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155–163. CrossRefGoogle Scholar
  24. 24.
    Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86(2):420–428CrossRefGoogle Scholar
  25. 25.
    R Core Team (2017) R: A language and environment for statistical computing.
  26. 26.
    Harris D, Horner K, Gröndahl K, Jacobs R, Helmrot E, Benic GI, Bornstein MM, Dawood A, Quirynen M (2012) E.A.O. guidelines for the use of diagnostic imaging in implant dentistry 2011. A consensus workshop organized by the European Association for Osseointegration at the Medical University of Warsaw. Clin Oral Impl Res 23(11):1243–1253. CrossRefGoogle Scholar
  27. 27.
    Bornstein MM, Scarfe WC, Vaughn VM et al (2014) Cone beam computed tomography in implant dentistry: a systematic review focusing on guidelines, indications, and radiation dose risks. Int J Oral Maxillofac Implants 29(Suppl):55–77. CrossRefGoogle Scholar
  28. 28.
    Goldmann M, Pearson AH, Darzenta N (1974) Reliability of radiographic interpretations. Oral Surg 38(2):287–293CrossRefGoogle Scholar
  29. 29.
    Kaffe I, Gratt BM (1988) Variations in the radiographic interpretation of the periapical dental region. J Endod 14(7):330–335. CrossRefGoogle Scholar
  30. 30.
    Park Y-B, Jeon H-S, Shim J-S, Lee KW, Moon HS (2011) Analysis of the anatomy of the maxillary sinus septum using 3-dimensional computed tomography. J Oral Maxillofac Surg 69(4):1070–1078. CrossRefGoogle Scholar
  31. 31.
    Chakraborty DP (1989) Maximum likelihood analysis of free-response receiver operating characteristic (FROC) data. Med Phys 16(4):561–568. CrossRefGoogle Scholar
  32. 32.
    Flahault A, Cadilhac M, Thomas G (2005) Sample size calculation should be performed for design accuracy in diagnostic test studies. J Clin Epidemiol 58(8):859–862. CrossRefGoogle Scholar
  33. 33.
    European Commission (2012) Radiation protection no 172: cone beam ct for dental and maxillofacial radiology. Evidence based guidelines: evidence based guidelines: a report prepared by the sedentexct project, 2011 v20: 1–139Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Oral and Maxillofacial Surgery and Plastic Surgery, Section of Oral RadiologyUniversity Medical Centre, Johannes Gutenberg University of MainzMainzGermany

Personalised recommendations