Skip to main content

Automated Detection of Dual p16/Ki67 Nuclear Immunoreactivity in Liquid-Based Pap Tests for Improved Cervical Cancer Risk Stratification


The Papanicolau (Pap) test is a routine cytological procedure for early detection of dysplastic lesions in cervical epithelium. A reliable screening method is crucial for triage of women at risk; however manual screening and interpretation are associated with relatively low sensitivity and substantial interobserver diagnostic variability. P16 and Ki67 biomarkers have been recently proposed as adjunctive tools in the diagnosis of high-risk human papillomavirus (hrHPV) associated dysplasias to supplement the morphological characteristics of cells by additional colorimetric features. In this study, an automated technique for the evaluation of dual p16/Ki67 immunoreactivity in cervical cell nuclei is introduced. Smears stained with p16 and Ki67 antibodies were digitized, and analyzed by algorithms we developed. Gradient-based radial symmetry operator and adaptive processing of symmetry image were employed to obtain the nuclear mask. This step was followed by the extraction of features including pixel data and immunoreactivity signature from each nucleus. The features were analyzed by two support vector machine classifiers to assign a nucleus into one of four types of immunoreactivity: p16 positive (p16+/Ki67), Ki67 positive (p16/Ki67+), dual p16/Ki67 positive (p16+/Ki67+) and negative (p16/Ki67), respectively. Results obtained by our method correlated well with readings by two cytopathologists (n = 18,068 cells); p16+/Ki67+ nuclei were classified with respective precisions of 77.1% and 82.6%. Specificity in identification of p16/Ki67 nuclei was better than 99.5%, and the sensitivity in detection of all immunopositive nuclei was 86.3 and 89.4%, respectively. We found that the quantitative characterization of immunoreactivity provided by the additional highlighting of classified nuclei can positively impact the efficacy and screening outcome of the Pap test.

This is a preview of subscription content, access via your institution.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7


  1. 1.

    Bose, S., H. Evans, L. Lantzy, K. Scharre, and E. Youssef. p16(INK4A) is a surrogate biomarker for a subset of human papilloma virus-associated dysplasias of the uterine cervix as determined on the Pap smear. Diagn. Cytopathol. 32:21–24, 2005.

    PubMed  Article  Google Scholar 

  2. 2.

    Botev, Z. I., J. F. Grotowski, and D. P. Kroese. Kernel density estimation via diffusion. Ann. Stat. 38:2916–2957, 2010.

    Article  Google Scholar 

  3. 3.

    Byers, T., J. Mouchawar, J. Marks, B. Cady, N. Lins, G. M. Swanson, D. G. Bal, and H. Eyre. The American Cancer Society challenge goals. How far can cancer rates decline in the U.S. by the year 2015? Cancer 86:715–727, 1999.

    PubMed  Article  CAS  Google Scholar 

  4. 4.

    Dunton, C. J., K. H. van Hoeven, A. J. Kovatich, R. E. Oliver, R. Q. Scacheri, J. R. Cater, and J. A. Carlson, Jr. Ki-67 antigen staining as an adjunct to identifying cervical intraepithelial neoplasia. Gynecol. Oncol. 64:451–455, 1997.

    PubMed  Article  CAS  Google Scholar 

  5. 5.

    Haralick, R. M., and L. G. Shapiro. Computer and Robot Vision. Reading, MA: Addison-Wesley, 1992.

    Google Scholar 

  6. 6.

    Juric, D., V. Mahovlic, S. Rajhvajn, A. Ovanin-Rakic, L. Skopljanac-Macina, A. Barisic, I. S. Projic, D. Babic, M. Susa, A. Corusic, and S. Oreskovic. Liquid-based cytology—new possibilities in the diagnosis of cervical lesions. Coll. Antropol. 34:19–24, 2010.

    PubMed  Google Scholar 

  7. 7.

    Longatto Filho, A., M. L. Utagawa, N. K. Shirata, S. M. Pereira, G. M. Namiyama, C. T. Kanamura, C. Santos Gda, M. A. de Oliveira, A. Wakamatsu, S. Nonogaki, C. Roteli-Martins, C. di Loreto, G. de Mattosinho Castro Ferraz Mda, M. Y. Maeda, V. A. Alves, and K. Syrjanen. Immunocytochemical expression of p16INK4A and Ki-67 in cytologically negative and equivocal pap smears positive for oncogenic human papillomavirus. Int. J. Gynecol. Pathol. 24:118–124, 2005.

    PubMed  Article  Google Scholar 

  8. 8.

    Loy, G., and A. Zelinsky. Fast radial symmetry for detecting points of interest. IEEE Trans. Pattern Anal. Mach. Intell. 25:959–973, 2003.

    Article  Google Scholar 

  9. 9.

    Mat-Isa, N. A. Automated edge detection technique for Pap smear images using moving K-means clustering and modified seed based region growing algorithm. Int. J. Comput. Internet Manag. 13:45–59, 2005.

    Google Scholar 

  10. 10.

    Meyer, J. L., D. W. Hanlon, B. T. Andersen, O. F. Rasmussen, and K. Bisgaard. Evaluation of p16INK4a expression in ThinPrep cervical specimens with the CINtec p16INK4a assay: correlation with biopsy follow-up results. Cancer 111:83–92, 2007.

    PubMed  Article  CAS  Google Scholar 

  11. 11.

    Nanda, K., D. C. McCrory, E. R. Myers, L. A. Bastian, V. Hasselblad, J. D. Hickey, and D. B. Matchar. Accuracy of the Papanicolaou test in screening for and follow-up of cervical cytologic abnormalities: a systematic review. Ann. Intern. Med. 132:810–819, 2000.

    PubMed  CAS  Google Scholar 

  12. 12.

    Nayar, R., and D. Solomon. Second edition of ‘The Bethesda System for reporting cervical cytology’—atlas, website, and Bethesda interobserver reproducibility project. Cytojournal 1:4, 2004.

    PubMed  Article  Google Scholar 

  13. 13.

    Perona, P., and J. Malik. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12:629–639, 1990.

    Article  Google Scholar 

  14. 14.

    Plissiti, M. E., C. Nikou, and A. Charchanti. Automated detection of cell nuclei in pap smear images using morphological reconstruction and clustering. IEEE Trans. Inf. Technol. Biomed. 15:233–241, 2011.

    PubMed  Article  Google Scholar 

  15. 15.

    Reagan, J. W., and M. J. Hamonic. The cellular pathology in carcinoma in situ; a cytohistopathological correlation. Cancer 9:385–402, 1956.

    PubMed  Article  CAS  Google Scholar 

  16. 16.

    Sahebali, S., C. E. Depuydt, G. A. Boulet, M. Arbyn, L. M. Moeneclaey, A. J. Vereecken, E. A. Van Marck, and J. J. Bogers. Immunocytochemistry in liquid-based cervical cytology: analysis of clinical use following a cross-sectional study. Int. J. Cancer 118:1254–1260, 2006.

    PubMed  Article  CAS  Google Scholar 

  17. 17.

    Sahebali, S., C. E. Depuydt, K. Segers, A. J. Vereecken, E. Van Marck, and J. J. Bogers. Ki-67 immunocytochemistry in liquid based cervical cytology: useful as an adjunctive tool? J. Clin. Pathol. 56:681–686, 2003.

    PubMed  Article  CAS  Google Scholar 

  18. 18.

    Schölkopf, B., C. J. C. Burges, and A. J. Smola. Advances in Kernel Methods: Support Vector Learning. Cambridge, MA: MIT, 1999.

    Google Scholar 

  19. 19.

    Scholzen, T., and J. Gerdes. The Ki-67 protein: from the known and the unknown. J. Cell. Physiol. 182:311–322, 2000.

    PubMed  Article  CAS  Google Scholar 

  20. 20.

    Schubert, J. M., B. Bird, K. Papamarkakis, M. Miljkovic, K. Bedrossian, N. Laver, and M. Diem. Spectral cytopathology of cervical samples: detecting cellular abnormalities in cytologically normal cells. Lab. Invest. 90:1068–1077, 2010.

    PubMed  Article  Google Scholar 

  21. 21.

    Siddiqi, A. M., H. Li, F. Faruque, W. Williams, K. Lai, M. Hughson, S. Bigler, J. Beach, and W. Johnson. Use of hyperspectral imaging to distinguish normal, precancerous, and cancerous cells. Cancer 114:13–21, 2008.

    PubMed  Article  Google Scholar 

  22. 22.

    Sirovich, B. E., and H. G. Welch. The frequency of Pap smear screening in the United States. J. Gen. Intern. Med. 19:243–250, 2004.

    PubMed  Article  Google Scholar 

  23. 23.

    Sobervilla, P., E. Montseny, F. Vaschetto, and E. Lerma. Fuzzy-based analysis of microscopic color cervical Pap smear images: nuclei detection. Int. J. Comput. Intell. Appl. 9:187–206, 2010.

    Article  Google Scholar 

  24. 24.

    Tsoumpou, I., M. Arbyn, M. Kyrgiou, N. Wentzensen, G. Koliopoulos, P. Martin-Hirsch, V. Malamou-Mitsi, and E. Paraskevaidis. p16(INK4a) immunostaining in cytological and histological specimens from the uterine cervix: a systematic review and meta-analysis. Cancer Treat. Rev. 35:210–220, 2009.

    PubMed  Article  CAS  Google Scholar 

  25. 25.

    Walts, A. E., and S. Bose. p16, Ki-67, and BD ProExC immunostaining: a practical approach for diagnosis of cervical intraepithelial neoplasia. Hum. Pathol. 40:957–964, 2009.

    PubMed  Article  CAS  Google Scholar 

  26. 26.

    Yang-Mao, S. F., Y. K. Chan, and Y. P. Chu. Edge enhancement nucleus and cytoplast contour detector of cervical smear images. IEEE Trans. Syst. Man. Cybern. B: Cybern. 38:353–366, 2008.

    Article  Google Scholar 

Download references


This work was supported in part by a grant from the Department of Surgery at Cedars-Sinai Medical Center, and in part by a NIH grant 5R21CA143618-02 (to AG). We also thank Dr Hunter Hardy M.D. for technical help in specimen imaging.

Conflict of interest

The authors declare that they have no conflict of interest.

Author information



Corresponding author

Correspondence to Arkadiusz Gertych.

Additional information

Associate Editor James Tunnell oversaw the review of this article.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Gertych, A., Joseph, A.O., Walts, A.E. et al. Automated Detection of Dual p16/Ki67 Nuclear Immunoreactivity in Liquid-Based Pap Tests for Improved Cervical Cancer Risk Stratification. Ann Biomed Eng 40, 1192–1204 (2012).

Download citation


  • Pap test
  • Cervical cancer screening
  • Immunocytochemistry
  • Computer analysis
  • Nuclei segmentation
  • Quantification