Discriminative Subtree Selection for NBI Endoscopic Image Labeling

  • Tsubasa Hirakawa
  • Toru Tamaki
  • Takio Kurita
  • Bisser Raytchev
  • Kazufumi Kaneda
  • Chaohui Wang
  • Laurent Najman
  • Tetsushi Koide
  • Shigeto Yoshida
  • Hiroshi Mieno
  • Shinji Tanaka
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10117)


In this paper, we propose a novel method for image labeling of colorectal Narrow Band Imaging (NBI) endoscopic images based on a tree of shapes. Labeling results could be obtained by simply classifying histogram features of all nodes in a tree of shapes, however, satisfactory results are difficult to obtain because histogram features of small nodes are not enough discriminative. To obtain discriminative subtrees, we propose a method that optimally selects discriminative subtrees. We model an objective function that includes the parameters of a classifier and a threshold to select subtrees. Then labeling is done by mapping the classification results of nodes of the subtrees to those corresponding image regions. Experimental results on a dataset of 63 NBI endoscopic images show that the proposed method performs qualitatively and quantitatively much better than existing methods.



This work was supported in part by JSPS KAKENHI grants numbers JP14J00223 and JP26280015.


  1. 1.
    Cancer Research, U.K.: Bowel cancer statistics (2015). Accessed 7 Aug 2016
  2. 2.
    Meining, A., Rösch, T., Kiesslich, R., Muders, M., Sax, F., Heldwein, W.: Inter- and intra-observer variability of magnification chromoendoscopy for detecting specialized intestinal metaplasia at the gastroesophageal junction. Endoscopy 36, 160–164 (2004)CrossRefGoogle Scholar
  3. 3.
    Mayinger, B., Oezturk, Y., Stolte, M., Faller, G., Benninger, J., Schwab, D., Maiss, J., Hahn, E.G., Muehldorfer, S.: Evaluation of sensitivity and inter- and intra-observer variability in the detection of intestinal metaplasia and dysplasia in barrett’s esophagus with enhanced magnification endoscopy. Scand. J. Gastroenterol. 41, 349–356 (2006)CrossRefGoogle Scholar
  4. 4.
    Oba, S., Tanaka, S., Oka, S., Kanao, H., Yoshida, S., Shimamoto, F., Chayama, K.: Characterization of colorectal tumors using narrow-band imaging magnification: combined diagnosis with both pit pattern and microvessel features. Scand. J. Gastroenterol. 45, 1084–1092 (2010)CrossRefGoogle Scholar
  5. 5.
    Takemura, Y., Yoshida, S., Tanaka, S., Kawase, R., Onji, K., Oka, S., Tamaki, T., Raytchev, B., Kaneda, K., Yoshihara, M., Chayama, K.: Computer-aided system for predicting the histology of colorectal tumors by using narrow-band imaging magnifying colonoscopy (with video). Gastrointest. Endosc. 75, 179–185 (2012)CrossRefGoogle Scholar
  6. 6.
    Tamaki, T., Yoshimuta, J., Kawakami, M., Raytchev, B., Kaneda, K., Yoshida, S., Takemura, Y., Onji, K., Miyaki, R., Tanaka, S.: Computer-aided colorectal tumor classification in NBI endoscopy using local features. Med. Image Anal. 17, 78–100 (2013)CrossRefGoogle Scholar
  7. 7.
    Kanao, H., Tanaka, S., Oka, S., Hirata, M., Yoshida, S., Chayama, K.: Narrow-band imaging magnification predicts the histology and invasion depth of colorectal tumors. Gastrointest. Endosc. 69, 631–636 (2009)CrossRefGoogle Scholar
  8. 8.
    Kominami, Y., Yoshida, S., Tanaka, S., Sanomura, Y., Hirakawa, T., Raytchev, B., Tamaki, T., Koide, T., Kaneda, K., Chayama, K.: Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy. Gastrointest. Endosc. 83, 643–649 (2016)CrossRefGoogle Scholar
  9. 9.
    Hirakawa, T., Tamaki, T., Raytchev, B., Kaneda, K., Koide, T., Kominami, Y., Yoshida, S., Tanaka, S.: SVM-MRF segmentation of colorectal NBI endoscopic images. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4739–4742 (2014)Google Scholar
  10. 10.
    Xia, G.S., Delon, J., Gousseau, Y.: Shape-based invariant texture indexing. Int. J. Comput. Vis. 88, 382–403 (2010)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Monasse, P., Guichard, F.: Fast computation of a contrast-invariant image representation. IEEE Trans. Image Process. 9, 860–872 (2000)CrossRefGoogle Scholar
  12. 12.
    Gross, S., Kennel, M., Stehle, T., Wulff, J., Tischendorf, J., Trautwein, C., Aach, T.: Polyp segmentation in NBI colonoscopy. In: Meinzer, H.P., Deserno, T.M., Handels, H., Tolxdorff, T. (eds.) Bildverarbeitung für die Medizin 2009, pp. 252–256. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  13. 13.
    Ganz, M., Yang, X., Slabaugh, G.: Automatic segmentation of polyps in colonoscopic narrow-band imaging data. IEEE Trans. Biomed. Eng. 59, 2144–2151 (2012)CrossRefGoogle Scholar
  14. 14.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 898–916 (2011)CrossRefGoogle Scholar
  15. 15.
    Collins, T., Bartoli, A., Bourdel, N., Canis, M.: Segmenting the uterus in monocular laparoscopic images without manual input. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 181–189. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24574-4_22 CrossRefGoogle Scholar
  16. 16.
    Bernal, J., Sánchez, J., Vilariño, F.: A region segmentation method for colonoscopy images using a model of polyp appearance. In: Vitrià, J., Sanches, J.M., Hernández, M. (eds.) IbPRIA 2011. LNCS, vol. 6669, pp. 134–142. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-21257-4_17 CrossRefGoogle Scholar
  17. 17.
    Hegadi, R.S., Goudannavar, B.A.: Interactive segmentation of medical images using grabcut. Int. J. Mach. Intell. 3, 168–171 (2011)CrossRefGoogle Scholar
  18. 18.
    Breier, M., Gross, S., Behrens, A., Stehle, T., Aach, T.: Active contours for localizing polyps in colonoscopic NBI image data (2011)Google Scholar
  19. 19.
    Figueiredo, I.N., Figueiredo, P.N., Stadler, G., Ghattas, O., Araujo, A.: Variational image segmentation for endoscopic human colonic aberrant crypt foci. IEEE Trans. Med. Imaging 29, 998–1011 (2010)CrossRefGoogle Scholar
  20. 20.
    Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10, 266–277 (2001)CrossRefzbMATHGoogle Scholar
  21. 21.
    Nosrati, M.S., Amir-Khalili, A., Peyrat, J.M., Abinahed, J., Al-Alao, O., Al-Ansari, A., Abugharbieh, R., Hamarneh, G.: Endoscopic scene labelling and augmentation using intraoperative pulsatile motion and colour appearance cues with preoperative anatomical priors. Int. J. Comput. Assist. Radiol. Surg. 11, 1409–1418 (2016)CrossRefGoogle Scholar
  22. 22.
    Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1915–1929 (2013)CrossRefGoogle Scholar
  23. 23.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2015)Google Scholar
  24. 24.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)Google Scholar
  25. 25.
    Liu, F., Lin, G., Shen, C.: CRF learning with CNN features for image segmentation. Pattern Recogn. 48, 2983–2992 (2015). Discriminative Feature Learning from Big Data for Visual RecognitionCrossRefGoogle Scholar
  26. 26.
    Noh, H., Hong, S., Han, B.: Learning deconvolution network for semantic segmentation. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1520–1528 (2015)Google Scholar
  27. 27.
    Jones, R.: Connected filtering and segmentation using component trees. Comput. Vis. Image Underst. 75, 215–228 (1999)CrossRefGoogle Scholar
  28. 28.
    Najman, L., Couprie, M.: Building the component tree in quasi-linear time. IEEE Trans. Image Process. 15, 3531–3539 (2006)CrossRefGoogle Scholar
  29. 29.
    Salembier, P., Garrido, L.: Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval. IEEE Trans. Image Process. 9, 561–576 (2000)CrossRefGoogle Scholar
  30. 30.
    Cousty, J., Najman, L.: Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts. In: Soille, P., Pesaresi, M., Ouzounis, G.K. (eds.) ISMM 2011. LNCS, vol. 6671, pp. 272–283. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-21569-8_24 CrossRefGoogle Scholar
  31. 31.
    Xu, Y., Géraud, T., Najman, L.: Two applications of shape-based morphology: blood vessels segmentation and a generalization of constrained connectivity. In: Hendriks, C.L.L., Borgefors, G., Strand, R. (eds.) ISMM 2013. LNCS, vol. 7883, pp. 390–401. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-38294-9_33 CrossRefGoogle Scholar
  32. 32.
    Dufour, A., Tankyevych, O., Naegel, B., Talbot, H., Ronse, C., Baruthio, J., Dokládal, P., Passat, N.: Filtering and segmentation of 3D angiographic data: advances based on mathematical morphology. Med. Image Anal. 17, 147–164 (2013)CrossRefGoogle Scholar
  33. 33.
    Perret, B., Collet, C.: Connected image processing with multivariate attributes: an unsupervised Markovian classification approach. Comput. Vis. Image Underst. 133, 1–14 (2015)CrossRefGoogle Scholar
  34. 34.
    Liu, G., Xia, G.S., Yang, W., Zhang, L.: Texture analysis with shape co-occurrence patterns. In: Pattern Recognition (ICPR), pp. 1627–1632 (2014)Google Scholar
  35. 35.
    He, C., Zhuo, T., Su, X., Tu, F., Chen, D.: Local topographic shape patterns for texture description. IEEE Sig. Process. Lett. 22, 871–875 (2015)CrossRefGoogle Scholar
  36. 36.
    Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: LIBLINEAR: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)zbMATHGoogle Scholar
  37. 37.
    Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26, 297–302 (1945)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Tsubasa Hirakawa
    • 1
  • Toru Tamaki
    • 1
  • Takio Kurita
    • 1
  • Bisser Raytchev
    • 1
  • Kazufumi Kaneda
    • 1
  • Chaohui Wang
    • 2
  • Laurent Najman
    • 2
  • Tetsushi Koide
    • 3
  • Shigeto Yoshida
    • 4
  • Hiroshi Mieno
    • 4
  • Shinji Tanaka
    • 5
  1. 1.Hiroshima UniversityHigashihiroshimaJapan
  2. 2.Laboratoire d’Informatique Gaspard-MongeUniversité Paris-EstChamps-sur-MarneFrance
  3. 3.Research Institute for Nanodevice and Bio Systems (RNBS)Hiroshima UniversityHigashihiroshimaJapan
  4. 4.Hiroshima General Hospital of West Japan Railway CompanyHiroshimaJapan
  5. 5.Hiroshima University HospitalHiroshimaJapan

Personalised recommendations