Semi–supervised Learning for Image Modality Classification

  • Alba García Seco de HerreraEmail author
  • Dimitrios Markonis
  • Ranveer Joyseeree
  • Roger Schaer
  • Antonio Foncubierta-Rodríguez
  • Henning Müller
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9059)


Searching for medical image content is a regular task for many physicians, especially in radiology. Retrieval of medical images from the scientific literature can benefit from automatic modality classification to focus the search and filter out non–relevant items. Training datasets are often unevenly distributed regarding the classes resulting sometimes in a less than optimal classification performance. This article proposes a semi–supervised learning approach applied using a k–Nearest Neighbours (k–NN) classifier to exploit unlabelled data and to expand the training set. The algorithmic implementation is described and the method is evaluated on the ImageCLEFmed modality classification benchmark. Results show that this approach achieves an improved performance over supervised k–NN and Random Forest classifiers. Moreover, medical case–based retrieval also obtains higher performance when using the classified modalities as filter. This shows that image types can be classified well using visual information and they can then be used in a variety of applciations.


Semi–supervised learning Medical image classification Crowdsourcing Case–based retrieval 



This work was partly supported by the EU \(7^{th}\) Framework Program in the context of the Khresmoi project (FP7–257528).


  1. 1.
    Bow, H.C., Dattilo, J.R., Jonas, A.M., Lehmann, C.U.: A crowdsourcing model for creating preclinical medical education study tools. Acad. Med. 88(6), 766–770 (2013)CrossRefGoogle Scholar
  2. 2.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  3. 3.
    Cano, J., Pérez-Cortés, J.-C., Arlandis, J., Llobet, R.: Training set expansion in handwritten character recognition. In: Caelli, T.M., Amin, A., Duin, R.P.W., Kamel, M.S., de Ridder, D. (eds.) SPR 2002 and SSPR 2002. LNCS, vol. 2396, pp. 548–556. Springer, Heidelberg (2002) CrossRefGoogle Scholar
  4. 4.
    Chakravarti, R., Meng, X.: A study of color histogram based image retrieval. In: Sixth International Conference on Information Technology: New Generations ITNG, pp. 1323–1328 (2009)Google Scholar
  5. 5.
    Chan, H.-P., Wei, J., Zhang, Y., Helvie, M.A., Moore, R.H., Sahiner, B., Hadjiiski, L., Kopans, D.B.: Computer-aided detection of masses in digital tomosynthesis mammography: comparison of three approaches. Med. Phy. 35(9), 4087–4095 (2008)CrossRefGoogle Scholar
  6. 6.
    Chapelle, O., Schölkopf, B., Zien, A., et al.: Semi-Supervised Learning, vol. 2. MIT press, Cambridge (2006)CrossRefGoogle Scholar
  7. 7.
    Chatzichristofis, S.A., Boutalis, Y.S.: CEDD: Color and edge directivity descriptor: a compact descriptor for image indexing and retrieval. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 312–322. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  8. 8.
    Chatzichristofis, S.A., Boutalis, Y.S.: FCTH: Fuzzy color and texture histogram: a low level feature for accurate image retrieval. In: Proceedings of the 9th International Workshop on Image Analysis for Multimedia Interactive Service, pp. 191–196 (2008)Google Scholar
  9. 9.
    Chhatkuli, A., Markonis, D., Foncubierta-Rodríguez, A., Meriaudeau, F., Müller, H.: Separating compound figures in journal articles to allow for subfigure classification. In: SPIE Medical Imaging (2013)Google Scholar
  10. 10.
    Csurka, G., Clinchant, S., Jacquet, G.: XRCE’s participation at medical image modality classification and ad-hoc retrieval task of ImageCLEFmed 2011. In: Working Notes of CLEF 2011 (2011)Google Scholar
  11. 11.
    García Seco de Herrera, A., Foncubierta-Rodríguez, A., Markonis, D., Schaer, R., Müller, H.: Crowdsourcing for medical image classification. In: Annual Congress SGMI 2014 (2014)Google Scholar
  12. 12.
    García Seco de Herrera, A., Kalpathy-Cramer, J., Fushman, D.D., Antani, S., Müller, H.: Overview of the ImageCLEF 2013 medical tasks. In: Working Notes of CLEF 2013 (Cross Language Evaluation Forum), September 2013Google Scholar
  13. 13.
    García Seco de Herrera, A., Markonis, D., Müller, H.: Bag–of–colors for biomedical document image classification. In: Greenspan, H., Müller, H., Syeda-Mahmood, T. (eds.) MCBR-CDS 2012. LNCS, vol. 7723, pp. 110–121. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  14. 14.
    García Seco de Herrera, A., Markonis, D., Schaer, R., Eggel, I., Müller, H.: The medGIFT group in ImageCLEFmed 2013. In: Working Notes of CLEF 2013 (Cross Language Evaluation Forum), September 2013Google Scholar
  15. 15.
    García Seco de Herrera, A., Schaer, R., Müller, H.: Comparing fusion techniques for the ImageCLEF 2013 medical case retrieval task. Comput. Med. Imaging Graph. 39, 46–54 (2015)CrossRefGoogle Scholar
  16. 16.
    Good, B.M., Andrew, I.S.: Crowdsourcing for bioinformatics. Bioinformatics 16(29), 1925–1933 (2013)CrossRefGoogle Scholar
  17. 17.
    Han, J., Ma, K.-K.: Fuzzy color histogram and its use in color image retrieval. IEEE Trans. Image Process. 11(8), 944–952 (2002)CrossRefGoogle Scholar
  18. 18.
    Hanbury, A., Boyer, C., Gschwandtner, M., Müller, H.: KHRESMOI: Towards a multi-lingual search and access system for biomedical information. In: Med-e-Tel, Luxembourg 2011, pp. 412–416 (2011)Google Scholar
  19. 19.
    Kalpathy-Cramer, J., García Seco de Herrera, A., Demner-Fushman, D., Antani, S., Bedrick, S., Müller, H.: Evaluating performance of biomedical image retrieval systems an overview of the medical image retrieval task at ImageCLEF 2004–2014. Comput. Med. Imaging Graph. 39, 55–61 (2015)CrossRefGoogle Scholar
  20. 20.
    Kitanovski, I., Dimitrovski, I., Loskovska, S.: FCSE at medical tasks of ImageCLEF 2013.In: Working Notes of CLEF 2013 (Cross Language Evaluation Forum), September 2013Google Scholar
  21. 21.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: Proceedings of the 2006 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 2169–2178. IEEE Computer Society, Washington, DC (2006)Google Scholar
  22. 22.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  23. 23.
    Markonis, D., Holzer, M., Dungs, S., Vargas, A., Langs, G., Kriewel, S., Müller, H.: A survey on visual information search behavior and requirements of radiologists. Methods Inf. Med. 51(6), 539–548 (2012)CrossRefGoogle Scholar
  24. 24.
    Mourão, A., Martins, F., Magalhães, J.: NovaSearch on medical ImageCLEF 2013. In: Working Notes of CLEF 2013 (Cross Language Evaluation Forum), September 2013Google Scholar
  25. 25.
    Müller, H., Michoux, N., Bandon, D., Geissbuhler, A.: A review of content-based image retrieval systems in medicine-clinical benefits and future directions. Int. J. Med. Inf. 73(1), 1–23 (2004)CrossRefGoogle Scholar
  26. 26.
    Ranard, B.L., Ha, Y.P., Meisel, Z.F., Asch, D.A., Hill, S.S., Becker, L.B., Seymour, A.K., Merchant, R.M.: Crowdsourcing-harnessing the masses to advance health and medicine, a systematic review. J. General Internal Med. 29(1), 187–203 (2014)CrossRefGoogle Scholar
  27. 27.
    Schaer, R., Markonis, D., Müller, H.: Architecture and applications of the parallel distributed image search engine (ParaDISE). In: FoRESEE 2014, 1st International Workshop on Future Search Engines at INFORMATIK 2014 (2014)Google Scholar
  28. 28.
    Tamura, H., Mori, S., Yamawaki, T.: Textural features corresponding to visual perception. IEEE Trans. Syst. Man Cybern. 8(6), 460–473 (1978)CrossRefGoogle Scholar
  29. 29.
    Vajda, S., You, D., Antani, S.K., Thoma, G.R.: Label the many with a few: Semi-automatic medical image modality discovery in a large image collection. In: 2014 IEEE Symposium on Computational Intelligence in Healthcare and e-health (CICARE), pp. 167–173. IEEE (2014)Google Scholar
  30. 30.
    Yip, K.Y., Gerstein, M.: Training set expansion: An approach to improving the reconstruction of biological networks from limited and uneven reliable interactions. Bioinformatics 25(2), 243–250 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Alba García Seco de Herrera
    • 1
    Email author
  • Dimitrios Markonis
    • 1
  • Ranveer Joyseeree
    • 1
    • 2
  • Roger Schaer
    • 1
  • Antonio Foncubierta-Rodríguez
    • 2
  • Henning Müller
    • 1
  1. 1.University of Applied Sciences Western Switzerland (HES–SO)SierreSwitzerland
  2. 2.Swiss Federal Institute of TechnologyZurichSwitzerland

Personalised recommendations