Advertisement

Automated Medical Image Modality Recognition by Fusion of Visual and Text Information

  • Noel Codella
  • Jonathan Connell
  • Sharath Pankanti
  • Michele Merler
  • John R. Smith
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8674)

Abstract

In this work, we present a framework for medical image modality recognition based on a fusion of both visual and text classification methods. Experiments are performed on the public ImageCLEF 2013 medical image modality dataset, which provides figure images and associated fulltext articles from PubMed as components of the benchmark. The presented visual-based system creates ensemble models across a broad set of visual features using a multi-stage learning approach that best optimizes per-class feature selection while simultaneously utilizing all available data for training. The text subsystem uses a pseudo-probabilistic scoring method based on detection of suggestive patterns, analyzing both the figure captions and mentions of the figures in the main text. Our proposed system yields state-of-the-art performance in all 3 categories of visual-only (82.2%), text-only (69.6%), and fusion tasks (83.5%).

Keywords

medical image modality image recognition image classification text visual fusion 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Frost & Sullivan: 2004 Healthcare Storage Report (2004), http://www.frost.com/c/10024/sublib/display-report.do
  2. 2.
    National Center for Health Statistics, http://www.cdc.gov/nchs/data/databriefs/db105.pdf
  3. 3.
    Muller, H., Seco de Herrera, A.G., Kalpathy-Cramer, J., Fushman, D.D., Antani, S., Eggel, I.: Overview of the ImageCLEF 2012 medical image retrieval and classification tasks. In: Third International Conference of the CLEF Initiative Workshop (2012)Google Scholar
  4. 4.
    Caputo, B., et al.: ImageCLEF 2013: The Vision, the Data and the Open Challenges. In: Forner, P., Müller, H., Paredes, R., Rosso, P., Stein, B. (eds.) CLEF 2013. LNCS, vol. 8138, pp. 250–268. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  5. 5.
    Seco de Herrera, A.G., Kalpathy-Cramer, J., Fushman, D.D., Antani, S., Muller, H.: Overview of the ImageCLEF 2013 medical tasks. In: American Medical Informatics Association (AMIA) ImageCLEF Medical Image Retrieval Workshop (2013)Google Scholar
  6. 6.
    Cao, L., Codella, N., Connell, J., Merler, M., Nguyen, Q., Pankanti, S., Smith, J.: IBM Multimedia Analytics @ ImageCLEF2013. In: American Medical Informatics Association (AMIA) ImageCLEF Medical Image Retrieval Workshop (2013)Google Scholar
  7. 7.
    Kitanovski, I., Dimitrovski, I., Loskovska, S.: FCSE at Medical Tasks of ImageCLEF 2013. In: American Medical Informatics Association (AMIA) ImageCLEF Medical Image Retrieval Workshop (2013)Google Scholar
  8. 8.
    Chawla, N., Bowyer, K., Hall, L., Kegelmeyer, P.: SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research 16, 321–357 (2002)zbMATHGoogle Scholar
  9. 9.
    van de Sande, K., Gevers, T., Snoek, C.: Evaluating Color Descriptors for Object and Scene Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(9), 1582–1596 (2010)CrossRefGoogle Scholar
  10. 10.
    Ahonen, T., Hadid, A., Pietikäinen, M.: Face Recognition with Local Binary Patterns. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 469–481. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  11. 11.
    Zhu, C., Bichot, C., Chen, L.: Multi-scale Color Local Binary Patterns for Visual Object Classes Recognition. In: 20th IAPR International Conference on Pattern Recognition (ICPR), pp. 3065–3068. IEEE Press, New York (2010)Google Scholar
  12. 12.
    Candès, E., Demanet, L., Donoho, D., Ying, L.: Fast Discrete Curvelet Transforms (2005), http://www.curvelet.org/
  13. 13.
    Image Retrieval in Medical Applications, http://ganymed.imib.rwth-aachen.de/irma/index_en.php
  14. 14.
    The Cancer Imaging Archive (TCIA), http://cancerimagingarchive.net/
  15. 15.
    Cornell University Vision and Analysis Group Public Image Databases, http://www.via.cornell.edu/visionx/simba/
  16. 16.
    Japanese Society of Radiological Technology, http://www.jsrt.or.jp/jsrt-db/eng.php

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Noel Codella
    • 1
  • Jonathan Connell
    • 1
  • Sharath Pankanti
    • 1
  • Michele Merler
    • 1
  • John R. Smith
    • 1
  1. 1.IBM T.J. Watson Research CenterYorktown HeightsUSA

Personalised recommendations