An Expert-in-the-loop Paradigm for Learning Medical Image Grouping

  • Xuan Guo
  • Qi Yu
  • Rui Li
  • Cecilia Ovesdotter Alm
  • Cara Calvelli
  • Pengcheng Shi
  • Anne Haake
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9651)

Abstract

Image grouping in knowledge-rich domains is challenging, since domain knowledge and expertise are key to transform image pixels into meaningful content. Manually marking and annotating images is not only labor-intensive but also ineffective. Furthermore, most traditional machine learning approaches cannot bridge this gap for the absence of experts’ input. We thus present an interactive machine learning paradigm that allows experts to become an integral part of the learning process. This paradigm is designed for automatically computing and quantifying interpretable grouping of dermatological images. In this way, the computational evolution of an image grouping model, its visualization, and expert interactions form a loop to improve image grouping. In our paradigm, dermatologists encode their domain knowledge about the medical images by grouping a small subset of images via a carefully designed interface. Our learning algorithm automatically incorporates these manually specified connections as constraints for re-organizing the whole image dataset. Performance evaluation shows that this paradigm effectively improves image grouping based on expert knowledge.

Keywords

Dermatological images Multimodal data Image grouping Visual analytics Interactive machine learning 

References

  1. 1.
    Aronson, A.R.: Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. In: American Medical Informatics Association Annual Symposium Proceedings, pp. 17–21. AMIA (2001)Google Scholar
  2. 2.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. NIPS 14, 585–591 (2001)Google Scholar
  3. 3.
    Cai, D., Bao, H., He, X.: Sparse concept coding for visual analysis. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2905–2910. IEEE (2011)Google Scholar
  4. 4.
    Choo, J., Lee, C., Reddy, C.K., Park, H.: UTOPIAN: user-driven topic modeling based on interactive nonnegative matrix factorization. IEEE Trans. Vis. Comput. Graph. 19(12), 1992–2001 (2013)CrossRefGoogle Scholar
  5. 5.
    Choo, J., Lee, H., Kihm, J., Park, H.: iVisClassifier: an interactive visual analytics system for classification based on supervised dimension reduction. In: IEEE Symposium on Visual Analytics Science and Technology (VAST), pp. 27–34. IEEE (2010)Google Scholar
  6. 6.
    Crossno, P.J., Dunlavy, D.M., Shead, T.M.: LSAView: A tool for visual exploration of latent semantic modeling. In: IEEE Symposium on Visual Analytics Science and Technology (VAST), pp. 83–90. IEEE (2009)Google Scholar
  7. 7.
    El-Hamdouchi, A., Willett, P.: Comparison of hierarchic agglomerative clustering methods for document retrieval. Comput. J. 32(3), 220–227 (1989)CrossRefGoogle Scholar
  8. 8.
    Gao, S., Tsang, I.W., Chia, L.-T., Zhao, P.: Local features are not lonely-Laplacian sparse coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3555–3561. IEEE (2010)Google Scholar
  9. 9.
    Guo, X., Li, R., Alm, C., Yu, Q., Pelz, J., Shi, P., Haake, A.: Infusing perceptual expertise, domain knowledge into a human-centered image retrieval system: a prototype application. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 275–278. ACM (2014)Google Scholar
  10. 10.
    Guo, X., Yu, Q., Alm, C.O., Calvelli, C., Pelz, J.B., Shi, P., Haake, A.R.: From spoken narratives to domain knowledge: mining linguistic data for medical image understanding. Artif. Intell. Med. 62(2), 79–90 (2014)CrossRefGoogle Scholar
  11. 11.
    Guo, X., Yu, Q., Li, R., Alm, C.O., Haake, A.R.: Fusing multimodal human expert data to uncover hidden semantics. In: Proceedings of the 7th Workshop on Eye Gaze in Intelligent Human Machine Interaction: Eye-Gaze and Multimodality, pp. 21–26. ACM (2014)Google Scholar
  12. 12.
    Holzinger, A.: Human-Computer Interaction and Knowledge Discovery (HCI-KDD): what is the benefit of bringing those two fields to work together? In: Cuzzocrea, A., Kittl, C., Simos, D.E., Weippl, E., Xu, L. (eds.) CD-ARES 2013. LNCS, vol. 8127, pp. 319–328. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  13. 13.
    Hu, Y., Boyd-Graber, J., Satinoff, B., Smith, A.: Interactive topic modeling. Mach. Learn. 95(3), 423–469 (2014)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: Advances in Neural Information Processing Systems, pp. 801–808 (2006)Google Scholar
  15. 15.
    Lee, H., Kihm, J., Choo, J., Stasko, J., Park, H.: iVisClustering: an interactive visual document clustering via topic modeling. In: Computer Graphics Forum, vol. 31, pp. 1155–1164. Wiley Online Library (2012)Google Scholar
  16. 16.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  17. 17.
    McCoy, W., Alm, C.O., Calvelli, C., Li, R., Pelz, J.B., Shi, P., Haake, A.: Annotation schemes to encode domain knowledge in medical narratives. In: Ide, N., Xia, F. (eds.) Proceedings of the 6th Linguistic Annotation Workshop, pp. 95–103. Association for Computational Linguistics, Stroudsburg (2012)Google Scholar
  18. 18.
    Vaidyanathan, P., Pelz, J., Li, R., Mulpuru, S., Wang, D., Shi, P., Calvelli, C., Haake, A.: Using human experts’ gaze data to evaluate image processing algorithms. In: Hemami, S., Pappas, T.N. (eds.) \(10^{th}\) IVMSP Workshop: Perception and Visual Signal Analysis, pp. 129–134. IEEE (2011)Google Scholar
  19. 19.
    Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(2579–2605), 85 (2008)MATHGoogle Scholar
  20. 20.
    Zheng, M., Bu, J., Chen, C., Wang, C., Zhang, L., Qiu, G., Cai, D.: Graph regularized sparse coding for image representation. IEEE Trans. Image Process. 20(5), 1327–1336 (2011)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Xuan Guo
    • 1
  • Qi Yu
    • 2
  • Rui Li
    • 2
  • Cecilia Ovesdotter Alm
    • 2
  • Cara Calvelli
    • 2
  • Pengcheng Shi
    • 2
  • Anne Haake
    • 2
  1. 1.B. Thomas Golisano College of Computing and Information SciencesRochesterUSA
  2. 2.Rochester Institute of TechnologyRochesterUSA

Personalised recommendations