Unsupervised Feature Learning for Endomicroscopy Image Retrieval
Learning the visual representation for medical images is a critical task in computer-aided diagnosis. In this paper, we propose Unsupervised Multimodal Graph Mining (UMGM) to learn the discriminative features for probe-based confocal laser endomicroscopy (pCLE) mosaics of breast tissue. We build a multiscale multimodal graph based on both pCLE mosaics and histology images. The positive pairs are mined via cycle consistency and the negative pairs are extracted based on geodetic distance. Given the positive and negative pairs, the latent feature space is discovered by reconstructing the similarity between pCLE and histology images. Experiments on a database with 700 pCLE mosaics demonstrate that the proposed method outperforms previous works on pCLE feature learning. Specially, the top-1 accuracy in an eight-class retrieval task is 0.659 which leads to 10% improvement compared with the state-of-the-art method.
KeywordsProbe-based laser endomicroscopy Histology Graph mining Feature learning
This work is partially supported by NSFC, China (No: 61572315,6151101179) and 973 Plan, China (No. 2015CB856004). Yun Gu is also supported by Chinese Scholarship Council (CSC). The tissue specimens were obtained from consented patients using the Imperial tissue bank ethical protocol following the R-12047 project.
- 2.Zhang, X., Su, H., Yang, L., Zhang, S.: Fine-grained histopathological image analysis via robust segmentation and large-scale retrieval. In: CVPR, pp. 5361–5368 (2015)Google Scholar
- 4.André, B., Vercauteren, T., Perchant, A., Buchner, A.M., Wallace, M.B., Ayache, N.: Endomicroscopic image retrieval and classification using invariant visual features. In: IEEE ISBI 2009, pp. 346–349. IEEE (2009)Google Scholar
- 5.Gu, Y., Yang, J., Yang, G.Z.: Multi-view multi-modal feature embedding for endomicroscopy mosaic classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 11–19 (2016)Google Scholar
- 6.Li, D., Hung, W.-C., Huang, J.-B., Wang, S., Ahuja, N., Yang, M.-H.: Unsupervised visual representation learning by graph-based consistent constraints. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 678–694. Springer, Cham (2016). doi: 10.1007/978-3-319-46493-0_41 CrossRefGoogle Scholar
- 7.Zhai, X., Peng, Y., Xiao, J.: Heterogeneous metric learning with joint graph regularization for cross-media retrieval. In: AAAI 2013, pp. 1198–1204 (2013)Google Scholar
- 10.Mignon, A., Jurie, F.: Cmml: a new metric learning approach for cross modal matching. In: Asian Conference on Computer Vision, p. 14 (2012)Google Scholar
- 11.Chang, T.P., Leff, D.R., Shousha, S., Hadjiminas, D.J., Ramakrishnan, R., Hughes, M.R., Yang, G.Z., Darzi, A.: Imaging breast cancer morphology using probe-based confocal laser endomicroscopy: towards a real-time intraoperative imaging tool for cavity scanning. Breast Cancer Res. Treat. 153(2), 299–310 (2015)CrossRefGoogle Scholar