Supervised Intra-embedding of Fisher Vectors for Histopathology Image Classification

  • Yang SongEmail author
  • Hang Chang
  • Heng Huang
  • Weidong Cai
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10435)


In this paper, we present a histopathology image classification method with supervised intra-embedding of Fisher vectors. Recently in general computer vision, Fisher encoding combined with convolutional neural network (ConvNet) has become popular as a highly discriminative feature descriptor. However, Fisher vectors have two intrinsic problems that could limit their performance: high dimensionality and bursty visual elements. To address these problems, we design a novel supervised intra-embedding algorithm with a multilayer neural network model to transform the ConvNet-based Fisher vectors into a more discriminative feature representation. We apply this feature encoding method on two public datasets, including the BreaKHis image dataset of benign and malignant breast tumors, and IICBU 2008 lymphoma dataset of three malignant lymphoma subtypes. The results demonstrate that our supervised intra-embedding method helps to enhance the ConvNet-based Fisher vectors effectively, and our classification results largely outperform the state-of-the-art approaches on these datasets.


  1. 1.
    Arandjelovic, R., Zisserman, A.: All about VLAD. In: CVPR, pp. 1578–1585 (2013)Google Scholar
  2. 2.
    Barker, J., et al.: Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles. Med. Image Anal. 30(1), 60–71 (2016)CrossRefGoogle Scholar
  3. 3.
    BenTaieb, A., Li-Chang, H., Huntsman, D., Hamarneh, G.: Automatic diagnosis of ovarian carcinomas via sparse multiresolution tissue representation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 629–636. Springer, Cham (2015). doi: 10.1007/978-3-319-24553-9_77 CrossRefGoogle Scholar
  4. 4.
    Cimpoi, M., et al.: Deep filter banks for texture recognition, description, and segmentation. Int. J. Compt. Vis. 118(1), 65–94 (2016)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Codella, N., et al.: Lymphoma diagnosis in histopathology using a multi-stage visual learning approach. In: SPIE, p. 97910H (2016)Google Scholar
  6. 6.
    Kandemir, M., Zhang, C., Hamprecht, F.A.: Empowering multiple instance histopathology cancer diagnosis by cell graphs. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8674, pp. 228–235. Springer, Cham (2014). doi: 10.1007/978-3-319-10470-6_29 CrossRefGoogle Scholar
  7. 7.
    Perronnin, F., Sánchez, J., Mensink, T.: Improving the Fisher kernel for large-scale image classification. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 143–156. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15561-1_11 CrossRefGoogle Scholar
  8. 8.
    Shamir, L., et al.: IICBU 2008: a proposed benchmark suite for biological image analysis. Med. Biol. Eng. Comput. 46(9), 943–947 (2008)CrossRefGoogle Scholar
  9. 9.
    Simonyan, K., et al.: Fisher vector faces in the wild. In: BMVC, pp. 1–12 (2013)Google Scholar
  10. 10.
    Song, Y., Li, Q., Huang, H., Feng, D., Chen, M., Cai, W.: Histopathology image categorization with discriminative dimension reduction of Fisher vectors. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 306–317. Springer, Cham (2016). doi: 10.1007/978-3-319-46604-0_22 CrossRefGoogle Scholar
  11. 11.
    Spanhol, F., et al.: Breast cancer histopathological image classification using convolutional neural networks. In: IJCNN, pp. 1–8 (2016)Google Scholar
  12. 12.
    Spanhol, F., et al.: A dataset for breast cancer histopathological image classification. IEEE Trans. Biomed. Eng 63(7), 1455–1462 (2016)CrossRefGoogle Scholar
  13. 13.
    Wang, J., MacKenzie, J.D., Ramachandran, R., Chen, D.Z.: Neutrophils identification by deep learning and voronoi diagram of clusters. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 226–233. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_27 CrossRefGoogle Scholar
  14. 14.
    Xing, F., et al.: An automatic learning-based framework for robust nucleus segmentation. IEEE Trans. Med. Imag. 35(2), 550–566 (2016)CrossRefGoogle Scholar
  15. 15.
    Xu, X., Lin, F., Ng, C., Leong, K.P.: Adaptive co-occurrence differential texton space for HEp-2 cells classification. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 260–267. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_31 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Yang Song
    • 1
    Email author
  • Hang Chang
    • 2
  • Heng Huang
    • 3
  • Weidong Cai
    • 1
  1. 1.School of Information TechnologiesUniversity of SydneySydneyAustralia
  2. 2.Biological Systems and Engineering DivisionLawrence Berkeley National LaboratoryBerkeleyUSA
  3. 3.Computer Science and EngineeringUniversity of Texas at ArlingtonArlingtonUSA

Personalised recommendations