Computer-Aided Diagnosis in Chest Radiography with Deep Multi-Instance Learning

  • Kang Qu
  • Xiangfei Chai
  • Tianjiao Liu
  • Yadong Zhang
  • Biao Leng
  • Zhang Xiong
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10637)


The Computer-Aided Diagnosis (CAD) for chest X-ray image has been investigated for many years. However, it has not been widely used since limited accuracy. Deep learning opens a new era for image recognition and classification. We propose a novel framework called Deep Multi-Instance Learning (DMIL) on chest radiographic images diagnosis, which combines deep learning and multi-instance learning. Besides, we preprocess images with the alignment based on the key points. This framework can effectively improve the diagnosis effect in the image level annotation. We quantify the framework on three datasets, respectively with different amounts and different classification tasks. The proposed framework obtained the AUC of 0.986, 0.873, 0.824 respectively in classification tasks of the enlarged heart, the pulmonary nodule, and the abnormal. The experiments we implement demonstrate that the proposed framework outperforms the other methods in various evaluation criteria.


Chest radiograph Deep learning Multi-Instance Learning Medical image 



This work is supported by the National Natural Science Foundation of China (No. 61472023) and the State Key Laboratory of Software Development Environment (No. SKLSDE-2016ZX-24).


  1. 1.
    Song, Y., Cai, W., Zhou, Y., Feng, D.D.: Feature-based image patch approximation for lung tissue classification. IEEE Trans. Med. Imag. 32(4), 797–808 (2013)CrossRefGoogle Scholar
  2. 2.
    Sorensen, L., Shaker, S.B., De Bruijne, M.: Quantitative analysis of pulmonary Emphysema using local binary patterns. IEEE Trans. Med. Imag. 29(2), 559–569 (2010)CrossRefGoogle Scholar
  3. 3.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  4. 4.
    Roth, H., Lu, L., Liu, J., Yao, J., Seff, A., Cherry, K., Kim, L., Summers, R.: Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans. Med. Imag. 35(5), 1170–1181 (2015)CrossRefGoogle Scholar
  5. 5.
    Bar, Y., Diamant, I., Wolf, L., Greenspan, H.: Deep learning with non-medical training used for chest pathology identification. In: SPIE Medical Imaging, pp. 94140V. International Society for Optics and Photonics (2015)Google Scholar
  6. 6.
    Bar, Y., Diamant, I., Wolf, L., Lieberman, S., Konen, E., Greenspan, H.: Chest pathology detection using deep learning with non-medical training. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pp. 294–297. IEEE (2015)Google Scholar
  7. 7.
    Albarqouni, S., Baur, C., Achilles, F., Belagiannis, V., Demirci, S., Navab, N.: AggNet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Trans. Med. Imag. 35(5), 1313–1321 (2016)CrossRefGoogle Scholar
  8. 8.
    Wu, J., Yu, Y., Huang, C., Yu, K.: Deep multiple instance learning for image classification and auto-annotation, pp. 3460–3469 (2015)Google Scholar
  9. 9.
    Zeng, T., Ji, S.: Deep convolutional neural networks for multi-instance multi-task learning. In: 2015 IEEE International Conference on Data Mining (ICDM), pp. 579–588. IEEE (2015)Google Scholar
  10. 10.
    Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 FPS via regressing local binary features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1685–1692 (2014)Google Scholar
  11. 11.
    Shin, H.C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., Summers, R.M.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imag. 35(5), 1285–1298 (2016)CrossRefGoogle Scholar
  12. 12.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  13. 13.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  14. 14.
    Melendez, J., van Ginneken, B., Maduskar, P., Philipsen, R.H., Reither, K., Breuninger, M., Adetifa, I.M., Maane, R., Ayles, H., Sánchez, C.I.: A novel multiple-instance learning-based approach to computer-aided detection of Tuberculosis on chest X-rays. IEEE Trans. Med. Imag. 34(1), 179–192 (2015)CrossRefGoogle Scholar
  15. 15.
    Ciompi, F., de Hoop, B., van Riel, S.J., Chung, K., Scholten, E.T., Oudkerk, M., de Jong, P.A., Prokop, M., van Ginneken, B.: Automatic classification of pulmonary peri-fissural nodules in Computed Tomography using an ensemble of 2D views and a convolutional neural network out-of-the-box. Med. Image Anal. 26(1), 195–202 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Kang Qu
    • 1
  • Xiangfei Chai
    • 2
  • Tianjiao Liu
    • 3
  • Yadong Zhang
    • 2
  • Biao Leng
    • 4
  • Zhang Xiong
    • 4
  1. 1.School of Computer Science and EngineeringBeihang UniversityBeijingChina
  2. 2.Huiying Medical Technology Inc. (Beijing)BeijingChina
  3. 3.Department of Electronic EngineeringTsinghua UniversityBeijingChina
  4. 4.State Key Laboratory of Software Development Environment, School of Computer Science and EngineeringBeihang UniversityBeijingChina

Personalised recommendations