Medical & Biological Engineering & Computing

, Volume 57, Issue 1, pp 107–121 | Cite as

A novel fused convolutional neural network for biomedical image classification

  • Shuchao Pang
  • Anan Du
  • Mehmet A. Orgun
  • Zhezhou YuEmail author
Original Article


With the advent of biomedical imaging technology, the number of captured and stored biomedical images is rapidly increasing day by day in hospitals, imaging laboratories and biomedical institutions. Therefore, more robust biomedical image analysis technology is needed to meet the requirement of the diagnosis and classification of various kinds of diseases using biomedical images. However, the current biomedical image classification methods and general non-biomedical image classifiers cannot extract more compact biomedical image features or capture the tiny differences between similar images with different types of diseases from the same category. In this paper, we propose a novel fused convolutional neural network to develop a more accurate and highly efficient classifier for biomedical images, which combines shallow layer features and deep layer features from the proposed deep neural network architecture. In the analysis, it was observed that the shallow layers provided more detailed local features, which could distinguish different diseases in the same category, while the deep layers could convey more high-level semantic information used to classify the diseases among the various categories. A detailed comparison of our approach with traditional classification algorithms and popular deep classifiers across several public biomedical image datasets showed the superior performance of our proposed method for biomedical image classification. In addition, we also evaluated the performance of our method in modality classification of medical images using the ImageCLEFmed dataset.

Graphical abstract

The graphical abstract shows the fused, deep convolutional neural network architecture proposed for biomedical image classification. In the architecture, we can clearly see the feature-fusing process going from shallow layers and the deep layers.


Biomedical image classification Convolutional neural networks Deep learning Deep feature Shallow feature 


Funding information

This work started while Shuchao Pang was a visiting scholar at Macquarie University from Mar. 2016 to Mar. 2017. This work has also been supported by the project of Science and Technology Development Plan of Jilin Province, China (Grant 20150204007GX) and Specialized Research Fund for the Doctoral Program of Higher Education of China (Grant 20120061110045).


  1. 1.
    Andreu-Perez J, Poon CC, Merrifield RD, Wong ST, Yang GZ (2015) Big data for health. IEEE JBHI 19(4):1193–1208Google Scholar
  2. 2.
    Depeursinge A, Van de Ville D, Platon A, Geissbuhler A, Poletti PA, Muller H (2012) Near-affine-invariant texture learning for lung tissue analysis using isotropic wavelet frames. IEEE Trans Inf Technol Biomed 16(4):665–675CrossRefGoogle Scholar
  3. 3.
    Song Y, Cai W, Huang H, Zhou Y, Feng DD, Wang Y, Fulham MJ, Chen M (2015) Large margin local estimate with applications to medical image classification. IEEE Trans Med Imaging 34(6):1362–1377CrossRefGoogle Scholar
  4. 4.
    Khachane MY, Ramteke RJ (2016) Modality based medical image classification. In Emerging research in computing, information, communication and applications 597-606Google Scholar
  5. 5.
    Ertuğrul ÖF, Kaya Y, Tekin R (2016) A novel approach for SEMG signal classification with adaptive local binary patterns. Med Biol Eng Comput 54(7):1137–1146CrossRefGoogle Scholar
  6. 6.
    Li Q, Cai W, Wang X, Zhou Y, Feng DD, Chen M (2014) Medical image classification with convolutional neural network. In Control Automation Robotics & Vision (ICARCV), 13th IEEE International Conference on 844–848Google Scholar
  7. 7.
    Gao Z, Zhang J, Zhou L, Wang L (2014) HEp-2 cell image classification with convolutional neural networks. In Pattern recognition techniques for indirect immunofluorescence images (I3A), 2014 1st IEEE Workshop on 24–28Google Scholar
  8. 8.
    Abbas Q, Fondon I, Sarmiento A, Jiménez S, Alemany P (2017) Automatic recognition of severity level for diagnosis of diabetic retinopathy using deep visual features. Med Biol Eng Comput:1–16Google Scholar
  9. 9.
    Ahn E, Kumar A, Kim J, Li C, Feng D, Fulham M (2016) X-ray image classification using domain transferred convolutional neural networks and local sparse spatial pyramid. IEEE 13th ISBI 855–858Google Scholar
  10. 10.
    Phan HTH, Kumar A, Kim J, Feng D (2016) Transfer learning of a convolutional neural network for HEp-2 cell image classification. IEEE 13th ISBI 1208–1211Google Scholar
  11. 11.
    Pang S, Yu Z, Orgun MA (2017) A novel end-to-end classifier using domain transferred deep convolutional neural networks for biomedical images. Comput Methods Prog Biomed 140:283–293CrossRefGoogle Scholar
  12. 12.
    Shi J, Wu J, Li Y, Zhang Q, Ying S (2017) Histopathological image classification with color pattern random binary hashing-based PCANet and matrix-form classifier. IEEE J Biomed Health Inform 21(5):1327–1337CrossRefGoogle Scholar
  13. 13.
    LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444CrossRefGoogle Scholar
  14. 14.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems 1097–1105Google Scholar
  15. 15.
    Ouyang W, Wang X, Zhang C, Yang X (2016) Factors in finetuning deep model for object detection with long-tail distribution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 864–873Google Scholar
  16. 16.
    Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 3431–3440Google Scholar
  17. 17.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556Google Scholar
  18. 18.
    Long M, Wang J, Jordan MI (2016) Deep transfer learning with joint adaptation networks. arXiv preprint arXiv:1605.06636Google Scholar
  19. 19.
    Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia 675–678Google Scholar
  20. 20.
    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 1–9Google Scholar
  21. 21.
    Murala S, Wu QM (2013) Peak valley edge patterns: a new descriptor for biomedical image indexing and retrieval. Proc. IEEE CVPR Workshops 444–449Google Scholar
  22. 22.
    Dubey SR, Singh S, Singh R (2015) Local bit-plane decoded pattern: a novel feature descriptor for biomedical image retrieval. IEEE J Biomed Health Inform 20(4):1139–1147CrossRefGoogle Scholar
  23. 23.
    NEMA–CT image database (2012) Available:
  24. 24.
    Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M, Tarbox L (2013) The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 26(6):1045–1057CrossRefGoogle Scholar
  25. 25.
    Marcus DS, Wang TH, Parker J, Csernansky JG, Morris JC, Buckner RL (2007) Open Access Series of Imaging Studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J Cogn Neurosci 19(9):1498–1507CrossRefGoogle Scholar
  26. 26.
    de Herrera AGS, Müller H, Bromuri S (2015) Overview of the ImageCLEF 2015 Medical Classification Task. In CLEF (Working Notes)Google Scholar
  27. 27.
    Kalpathy-Cramer J, de Herrera AGS, Demner-Fushman D, Antani S, Bedrick S, Müller H (2015) Evaluating performance of biomedical image retrieval systems—an overview of the medical image retrieval task at ImageCLEF 2004–2013. Comput Med Imaging Graph 39:55–61CrossRefGoogle Scholar
  28. 28.
    Pelka O, Friederich CM (2015) FHDO Biomedical Computer Science Group at Medical Classification Task of ImageCLEF 2015. In CLEF (Working Notes)Google Scholar
  29. 29.
    Cirujeda P, Binefa X (2015) Medical image classification via 2D color feature based covariance descriptors. In CLEF (Working Notes)Google Scholar
  30. 30.
    Lyndon D, Kumar A, Kim J, Leong PHW, Feng D (2015) Convolutional neural networks for subfigure classification. In CLEF (Working Notes)Google Scholar
  31. 31.
    Van DM, Laurens (2014) Accelerating t-SNE using tree-based algorithms. J Mach Learn Res 15(1):3221–3245Google Scholar
  32. 32.
    Rippel O, Paluri M, Dollar P, Bourdev L (2015) Metric learning with adaptive density discrimination. arXiv preprint arXiv:1511.05939Google Scholar

Copyright information

© International Federation for Medical and Biological Engineering 2018

Authors and Affiliations

  1. 1.Department of Computational Intelligence, College of Computer Science and TechnologyJilin UniversityChangchunChina
  2. 2.Department of ComputingMacquarie UniversitySydneyAustralia
  3. 3.China Mobile (HangZhou) Information Technology Co., LtdHangzhouChina

Personalised recommendations