Skip to main content

Synthetic image augmentation with generative adversarial network for enhanced performance in protein classification

Biomedical Engineering Letters volume 10, pages 443–452 (2020)Cite this article

Abstract

Proteins are complex macromolecules accountable for the biological processes in the cell. In biomedical research, the images of protein are extensively used in medicine. The rate at which these images are produced makes it difficult to evaluate them manually and hence there exists a need to automate the system. The quality of images is still a major issue. In this paper, we present the use of different image enhancement techniques that improves the contrast of these images. Besides the quality of images, the challenge of gathering such datasets in the field of medicine persists. We use generative adversarial networks for generating synthetic samples to ameliorate the results of CNN. The performance of the synthetic data augmentation was compared with the classic data augmentation on the classification task, an increase of 2.7% in Macro F1 and 2.64% in Micro F1 score was observed. Our best results were obtained by the pretrained Inception V4 model that gave a fivefold cross-validated macro F1 of 0.603. The achieved results are contrasted with the existing work and comparisons show that the proposed method outperformed.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. Kraus OZ, Ba JL, Frey BJ. Classifying and segmenting microscopy images with deep multiple instance learning. Bioinformatics. 2016;32(12):i52–9.

    Article  Google Scholar 

  2. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44.

    Article  Google Scholar 

  3. Pärnamaa T, Parts L. Accurate classification of protein subcellular localization from high-throughput microscopy images using deep learning. Genes Genomes Genet. 2017;7(5):1385–92.

    Google Scholar 

  4. Tan J, Ung M, Cheng C, Greene CS. Unsupervised feature construction and knowledge extraction from genome-wide assays of breast cancer with denoising autoencoders. In: Pacific symposium on biocomputing co-chairs (2014). pp. 132–143.

  5. Angermueller C, Pärnamaa T, Parts L, Stegle O. Deep learning for computational biology. Mol Syst Biol. 2016;12:7.

    Article  Google Scholar 

  6. Sønderby SK, Sønderby CK, Nielsen H, Winther O. Convolutional LSTM networks for subcellular localization of proteins. In: International conference on algorithms for computational biology (2015 Aug 4), pp. 68–80. Springer, Cham.

  7. Almagro Armenteros JJ, Sønderby CK, Sønderby SK, Nielsen H, Winther O. DeepLoc: prediction of protein subcellular localization using deep learning. Bioinformatics. 2017;33(21):3387–95.

    Article  Google Scholar 

  8. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition (2014 Sep 4). arXiv preprint arXiv:1409.1556.

  9. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition 2016, pp. 770–778.

  10. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA. Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-first AAAI conference on artificial intelligence (2017 Feb 12).

  11. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (2017), pp. 4700–4708.

  12. Zoph B, Vasudevan V, Shlens J, Le QV. Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (2018), pp. 8697–8710.

  13. Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging. 2016;35(5):1285–98.

    Article  Google Scholar 

  14. Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J. Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE Trans Med Imaging. 2016;35(5):1299–312.

    Article  Google Scholar 

  15. Azizpour H, Sharif Razavian A, Sullivan J, Maki A, Carlsson S. From generic to specific deep representations for visual recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops (2015), pp. 36–45.

  16. Penatti OA, Werneck RD, de Almeida WR, Stein BV, Pazinato DV, Júnior PR, Torres RD, Rocha A. Mid-level image representations for real-time heart view plane classification of echocardiograms. Comput Biol Med. 2015;1(66):66–81.

    Article  Google Scholar 

  17. Sharif Razavian A, Azizpour H, Sullivan J, Carlsson S. CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops (2014), pp. 806–813.

  18. Pizer SM, Amburn EP, Austin JD, Cromartie R, Geselowitz A, Greer T, ter Haar Romeny B, Zimmerman JB, Zuiderveld K. Adaptive histogram equalization and its variations. Comput Vis Graph Image Process. 1987;39(3):355–68.

    Article  Google Scholar 

  19. Zuiderveld K. Contrast limited adaptive histogram equalization. In: Graphics gems IV (1994 Aug 1), pp. 474–485. Academic Press Professional, Inc.

  20. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems (2012), pp. 1097–1105.

  21. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In: Advances in neural information processing systems (2014), pp. 2672–2680.

  22. Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks (2015 Nov 19). arXiv preprint arXiv:1511.06434.

  23. Denton EL, Chintala S, Fergus R. Deep generative image models using a laplacian pyramid of adversarial networks. InAdvances in neural information processing systems (2015), pp. 1486–1494.

  24. Mirza M, Osindero S. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784. 2014 Nov 6.

  25. Odena A, Olah C, Shlens J. Conditional image synthesis with auxiliary classifier gans. In: Proceedings of the 34th international conference on machine learning-volume 70 (2017 Aug 6), pp. 2642–2651. JMLR. org.

  26. Costa P, Galdran A, Meyer MI, Niemeijer M, Abràmoff M, Mendonça AM, Campilho A. End-to-end adversarial retinal image synthesis. IEEE Trans Med Imaging. 2017;37(3):781–91.

    Article  Google Scholar 

  27. Dai W, Dong N, Wang Z, Liang X, Zhang H, Xing EP. Scan: Structure correcting adversarial network for organ segmentation in chest x-rays. In: Deep learning in medical image analysis and multimodal learning for clinical decision support (2018 Sep 20), pp. 263–273. Springer, Cham.

  28. Xue Y, Xu T, Zhang H, Long LR, Huang X. Segan: adversarial network with multi-scale l 1 loss for medical image segmentation. Neuroinformatics. 2018;16(3–4):383–92.

    Article  Google Scholar 

  29. Nie D, Trullo R, Lian J, Petitjean C, Ruan S, Wang Q, Shen D. Medical image synthesis with context-aware generative adversarial networks. In: International conference on medical image computing and computer-assisted intervention (2017 Sep 10), pp. 417–425. Springer, Cham.

  30. Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: International conference on information processing in medical imaging (2017 Jun 25), pp. 146–157. Springer, Cham.

  31. Alex V, KP MS, Chennamsetty SS, Krishnamurthi G. Generative adversarial networks for brain lesion detection. In: Medical imaging 2017: image processing (2017 Feb 24), Vol. 10133, p. 101330G. International Society for Optics and Photonics.

  32. Ben-Cohen A, Klang E, Raskin SP, Amitai MM, Greenspan H. Virtual PET images from CT data using deep convolutional networks: initial results. In: International workshop on simulation and synthesis in medical imaging (2017 Sep 10), pp. 49–57. Springer, Cham.

  33. Litjens G, Kooi T, Bejnordi BE, Setio AA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, Sánchez CI. A survey on deep learning in medical image analysis. Med Image Anal. 2017;1(42):60–88.

    Article  Google Scholar 

  34. Greenspan H, Van Ginneken B, Summers RM. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans Med Imaging. 2016;35(5):1153–9.

    Article  Google Scholar 

  35. Shi J, Zhou S, Liu X, Zhang Q, Lu M, Wang T. Stacked deep polynomial network based representation learning for tumor classification with small ultrasound image dataset. Neurocomputing. 2016;19(194):87–94.

    Article  Google Scholar 

  36. Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X. Improved techniques for training gans. In: Advances in neural information processing systems (2016), pp. 2234–2242.

  37. Kaggle. Human protein atlas image classification challenge. (2019). https://www.kaggle.com/c/human-protein-atlas-image-classification. Online Accessed on 24 Oct 2018.

  38. Protein Atlas. The human protein atlas. https://www.proteinatlas.org. Online Accessed on 24 Oct 2018.

  39. Lin TY, Goyal P, Girshick R, He K, Dollár P. Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision (2017), pp. 2980–2988.

  40. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process. 2004;13(4):600–12.

    Article  Google Scholar 

  41. Tian Y. TUNet: Incorporating segmentation maps to improve classification (2019 Jan 27). arXiv preprint arXiv:1901.11379.

Download references

Acknowledgements

The authors are thankful to ABV-Indian Institute of Information Technology and Management, Gwalior for providing resources for this research work. This research work has not taken funding from any source.

Author information

Affiliations

  1. Soft Computing and Expert Systems Laboratory, ABV-IIITM, Gwalior, M.P., 474015, India

    Rohit Verma, Raj Mehrotra, Chinmay Rane, Ritu Tiwari & Arun Kumar Agariya

Authors
  1. Rohit Verma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raj Mehrotra
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chinmay Rane
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ritu Tiwari
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Arun Kumar Agariya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rohit Verma.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 60 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Verma, R., Mehrotra, R., Rane, C. et al. Synthetic image augmentation with generative adversarial network for enhanced performance in protein classification. Biomed. Eng. Lett. 10, 443–452 (2020). https://doi.org/10.1007/s13534-020-00162-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13534-020-00162-9

Keywords

  • Image enhancement
  • Generative adversarial network
  • Protein Image classification
  • Convolutional neural network
  • Transfer learning