Skip to main content
SpringerLink
Log in

Multi-class imbalanced image classification using conditioned GANs

  • Regular Paper
  • Published:
International Journal of Multimedia Information Retrieval Aims and scope Submit manuscript

Abstract

The problem of data skewness, class imbalance, data scarcity and noise limits the application of machine learning and deep learning models in applications like anomaly detection, fraud detection, intrusion detection, fault diagnosis, machine-to-machine communication, etc. Performance of supervised learning leans towards majority class and fails to generalize testing data in class imbalance and noisy data problems. Using neural-based data augmentation techniques for data generation and deep convolutional models for classification would enhance the performance of the applications mentioned above. Recently, GANs (generative adversarial networks) showed significant improvements in generating images. In this paper, a model that uses a conditioned deep convolutional GAN and an auxiliary classifier are proposed to tackle the aforementioned issues. The conditioned GAN is used for data generation of minority classes images and noisy images. Another auxiliary deep convolutional model is employed for the classification of images on data augmented dataset. Also, a multi-hinge loss is employed in both the data augmentation and classification tasks. The effectiveness of the proposed model is investigated on the quality of generated images and classification metrics using four publicly available popular datasets: MNIST, EMNIST, CIFAR10, and SVHN. The proposed model has shown significant improvements over the other often used models of data augmentation and multi-class imbalance image classification in terms of generated samples and classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Ali A, Shamsuddin SM, Ralescu AL et al (2015) Classification with class imbalance problem: a review. Int J Advance Soft Compu Appl 7(3):176–204

    Google Scholar 

  2. Ali-Gombe A, Elyan E (2019) MFC-GAN: class-imbalanced dataset classification using multiple fake class generative adversarial network. Neurocomputing 361:212–221

    Article  Google Scholar 

  3. Ali-Gombe A, Elyan E, Savoye Y, Jayne C (2018) Few-shot classifier GAN. In: 2018 International Joint Conference on Neural Networks (IJCNN), IEEE, pp 1–8

  4. Antipov G, Baccouche M, Dugelay JL (2017) Face aging with conditional generative adversarial networks. In: 2017 IEEE international conference on image processing (ICIP), IEEE, pp 2089–2093

  5. Brock A, Lim T, Ritchie JM, Weston N (2016) Neural photo editing with introspective adversarial networks

  6. Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C (2012) Dbsmote: density-based synthetic minority over-sampling technique. Appl Intell 36(3):664–684

    Article  Google Scholar 

  7. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. J Art Intell Res 16:321–357

    MATH  Google Scholar 

  8. Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8789–8797

  9. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680

  10. Han H, Wang WY, Mao BH (2005) Borderline-smote: a new over-sampling method in imbalanced data sets learning. In: International conference on intelligent computing, Springer, pp 878–887

  11. He H, Bai Y, Garcia EA, Li S (2008) Adasyn: Adaptive synthetic sampling approach for imbalanced learning. In: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), IEEE, pp 1322–1328

  12. Im DJ, Kim CD, Jiang H, Memisevic R (2016) Generating images with recurrent adversarial networks. arXiv preprint

  13. Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134

  14. Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of gans for improved quality, stability, and variation.

  15. Kavalerov I, Czaja W, Chellappa R (2019) cgans with multi-hinge loss.

  16. Kim T, Cha M, Kim H, Lee JK, Kim J (2017) Learning to discover cross-domain relations with generative adversarial networks. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70, JMLR. org, pp 1857–1865

  17. Krawczyk B (2016) Learning from imbalanced data: open challenges and future directions. Prog Art Intell 5(4):221–232

    Article  Google Scholar 

  18. Kupyn O, Budzan V, Mykhailych M, Mishkin D, Matas J (2018) Deblurgan: Blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8183–8192

  19. Kwak H, Zhang BT (2016) Generating images part by part with composite generative adversarial networks.

  20. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, et al. (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690

  21. Li C, Wand M (2016) Precomputed real-time texture synthesis with markovian generative adversarial networks. In: European conference on computer vision, Springer, pp 702–716

  22. Li J, Liang X, Wei Y, Xu T, Feng J, Yan S (2017) Perceptual generative adversarial networks for small object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1222–1230

  23. Liu MY, Tuzel O (2016) Coupled generative adversarial networks. In: Advances in neural information processing systems, pp 469–477

  24. Liu Y, Peng Y, Wang B, Yao S, Liu Z (2017) Review on cyber-physical systems. IEEE/CAA J Automatica Sinica 4(1):27–40

    Article  Google Scholar 

  25. Masi I, Trán AT, Hassner T, Leksut JT, Medioni G (2016) Do we really need to collect millions of faces for effective face recognition? In: European Conference on Computer Vision, Springer, pp 579–596

  26. Mundra K, Modpur R, Chattopadhyay A, Kar IN (2020) Adversarial image detection in cyber-physical systems. In: Proceedings of the 1st ACM Workshop on Autonomous and Intelligent Mobile Systems, pp 1–5

  27. Perarnau G, Van De Weijer J, Raducanu B, Álvarez JM (2016) Invertible conditional gans for image editing.

  28. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks.

  29. Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G (2017) Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: International conference on information processing in medical imaging, Springer, pp 146–157

  30. Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):60

    Article  Google Scholar 

  31. Taigman Y, Polyak A, Wolf L (2016) Unsupervised cross-domain image generation.

  32. Taylor L, Nitschke G (2017) Improving deep learning using generic data augmentation.

  33. Wang TC, Liu MY, Zhu JY, Tao A, Kautz J, Catanzaro B (2018) High-resolution image synthesis and semantic manipulation with conditional GANs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8798–8807

  34. Wu J, Zhang C, Xue T, Freeman B, Tenenbaum J (2016) Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. In: Advances in neural information processing systems, pp 82–90

  35. Yoo D, Kim N, Park S, Paek AS, Kweon IS (2016) Pixel-level domain transfer. In: European Conference on Computer Vision, Springer, pp 517–532

  36. Zhang H, Xu T, Li H, Zhang S, Wang X, Huang X, Metaxas DN (2017) Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 5907–5915

  37. Zhu JY, Krähenbühl P, Shechtman E, Efros AA (2016) Generative visual manipulation on the natural image manifold. In: European Conference on Computer Vision, Springer, pp 597–613

  38. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 2223–2232

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, Tamilnadu, India

    M R Pavan Kumar

  2. School of Information Technology and Engineering, Vellore Institute of Technology, Vellore, Tamilnadu, India

    Prabhu Jayagopal

Authors
  1. M R Pavan Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prabhu Jayagopal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Prabhu Jayagopal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pavan Kumar, M.R., Jayagopal, P. Multi-class imbalanced image classification using conditioned GANs. Int J Multimed Info Retr 10, 143–153 (2021). https://doi.org/10.1007/s13735-021-00213-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13735-021-00213-6

Keywords

Search

Navigation