Abstract
Image generation is a hot topic in the field of machine learning and computer vision. As a representative of its algorithm, the Generative Adversarial Network (GAN) has the problem of mode collapse in practice. The proposed Dual Discriminator Weighted Mixture Generative Adversarial Network (D2WMGAN) approach can cope with this problem. On the one hand, the D2WMGAN uses the mixed distribution of multiple generators to approximate the real distribution, in order to prevent the extreme situation that multiple generators learn the same distribution and generate the same class of samples, with a classifier to play games with generators to make different generators learn different distributions. On the other hand, the objective function of D2WMGAN weights the Kullback–Leibler (KL) divergence and the reverse KL divergence, and uses their complementary characteristics to improve the quality and diversity of samples from the generators. Then, the theoretical conditional optimality of the D2WMGAN is proved theoretically, which shows that multiple generators can learn the real data distribution in the case of the optimal discriminator and classifier. Finally, extensive experiments are conducted on a large amount of synthetic data and real-world large-scale datasets (such as, CIFAR-10 and MNIST), and the commonly used GAN evaluation indicators (Wasserstein distance, JS divergence, Inception score, and Frechet Inception Distance) are introduced for comparative analysis. Experimental results show that the proposed D2WMGAN approach can better learn multiple mode data, generate rich realistic samples, and effectively solve the problem of mode collapse.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Code availability
The code that support the findings of this study are available from the corresponding author upon reasonable request.
References
Arjovsky M, Bottou L (2017) Towards principled methods for training generative adversarial networks. In: Proceedings of International Conference on Learning Representations (ICLR)
Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: Proceedings of International Conference on Machine Learning, PMLR, pp 214–223
Dowson DC, Landau BV (1982) The Fréchet distance between multivariate normal distributions. J Multivar Anal 12(3):450–455
Fiore U, Santis AD, Perla F, Zanetti P, Palmieri F (2019) Using generative adversarial networks for improving classification effectiveness in credit card fraud detection. Inf Sci 479:448–455
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in Neural Information Processing Systems, Curran Associates, pp 2672–2680
Guo Z, Wan Y, Ye H (2019) A data imputation method for multivariate time series based on generative adversarial network. Neurocomputing 360:185–197
Hoang Q, Nguyen TD, Le T, Phung D (2018) MGAN: Training generative adversarial nets with multiple generators. In: Proceedings of International Conference on Learning Representations (ICLR)
Hu R, Cui X (2021) Application of single frame image super-resolution algorithm based on generative adversarial network in tennis motion image resolution. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-021-03100-4
Hu H, Miao C, Hu WW (2018) Generative adversarial networks-and ResNets-based framework for image translation with super-resolution. J Electron Imaging 27(6):063018
Jin Q, Luo X, Shi Y, Kita K (2019) Image generation method based on improved condition GAN. In: Proceedings of 2019 6th International Conference on Systems and Informatics (ICSAI), IEEE, pp 1290–1294
Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Tech Rep TR-2009 1(4):54–57
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp 1097–1105
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. In: Proceedings of the IEEE, pp 2278–2324
Lee CY, Shon JG, Park JS (2021) An edge detection–based eGAN model for connectivity in ambient intelligence environments. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-021-03261-2
Lei N, An D, Guo Y, Su K, Liu S, Luo Z, Yan ST, Gu XF (2020) A geometric understanding of deep learning. Engineering 6(3):361–374
Li N, Zheng Z, Zhang S, Yu Z, Zheng H, Zheng B (2018a) The synthesis of unpaired underwater images using a multistyle generative adversarial network. IEEE Access 6:54241–54257
Li Z, Nie F, Chang X, Nie L, Zhang H, Yang Y (2018b) Rank-constrained spectral clustering with flexible embedding. IEEE Trans Neural Netw Learn Syst 29(12):6073–6082
Li Z, Nie F, Chang X, Yang Y, Zhang C, Sebe N (2018c) Dynamic affinity graph construction for spectral clustering using multiple features. IEEE Trans Neural Netw Learn Syst 29(12):6323–6332
Li Y, Xiao N, Ouyang W (2019) Improved generative adversarial networks with reconstruction loss. Neurocomputing 323:363–372
Li Y, Zhao K, Ren F, Wang B, Zhao J (2020) Research on super-resolution image reconstruction based on low-resolution infrared sensor. IEEE Access 8:69186–69199
Li P, Li Z, Pang X, Wang H, Lin W, Wu W (2021) Multi-scale residual denoising GAN model for producing super-resolution CTA images. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-021-03009-y
Liu G, Li X, Wei J (2021a) Large-area damage image restoration algorithm based on generative adversarial network. Neural Comput Appl 33(10):4651–4661
Liu X, Gao Z, Chen BM (2021b) IPMGAN: integrating physical model and generative adversarial network for underwater image enhancement. Neurocomputing 453:538–551
Liu B, Gao N, Huang MT, Liu H, Wang JT (2021c) On the effectiveness of dual discriminator weighted generative adversarial network. J Electron Imaging 30(3):1–17
Liu L, Muelly M, Deng J, Pfister T, Li J (2019) Generative modeling for small-data object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp 6073–6081
Ma C, Zhu J, Li Y, Li J, Jiang Y, Li X (2020) Single image super resolution via wavelet transform fusion and SRFeat network. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-020-02065-0
Menéndez ML, Pardo JA, Pardo L, Pardo MC (1997) The Jensen-shannon divergence. J Franklin Inst 334(2):307–318
Metz L, Poole B, Pfau D, Sohl-Dickstein J (2016) Unrolled generative adversarial networks. In: Proceedings of International Conference on Learning Representations (ICLR), pp 1–25
Nguyen T, Le T, Vu H (2017) Dual discriminator generative adversarial nets. In: Proceedings of International Conference on Neural Information Processing Systems (NIPS), pp 2671–2681
Nowozin S, Cseke B, Tomioka R (2016) f-gan: Training generative neural samplers using variational divergence minimization. In: Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS), pp 271–279
Raiber F, Kurland O (2017) Kullback-leibler divergence revisited. In: Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval, pp 117–124
Rajasenbagam T, Jeyanthi S, Arun Pandian J (2021) Detection of pneumonia infection in lungs from chest X-ray images using deep convolutional neural network and content-based image retrieval techniques. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-021-03075-2
Ren P, Xiao Y, Chang X, Huang PY, Li Z, Chen X, Wang X (2021) A comprehensive survey of neural architecture search: challenges and solutions. ACM Comput Surv (CSUR) 54(4):1–34
Rubner Y, Tomasi C, Guibas LJ (2000) The earth mover’s distance as a metric for image retrieval. Int J Comput Vision 40(2):99–121
Sajja TK, Kalluri HK (2021) Image classification using regularized convolutional neural network design with dimensionality reduction modules: RCNN–DRM. J Ambient Intell Humaniz Comput 12:9423–9434
Salimans T, Goodfellow I, Zaremba W (2016) Improved techniques for training GANs. In: Advances in Neural Information Processing Systems, pp 2234–2242
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2818–2826
Xiang P, Wang L, Cheng J, Zhang B, Wu JJ (2017) A deep network architecture for image inpainting. In: Proceedings of 2017 3rd IEEE International Conference on Computer and Communications (ICCC), IEEE, pp 1851–1856
Xu L, Zeng X, Li W, Huang Z (2020) Multi-granularity generative adversarial nets with reconstructive sampling for image inpainting. Neurocomputing 402:220–234
Yan C, Chang X, Luo M, Zheng Q, Zhang X, Li Z, Nie F (2020) Self-weighted robust LDA for multiclass classification with edge classes. ACM Trans Intell Syst Technol (TIST) 12(1):1–19
Yang T, Chang X, Su H, Crombez N, Yan Z (2020) Raindrop removal with light field image using image inpainting. IEEE Access 8:58416–58426
Zhang CL, Luo JH, Wei XS, Wu J (2017) In defense of fully connected layers in visual representation transfer. In: Proceedings of Pacific Rim Conference on Multimedia. Springer, Cham, pp 807–817
Acknowledgements
Research supported in part by grant for the Key Research and Development Program of Shaanxi (2021GY-131), Yulin Science and Technology Plan Project (CXY-2020-037), Xi’an Science and Technology Plan Project (2020KJRC0068), Scientific Research Program Funded by Shaanxi Provincial Education Department (18JK1005), Key R & D Projects of Shaanxi Province (2019GY-097), and Innovation Capability Support Program of Shaanxi (2020TD-021).
Author information
Authors and Affiliations
Contributions
BL: conceptualization, methodology, writing—review and editing, and funding acquisition. LW: software, validation, investigation, resources, data curation, and writing—original draft preparation. JW and JZ: formal analysis. JW: supervision and project administration. JZ: visualization.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
Ethics approval and consent to participate
Not applicable.
Consent to publication
All authors have read this manuscript and would like to have it considered exclusively for publication.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liu, B., Wang, L., Wang, J. et al. Dual Discriminator Weighted Mixture Generative Adversarial Network for image generation. J Ambient Intell Human Comput 14, 10013–10025 (2023). https://doi.org/10.1007/s12652-021-03667-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-021-03667-y