VARGAN: variance enforcing network enhanced GAN

Abstract

Generative adversarial networks (GANs) are one of the most widely used generative models. GANs can learn complex multi-modal distributions, and generate real-like samples. Despite the major success of GANs in generating synthetic data, they might suffer from unstable training process, and mode collapse. In this paper, we propose a new GAN architecture called variance enforcing GAN (VARGAN), which incorporates a third network to introduce diversity in the generated samples. The third network measures the diversity of the generated samples, which is used to penalize the generator’s loss for low diversity samples. The network is trained on the available training data and undesired distributions with limited modality. On a set of synthetic and real-world image data, VARGAN generates a more diverse set of samples compared to the recent state-of-the-art models. High diversity and low computational complexity, as well as fast convergence, make VARGAN a promising model to alleviate mode collapse.

Appendices

Appendix A: Selection of MCR formulation and its parameters

We have explored three different formulations for the MCR values. Equation (5) presents a linear relationship between MCR value and percentage of covered modes.

$$ MCR(n) = \begin{cases} \frac{n}{N} & n = 2,\hdots, N-1, \enspace n \mid B\\ 0 & n = 1\\ \end{cases} $$

(5)

Equation (6) models a relationship that achieves early convergence to reward the model with low generator error.

$$ MCR(n) = \begin{cases} \frac{\mathrm{L}}{\mathrm{1} + S_{1} \cdot e^{-\frac{n}{N} \cdot S_{2}}} & n = 2,\hdots, N-1, \enspace n \mid B\\ 0 & n = 1\\ \end{cases} $$

(6)

Equation (7) models a relationship where it avoids rewarding the generator too early to keep it motivated.

$$ MCR(n) = \begin{cases} \frac{\mathrm{L}}{\mathrm{1} + S_{1} \cdot e^{-\frac{n-N}{N} \cdot S_{2}}} & n = 2,\hdots, N-1, \enspace n \mid B\\ 0 & n = 1\\ \end{cases} $$

(7)

Figure 12 shows the trajectory of different formulations based on the percentage of covered modes.

Fig. 12

MCR value based on number of generated modes divided by target modes for different equations. Left, middle and right figures in order present (5), (6) and (7)

Firstly, we have experimented with constant values of L, S₁ and S₂ for different formulations on synthetic data with 36 modes. Our results illustrated in Fig. 13 indicate that (6) is presenting a suitable trajectory for MCR values over the percentage of covered modes.

Fig. 13

Comparison of different MCR formulations on synthetic 2D grid data with 36 modes averaged over 5 repeats

Finally, we have used different values of L, S₁ and S₂ on synthetic data with 36 modes. The hyperparameter optimization results are illustrated in Fig. 14.

Fig. 14

Comparison of different hyperparameters for (6) on synthetic 2D grid data with 36 modes averaged over 5 repeats

Equation (6) with L, S₁ and S₂ of 1, 10 and 5 shows an early convergence for all the metrics compared to other formulations and hyperparameters. As shown in Fig. 12, L, S₁ and S₂ of 1, 10 and 5 enforce a low initial MCR value with moderate speed of convergence to MCR value of one. In other words, VARGAN performance does not improve by defining a large MCR value for initial low mode coverage or a fast convergence trajectory.

B Architecture of models

In this section, a detailed summary of GAN architectures is presented. Table 7 presents the feed-forward model architecture used for all the datasets.

Table 7 Structure of FF GAN model

Table 8 shows the convolutional model architecture used for stacked MNIST dataset. We have modified the architecture for other GAN variants to implement the specific details of their methodology.

Table 8 Structure of convolutional GAN model for stacked MNIST dataset

Convolutional model architecture used for EES dataset is reported in Table 9. Number of convolutional layers is changed to implement both 9 × 9 and 19 × 19 designs.

Table 9 Structure of convolutional GAN model for EES dataset

C Detailed results

In this section, performance comparison of GAN models over the epochs is presented. Figures 15 and 16 illustrate the convergence of the models for synthetic data with 8 and 25 modes based on different performance metrics and over epochs. VARGAN and GDPP models seem to have early convergence in the beginning epochs for synthetic data with 8 modes. VARGAN shows great early convergence on synthetic data with 25 modes as well, and the rest of the models follow it by a large gap.

Fig. 15

Comparison of different GAN models on synthetic 2D ring data with 8 modes averaged over 5 repeats

Fig. 16

Comparison of different GAN models on synthetic 2D grid data with 25 modes averaged over 5 repeats

Figure 17 presents the convergence of performance metrics for convolutional GAN models on stacked MNIST data. MGGAN, VARGAN and PacVARGAN show great early convergence over all the metrics. Moreover, VARGAN and PacVARGAN present lower standard deviations compared to MGGAN.

Fig. 17

Performance comparison of different GAN models with convolutional GAN structure for stacked MNIST dataset averaged over 5 repeats