Addressing GAN Training Instabilities via Tunable Classification Losses

Generative adversarial networks (GANs), modeled as a zero-sum game between a generator (G) and a discriminator (D), allow generating synthetic data with formal guarantees. Noting that D is a classifier, we begin by reformulating the GAN value function using class probability estimation (CPE) losses. We prove a two-way correspondence between CPE loss GANs and f-GANs which minimize f-divergences. We also show that all symmetric f-divergences are equivalent in convergence. In the finite sample and model capacity setting, we define and obtain bounds on estimation and generalization errors. We specialize these results to α -GANs, defined using α -loss, a tunable CPE loss family parametrized by α in (0,∞ ]. We next introduce a class of dual-objective GANs to address training instabilities of GANs by modeling each player's objective using α -loss to obtain (α_D,α_G) -GANs. We show that the resulting non-zero sum game simplifies to minimizing an f-divergence under appropriate conditions on (α_D,α_G). Generalizing this dual-objective formulation using CPE losses, we define and obtain upper bounds on an appropriately defined estimation error. Finally, we highlight the value of tuning (α_D,α_G) in alleviating training instabilities for the synthetic 2D Gaussian mixture ring as well as the large publicly available Celeb-A and LSUN Classroom image datasets.