生成式对抗网络的不稳定性分析及其处理技术

Instability analysis for generative adversarial networks and its solving techniques

生成式对抗网络(generative adversarial networks,GANs)训练的不稳定性问题一直是GANs研究领域最具挑战性的问题之一.目前,仍未从理论上找到影响GANs训练稳定性的根本原因及有效的解决办法.本文通过理论分析发现,GANs训练的不稳定性主要是由于训练最优判别器与最小化生成器之间相互矛盾所致.经逐步分析得出,控制判别器的Lipschitz常数是解决GANs不稳定性问题的关键,进而提出一种有针对性的梯度惩罚技术来解决此问题.最后,本文从损失函数的振荡幅度(收敛性)、梯度总体变化趋势,以及网络整体性能3个方面进行了全面对比实验.结果显示,本文所提出的惩罚技术对处理GANs训练的不稳定性问题具有显著的效果.

Training instability in generative adversarial networks(GANs) remains one of the most challenging problems, for which both the theoretical root and an effective solution are needed. In this study, we theoretically determined that the mutual contradiction between training the optimal discriminator and minimizing the generator leads to training instability in GANs. To address this problem, we propose a targeted gradient penalty technique. Unlike other penalty techniques, we penalize the Lipschitz constant of the discriminator, which is the key to dealing with the instability problem(this amounts to controlling the Lipschitz constant of the discriminator). We performed a series of experimental comparisons from three different perspectives: the oscillation amplitude of the loss function(convergence), the general variation trend of the gradient, and the holistic performance of the network. The results demonstrated that the proposed technique has a significant and positive effect on the training instability in GANs.