基于预测向心加速的生成对抗网络训练

Training GAN with predictive centripetal acceleration

为缓解生成对抗网络(generative adversarial networks,GAN)训练过程中的极限循环行为,本文受向心加速算法及Liang和Stokes(2019)的修正的预测方法(modified predictive method,MPM)的启发,基于对匀速圆周运动的几何观察提出了预测向心加速算法(predictive centripetal acceleration algorithm,PCA).首先,在二元线性博弈(特殊的GAN)上证明了PCA的最后一次迭代收敛性.然后,将PCA分别与随机梯度下降(stochastic gradient descent,SGD)算法和自适应性矩估计(adaptive moment estimation,Adam)算法结合,提出了随机PCA(stochastic PCA,SPCA)和PCA-Adam用于实际训练GAN.最后,在二元线性博弈、多元Gauss分布以及CIFAR10和Celeb A数据集上的实验分别验证了所提出算法的有效性.

To alleviate the issue of limit cycle behavior in training generative adversarial networks(GAN),in this paper,we draw inspiration from the centripetal acceleration algorithm and the modified predictive method(MPM)by Liang and Stokes(2019).Building upon geometric observation of uniform circular motion,we propose the predictive centripetal acceleration algorithm(PCA).First and foremost,we prove the last-iterate convergence of the PCA on the bilinear game,which is a special case of the GAN.Besides,by combining PCA with the stochastic gradient descent(SGD)algorithm and adaptive moment estimation(Adam)algorithm,we propose two variants,which are called stochastic PCA(SPCA)and PCA-Adam,for the practical training GAN.Last but not least,experiments conducted on the bilinear game,multivariate Gaussian distribution,and CIFAR10 and CelebA datasets validate the effectiveness of the proposed algorithm.