CycleGAN-SN:结合谱归一化和CycleGAN的图像风格化算法

CycleGAN-SN:Image Stylization Algorithm Combining Spectral Normalization and CycleGAN

为解决CycleGAN算法图像风格化质量不高、网络稳定性不强的问题,提出了CycleGAN-SN算法。在CycleGAN算法判别网络的每一个卷积层后添加谱归一化层,通过幂迭代法估算卷积层参数矩阵的谱范数,采用随机梯度下降法更新卷积层参数。由于参数在每一次更新中的变化量很小,只需迭代一次即可快速估算出矩阵的最大奇异值。根据得到的最大奇异值,对卷积层参数进行归一化处理,使得整个判别网络满足1-Lipschitz连续。在4个常用风格图像数据集上进行实验,并与CycleGAN算法进行对比,结果表明:所提算法能够在保留原有图像细节的基础上,生成色彩鲜艳、纹理清晰、风格渲染充分的风格化图像;在训练过程中的损失函数振荡幅度小,能够使用更大的学习率进行训练,稳定性较强;能够有效减少网络收敛所需的步数,收敛速度较快;在测试阶段一次性风格化751幅图像时,时间最多仅增加0.63 s,几乎没有额外的时间消耗。

To solve the problems of lower image stylization quality of CycleGAN algorithm and weaker network stability,a CycleGAN-SN algorithm is proposed.This algorithm adds a spectral normalization layer after each convolutional layer of the discriminator network of CycleGAN algorithm,then estimates the spectral norm of the parameter matrix of the convolutional layer by a power iteration method,and updates the parameters of the convolutional layer with a random gradient descent method.The parameter change in each update is small,so it only needs to iterate once to estimate the maximum singular value of the matrix.The maximum singular value obtained is used to normalize the parameters of the convolution layer,which makes the entire discriminative network meet the 1-Lipschitz continuity.Experiments are performed on 4 commonly used style image datasets.Compared with CycleGAN algorithm,the results show that the proposed algorithm can generate higher-quality stylized pictures with bright colors,clear textures,and sufficient style renderings while retaining the details of the original pictures.The loss function of the proposed algorithm in the training process has a small oscillation amplitude.The network can be trained at higher learning rate and with greater stability.The proposed algorithm effectively reduces the number of steps required for network convergence and has a faster convergence rate.The proposed algorithm stylizes 751 images at a time during the test phase with protracting the period only by a maximum of 0.63 seconds,almost no extra time consumption.