基于条件残差生成对抗网络的风景图生成

Landscape image generation based on conditional residual generative adversarial network

风景图像的语义分割图中包含天空、白云、山川、树木、河流等大量类别信息,针对语义分割图中存在的信息类别过多、不同区域间的色彩变换不明显等问题,现有方法生成的风景图像在清晰度和真实性上效果并不理想。因此提出了一种基于条件残差生成对抗网络(CRGAN)方法,用于生成清晰度更高和内容更真实的风景图像。首先,优化生成器网络的上采样和下采样结构,提升生成器对语义分割图的特征提取效果。其次,在编码器和解码器之间使用跳跃连接传递语义分割图的特征信息,防止特征信息在编码器中传递丢失,保留特征信息的完整性。最后,在网络的编码器和解码器之间添加残差模块,以便更好地提取、传输和保留语义信息。此外,方法中采用均方差(MSE)提升语义分割图和生成图像之间的相似度。实验结果表明,相较于pix2pix和cyclegan方法,CRGAN生成的图像在FID指标中分别增加了26.769和119.333,有效提升了风景图像的清晰度和真实性。同时使用公共数据集验证了CRGAN的泛用性和有效性。

The semantic segmentation map of landscape image encompasses a large number of categorical information such as the sky,white clouds,mountains,rivers,and trees.In view of the challenges presented by the numerous information categories in the semantic segmentation map and the subtle color transformations between different regions,the landscape images generated by current methods are deficient in terms of both clarity and authenticity.Consequently,a method based on conditional residual generation adversarial network(CRGAN)was proposed to generate landscape images with a higher resolution and more realistic content.Firstly,the proposed method involved the upsampling and downsampling structures of the generator network to enhance the feature extraction effect of the generator on the semantic segmentation graph.Secondly,skip connections were utilized between the encoder and decoder to transmit the feature information from the semantic segmentation graph,ensuring the integrity of such information was retained,and not lost in the encoder.Finally,a residual module was added between the encoder and decoder of the network,facilitating better extraction,transmission,and retention of semantic information.In addition,the mean square error(MSE)was employed to enhance the similarity between semantically segmented graphs and generated images.The experimental results demonstrated that compared with pix2pix and cyclegan methods,the FID index of images generated by CRGAN increased by 26.769 and 119.333,respectively.This improvement effectively enhanced the clarity and authenticity of landscape images.The universality and validity of CRGAN were also validated using a common dataset.