基于隐层傅里叶卷积的非平稳纹理合成方法

Hidden layer Fourier convolution for non-stationary texture synthesis

随着深度学习在计算机视觉领域取得的巨大成功,基于示例的纹理合成研究得到了长足的发展.当下主流纹理合成模型往往采用神经网络方法,其通常包含卷积层和上采样层、下采样层等局部组件,并不适用于捕捉非平稳纹理中的不规则结构特征.受频率域与空间域的对偶性质的启发,提出了一种基于隐层傅里叶卷积的非平稳纹理合成方法.该方法以生成对抗网络为基础架构,沿着隐层通道进行特征拆分,搭建图像域局部分支和频率域全局分支,进而兼顾视觉感知和结构信息.实验表明,该方法能够处理结构上极具挑战的非平稳纹理样本,相较于目前最优方法而言,在大尺度结构的学习与扩展上取得了更好的效果.

The remarkable achievements of deep learning in computer vision have led to significant development in example-based texture synthesis.The texture synthesis model using neural networks mainly includes local components,such as convolution and up/down sampling,which is unsuitable for capturing irregular structural attributes in non-stationary textures.Inspired by the frequency and space domain duality,a non-stationary texture synthesis method based on hidden layer Fourier convolution is proposed in this study.The proposed method uses the generative adversarial network as the basic architecture,performs feature splitting along the channel in the hidden layer,and builds a local branch in the image domain and a global branch in the frequency domain to consider visual perception and structural information.Experimental results show that this method can handle structurally challenging non-stationary texture exemplars.Compared with state-of-the-art methods,the method yielded better results in the learning and expansion of large-scale structures.