内容结构保持的图像风格迁移方法

Content Structure Preserved Image Style Transfer Method

现有的风格迁移算法大多是实现艺术作品到真实场景,如照片等的转换任务。在这种任务设定下,图像内容的边界等结构信息可能会出现一定程度的变化,然而这种变化不利于下游任务的完成,因而不适用于大部分工业场景。为此提出一种新的结构保持的风格迁移方法,该方法分为频域约束的图像生成模块和基于memory bank机制的语义匹配模块。前者用以保证转换前后图像的整体语义结构一致性,而后者保证了图像的语义与风格匹配,从而物体内部结构一致。为了验证该方法在工业场景中的迁移效果,采用光伏板数据集,在虚拟到真实的迁移方向上,所提出的风格迁移算法能在完成风格转换的同时较好地解决图像结构形变问题,进而满足后续任务的需求。

Most existing style transfer algorithms realize the conversion task from art works to real scenes,such as photos.Under this experiment setting,the content structure of the image may change to a certain extent.However,this change is not conducive to the completion of downstream tasks,so it is not available for most industrial scenes.Therefore,a new structure preserving style transfer method is proposed,which contains a frequency domain constrained image generation module and a semantic matching module based on memory bank.The former is used to ensure the consistency of the overall semantic structure of the image before and after transfer,while the latter ensures the semantic and style matching,so that the internal structure of the object is consistent.By comparing the frequency domain similarity between the original image and the migrated image and the similarity between the generated image and the corresponding category in memory bank,the purpose of keeping the image structure unchanged is achieved.In order to verify the effect this method produced in the industrial scene,the photovoltaic panel dataset is used.The proposed style transfer algorithm can not only complete the style conversion,but also better resolve the problem of image structure deformation,so as to meet the needs of subse-quent tasks.