基于田字格变换的自监督汉字字体生成

Self-supervised Chinese font generation based on square-block transformation

近年来,汉字自动生成因其在艺术字体生成、个性化字体设计,以及书法作品生成等问题中的广泛应用而引起了大量关注.当前主流的汉字字体自动生成方法主要基于非配对数据和深度生成模型如生成对抗网络等.然而,这些主流的深度汉字字体生成方法通常忽略了汉字本身的结构信息,导致在提取特征时缺乏相应指导,且在训练过程中容易出现模式坍塌现象,从而在生成汉字质量方面亟待进一步提高.针对该问题,本文受汉字田字格书写的启发,提出一种基于田字格变换的自监督方法来指导网络模型提取更高质量的特征,进而提升汉字生成效果,需要特别指出的是所设计的田字格几何变换无需改变现有模型网络且不增加任何人工成本,因此潜在可嵌入许多已有深度汉字字体生成模型.所提自监督学习方法的有效性在一系列实验中得到验证.实验结果表明,在嵌入所提的自监督学习任务后,当前流行的基于CycleGAN的深度汉字生成模型在生成效果和训练稳定性等方面都有较大提升,并且模式坍塌现象得到改善.与现有其他深度汉字字体生成方法相比,所提基于田字格几何变换的自监督方法提高了生成汉字质量,并且在生成汉字内容准确率、FID值、L1损失和IOU这4个评价指标上均有一定提升.

The generation of Chinese fonts has attracted much attention due to its wide range of applications such as artistic font generation, personalized font design, and calligraphy generation. The predominated Chinese font generation methods are mainly based on unpaired data and deep generative models such as generative adversarial networks. However, the existing deep methods for Chinese font generation commonly ignore the special structure information of Chinese characters, resulting in the lack of guidance during the feature extraction,and usually suffer from the mode collapse issue during the training procedure. Thus, the generation quality of Chinese characters needs to be improved. In order to solve this problem, this paper proposes a squared-block transformation-based self-supervised method to guide the model network to extract features with higher quality,and thus the proposed method significantly improves the performance on the generation of Chinese character fonts. It should be pointed out that the suggested squared-block geometric transformation does not require any modification of existing model networks and any additional human-labor cost. Thus, it can be adapted to many existing deep generative models for Chinese font generation. The effectiveness of the suggested self-supervised method is demonstrated by a series of experiments. The experiment results show that when equipped with the suggested squared-block transformation-based method, the popular CycleGAN-based deep model for Chinese font generation can significantly improve the quality of generated characters and stabilize the training procedure, while reducing the mode collapse. Moreover, when compared to other existing deep methods, the proposed method also outperforms them in terms of four evaluation metrics such as the content accuracy, FID, L1 loss, and IOU,as well as the quality of generated characters.