基于序列到序列预训练语言模型的楹联自动生成算法

Automatic Chinese Couplets Generation via Sequence-to-Sequence Pre-trained Neural Language Model

为了降低楹联文化的学习门槛,激发年轻人对楹联文化的兴趣,提出了一种基于序列到序列预训练神经网络语言模型的楹联自动生成算法。该算法将楹联应对任务建模为一个序列到序列的生成问题,将楹联的上联作为输入,并自递归地(auto-regressively)生成出符合楹联标准要求的下联。与现有神经网络方法不同,该算法模型在楹联生成任务上的训练前,在大规模无监督语料上进行预训练(pre-train),在楹联监督数据上进行微调(fine-tune)。在公开数据集上的实验证明,该算法在测试集上的BLEU值与人工评估指标相对基线模型均有明显提升,证明了该算法的有效性。

In order to reduce the obstacles of writing Chinese couplets, and stimulate young people’s interest in couplet culture, this paper proposes an automatic couplet generation algorithm based on sequence-to-sequence pre-trained neural network language model. The algorithm models the task as a sequence-to-sequence generation problem, takes the first line of the couplet as input, and auto-regressively generates the second line that meets the requirements of the Chinese couplet standard. The pre-training model used by the algorithm is composed of Transformers. During training, it is pre-trained on large-scale unsupervised corpus, and fine-tuned on the supervised data of Chinese couplets. Experiments on the public dataset show that the BLEU score and human evaluation score on test dataset are improved obviously from the baseline model, which demonstrate the effectiveness of the algorithm.