基于对比学习的表达式偏序关系建模

Modeling Partial Order Relation in Math Expression by Contrastive Learning

数学公式解题任务要求模型根据数学问题生成表达式用于解答。该任务的主流方法是将目标表达式当作文本序列来生成。然而,这一设定导致模型忽略了表达式树作为树形结构所带有的偏序关系,如交换律、分配律等。这不仅降低了模型对表达式生成的学习效率,也减弱了模型的泛化能力。为解决这一问题,该文提出一种基于对比学习的表达式偏序关系建模方法。该方法的核心做法是在模型训练时,对表达式树做微调扰动,产生和原有表达式等价和不等价的正样本和负样本,并通过对比学习最小化原式和等价式子之间的距离,且最大化与不等价负样本式子之间的距离。在公开数据集Math23K和MAWPS上的对比实验表明,该文方法相对于基线模型具有显著性能提升。

In math word problem solving task,mainstream methods treat target expression as text sequence.The consequence of this setting is that the model cannot capture the partial order relation of expression trees,such as the law of exchange or the distributive law.This hinders models'learning efficiency in generating expression,and also reduces their generalization capability.To address this problem,this paper proposes a method to model expression tree's partial order relation by contrastive learning.The core practice is to generate both positive and negative sam-ples when training models.The samples are obtained by modifying the original expression tree.The training objec-tive is to minimize the distance between original expression with equivalent samples and maximize the distance with inequivalent ones.Based on experiments performed on Math23k and MAWPS,our model outperforms the baseline models by a significant margin.