基于平衡损失和多级注意力的溯因推理方法

Abductive reasoning based on balanced loss and multi-level attention

针对溯因自然语言推理任务(aNLI)中存在的问题,即正确假设之间存在一定独立性,且对推理的贡献存在不一致性,设计一种“平衡正样本softmax聚焦损失”,调整正确假设概率影响程度,并平衡样本损失权重.此外,在aNLI中,正样本与负样本之间的关联性往往体现在特定的短语上,这些短语对判断样本的合理性至关重要.因此设计多级注意力模型,通过多层次的注意力机制逐步细化,从而实现对短语级特征的深层次关注.这个新模型被命名为平衡损失多级注意力MAT-Ball模型.结果表明,MAT-Ball模型在Roberta-large预训练模型上取得了最高的性能,与公开可获得代码的方法相比,ACC和AUC结果分别增加了约1%和0.5%.同时,研究比较了在低资源和损失收敛性方面的性能,证明了所提出的方法的效率和鲁棒性.

To solve the problem in abductive natural language reasoning task(aNLI),where is a certain degree of independence between correct hypotheses and inconsistent contributions to reasoning,a balanced positive sample softmax focal loss is designed.This loss function adjusts the influence of correct hypothesis probability and balances sample loss weight.In addition,in aNLI,the correlation between positive and negative samples is often reflected in specific phrases,which are essential to judge the rationality of the sample.Therefore,a multi-level attention model is designed to achieve deep attention to phrase-level features through multi-level attention mechanism refinement.This new model was named aNLI:Multi-level Attention with Balanced Loss(MAT-Ball)model.The experimental results show that MAT-Ball has achieved the highest performance on the RoBERTa-large pretrained model,with ACC and AUC results increased by about 1%and 0.5%respectively compared to publicly available codes.Meanwhile,the efficiency and robustness of the proposed method are demonstrated by comparing the performance in terms of low resources and loss convergence.