ACRank:在神经排序模型中引入检索公理知识

ACRank:Injecting IR Axiomatic Knowledge to Neural Ranking Models

传统的信息检索(Information Retrieval,IR)是知识驱动的方法,如以BM25、LMIR等为代表的检索模型在设计过程中考虑词频、逆文档频率、文档长度等关键因素计算查询-文档的相关性得分.这些关键因素被总结为IR公理,在传统模型的设计和评价中起到了至关重要的作用.如词频规则认为有更多查询词的文档更相关.与之相对,数据驱动的神经排序模型基于大量的标注数据与精巧的神经网络结构自动学习相关性评分函数,带来了显著的排序精度提升.传统IR公理知识是否能用来提升神经排序模型的效果是一个值得研究的重要问题且已有学者进行了初步探索,其首先通过公理指导增强数据生成,然后利用生成的标注数据直接训练神经网络.但IR公理的形式是通过比较匹配信号的强弱给出两个文档间相对的相关关系,而非直接给出文档的相关度标签.针对这一问题,本文提出了一种通过对比学习将IR公理知识引入神经排序模型的框架,称为ACRank.ACRank利用信息检索公理生成增强数据,抽取不同文档的匹配信号,利用对比学习拉开匹配信号间差距,使正样本匹配信号强于负样本,通过上述方式,ACRank将IR公理知识自然地融入到数据驱动的神经排序模型中.ACRank作为通用框架,可应用于不同规则,本文选择词频规则进行实验,基于大规模公开数据集上的实验结果表明,ACRank能够有效提升已有神经检索模型如BERT的排序精度,相关分析实验验证了该框架的有效性.

Traditional information retrieval(IR)models such as BM25 and language models for IR(LMIR)are knowledge-driven approaches that primarily focus on the term frequency,document length,and inverse document frequency.These models derive the relevance score of a document based on a combination of these factors.Experts summarize a set of IR axioms that these models should follow.The axioms describe the characteristics that an ideal IR model should have.One such famous axiom is the TFC constraint,which stands for“Term Frequency Constraint”.According to this constraint,a document that contains more occurrences of the query terms should be considered more relevant.This means that the model should prioritize documents that have a high frequency of search terms over those with a lower frequency.Recently,there has been a significant amount of research and development focused on data-driven approaches to IR,particularly using neural models.These models can learn ranking functions from labeled data and have been extensively studied and improved in recent years.Researchers have preliminarily explored whether axiomatic knowledge can improve the neural ranking model.One approach that has been explored is to generate augmented data that follows IR axioms and use these data to train the neural models.By incorporating these axioms,it is believed that the models can learn to better distinguish between relevant and irrelevant documents.However,it is worth noting that IR axioms only provide information on the relative relevance of two documents based on the difference in their matching signals.They do not provide a definitive determination on whether a document is relevant or not.To better incorporate IR axioms’comparison nature of matching signals and inspired by the data generation process in contrastive learning,we proposed a framework that enhances axiomatic knowledge in the neural ranking model via contrastive learning,named ACRank.ACRank generates documents guided by the IR axioms that have similar content but differ in other features.By presenting these pairs to the neural model and training it to extract matching signals between them based on axiomatic knowledge.The key differences in matching signals between the generated documents and the original documents are highlighted through optimizing contrastive loss.Through this approach,the model can learn the comparison nature of the matching signals mentioned by IR axioms.This allows the model to better understand IR axioms for determining relevance and to leverage this knowledge when ranking documents.In this way,the IR axiomatic knowledge is naturally transferred to the models.Overall,the proposed framework is a general framework that can be applied to different axioms.It is a promising way to enhance the effectiveness of neural ranking models by leveraging insights from IR axioms.By combining the strengths of knowledge-driven and data-driven approaches,we hope to develop more effective search systems for a wide range of applications.To test the effectiveness of the framework,we use the TFC constraint.Experimental results showed that the proposed framework was able to effectively improve the ranking model in comparison to BERT and others.The analysis showed that the framework was able to significantly improve the effectiveness of the model.