基于错误驱动的语义文法自动扩展学习方法研究

Automatic Error-driven Learning Method of Semantic Grammar

面向领域的自然语言理解技术是垂直搜索引擎、领域相关问答系统等应用的核心技术之一.本文在已构建的基于本体和语义文法的自然语言理解系统的基础上,提出一种基于错误驱动的语义文法自动扩展学习方法,对于解析错误的句子,利用核心文法生成部分解析树,按照打分函数选择一组最佳的部分解析树,利用预测模型预测部分解析树的上层节点并试图构建完整的解析树,从而学习得到新的文法规则,对于学习得到的不同类型的规则进行验证并更新核心文法库,通过对句子的可学习性度量来筛选学习对象,从而提高文法扩展学习的整体质量和效率.分别在两个不同规模的领域数据集进行了测试,在交互式学习范式下,测试对比了学习算法在不同规模领域的学习效率,在批量学习范式下,测试对比了更新后的文法和核心文法在两个领域数据集上的准确率和识别率等性能指标.实验结果表明,本文所提出的方法是有效的.

Domain-specific natural language understanding technology is one of the core technology of vertical search engines,domain-specific question answering system and other applications.This research focus on a novel constrained semantic grammar and its automatic learning methods based on an existing domain-specific question answering system.An error-driven learning method of semantic grammar is proposed.The method first partially parses the ungrammatical sentence based on the core semantic grammar,then it attempts to build a complete parse tree,including predicting the top-level node of the partial parsing tree,generating and verifying hypotheses of new grammar rules.Learnability metrics is used to filter sentences in the training corpus to improve the overall quality and efficiency of grammar extending algorithm.The proposed algorithm is applied to two domains of different scales.In the interactive learning paradigm,learning efficiency are compared in different domains.In the batch learning paradigm,the paper compares the accuracy,MRR and recognition rate of the extended grammar and core grammar on twodatasets.The test results show that the proposed method is effective.