摘要
针对关系抽取中句子内部单词干扰产生的噪声问题,提出了一种基于软阈值模块的残差收缩网络的改进算法。首先,在残差网络的各个特征通道训练阈值,该阈值具备两个特点,一是其绝对值不能过大,过大将会剔除有效信息;二是该阈值对于不同的输入训练有不同的结果。然后,根据软阈值化的特性,将通道特征中小于阈值的部分删除,大于阈值的部分减小,相较于直接删除负面特性,软阈值可以保存负面特性中有用的信息。最后,额外加入注意力模块优化模型,该模块可以降低远程监督中错误标签问题对实验的影响。选取分段卷积神经网络(PCNN)、双向长短期记忆神经(BiLSTM)网络和普通残差网络(ResNet)作为基线模型进行对比实验,实验结果表明,所提模型的精确率-召回率曲线包含了基线模型的曲线,且F1值相较于基准模型分别提高了6.0个百分点、3.9个百分点和1.4个百分点,验证了加入软阈值化的网络模型可以通过减少句内噪声的方式提高关系抽取的准确性。
An improved algorithm based on residual shrinkage network with soft threshold module was proposed to solve the problem of noise caused by interference between words within a sentence in relation extraction. Firstly, the threshold was trained in each feature channel of the residual network. The threshold had two characteristics: first, its absolute value would not be too large, if it was too large, effective information would be eliminated;second, the threshold had different results for different input training. Secondly, according to the characteristics of soft threshold, the channel features lower than the threshold were deleted, and those higher than the threshold were reduced. Compared with direct deletion of negative features, soft threshold was able to save useful information of negative features. Finally, an optimization model of attention module was added to reduce the influence of mislabeling problem in distant supervision. Piecewise Convolutional Neural Network(PCNN), Bi-directional Long Short-Term Memory(BiLSTM) network and ordinary Residual Network(ResNet) were selected as baseline models for comparison experiments. Experimental results show that the precision-recall curves of the proposed model include the curves of other models and the F1 scores of the proposed model are increased by 6. 0percentage points, 3. 9 percentage points and 1. 4 percentage points respectively compared to the baseline models, which verifies that addition of soft thresholding network model can improve accuracy of relation extraction by reducing in-sentence noise.
作者
袁泉
薛书鑫
YUAN Quan;XUE Shuxin(School of Communication and Information Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China;Research Center of New Communication Technology Applications,Chongqing University of Posts and Telecommunications,Chongqing 400065,China)
出处
《计算机应用》
CSCD
北大核心
2022年第10期3040-3045,共6页
journal of Computer Applications
关键词
残差网络
远程监督
注意力机制
关系抽取
软阈值化
residual network
distant supervision
attention mechanism
relation extraction
soft thresholding