Transformer优化及其在苹果病虫命名实体识别中的应用

Transformer Optimization and Application in Named Entity Recognition of Apple Diseases and Pests

为提高苹果生产领域实体识别的准确性,提出一种新的Transformer优化模型。首先,为解决苹果生产数据集的缺失,基于苹果栽培领域园艺专家的知识经验,创建以苹果病虫害为主的产业数据集。通过字向量与词向量的拼接,提高文本语义表征的准确性;随后,为防止位置信息缺失,引入具有方向和距离感知的注意力机制,平均集成BiLSTM的上下文长距离依赖特征;最后,结合条件随机场(Conditional random fields, CRF)约束上下文标注结果,最终得到Transformer优化模型。实验结果表明,所提方法在苹果病虫命名实体识别中的F1值可达92.66%,可为农业命名实体的准确智能识别提供技术手段。

Aiming to improve the accuracy of entity identification in apple production field,a new Transformer optimization model was proposed.Firstly,in order to address the lack of apple production dataset,a corpus focusing on diseases and pests was constructed based on the knowledge and experience of horticultural experts in related field of apple cultivation.The accuracy of semantic representation of text was improved by combining word vector and character vector.Secondly,since the location information was crucial to text semantics,but the traditional Transformer model lacks the directionality of location information,in order to take advantage of the location features of text,an attention mechanism with direction and distance perception was introduced in Transformer encoder.And the contextual long⁃distance dependence features of BiLSTM was integrated on average to enhance semantic representation.Lastly,with imposing restrictions on labeling results by conditional random fields(CRF),the Transformer optimization model was obtained.The experimental results showed that the F1 score of the proposed method was 92.66%in Chinese named entity recognition of Apple diseases and pests.It indicated that the method proposed could effectively identify the named entities of apple diseases and pest,and provide a technical means for the accurate and intelligent identification of other agricultural named entities.