关系抽取是信息获取中一项关键技术。句子结构树能够捕获单词之间的长距离依赖关系,已被广泛用于关系抽取任务中。但是,现有方法存在过度依赖句子结构树本身信息而忽略外部信息的不足。本文提出一种新型的图神经网络模型,即注意力图长...关系抽取是信息获取中一项关键技术。句子结构树能够捕获单词之间的长距离依赖关系,已被广泛用于关系抽取任务中。但是,现有方法存在过度依赖句子结构树本身信息而忽略外部信息的不足。本文提出一种新型的图神经网络模型,即注意力图长短时记忆神经网络(attention graph long short term memory neural network,AGLSTM)。该模型采用一种软修剪策略自动学习对关系抽取有用的句子结构信息;通过引入注意力机制,结合句法图信息学习句子的结构特征;并设计一种新型的图长短时记忆神经网络,使得模型能够更好地融合句法图信息和句子的时序信息。与10种典型的关系抽取方法进行对比,实验验证了该模型的优异性能。展开更多
工业控制场景下5G-A终端传输时延是确定性网络能力的直接表征之一,时延预测对提高网络确定性至关重要。由于传输时延序列的不稳定性和随机性,单一模型难以准确预测。针对该问题,提出一种基于优化变分模态分解(Variational Mode Decompos...工业控制场景下5G-A终端传输时延是确定性网络能力的直接表征之一,时延预测对提高网络确定性至关重要。由于传输时延序列的不稳定性和随机性,单一模型难以准确预测。针对该问题,提出一种基于优化变分模态分解(Variational Mode Decomposition, VMD)和卷积注意力长短时记忆网络(Convolutional Attention Long Short Term Memory Network, CA-LSTM)的传输时延预测方法。为提高VMD的分解性能,利用相关系数检验法确定时延序列分解的模态数,并利用蝗虫优化寻优分解的惩罚因子和保真度系数;设计CA-LSTM网络,借助卷积滤波器以及注意力机制使得网络具备分辨时延特征重要程度的能力;将各模态预测值重建成一维时延值得到预测结果。实验研究结果表明,优化VDM能够将5G终端传输时延序列有效分解,结合CA-LSTM模型相比于经典LSTM在MSE、RMSE和MAE上分别提升了37.1%、21.3%和23.6%。展开更多
为了对高校学术活动进行信息提取和文本分类,基于结合注意力机制的长短期记忆网络(Attention-Based Long Short-Term Memory)特征选择模型,构建了高校学术活动分类系统。通过大量分析高校学术活动语料的特点,准确抽取学术活动的相关内容...为了对高校学术活动进行信息提取和文本分类,基于结合注意力机制的长短期记忆网络(Attention-Based Long Short-Term Memory)特征选择模型,构建了高校学术活动分类系统。通过大量分析高校学术活动语料的特点,准确抽取学术活动的相关内容,改善了文本数据质量;提出了一种改进的Attention-Based LSTM特征选择模型,降低了数据维度,有效地突出了重点信息。实验结果表明,该方法提高了分类的准确率,其分类效果明显优于普通LSTM(Long Short-Term Memory)模型和传统模型的处理结果。展开更多
To fully make use of information from different representation subspaces,a multi-head attention-based long short-term memory(LSTM)model is proposed in this study for speech emotion recognition(SER).The proposed model ...To fully make use of information from different representation subspaces,a multi-head attention-based long short-term memory(LSTM)model is proposed in this study for speech emotion recognition(SER).The proposed model uses frame-level features and takes the temporal information of emotion speech as the input of the LSTM layer.Here,a multi-head time-dimension attention(MHTA)layer was employed to linearly project the output of the LSTM layer into different subspaces for the reduced-dimension context vectors.To provide relative vital information from other dimensions,the output of MHTA,the output of feature-dimension attention,and the last time-step output of LSTM were utilized to form multiple context vectors as the input of the fully connected layer.To improve the performance of multiple vectors,feature-dimension attention was employed for the all-time output of the first LSTM layer.The proposed model was evaluated on the eNTERFACE and GEMEP corpora,respectively.The results indicate that the proposed model outperforms LSTM by 14.6%and 10.5%for eNTERFACE and GEMEP,respectively,proving the effectiveness of the proposed model in SER tasks.展开更多
Aiming at the relation linking task for question answering over knowledge base,especially the multi relation linking task for complex questions,a relation linking approach based on the multi-attention recurrent neural...Aiming at the relation linking task for question answering over knowledge base,especially the multi relation linking task for complex questions,a relation linking approach based on the multi-attention recurrent neural network(RNN)model is proposed,which works for both simple and complex questions.First,the vector representations of questions are learned by the bidirectional long short-term memory(Bi-LSTM)model at the word and character levels,and named entities in questions are labeled by the conditional random field(CRF)model.Candidate entities are generated based on a dictionary,the disambiguation of candidate entities is realized based on predefined rules,and named entities mentioned in questions are linked to entities in knowledge base.Next,questions are classified into simple or complex questions by the machine learning method.Starting from the identified entities,for simple questions,one-hop relations are collected in the knowledge base as candidate relations;for complex questions,two-hop relations are collected as candidates.Finally,the multi-attention Bi-LSTM model is used to encode questions and candidate relations,compare their similarity,and return the candidate relation with the highest similarity as the result of relation linking.It is worth noting that the Bi-LSTM model with one attentions is adopted for simple questions,and the Bi-LSTM model with two attentions is adopted for complex questions.The experimental results show that,based on the effective entity linking method,the Bi-LSTM model with the attention mechanism improves the relation linking effectiveness of both simple and complex questions,which outperforms the existing relation linking methods based on graph algorithm or linguistics understanding.展开更多
文摘关系抽取是信息获取中一项关键技术。句子结构树能够捕获单词之间的长距离依赖关系,已被广泛用于关系抽取任务中。但是,现有方法存在过度依赖句子结构树本身信息而忽略外部信息的不足。本文提出一种新型的图神经网络模型,即注意力图长短时记忆神经网络(attention graph long short term memory neural network,AGLSTM)。该模型采用一种软修剪策略自动学习对关系抽取有用的句子结构信息;通过引入注意力机制,结合句法图信息学习句子的结构特征;并设计一种新型的图长短时记忆神经网络,使得模型能够更好地融合句法图信息和句子的时序信息。与10种典型的关系抽取方法进行对比,实验验证了该模型的优异性能。
文摘工业控制场景下5G-A终端传输时延是确定性网络能力的直接表征之一,时延预测对提高网络确定性至关重要。由于传输时延序列的不稳定性和随机性,单一模型难以准确预测。针对该问题,提出一种基于优化变分模态分解(Variational Mode Decomposition, VMD)和卷积注意力长短时记忆网络(Convolutional Attention Long Short Term Memory Network, CA-LSTM)的传输时延预测方法。为提高VMD的分解性能,利用相关系数检验法确定时延序列分解的模态数,并利用蝗虫优化寻优分解的惩罚因子和保真度系数;设计CA-LSTM网络,借助卷积滤波器以及注意力机制使得网络具备分辨时延特征重要程度的能力;将各模态预测值重建成一维时延值得到预测结果。实验研究结果表明,优化VDM能够将5G终端传输时延序列有效分解,结合CA-LSTM模型相比于经典LSTM在MSE、RMSE和MAE上分别提升了37.1%、21.3%和23.6%。
基金The National Natural Science Foundation of China(No.61571106,61633013,61673108,81871444).
文摘To fully make use of information from different representation subspaces,a multi-head attention-based long short-term memory(LSTM)model is proposed in this study for speech emotion recognition(SER).The proposed model uses frame-level features and takes the temporal information of emotion speech as the input of the LSTM layer.Here,a multi-head time-dimension attention(MHTA)layer was employed to linearly project the output of the LSTM layer into different subspaces for the reduced-dimension context vectors.To provide relative vital information from other dimensions,the output of MHTA,the output of feature-dimension attention,and the last time-step output of LSTM were utilized to form multiple context vectors as the input of the fully connected layer.To improve the performance of multiple vectors,feature-dimension attention was employed for the all-time output of the first LSTM layer.The proposed model was evaluated on the eNTERFACE and GEMEP corpora,respectively.The results indicate that the proposed model outperforms LSTM by 14.6%and 10.5%for eNTERFACE and GEMEP,respectively,proving the effectiveness of the proposed model in SER tasks.
基金The National Natural Science Foundation of China(No.61502095).
文摘Aiming at the relation linking task for question answering over knowledge base,especially the multi relation linking task for complex questions,a relation linking approach based on the multi-attention recurrent neural network(RNN)model is proposed,which works for both simple and complex questions.First,the vector representations of questions are learned by the bidirectional long short-term memory(Bi-LSTM)model at the word and character levels,and named entities in questions are labeled by the conditional random field(CRF)model.Candidate entities are generated based on a dictionary,the disambiguation of candidate entities is realized based on predefined rules,and named entities mentioned in questions are linked to entities in knowledge base.Next,questions are classified into simple or complex questions by the machine learning method.Starting from the identified entities,for simple questions,one-hop relations are collected in the knowledge base as candidate relations;for complex questions,two-hop relations are collected as candidates.Finally,the multi-attention Bi-LSTM model is used to encode questions and candidate relations,compare their similarity,and return the candidate relation with the highest similarity as the result of relation linking.It is worth noting that the Bi-LSTM model with one attentions is adopted for simple questions,and the Bi-LSTM model with two attentions is adopted for complex questions.The experimental results show that,based on the effective entity linking method,the Bi-LSTM model with the attention mechanism improves the relation linking effectiveness of both simple and complex questions,which outperforms the existing relation linking methods based on graph algorithm or linguistics understanding.
