Sentiment analysis,commonly called opinion mining or emotion artificial intelligence(AI),employs biometrics,computational linguistics,nat-ural language processing,and text analysis to systematically identify,extract,m...Sentiment analysis,commonly called opinion mining or emotion artificial intelligence(AI),employs biometrics,computational linguistics,nat-ural language processing,and text analysis to systematically identify,extract,measure,and investigate affective states and subjective data.Sentiment analy-sis algorithms include emotion lexicon,traditional machine learning,and deep learning.In the text sentiment analysis algorithm based on a neural network,multi-layer Bi-directional long short-term memory(LSTM)is widely used,but the parameter amount of this model is too huge.Hence,this paper proposes a Bi-directional LSTM with a trapezoidal structure model.The design of the trapezoidal structure is derived from classic neural networks,such as LeNet-5 and AlexNet.These classic models have trapezoidal-like structures,and these structures have achieved success in the field of deep learning.There are two benefits to using the Bi-directional LSTM with a trapezoidal structure.One is that compared with the single-layer configuration,using the of the multi-layer structure can better extract the high-dimensional features of the text.Another is that using the trapezoidal structure can reduce the model’s parameters.This paper introduces the Bi-directional LSTM with a trapezoidal structure model in detail and uses Stanford sentiment treebank 2(STS-2)for experiments.It can be seen from the experimental results that the trapezoidal structure model and the normal structure model have similar performances.However,the trapezoidal structure model parameters are 35.75%less than the normal structure model.展开更多
目的:利用深度学习方法自动抽取中文生物医学文本中的开放式概念关系,以增强生物医学文本理解及医学知识网络构建。方法:使用BiLSTM-CRF模型从中文生物医学文献数据中抽取以句子上下文短语描述的开放式概念关系,并与基于条件随机场(Cond...目的:利用深度学习方法自动抽取中文生物医学文本中的开放式概念关系,以增强生物医学文本理解及医学知识网络构建。方法:使用BiLSTM-CRF模型从中文生物医学文献数据中抽取以句子上下文短语描述的开放式概念关系,并与基于条件随机场(Conditional Random Fields,CRF)和基于长短时记忆网络(Long Short-Term Memory,LSTM)的方法进行对比分析。结果:基于BiLSTM-CRF的中文生物医学开放式概念关系抽取方法取得F1值为0.5221,显著高于基于CRF模型的方法(F1值为0.2353)和基于LSTM模型的方法(F1值为0.3355)。结论:与单独使用CRF模型或LSTM模型的方法相比,基于BiLSTM-CRF的开放式概念关系抽取方法具有更好的鲁棒性和泛化性,对于生物医学文本理解、医学知识网络构建等研究具有借鉴意义。展开更多
Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the ...Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the increasing size and complexity of these models have led to increased training costs and reduced efficiency.This study aims to minimize the inference time of such models while maintaining computational performance.It also proposes a novel Distillation model for PAL-BERT(DPAL-BERT),specifically,employs knowledge distillation,using the PAL-BERT model as the teacher model to train two student models:DPAL-BERT-Bi and DPAL-BERTC.This research enhances the dataset through techniques such as masking,replacement,and n-gram sampling to optimize knowledge transfer.The experimental results showed that the distilled models greatly outperform models trained from scratch.In addition,although the distilled models exhibit a slight decrease in performance compared to PAL-BERT,they significantly reduce inference time to just 0.25%of the original.This demonstrates the effectiveness of the proposed approach in balancing model performance and efficiency.展开更多
基金supported by Yunnan Provincial Education Department Science Foundation of China under Grant construction of the seventh batch of key engineering research centers in colleges and universities(Grant Project:Yunnan College and University Edge Computing Network Engineering Research Center).
文摘Sentiment analysis,commonly called opinion mining or emotion artificial intelligence(AI),employs biometrics,computational linguistics,nat-ural language processing,and text analysis to systematically identify,extract,measure,and investigate affective states and subjective data.Sentiment analy-sis algorithms include emotion lexicon,traditional machine learning,and deep learning.In the text sentiment analysis algorithm based on a neural network,multi-layer Bi-directional long short-term memory(LSTM)is widely used,but the parameter amount of this model is too huge.Hence,this paper proposes a Bi-directional LSTM with a trapezoidal structure model.The design of the trapezoidal structure is derived from classic neural networks,such as LeNet-5 and AlexNet.These classic models have trapezoidal-like structures,and these structures have achieved success in the field of deep learning.There are two benefits to using the Bi-directional LSTM with a trapezoidal structure.One is that compared with the single-layer configuration,using the of the multi-layer structure can better extract the high-dimensional features of the text.Another is that using the trapezoidal structure can reduce the model’s parameters.This paper introduces the Bi-directional LSTM with a trapezoidal structure model in detail and uses Stanford sentiment treebank 2(STS-2)for experiments.It can be seen from the experimental results that the trapezoidal structure model and the normal structure model have similar performances.However,the trapezoidal structure model parameters are 35.75%less than the normal structure model.
文摘目的:利用深度学习方法自动抽取中文生物医学文本中的开放式概念关系,以增强生物医学文本理解及医学知识网络构建。方法:使用BiLSTM-CRF模型从中文生物医学文献数据中抽取以句子上下文短语描述的开放式概念关系,并与基于条件随机场(Conditional Random Fields,CRF)和基于长短时记忆网络(Long Short-Term Memory,LSTM)的方法进行对比分析。结果:基于BiLSTM-CRF的中文生物医学开放式概念关系抽取方法取得F1值为0.5221,显著高于基于CRF模型的方法(F1值为0.2353)和基于LSTM模型的方法(F1值为0.3355)。结论:与单独使用CRF模型或LSTM模型的方法相比,基于BiLSTM-CRF的开放式概念关系抽取方法具有更好的鲁棒性和泛化性,对于生物医学文本理解、医学知识网络构建等研究具有借鉴意义。
基金supported by Sichuan Science and Technology Program(2023YFSY0026,2023YFH0004).
文摘Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the increasing size and complexity of these models have led to increased training costs and reduced efficiency.This study aims to minimize the inference time of such models while maintaining computational performance.It also proposes a novel Distillation model for PAL-BERT(DPAL-BERT),specifically,employs knowledge distillation,using the PAL-BERT model as the teacher model to train two student models:DPAL-BERT-Bi and DPAL-BERTC.This research enhances the dataset through techniques such as masking,replacement,and n-gram sampling to optimize knowledge transfer.The experimental results showed that the distilled models greatly outperform models trained from scratch.In addition,although the distilled models exhibit a slight decrease in performance compared to PAL-BERT,they significantly reduce inference time to just 0.25%of the original.This demonstrates the effectiveness of the proposed approach in balancing model performance and efficiency.
