A differentiable neural computer(DNC)is analogous to the Von Neumann machine with a neural network controller that interacts with an external memory through an attention mechanism.Such DNC’s offer a generalized metho...A differentiable neural computer(DNC)is analogous to the Von Neumann machine with a neural network controller that interacts with an external memory through an attention mechanism.Such DNC’s offer a generalized method for task-specific deep learning models and have demonstrated reliability with reasoning problems.In this study,we apply a DNC to a language model(LM)task.The LM task is one of the reasoning problems,because it can predict the next word using the previous word sequence.However,memory deallocation is a problem in DNCs as some information unrelated to the input sequence is not allocated and remains in the external memory,which degrades performance.Therefore,we propose a forget gatebased memory deallocation(FMD)method,which searches for the minimum value of elements in a forget gate-based retention vector.The forget gatebased retention vector indicates the retention degree of information stored in each external memory address.In experiments,we applied our proposed NTM architecture to LM tasks as a task-specific example and to rescoring for speech recognition as a general-purpose example.For LM tasks,we evaluated DNC using the Penn Treebank and enwik8 LM tasks.Although it does not yield SOTA results in LM tasks,the FMD method exhibits relatively improved performance compared with DNC in terms of bits-per-character.For the speech recognition rescoring tasks,FMD again showed a relative improvement using the LibriSpeech data in terms of word error rate.展开更多
This paper reviews the theory of language attrition,which refers to the loss or degradation of second language skills due to lack of using the second language for a certain period of time.Although our country attaches...This paper reviews the theory of language attrition,which refers to the loss or degradation of second language skills due to lack of using the second language for a certain period of time.Although our country attaches great importance to English language teaching,most of college English majors use English far less frequently than that of Chinese in real life,which makes them easily influenced by language attrition.Therefore,it is of great significance for college English majors to improve the efficiency of English vocabulary memory from the perspective of language attrition combined with Forgetting.This thesis consists of three parts.Chapter one is an analysis the concept of language attrition and Forgetting.Chapter two describes and analyzes the existing problems in current vocabulary memory among the college English majors via a questionnaire survey.The final chapter puts forward some corresponding countermeasures to help college English majors get rid of the influence of language attrition on vocabulary learning.展开更多
In this paper, we discuss questions of creating an electronic intellectual analogue of a human being. We introduce a mathematical concept of stimulus generating emotions. We also introduce a definition of logical thin...In this paper, we discuss questions of creating an electronic intellectual analogue of a human being. We introduce a mathematical concept of stimulus generating emotions. We also introduce a definition of logical thinking of robots and a notion of efficiency coefficient to describe their efficiency of rote (mechanical) memorizing. The paper proves theorems describing properties of permanent conflicts between logical and emotional thinking of robots with a nonabsolute rote memory.展开更多
Properties that are similar to the memory and learning functions in biological systems have been observed and reported in the experimental studies of memristors fabricated by different materials. These properties incl...Properties that are similar to the memory and learning functions in biological systems have been observed and reported in the experimental studies of memristors fabricated by different materials. These properties include the forgetting effect, the transition from short-term memory（STM） to long-term memory（LTM）, learning-experience behavior, etc. The mathematical model of this kind of memristor would be very important for its theoretical analysis and application design.In our analysis of the existing memristor model with these properties, we find that some behaviors of the model are inconsistent with the reported experimental observations. A phenomenological memristor model is proposed for this kind of memristor. The model design is based on the forgetting effect and STM-to-LTM transition since these behaviors are two typical properties of these memristors. Further analyses of this model show that this model can also be used directly or modified to describe other experimentally observed behaviors. Simulations show that the proposed model can give a better description of the reported memory and learning behaviors of this kind of memristor than the existing model.展开更多
心理咨询场景下的情感分类旨在获得咨询者话语的情感倾向,为建立心理咨询AI助手提供支持。现有的方法利用语境信息获取文本情感倾向,但未考虑对话记录中当前句与前向近邻句之间的情感传递。针对这一问题,提出一种基于交互注意力(AOA)机...心理咨询场景下的情感分类旨在获得咨询者话语的情感倾向,为建立心理咨询AI助手提供支持。现有的方法利用语境信息获取文本情感倾向,但未考虑对话记录中当前句与前向近邻句之间的情感传递。针对这一问题,提出一种基于交互注意力(AOA)机制的心理咨询文本情感分类模型,根据时序对历史情感词分配权重,进而提高分类准确率。利用构建的心理健康情感词典分别提取对话双方的历史情感词序列,再将当前句和历史情感词序列输入到双向长短期记忆(BiLSTM)网络获取对应的特征向量,并利用艾宾浩斯遗忘曲线对历史情感词序列分配权重。通过AOA机制获得惯性特征和交互特征,并结合文本特征输入到分类层计算情感倾向概率。在公开数据集Emotional First Aid Dataset上的实验结果表明,相较于Caps-DGCN(Capsule network and Directional Graph Convolutional Network)模型,所提模型的F1值提高了1.55%。可见,所提模型可以有效提升心理咨询文本的情感分类效果。展开更多
基金supported by the ICT R&D By the Institute for Information&communications Technology Promotion(IITP)grant funded by the Korea government(MSIT)[Project Number:2020-0-00113,Project Name:Development of data augmentation technology by using heterogeneous information and data fusions].
文摘A differentiable neural computer(DNC)is analogous to the Von Neumann machine with a neural network controller that interacts with an external memory through an attention mechanism.Such DNC’s offer a generalized method for task-specific deep learning models and have demonstrated reliability with reasoning problems.In this study,we apply a DNC to a language model(LM)task.The LM task is one of the reasoning problems,because it can predict the next word using the previous word sequence.However,memory deallocation is a problem in DNCs as some information unrelated to the input sequence is not allocated and remains in the external memory,which degrades performance.Therefore,we propose a forget gatebased memory deallocation(FMD)method,which searches for the minimum value of elements in a forget gate-based retention vector.The forget gatebased retention vector indicates the retention degree of information stored in each external memory address.In experiments,we applied our proposed NTM architecture to LM tasks as a task-specific example and to rescoring for speech recognition as a general-purpose example.For LM tasks,we evaluated DNC using the Penn Treebank and enwik8 LM tasks.Although it does not yield SOTA results in LM tasks,the FMD method exhibits relatively improved performance compared with DNC in terms of bits-per-character.For the speech recognition rescoring tasks,FMD again showed a relative improvement using the LibriSpeech data in terms of word error rate.
文摘This paper reviews the theory of language attrition,which refers to the loss or degradation of second language skills due to lack of using the second language for a certain period of time.Although our country attaches great importance to English language teaching,most of college English majors use English far less frequently than that of Chinese in real life,which makes them easily influenced by language attrition.Therefore,it is of great significance for college English majors to improve the efficiency of English vocabulary memory from the perspective of language attrition combined with Forgetting.This thesis consists of three parts.Chapter one is an analysis the concept of language attrition and Forgetting.Chapter two describes and analyzes the existing problems in current vocabulary memory among the college English majors via a questionnaire survey.The final chapter puts forward some corresponding countermeasures to help college English majors get rid of the influence of language attrition on vocabulary learning.
文摘In this paper, we discuss questions of creating an electronic intellectual analogue of a human being. We introduce a mathematical concept of stimulus generating emotions. We also introduce a definition of logical thinking of robots and a notion of efficiency coefficient to describe their efficiency of rote (mechanical) memorizing. The paper proves theorems describing properties of permanent conflicts between logical and emotional thinking of robots with a nonabsolute rote memory.
文摘Properties that are similar to the memory and learning functions in biological systems have been observed and reported in the experimental studies of memristors fabricated by different materials. These properties include the forgetting effect, the transition from short-term memory（STM） to long-term memory（LTM）, learning-experience behavior, etc. The mathematical model of this kind of memristor would be very important for its theoretical analysis and application design.In our analysis of the existing memristor model with these properties, we find that some behaviors of the model are inconsistent with the reported experimental observations. A phenomenological memristor model is proposed for this kind of memristor. The model design is based on the forgetting effect and STM-to-LTM transition since these behaviors are two typical properties of these memristors. Further analyses of this model show that this model can also be used directly or modified to describe other experimentally observed behaviors. Simulations show that the proposed model can give a better description of the reported memory and learning behaviors of this kind of memristor than the existing model.
文摘心理咨询场景下的情感分类旨在获得咨询者话语的情感倾向,为建立心理咨询AI助手提供支持。现有的方法利用语境信息获取文本情感倾向,但未考虑对话记录中当前句与前向近邻句之间的情感传递。针对这一问题,提出一种基于交互注意力(AOA)机制的心理咨询文本情感分类模型,根据时序对历史情感词分配权重,进而提高分类准确率。利用构建的心理健康情感词典分别提取对话双方的历史情感词序列,再将当前句和历史情感词序列输入到双向长短期记忆(BiLSTM)网络获取对应的特征向量,并利用艾宾浩斯遗忘曲线对历史情感词序列分配权重。通过AOA机制获得惯性特征和交互特征,并结合文本特征输入到分类层计算情感倾向概率。在公开数据集Emotional First Aid Dataset上的实验结果表明,相较于Caps-DGCN(Capsule network and Directional Graph Convolutional Network)模型,所提模型的F1值提高了1.55%。可见,所提模型可以有效提升心理咨询文本的情感分类效果。
