Event-related potential (ERP) studies demonstrated that emotional state immediately affects the processing of different linguistic mismatches (e.g., semantic and syntactic mismatches) in sentences. Furthermore, mood h...Event-related potential (ERP) studies demonstrated that emotional state immediately affects the processing of different linguistic mismatches (e.g., semantic and syntactic mismatches) in sentences. Furthermore, mood has been shown to impact discourse processing. In these studies, a strong expectancy was evoked by a linguistic context. In the present study, a strong expectancy was induced by a visual (non-linguistic) context paired with a sentence. A spatial array was followed by a sentence that matched (□○—The square stands in front of the circle) or mismatched the picture (intra-dimensional mismatch: □○—The square stands behind the circle;extra-dimensional mismatch: □○—The square stands above the circle). The main question was whether mood effects on the processing of visually induced expectancies are similar or different from mood effects on the processing of linguistically induced expectancies. To this aim, we presented conceptual (mis)matches that differed in saliency: intra-dimensional vs. extra-dimensional mismatches which are both reported to elicit an N2b/N400 and P600, reflecting reanalysis. EEG was recorded while participants read for comprehension. Mood (happy vs. sad) was effectively induced by film clips. Neither for N2b/N400 nor for P600, an interaction between mood and match was observed. Thus, in contrast with studies investigating the effects of pure linguistic mismatches (linguistic context paired with unexpected lexical item), mood did not modulate the processing of conceptual mismatches. However, a main effect of match revealed different ERP patterns for the two mismatches. While an increase in N2b/ N400 occurred for the intra-dimensional and extra-dimensional mismatches, a P600 only occurred to the extra-dimensional (more salient) mismatches. This finding is taken as support that being in an emotional state (positive or negative mood) influences processes of reanalysis. That is, when being in an emotional state, a reanalysis (monitoring) process is exclusively triggered by salient mismatches.展开更多
篇章关系抽取旨在识别篇章中实体对之间的关系.相较于传统的句子级别关系抽取,篇章级别关系抽取任务更加贴近实际应用,但是它对实体对的跨句子推理和上下文信息感知等问题提出了新的挑战.本文提出融合实体和上下文信息(Fuse entity and ...篇章关系抽取旨在识别篇章中实体对之间的关系.相较于传统的句子级别关系抽取,篇章级别关系抽取任务更加贴近实际应用,但是它对实体对的跨句子推理和上下文信息感知等问题提出了新的挑战.本文提出融合实体和上下文信息(Fuse entity and context information,FECI)的篇章关系抽取方法,它包含两个模块,分别是实体信息抽取模块和上下文信息抽取模块.实体信息抽取模块从两个实体中自动地抽取出能够表示实体对关系的特征.上下文信息抽取模块根据实体对的提及位置信息,从篇章中抽取不同的上下文关系特征.本文在三个篇章级别的关系抽取数据集上进行实验,效果得到显著提升.展开更多
远程监督关系抽取通过自动对齐自然语言文本与知识库生成带有标签的训练数据集,解决样本人工标注的问题。目前的远程监督研究大多没有关注到长尾(long-tail)数据,因此远程监督得到的大多数句包中所含句子太少,不能真实全面地反映数据的...远程监督关系抽取通过自动对齐自然语言文本与知识库生成带有标签的训练数据集,解决样本人工标注的问题。目前的远程监督研究大多没有关注到长尾(long-tail)数据,因此远程监督得到的大多数句包中所含句子太少,不能真实全面地反映数据的情况。因此,提出基于位置-类型注意力机制和图卷积网络的远程监督关系抽取模型PG+PTATT。利用图卷积网络GCN聚合相似句包的隐含高阶特征,并对句包进行优化以此得到句包更丰富全面的特征信息;同时构建位置-类型注意力机制PTATT,以解决远程监督关系抽取中错误标签的问题。PTATT利用实体词与非实体词的位置关系以及类型关系进行建模,减少噪声词带来的影响。提出的模型在New York Times数据集上进行实验验证,实验结果表明提出的模型能够有效解决远程监督关系抽取中存在的问题;同时,能够有效提升关系抽取的正确率。展开更多
Transformers have been widely studied in many natural language processing (NLP) tasks, which can capture the dependency from the whole sentence with a high parallelizability thanks to the multi-head attention and the ...Transformers have been widely studied in many natural language processing (NLP) tasks, which can capture the dependency from the whole sentence with a high parallelizability thanks to the multi-head attention and the position-wise feed-forward network. However, the above two components of transformers are position-independent, which causes transformers to be weak in modeling sentence structures. Existing studies commonly utilized positional encoding or mask strategies for capturing the structural information of sentences. In this paper, we aim at strengthening the ability of transformers on modeling the linear structure of sentences from three aspects, containing the absolute position of tokens, the relative distance, and the direction between tokens. We propose a novel bidirectional Transformer with absolute-position aware relative position encoding (BiAR-Transformer) that combines the positional encoding and the mask strategy together. We model the relative distance between tokens along with the absolute position of tokens by a novel absolute-position aware relative position encoding. Meanwhile, we apply a bidirectional mask strategy for modeling the direction between tokens. Experimental results on the natural language inference, paraphrase identification, sentiment classification and machine translation tasks show that BiAR-Transformer achieves superior performance than other strong baselines.展开更多
文摘Event-related potential (ERP) studies demonstrated that emotional state immediately affects the processing of different linguistic mismatches (e.g., semantic and syntactic mismatches) in sentences. Furthermore, mood has been shown to impact discourse processing. In these studies, a strong expectancy was evoked by a linguistic context. In the present study, a strong expectancy was induced by a visual (non-linguistic) context paired with a sentence. A spatial array was followed by a sentence that matched (□○—The square stands in front of the circle) or mismatched the picture (intra-dimensional mismatch: □○—The square stands behind the circle;extra-dimensional mismatch: □○—The square stands above the circle). The main question was whether mood effects on the processing of visually induced expectancies are similar or different from mood effects on the processing of linguistically induced expectancies. To this aim, we presented conceptual (mis)matches that differed in saliency: intra-dimensional vs. extra-dimensional mismatches which are both reported to elicit an N2b/N400 and P600, reflecting reanalysis. EEG was recorded while participants read for comprehension. Mood (happy vs. sad) was effectively induced by film clips. Neither for N2b/N400 nor for P600, an interaction between mood and match was observed. Thus, in contrast with studies investigating the effects of pure linguistic mismatches (linguistic context paired with unexpected lexical item), mood did not modulate the processing of conceptual mismatches. However, a main effect of match revealed different ERP patterns for the two mismatches. While an increase in N2b/ N400 occurred for the intra-dimensional and extra-dimensional mismatches, a P600 only occurred to the extra-dimensional (more salient) mismatches. This finding is taken as support that being in an emotional state (positive or negative mood) influences processes of reanalysis. That is, when being in an emotional state, a reanalysis (monitoring) process is exclusively triggered by salient mismatches.
文摘篇章关系抽取旨在识别篇章中实体对之间的关系.相较于传统的句子级别关系抽取,篇章级别关系抽取任务更加贴近实际应用,但是它对实体对的跨句子推理和上下文信息感知等问题提出了新的挑战.本文提出融合实体和上下文信息(Fuse entity and context information,FECI)的篇章关系抽取方法,它包含两个模块,分别是实体信息抽取模块和上下文信息抽取模块.实体信息抽取模块从两个实体中自动地抽取出能够表示实体对关系的特征.上下文信息抽取模块根据实体对的提及位置信息,从篇章中抽取不同的上下文关系特征.本文在三个篇章级别的关系抽取数据集上进行实验,效果得到显著提升.
文摘远程监督关系抽取通过自动对齐自然语言文本与知识库生成带有标签的训练数据集,解决样本人工标注的问题。目前的远程监督研究大多没有关注到长尾(long-tail)数据,因此远程监督得到的大多数句包中所含句子太少,不能真实全面地反映数据的情况。因此,提出基于位置-类型注意力机制和图卷积网络的远程监督关系抽取模型PG+PTATT。利用图卷积网络GCN聚合相似句包的隐含高阶特征,并对句包进行优化以此得到句包更丰富全面的特征信息;同时构建位置-类型注意力机制PTATT,以解决远程监督关系抽取中错误标签的问题。PTATT利用实体词与非实体词的位置关系以及类型关系进行建模,减少噪声词带来的影响。提出的模型在New York Times数据集上进行实验验证,实验结果表明提出的模型能够有效解决远程监督关系抽取中存在的问题;同时,能够有效提升关系抽取的正确率。
基金supported by the Key Development Program of the Ministry of Science and Technology(2019YFF0303003)the National Natural Science Foundation of China(Grant No.61976068)“Hundreds,Millions”Engineering Science and Technology Major Special Project of Heilongjiang Province(2020ZX14A02).
文摘Transformers have been widely studied in many natural language processing (NLP) tasks, which can capture the dependency from the whole sentence with a high parallelizability thanks to the multi-head attention and the position-wise feed-forward network. However, the above two components of transformers are position-independent, which causes transformers to be weak in modeling sentence structures. Existing studies commonly utilized positional encoding or mask strategies for capturing the structural information of sentences. In this paper, we aim at strengthening the ability of transformers on modeling the linear structure of sentences from three aspects, containing the absolute position of tokens, the relative distance, and the direction between tokens. We propose a novel bidirectional Transformer with absolute-position aware relative position encoding (BiAR-Transformer) that combines the positional encoding and the mask strategy together. We model the relative distance between tokens along with the absolute position of tokens by a novel absolute-position aware relative position encoding. Meanwhile, we apply a bidirectional mask strategy for modeling the direction between tokens. Experimental results on the natural language inference, paraphrase identification, sentiment classification and machine translation tasks show that BiAR-Transformer achieves superior performance than other strong baselines.
