Affective brain-computer interfaces have become an increasingly important topic to achieve emotional intelligence in human–machine collaboration.However,due to the complexity of electroencephalogram(EEG)signals and t...Affective brain-computer interfaces have become an increasingly important topic to achieve emotional intelligence in human–machine collaboration.However,due to the complexity of electroencephalogram(EEG)signals and the individual differences in emotional response,it is still a great challenge to design a reliable and effective model.Considering the influence of personality traits on emotional response,it would be helpful to integrate personality information and EEG signals for emotion recognition.This study proposes a personalityguided attention neural network that can use personality information to learn effective EEG representations for emotion recognition.Specifically,we first use a convolutional neural network to extract rich temporal and regional representations of EEG signals,and a special convolution kernel is designed to learn inter-and intra-regional correlations simultaneously.Second,inspired by the fact that electrodes within distinct brain scalp regions play different roles in emotion recognition,a personality-guided regional-attention mechanism is proposed to further explore the contributions of electrodes within a region and between regions.Finally,attention-based long short-term memory is designed to explore the temporal dynamics of EEG signals.Experiments on the AMIGOS dataset,which is a dataset for multimodal research for affect,personality traits,and mood on individuals and groups,show that the proposed method can significantly improve the performance of subject-independent emotion recognition and outperform state-of-the-art methods.展开更多
基金Project supported by the National Key R&D Program of China(No.2019YFA0706200)the National Natural Science Foundation of China(Nos.62072219 and 61632014)the National Basic Research Program(973)of China(No.2014CB744600)。
文摘Affective brain-computer interfaces have become an increasingly important topic to achieve emotional intelligence in human–machine collaboration.However,due to the complexity of electroencephalogram(EEG)signals and the individual differences in emotional response,it is still a great challenge to design a reliable and effective model.Considering the influence of personality traits on emotional response,it would be helpful to integrate personality information and EEG signals for emotion recognition.This study proposes a personalityguided attention neural network that can use personality information to learn effective EEG representations for emotion recognition.Specifically,we first use a convolutional neural network to extract rich temporal and regional representations of EEG signals,and a special convolution kernel is designed to learn inter-and intra-regional correlations simultaneously.Second,inspired by the fact that electrodes within distinct brain scalp regions play different roles in emotion recognition,a personality-guided regional-attention mechanism is proposed to further explore the contributions of electrodes within a region and between regions.Finally,attention-based long short-term memory is designed to explore the temporal dynamics of EEG signals.Experiments on the AMIGOS dataset,which is a dataset for multimodal research for affect,personality traits,and mood on individuals and groups,show that the proposed method can significantly improve the performance of subject-independent emotion recognition and outperform state-of-the-art methods.
