Image Emotion Classification Network Based on Multilayer Attentional Interaction,Adaptive Feature Aggregation

The image emotion classification task aims to use the model to automatically predict the emotional response of people when they see the image.Studies have shown that certain local regions are more likely to inspire an emotional response than the whole image.However,existing methods perform poorly in predicting the details of emotional regions and are prone to overfitting during training due to the small size of the dataset.Therefore,this study proposes an image emotion classification network based on multilayer attentional interaction and adaptive feature aggregation.To perform more accurate emotional region prediction,this study designs a multilayer attentional interaction module.The module calculates spatial attention maps for higher-layer semantic features and fusion features through amultilayer shuffle attention module.Through layer-by-layer up-sampling and gating operations,the higher-layer features guide the lower-layer features to learn,eventually achieving sentiment region prediction at the optimal scale.To complement the important information lost by layer-by-layer fusion,this study not only adds an intra-layer fusion to the multilayer attention interaction module but also designs an adaptive feature aggregation module.The module uses global average pooling to compress spatial information and connect channel information from all layers.Then,the module adaptively generates a set of aggregated weights through two fully connected layers to augment the original features of each layer.Eventually,the semantics and details of the different layers are aggregated through gating operations and residual connectivity to complement the lost information.To reduce overfitting on small datasets,the network is pre-trained on the FI dataset,and further weight fine-tuning is performed on the small dataset.The experimental results on the FI,Twitter I and Emotion ROI(Region of Interest)datasets show that the proposed network exceeds existing image emotion classification methods,with accuracies of 90.27%,84.66%and 84.96%.

Neural Mechanism in Emotion Regulation by Simultaneous Recording of EEG and fMRI

The combination of electroencephalogram (EEG) and functional magnetic resonance imaging(fMRI) is a very attractive aim in neuroscience in order to achieve both high temporal and spatial resolution for the non-invasive study of cognitive brain function. In this paper, we record simultaneous EEG-fMRI of the same subject in emotional processing experiment in order to explore the characteristics of different emotional picture processing, and try to find the difference of the subjects' brain hemisphere while viewing different valence emotional pictures. The late positive potential(LPP) is a reliable electrophysiological index of emotional perception in humans. According to the analysis results, the slow-wave LPP and visual cortical blood oxygen level-dependent (BOLD) signals are both modulated by the rated intensity of picture arousal. The amplitude of the LPP correlate significantly with BOLD intensity in visual cortex, amygdala, temporal area, prefrontal and central areas across picture contents.