基于时间分段网络并融合上下文信息的视频情感识别

On video emotion recognition based on temporal segment networks and fusion of context information

提出一种基于时间分段网络并融合上下文信息的视频情感识别模型.该模型由2个并行的时间分段网络组成,分别用于提取视频中的脸部信息和上下文信息的时空特征并计算情感类别分数.将2个网络的计算结果进行决策融合,得到整个视频的情感类别.在2个视频情感数据库CHEAVD和AFEW上训练并测试了该模型,同时与其他现有方法进行比较.所提模型在CHEAVD上获得了54.2%的ACC和45.6%的MAP,在AFEW上获得了53.8%的ACC,识别性能显著高于数据库的基线,并且优于其他现有方法.

A video emotion recognition model based on temporal segment networks and fused with context information is proposed.Themodel consists of two parallel temporal segment networks,which are used to extract the spatiotemporal features of face and context information respectively in the video and calculate the emotion class score.By merging the calculation results of the two networks,the emotion category of the video is finally obtained.The model was trained and tested on two challenging video emotion databases CHEAVD and AFEW,and is compared with other existing methods.The presented model obtained 54.2%ACC and 45.6%MAP on CHEAVD,and 53.8%ACC on AFEW.The recognition performance is significantly higher than the database baseline,and better than other existingmethods.