时序动作单元感知的开集动作识别

Temporal Action Unit Perception Based Open Set Action Recognition

开集动作识别任务要求模型不仅能准确识别训练集中的类别,还能拒绝训练集上未出现的未知类动作.目前,大多数方法都将动作视为一个整体,忽略动作本身可被分解为更细粒度的动作单元.为此,文中提出时序动作单元感知的开集动作识别方法.首先,设计动作单元关系模块,学习细粒度的动作单元特征,得到动作和动作单元的关系模式,并通过已知类动作和未知类动作在动作单元上不同的激活程度识别未知类动作.然后,设计动作单元时序模块,建模动作单元的时序信息,研究动作单元的时序性,进一步区分因为外观相似而被混淆的已知类动作和未知类动作.最后,综合考虑关系模式与动作单元时序信息,使模型具备区分已知类动作和未知类动作的能力.在3个动作识别数据集上的实验表明,文中方法性能较优.

In open set action recognition tasks,a model is requested to identify categories within the training set accurately and reject unknown actions that never appear in the training set.Currently,most of the methods treat the action as a whole,ignoring the fact that the action can be decomposed into finer-grained action units.To address this issue,a method for temporal action unit perception based open set action recognition is proposed in this paper.Firstly,an action unit relationship module is designed to learn fine-grained features of action units,and thus the relational pattern between actions and action units is obtained.The unknown actions are identified according to the different degrees of activation of known and unknown actions on action units.Secondly,an action unit temporal module is designed to model the temporal information of action units.The temporal characteristics of action units are explored to further distinguish between known actions and unknown actions that are visually similar but confusable with each other.Finally,with comprehensive consideration of both relational patterns and temporal information of action units,the model is equipped with the capability of distinguishing known actions from unknown actions.Experimental results on three action recognition datasets demonstrate the superior performance of the proposed method.