Human-Object Interaction Recognition Based on Modeling Context

This paper proposes a method to recognize human-object interactions by modeling context between human actions and interacted objects.Human-object interaction recognition is a challenging task due to severe occlusion between human and objects during the interacting process.Since that human actions and interacted objects provide strong context information,i.e.some actions are usually related to some specific objects,the accuracy of recognition is significantly improved for both of them.Through the proposed method,both global and local temporal features from skeleton sequences are extracted to model human actions.In the meantime,kernel features are utilized to describe interacted objects.Finally,all possible solutions from actions and objects are optimized by modeling the context between them.The results of experiments demonstrate the effectiveness of our method.

Graph-based method for human-object interactions detection

Human-object interaction(HOIs)detection is a new branch of visual relationship detection,which plays an important role in the field of image understanding.Because of the complexity and diversity of image content,the detection of HOIs is still an onerous challenge.Unlike most of the current works for HOIs detection which only rely on the pairwise information of a human and an object,we propose a graph-based HOIs detection method that models context and global structure information.Firstly,to better utilize the relations between humans and objects,the detected humans and objects are regarded as nodes to construct a fully connected undirected graph,and the graph is pruned to obtain an HOI graph that only preserving the edges connecting human and object nodes.Then,in order to obtain more robust features of human and object nodes,two different attention-based feature extraction networks are proposed,which model global and local contexts respectively.Finally,the graph attention network is introduced to pass messages between different nodes in the HOI graph iteratively,and detect the potential HOIs.Experiments on V-COCO and HICO-DET datasets verify the effectiveness of the proposed method,and show that it is superior to many existing methods.

An Intelligent Framework for Recognizing Social Human-Object Interactions

Human object interaction(HOI)recognition plays an important role in the designing of surveillance and monitoring systems for healthcare,sports,education,and public areas.It involves localizing the human and object targets and then identifying the interactions between them.However,it is a challenging task that highly depends on the extraction of robust and distinctive features from the targets and the use of fast and efficient classifiers.Hence,the proposed system offers an automated body-parts-based solution for HOI recognition.This system uses RGB(red,green,blue)images as input and segments the desired parts of the images through a segmentation technique based on the watershed algorithm.Furthermore,a convex hullbased approach for extracting key body parts has also been introduced.After identifying the key body parts,two types of features are extracted.Moreover,the entire feature vector is reduced using a dimensionality reduction technique called t-SNE(t-distributed stochastic neighbor embedding).Finally,a multinomial logistic regression classifier is utilized for identifying class labels.A large publicly available dataset,MPII(Max Planck Institute Informatics)Human Pose,has been used for system evaluation.The results prove the validity of the proposed system as it achieved 87.5%class recognition accuracy.

基于自适应三线性池化网络的细粒度图像分类

细粒度图像分类的关键在于提取图像中微妙的特征。现有基于弱监督方式的细粒度图像识别方法大多使用专家标注的边界注释辅助定位关键区域,存在标注成本高、训练过程复杂等问题。基于弱监督的双线性卷积神经网络方法因其学习到的特征空间更符合细粒度图像特性而具有一定的有效性,但忽略了层间的相互作用。针对细粒度图像识别领域存在的关键区域识别困难和层间交互关联弱的问题,融合二阶协方差通道注意力机制、自适应特征掩码与自适应三线性池化,提出自适应三线性池化网络ATP-Net,用于细粒度图像分类任务。通过二阶协方差通道注意力机制学习通道上的注意力向量,构建自适应特征掩码模块学习空间维上的注意力矩阵,设计自适应三线性池化模块学习特征的最终表示,以充分利用空间维、通道维上的信息。在CUB-200、Cars-196和Aircraft-1003个细粒度图像分类数据集上的实验结果表明,ATP-Net的分类精度分别为89.30%、94.20%和91.80%。

人物交互检测研究进展综述

作为目标检测、行为识别、视觉关系检测的交叉学科,人物交互(human-object interaction,HOI)检测旨在识别特定应用场景下人与物体的相互关系.本文对基于图像的人物交互检测研究成果进行了系统总结及论述.首先,从交互关系建模的原理出发,把人物交互检测方法分为基于全局实例和基于局部实例两类,并对代表性方法进行了详细阐述和分析;进而,根据所采用视觉特征的差异将基于全局实例的方法进行细分,包括融合空间位置信息、融合外观信息与融合人体姿态信息;然后,讨论了零样本学习、弱监督学习以及Transformer模型在人物交互检测中的应用;最后,从交互类别、视觉干扰以及运动视角三方面出发,总结了人物交互检测面临的挑战,并指出领域泛化、实时检测和端到端网络是未来发展的趋势.

深度学习的人-物体交互检测研究进展

人-物体交互检测(HOI),就是把图像作为输入,检测出图像中存在交互行为的人和物体以及他们之间的交互动词。它是计算机视觉范畴里继目标检测、图像分割和目标跟踪之后又一新任务,旨在对图像进行更深层的理解。针对目前基于深度学习的HOI检测综述性文章的空白,以HOI检测方法的发展历程为主线,对基于深度学习的HOI检测方法进行了分类与分析。首先简要总结了早期的技术方法,然后根据模型结构将现有算法分为两阶段方法和一阶段方法并对一些代表性算法进行分析介绍。将两阶段方法分为融入注意力、图模型以及姿势和身体部位三类进行重点论述,总结了每类方法的基本思想与优缺点。此外,还详细介绍了HOI检测任务的实验评价指标、基准数据集和大多数现有方法的实验结果,对不同类别的方法取得的结果进行说明。最后对该技术面临的主要挑战进行总结分析并对未来发展趋势进行展望。

任务感知双原型网络的人物交互少样本识别

人物交互(HOI)识别是计算机视觉领域的重要研究热点。随着深度学习在图像分类任务中的巨大成功,人物交互识别任务也取得重大进展,但样本不平衡和组合爆炸问题仍是制约当前人物交互识别任务性能的关键挑战。由此,将人物交互识别任务与少样本学习相结合,将人物交互识别任务定义为一个少样本任务,并提出了任务感知双原型网络(TDP-Net)来解决少样本人物交互任务。具体地,首先使用图方法为每个任务生成语义感知的任务表示作为任务的先验信息,并使用语义图注意力模块(SGA-Module)生成注意力权重,对特征图中不同区域进行不同重要程度的关注,以适应不同任务条件下的映射关系,实现在新任务中自动推理。此外,还设计了一个双路原型模块(DP-Module)以分别产生交互类别的动作类原型和物体类原型,并分别对动词和名词进行分类。通过分别为动作和物体建立类原型,有效地分离了动作和物体间复杂的视觉关系。同时由于人物交互类别之间具有相似性,可通过重新组合动作和物体类别将知识迁移到新的交互类别中。实验结果表明,该模型在人物交互少样本任务上的平均准确率比基线方法在两个实验设置上分别提高了3.2个百分点和15.7个百分点,验证了TDP-Net在少样本人物交互任务中的有效性。

基于深度学习的人—物交互关系检测综述

人—物交互关系检测旨在通过精细化定位图像或视频中产生特定动作行为的人,以及与其产生交互关系的物体,并识别人和物体之间的动作关系来理解和分析人体的行为。人—物交互关系检测是一个非常具有实际应用意义和前瞻性的研究方向,是高层视觉理解的关键基石。随着深度学习的发展,基于深度学习的研究方法引领了近期人—物交互关系检测研究的进步。本文一方面分析空域人—物交互关系检测任务,从数据内容场景、标注粒度两个方面总结和分析当下数据库和基准。然后从两阶段分段式方法和单阶段端到端式方法两个流派出发系统性地阐述当前检测方法的发展现状,分析两个流派方法的特性和优劣,厘清该领域方法的发展路线。其中,两阶段方法包括多流模型和图模型两种主要范式,而单阶段模型包括基于框的范式、基于关系点的范式和基于查询的范式。另一方面,对时空域人—物交互关系检测任务进行总结,分析现有时空域交互关系数据集构造与特性和现有基线算法的优劣。最后对未来的研究方向进行展望。