Human-object interaction(HOIs)detection is a new branch of visual relationship detection,which plays an important role in the field of image understanding.Because of the complexity and diversity of image content,the d...Human-object interaction(HOIs)detection is a new branch of visual relationship detection,which plays an important role in the field of image understanding.Because of the complexity and diversity of image content,the detection of HOIs is still an onerous challenge.Unlike most of the current works for HOIs detection which only rely on the pairwise information of a human and an object,we propose a graph-based HOIs detection method that models context and global structure information.Firstly,to better utilize the relations between humans and objects,the detected humans and objects are regarded as nodes to construct a fully connected undirected graph,and the graph is pruned to obtain an HOI graph that only preserving the edges connecting human and object nodes.Then,in order to obtain more robust features of human and object nodes,two different attention-based feature extraction networks are proposed,which model global and local contexts respectively.Finally,the graph attention network is introduced to pass messages between different nodes in the HOI graph iteratively,and detect the potential HOIs.Experiments on V-COCO and HICO-DET datasets verify the effectiveness of the proposed method,and show that it is superior to many existing methods.展开更多
Mobile device research has been increasing rapidly in Human Computer Interaction （HCI） recently. Following this trend, this paper proposes a user-centered interface, which has been designed, completely installed and...Mobile device research has been increasing rapidly in Human Computer Interaction （HCI） recently. Following this trend, this paper proposes a user-centered interface, which has been designed, completely installed and independently run on a mobile phone. Video signal is steamed through its camera as the image input to the interface by employing techniques of image processing, computer vision and graphics to identify automatically absolute positions of human face, neck and two hands. A paradigm is also put up theoretically. And it embeds this interface to perceive the human postures and convert them into relaxed comic character according to its context.展开更多
基金Project(51678075)supported by the National Natural Science Foundation of ChinaProject(2017GK2271)supported by the Hunan Provincial Science and Technology Department,China。
文摘Human-object interaction(HOIs)detection is a new branch of visual relationship detection,which plays an important role in the field of image understanding.Because of the complexity and diversity of image content,the detection of HOIs is still an onerous challenge.Unlike most of the current works for HOIs detection which only rely on the pairwise information of a human and an object,we propose a graph-based HOIs detection method that models context and global structure information.Firstly,to better utilize the relations between humans and objects,the detected humans and objects are regarded as nodes to construct a fully connected undirected graph,and the graph is pruned to obtain an HOI graph that only preserving the edges connecting human and object nodes.Then,in order to obtain more robust features of human and object nodes,two different attention-based feature extraction networks are proposed,which model global and local contexts respectively.Finally,the graph attention network is introduced to pass messages between different nodes in the HOI graph iteratively,and detect the potential HOIs.Experiments on V-COCO and HICO-DET datasets verify the effectiveness of the proposed method,and show that it is superior to many existing methods.
基金supported by the MKE(The Ministry of Knowledge Economy),Korea,under the ITRC(Information Technology Research Center) support program supervised by the NIPA(National ITIndustry Promotion Agency)(NIPA-2009-(C1090-0902-0007))the Soongsil University BK21 Digital Media Division
文摘Mobile device research has been increasing rapidly in Human Computer Interaction （HCI） recently. Following this trend, this paper proposes a user-centered interface, which has been designed, completely installed and independently run on a mobile phone. Video signal is steamed through its camera as the image input to the interface by employing techniques of image processing, computer vision and graphics to identify automatically absolute positions of human face, neck and two hands. A paradigm is also put up theoretically. And it embeds this interface to perceive the human postures and convert them into relaxed comic character according to its context.
