一种空间关系增强的场景图生成方法

A Scene Graph Generation Method With Enhanced Spatial Relationship

为了充分利用目标间的空间信息,更准确描述场景目标之间的关系,提出一种空间关系增强的场景图生成方法。该方法主要贡献包括目标间的关系统计和空间关系增强两个方面。其一,通过数据库建立目标对的关系数值矩阵,利用关系数值矩阵简化目标对的数量,以便目标间的关系检测;其二,由目标对的坐标信息计算两者之间的相对大小、相对位置和交并比,从而增强目标间的空间关系。实验结果表明,在Visual Genome数据集上,提出方法比Neural Motifs模型在场景图生成、场景图分类和谓词分类任务上均有提升。

To make full use of the spatial information between the targets and more accurately describe the relationship,a scene graph generation method with enhanced spatial relationship is proposed.The main contributions of this method include statistics of the relationship between the targets and the enhancement of the spatial relationship.The relationship statistics module between targets establishes the relationship value matrix of the target pair through the database,and the number of target pairs is simplified by the relationship value matrix,so as to facilitate the relationship detection.The spatial relationship enhancement module uses the coordinate information of the target pair to calculate the relative size,relative position and intersection ratio between the targets,and this spatial information are enhanced the relationship between the targets.The experimental results show that the proposed method is better than the Neural Motifs model in the tasks of scene graph generation,scene graph classification and predicate classification on the visual genome dataset.