基于混合智能的街景影像知识提取方法

Research on knowledge extraction from street scene images based on hybrid intelligence

针对街景影像目标的智能化提取难题,本文提出了一种基于混合智能的街景影像知识提取方法(K-CAPSNet)。首先,在现有全景分割网络的基础上,同时关注街景影像的通道信息和空间信息,发展了一种联合注意力机制的全景分割网络,以提高目标分割精度;其次,将人们在生产、生活中形成的街景知识融入街景影像认知过程,借助先验知识设置目标标记阈值,对分割结果进行优化;然后,进一步根据街景影像先验知识验证街景目标之间的拓扑关系并利用深度信息进行空间关系知识挖掘;最后,采用语义模板对街景目标类型、数量及空间关系进行描述和表达。试验表明,相较于基线网络,本文方法在全景分割质量和识别质量方面都有明显提升,较好地实现了对街景影像知识的提取与表达。

This study presents a hybrid intelligence-based approach,named K-CAPSNet,for extracting knowledge from streetscape images.To tackle the challenge of intelligent extraction of streetscape image objects,we develop a panoramic segmentation network with a joint attention mechanism that integrates both channel information and spatial information of streetscape images.This improves the object segmentation accuracy.Additionally,we incorporate streetscape knowledge,which is formed by people in production and life,into the streetscape image cognition process.We set the object marking threshold using a priori knowledge to optimize the segmentation results.Moreover,we utilize the a priori knowledge of streetscape images to verify the topological relationship between streetscape objects and to mine spatial relationship knowledge using depth information.Finally,we employ semantic templates to describe and express the type,number,and spatial relationship between streetscape objects.The experimental results demonstrate that our method outperforms the baseline network and significantly improves the quality of panoramic segmentation and recognition,thereby achieving better extraction and expression of the knowledge of streetscape images.