高低密度多维视角多元信息融合人群计数方法

High and low density multi-dimension perspective multivariate information fusion crowd counting method

针对人群密度在二维图像中随图像视角变化呈现较大差异、特征空间多尺度信息丢失等问题,提出一种多维视角多元信息融合(MDPMIF)的人群密度估计方法.首先,由“上-左-右-下”的方向对视角变化进行信息编码,通过递进聚合方式捕获深层次全局上下文信息,同步提取多维度视角的尺度关系特征;然后,设计联合学习策略获取全局尺度关系特征,并将全局上下文表达、全局尺度关系特征集成,得到更全面的视角变换描述;最后,采用语义嵌入方式实现高、低阶特征相互补充,增强输出密度图的质量.同时,真实场景下的人群聚集模式存在差异,单纯密度图方法易对图像中的低聚集部分造成人群计数高估,基于此,提出一种高低密度多维视角多元信息融合人群计数网络.设计高低密度区分策略对MDPMIF输出进行高低密度区域自适应划分,高密区域保持MDPMIF网络估计结果,低密区域采用检测方法实现人群计数修正,提高模型的鲁棒性.实验结果表明,所提出方法的性能优于对比方法.

A crowd density estimation method with multi-dimensional perspective multivariate information fusion(MDPMIF) is proposed for the problems that crowd density in two-dimensional images presents large differences with image viewpoint changes and multi-scale information loss in feature space.Firstly,the information of perspective change is encoded from ‘up-left-right-bottom ’ direction,and the deep global contextual information is captured by progressive aggregation,and the scale relationship features of multi-dimensional perspective are extracted simultaneously.After that,a joint learning strategy is designed to obtain global scale relationship features and integrate global contextual expressions and global scale relationship features to obtain a more comprehensive description of perspective transformation.Finally,semantic embeddin g is used to realize the high and low order features to complement each other and enhance the quality of the output density map.Meanwhile,there are differences in crowd aggregation patterns in real scenes,and the simple density map method is prone to overestimate crowd counts for the low aggregation part of the image.Based on this,a high and low density multi-dimensional perspective multivariate information fusion crowd counting network(HLMMNet) is proposed on the basis of the MDPMIF network.A high and low density differentiation strategy is designed to adaptively divide the MDPMIF output into high and low density regions,keeping the MDPMIF network estimation results in the high density regions and using detection methods to achieve crowd counting correction in the low density regions,improving the robustness of the model.The experimental results show that the performance of this method is superior to other comparative methods.