基于互信息熵和NetVLAD的视频关键帧提取方法

Video Keyframe Extraction Based on Mutual Information Entropy and NetVLAD

针对现有关键帧提取方法时间复杂度高、漏检率大、忽略视频语义信息等问题,提出一种基于互信息熵和局部聚合描述符向量网络(vector of local aggregated descriptors net,NetVLAD)的视频关键帧提取方法。首先计算视频帧互信息熵,将视频划分为视频子集;然后通过NetVLAD进行视频帧的特征提取与聚类,根据最近邻匹配算法计算帧间距离,提取候选关键帧;最后通过感知哈希减少冗余度,得到关键帧集合。基于UAV-123数据集进行了实验分析,结果表明,该方法高鲁棒地提高了关键帧的提取效率,保证了高保真度的同时降低了关键帧的冗余。

To solve the problems of existing key frame extraction methods, such as high time complexity, high miss rate and video semantic information neglect, we propose a video keyframe extraction method based on mutual information entropy and vector of local aggregated descriptors net(NetVLAD). First, we calculate the mutual information entropy of video frames and divide the video into video subsets. Then, feature extraction and clustering of video frames are carried out by NetVLAD. The similarity between frames is calculated by the nearest neighbor matching algorithm, and candidate keyframes are extracted. Finally, the redundancy is reduced by perceptual hashing, and the keyframe set is obtained. Experimental analysis based on UAV-123 data set proves that the proposed method improves the extraction efficiency of keyframes with high robustness and reduces the redundancy of key frames with high fidelity.