Background Image matching is crucial in numerous computer vision tasks such as 3D reconstruction and simultaneous visual localization and mapping.The accuracy of the matching significantly impacted subsequent studies....Background Image matching is crucial in numerous computer vision tasks such as 3D reconstruction and simultaneous visual localization and mapping.The accuracy of the matching significantly impacted subsequent studies.Because of their local similarity,when image pairs contain comparable patterns but feature pairs are positioned differently,incorrect recognition can occur as global motion consistency is disregarded.Methods This study proposes an image-matching filtering algorithm based on global motion consistency.It can be used as a subsequent matching filter for the initial matching results generated by other matching algorithms based on the principle of motion smoothness.A particular matching algorithm can first be used to perform the initial matching;then,the rotation and movement information of the global feature vectors are combined to effectively identify outlier matches.The principle is that if the matching result is accurate,the feature vectors formed by any matched point should have similar rotation angles and moving distances.Thus,global motion direction and global motion distance consistencies were used to reject outliers caused by similar patterns in different locations.Results Four datasets were used to test the effectiveness of the proposed method.Three datasets with similar patterns in different locations were used to test the results for similar images that could easily be incorrectly matched by other algorithms,and one commonly used dataset was used to test the results for the general image-matching problem.The experimental results suggest that the proposed method is more accurate than other state-of-the-art algorithms in identifying mismatches in the initial matching set.Conclusions The proposed outlier rejection matching method can significantly improve the matching accuracy for similar images with locally similar feature pairs in different locations and can provide more accurate matching results for subsequent computer vision tasks.展开更多
When training a stereo matching network with a single training dataset, the network may overly rely on the learned features of the single training dataset due to differences in the training dataset scenes, resulting i...When training a stereo matching network with a single training dataset, the network may overly rely on the learned features of the single training dataset due to differences in the training dataset scenes, resulting in poor performance on all datasets. Therefore, feature consistency between matched pixels is a key factor in solving the network’s generalization ability. To address this issue, this paper proposed a more widely applicable stereo matching network that introduced whitening loss into the feature extraction module of stereo matching, and significantly improved the applicability of the network model by constraining the variation between salient feature pixels. In addition, this paper used a GRU iterative update module in the disparity update calculation stage, which expanded the model’s receptive field at multiple resolutions, allowing for precise disparity estimation not only in rich texture areas but also in low texture areas. The model was trained only on the Scene Flow large-scale dataset, and the disparity estimation was conducted on mainstream datasets such as Middlebury, KITTI 2015, and ETH3D. Compared with earlier stereo matching algorithms, this method not only achieves more accurate disparity estimation but also has wider applicability and stronger robustness.展开更多
近年来,无人机因体积小、灵活性好等优势被广泛应用在车辆跟踪领域。当无人机在高空飞行时,其捕捉的图像中车辆目标存在像素点少、拥挤以及被遮挡的情况。现有的多目标跟踪研究方法在车辆被遮挡过程中发生非线性运动时,使用卡尔曼滤波预...近年来,无人机因体积小、灵活性好等优势被广泛应用在车辆跟踪领域。当无人机在高空飞行时,其捕捉的图像中车辆目标存在像素点少、拥挤以及被遮挡的情况。现有的多目标跟踪研究方法在车辆被遮挡过程中发生非线性运动时,使用卡尔曼滤波预测,会出现车辆位置预测不准确的问题。为了解决这些问题,采用先检测后跟踪(tracking by detection,TBD)范式,对YOLOv8检测算法进行改进,在网络结构中引入了BiFormer稀疏动态注意力模块,用于提取小目标特征信息。同时使用轻量级上采样算子CARAFE替换原最近邻插值上采样,减少上采样过程中小目标特征丢失的问题。提出一种轻量化跟踪模型FA-SORT,针对SORT算法提出三点改进:改进KF、添加速度方向一致性匹配和检测值匹配。在自制地组合了多个车辆数据集上验证改进的YOLOv8算法。实验结果表明,与YOLOv8相比,精确率(precision)提高了0.97%,召回率(recall)提高了0.898%。对所提出的FA-SORT算法使用UAVDT数据集进行验证,结果表明,与现有的多目标跟踪算法相比,HOTA指标首个达到70.05%,IDF1达到87.45%,跟踪速度达到29.93 FPS。验证了FA-SORT跟踪算法在多车辆跟踪任务中的优越性。展开更多
基金Supported by the Natural Science Foundation of China(62072388,62276146)the Industry Guidance Project Foundation of Science technology Bureau of Fujian province(2020H0047)+2 种基金the Natural Science Foundation of Science Technology Bureau of Fujian province(2019J01601)the Creation Fund project of Science Technology Bureau of Fujian province(JAT190596)Putian University Research Project(2022034)。
文摘Background Image matching is crucial in numerous computer vision tasks such as 3D reconstruction and simultaneous visual localization and mapping.The accuracy of the matching significantly impacted subsequent studies.Because of their local similarity,when image pairs contain comparable patterns but feature pairs are positioned differently,incorrect recognition can occur as global motion consistency is disregarded.Methods This study proposes an image-matching filtering algorithm based on global motion consistency.It can be used as a subsequent matching filter for the initial matching results generated by other matching algorithms based on the principle of motion smoothness.A particular matching algorithm can first be used to perform the initial matching;then,the rotation and movement information of the global feature vectors are combined to effectively identify outlier matches.The principle is that if the matching result is accurate,the feature vectors formed by any matched point should have similar rotation angles and moving distances.Thus,global motion direction and global motion distance consistencies were used to reject outliers caused by similar patterns in different locations.Results Four datasets were used to test the effectiveness of the proposed method.Three datasets with similar patterns in different locations were used to test the results for similar images that could easily be incorrectly matched by other algorithms,and one commonly used dataset was used to test the results for the general image-matching problem.The experimental results suggest that the proposed method is more accurate than other state-of-the-art algorithms in identifying mismatches in the initial matching set.Conclusions The proposed outlier rejection matching method can significantly improve the matching accuracy for similar images with locally similar feature pairs in different locations and can provide more accurate matching results for subsequent computer vision tasks.
文摘When training a stereo matching network with a single training dataset, the network may overly rely on the learned features of the single training dataset due to differences in the training dataset scenes, resulting in poor performance on all datasets. Therefore, feature consistency between matched pixels is a key factor in solving the network’s generalization ability. To address this issue, this paper proposed a more widely applicable stereo matching network that introduced whitening loss into the feature extraction module of stereo matching, and significantly improved the applicability of the network model by constraining the variation between salient feature pixels. In addition, this paper used a GRU iterative update module in the disparity update calculation stage, which expanded the model’s receptive field at multiple resolutions, allowing for precise disparity estimation not only in rich texture areas but also in low texture areas. The model was trained only on the Scene Flow large-scale dataset, and the disparity estimation was conducted on mainstream datasets such as Middlebury, KITTI 2015, and ETH3D. Compared with earlier stereo matching algorithms, this method not only achieves more accurate disparity estimation but also has wider applicability and stronger robustness.
文摘近年来,无人机因体积小、灵活性好等优势被广泛应用在车辆跟踪领域。当无人机在高空飞行时,其捕捉的图像中车辆目标存在像素点少、拥挤以及被遮挡的情况。现有的多目标跟踪研究方法在车辆被遮挡过程中发生非线性运动时,使用卡尔曼滤波预测,会出现车辆位置预测不准确的问题。为了解决这些问题,采用先检测后跟踪(tracking by detection,TBD)范式,对YOLOv8检测算法进行改进,在网络结构中引入了BiFormer稀疏动态注意力模块,用于提取小目标特征信息。同时使用轻量级上采样算子CARAFE替换原最近邻插值上采样,减少上采样过程中小目标特征丢失的问题。提出一种轻量化跟踪模型FA-SORT,针对SORT算法提出三点改进:改进KF、添加速度方向一致性匹配和检测值匹配。在自制地组合了多个车辆数据集上验证改进的YOLOv8算法。实验结果表明,与YOLOv8相比,精确率(precision)提高了0.97%,召回率(recall)提高了0.898%。对所提出的FA-SORT算法使用UAVDT数据集进行验证,结果表明,与现有的多目标跟踪算法相比,HOTA指标首个达到70.05%,IDF1达到87.45%,跟踪速度达到29.93 FPS。验证了FA-SORT跟踪算法在多车辆跟踪任务中的优越性。
文摘异源图像配准中,由于图像的成像机理差异,图像像素强度关联和旋转畸变是不可避免的两大问题,针对图像像素强度关联问题,提出了基于辐射不变特征变换(radiation-variation insensitive feature transform,RIFT)的图像配准算法,对图像间像素关联差异小的图像对配准有良好的精度,但对旋转畸变图像会产生较多错误匹配。对于旋转畸变问题,传统的ORB(oriented fast and rotated brief)算法,对旋转图像的配准有一定的稳定性,但对于强度变化不明显的图像对,特征点检测质量较低,配准精度不理想。因此本文将相位一致性(phase consistency,PC)融合进ORB算法,利用相位信息代替传统的图像强度信息,再构造旋转不变性BRIEF特征描述子,对图像像素强度变化和旋转畸变均具有鲁棒性。用图像像素强度关联不明显的红外图像和可见光图像进行配准实验,本文算法针对不同旋转幅度的图像的配准精度较高,RMSE稳定在1.7~2.1,优于RIFT算法,在特征点检测数量、配准精度和效率等性能上均有良好性能。
