基于MatchNet和多点匹配约束的可见光-SAR图像匹配

Optical⁃SAR image matching based on MatchNet and multi⁃point matching constraint

基于可见光和SAR成像的异源图像匹配是实现复杂场景下飞行器视觉导航的关键。由于成像机理的差异,可见光-SAR图像匹配需要面对成像几何特性差异、非线性辐射失真以及噪声干扰等挑战。近年来,基于深度学习的图像匹配方法表现出比传统方法更优的场景适应性,但现有深度学习方法仍未完全解决上述几个挑战,还无法完全满足视觉导航中的高精度匹配需求。基于经典的MatchNet匹配框架,针对其单个模板匹配精度不足的问题,提出一种多点匹配约束的匹配方法(MMC-MatchNet)。方法主要分为2步:第1步生成子模板并利用MatchNet进行匹配,第2步基于多点几何约束剔除误匹配子模板并优化匹配结果。该方法在保持MatchNet输入尺寸不变的同时提升了网络对多尺寸可见光-SAR图像的匹配精度。此外,还提出了一种多级数据集构建方法来训练MatchNet。与随机采样构建负样本的方式相比,所提方法更关注于匹配任务中的困难样本,是一种适用于图像匹配的数据增强手段。对MMC-MatchNet在2种可见光-SAR图像匹配数据上进行了测试,取得了比NCC、NMI、HOPC以及单模板MatchNet方法更好的匹配结果,在与HOPC匹配精度相似的前提下,MMC-MatchNet可大幅提高匹配正确率(CMR)。同时,多点匹配约束和多级数据构建方法还适用于其他基于Patch的图像匹配模型。

Multimodal image matching between optical and SAR images is critical for visual navigation of the aircraft flying over complex areas.Due to different imaging mechanisms,accurate optical-SAR image matching faces several challenges,such as imaging geometry differences between optical and SAR,nonlinear radiation distortion of SAR,and noise interference.Although deep learning-based image matching methods have shown better adaptability than traditional methods,they still cannot fully solve the above-mentioned challenges and their matching accuracy may be not enough for visual navigation tasks.This paper proposes a new image matching method called Multi-point Matching Constraint MatchNet(MMC-MatchNet)based on the typical MatchNet framework,so as to improve the matching accuracy of the single template matching using MatchNet.The method consists of two main steps:sub-templates sampling and matching using MatchNet,and mismatched sub-templates removal based on multi-point geometric constraint.The original input image size of the MatchNet is maintained,while the matching accuracy for optical-SAR image matching is improved.This paper also proposes a multi-level training dataset generation method to train the MMCMatchNet.Compared to the random sampling method,our method focuses on the difficult samples in image matching,and can be seen as a special data augmentation method for image matching.MMC-MatchNet is tested on two multi-modal image matching datasets,outperforming NCC,NMI,HOPC and the single-template MatchNet.With similar matching accuracy to that of HOPC,MMC-MatchNet can improve the Correctly Matching Rate(CMR).The method based on multi-point matching constraint and multi-level dataset generation can be easily extend to other patchbased image matching models.