基于LoFTR算法的图像拼接算法研究与实现

Image Mosaic Research and Realization Based on LoFTR Algorithm

针对传统算法图像匹配准方法提取特征点不精确、鲁棒性低、低纹理下很难识别到特征点等问题,提出了一种新的局部图像特征匹配方法,替代传统的顺序执行图像特征检测,描述和匹配的步骤。首先在原图像提取分辨率为1/8的粗略特征,然后平铺为一维向量,并为其添加位置编码,将组合结果输入Transformers模块中的自注意力层和交叉注意力层,最后输入可微分匹配层后得到置信矩阵,为该矩阵设置阈值和相互最近标准,从而得到粗略的匹配预测。其次是在精细层次上细化良好的匹配,在建立精匹配之后,通过变换矩阵到统一的坐标下,实现图像重叠区域对齐,最后通过加权平局融合算法对图像进行融合,实现对图像的无缝拼接。使用Transformers中的自注意力层和交叉注意力层来获取图像的特征描述符。实验结果表明,在特征点提取方面,LoFTR算法比传统的SIFT算法,无论在低纹理区域还是纹理比较丰富的区域提取的都更精确,同时使用此方法得到的拼接效果比传统经典算法拼接的效果更好。

Aiming at the problem of inaccurate extraction of feature points by the traditional image matching method,low robustness,and problems such as difficulty in identifying feature points in area with poor texture,a new local image feature matching method,which replaces the traditional sequential image feature detection,description and matching steps was proposed.Firstly,the coarse features with a resolution of 1/8 from the original image were extracted,then the extracted features were tiled to a one-dimensional vector,and the positional encoding was added to it.Then the combined results were input into the self-attention layer and cross-attention layer in the Transformer module.A differentiable matching layer was used to match the transformed features and a confidence matrix was get.The Coarse-Level matching prediction were selected according to the confidence threshold and mutual-nearest-neighbor criteria.Secondly the fine matching was refined at the Fine-level match,after the Fine-level match was established,the image overlapped area was aligned by transforming the matrix to a unified coordinate,and the image was fused by the weighted fusion algorithm to realize the unification of seamless mosaic of images.The feature descriptors of the image were obtained by using the self-attention layer and cross-attention layer in Transformers.Finally,the experimental results show that in terms of feature point extraction,LoFTR algorithm is more accurate than the traditional SIFT algorithm in both low-texture regions and regions with rich textures.At the same time,the image mosaic effect obtained by LoFTR is better than the traditional classic algorithms.