针对大视差图像拼接的显性子平面配准

Dominant sub-plane registration algorithm for large parallax image stitching

目的针对图像拼接中大视差图像难以配准的问题,提出一种显性子平面自动配准算法。方法假设大视差图像包含多个显性子平面且每个平面内所含特征点密集分布。对该假设进行了验证性实验。所提算法以特征点分布为依据,通过聚类算法实现子平面分割,进而对子平面进行局部配准。首先,使用层次聚类算法对已匹配的特征点聚类,通过一种本文设计的拼接误差确定分组数目,并以各组特征点的聚类中心为新的聚类中心对重叠区域再聚类,分割出目标图像的显性子平面。然后,求解每个显性子平面的投影参数,并采用就近原则分配非重叠区域的单应性矩阵。结果采用公共数据集对本文算法进行测试,并与Auto-Stitching、微软Image Composite Editor两种软件及全局投影拼接方法(Baseline)、尽可能投影算法(APAP)进行对比,采用均方根误差作为配准精度的客观评判标准。实验结果表明,该算法在拼接大视差图像时,能有效地配准局部区域,解决软件和传统方法由误配准引起的鬼影、错位等问题。其均方根误差比Baseline方法平均减小55%左右。与APAP算法相比,均方根误差平均相差10%左右,但可视化配准效果相同且无需调节复杂参数,可实现自动配准。结论提出的显性子平面自动配准算法,通过分割图像所含子平面进而实现局部配准。该方法具有较高的配准精度,在大视差图像配准方面,优于部分软件及算法,可应用于图像拼接中大视差图像的自动配准。

Objective Image stitching technology can be used to combine multiple narrow-angle images with overlapping regions into a one wide-angle image and has a wide range of applications. Image registration is the core method of image mo- saic technology. The traditional registration method requires shooting a picture to meet the ideal condition in which the pho- tographing device is located in a fixed point in a three-dimensional space. However, the ideal condition is often hard to achieve, which results in a certain parallax between images. Image registration with a large parallax is highly difficult, which affects the quality of image mosaic. This paper proposes a novel dominant sub-plane automatic registration algorithm to achieve a large parallax image registration for image stitching. Method We propose the hypothesis that larze parallax images contain multiple dominant sub-planes and that feature points contained in each plane are densely distributed. To prove the hypothesis, we register the images after clustering feature points. Experimental result shows that the hypothesis is appli- cable to large parallax images. The proposed algorithm based on the distribution of feature points performs the local registration on the subplane segmented by clustering. First, feature points of the input images are extracted by scale-invariant fea- ture transform, and the nearest neighbor principle is used to pre-match the feature points. The random sample consensus al- gorithm is used to remove the mismatched feature points, and the overlapping area of the image is delimited by feature points. A canvas that is the same as the overlap region is initialized, and all correctly matched feature points are mapped to the canvas. The hierarchical clustering algorithm is used to group the matched feature points. The grouping number is determined by the stitching error designed in this study. The overlapping regions are re-clustered by taking the clustering centers of each group feature point as new clustering centers to obtain the dominant sub-plane. The projection parameters of each sub-plane are solved through their feature points, and the projection matrix of the non-overlapping region is assigned by the nearest neighbor principle. The proposed algorithm needs to project the image multiple times for different regions. Thus, the use of traditional bilinear interpolation in the projection process has a significant boundary effect, and the nearest neighbor interpolation method can effectively overcome this problem. Visual effects are improved significantly. Result The algorithm is tested on public datasets and compared with two commercial software, auto-stitching and Microsoft Image Com- posite Editor （ICE）, and two common methods, namely, baseline and as-projective-as-possible algorithm （APAP） . Root mean square error （RMSE） was used as an objective criterion for registration accuracy. Experimental results show that, compared with the baseline algorithm, the average RMSE of the proposed algorithm is reduced by approximately 55% . Many misaligned areas are aligned. Compared with the APAP algorithm, the accuracy is close, and the visual effect of reg- istration is the same. The method in this paper can realize automatic registration without adjusting complex parameters. Compared with auto-stitching and Microsoft ICE, the method can effectively register the local area and eliminate the ghos- ting and dislocation caused by misalignment when stitching large parallax images. Conclusion The proposed dominant sub- plane automatic registration algorithm achieves local registration by segmenting sub-planes contained in the image. Using the sub-plane local registration method can effectively overcome the global registration elTor. The area prone to ghosting and other issues can often be separated as a separate plane registration area that improves the accuracy of the local registration. This method has high registration accuracy for large parallax images and is better than some commercial software and ad- vanced algorithms. The method can be applied to the registration of large parallax images in image mosaic. The proposed method is based on plane approximation and allocates the projection matrix of the non-overlapping region using the proximity principle. Thus, some accumulative distortion may be observed in the non-overlapping region of the images because of projec- tion transformation, which affects the mosaic of multiple images. This paper combines some excellent image transformation methods to address projection distortion in non-overlapping regions and apply the algorithm in a panoramic image mosaic.