组合全景相机多圆柱投影全景拼接方法

Multicylinder Projection Stitching Method of Multicamera Panoramic System

针对360°全景视场的物距不一致性、多相机一个柱面的全景柱面投影方程导致的全景影像拼接误差,以及直接全景拼接方法容易导致“空白”像素的问题,本文提出了一种基于多圆柱投影优化的多相机组合全景柱面间接拼接方法。利用低成本网络摄像机设计一个八组合全景相机系统,根据多镜头组合型全景相机的严格成像原理,推导出一套完整的组合全景平面投影与子相机透视投影的间接映射方程;以相邻相机同名特征点的投影一致性为约束条件、以投影误差最小为目标函数,采用梯度下降法求解多个圆柱半径的最优解,将一个圆柱投影的方法拓展为多圆柱投影。实验结果表明,在360°全景视场不一致的情况下,多圆柱组合全景拼接的各重叠区域的最大投影误差不超过1 pixel,均方根误差(RMSE)优于0.27 pixel,拼接精度明显高于一个圆柱投影的传统组合全景拼接方法,达到了视觉上的几何一致性且无空白像素。

Objective Panoramic imaging technology has been extensively employed in traffic supervision,robot navigation,mobile measurement,etc.Several panoramic systems have been developed by combining numerous cameras to offer a horizontal field of view of 360°and a vertical field of view ranging from decades degrees to 180°,with the development of sensors and computing abilities.The panoramic images can be created from the combined panoramic systems using two approaches.The first solution may be transformation,registration,and blending only based on the feature points.The approach often has a strong dependence on the feature points in the surrounding environment.Alternatively,the panoramic stitching approach depends on the direct or indirect transformation between the cylindrical or sphere and perspective projection.The second approach is more robust and effective than the first approach because of its independence of the feature points.However,the panoramic imaging would experience the mosaic error and geometric inconsistency of vision due to the surrounding environment’s unknown depth.To solve the difficulties of panoramic mosaicking,empty pixels,and multiple pixel projection in one pixel caused by the traditional panoramic stitching approaches,a multicylinder projection-based mosaicking approach is proposed to ensure that panoramic images are produced effortlessly.Methods In this study,we proposed an indirect panoramic cylindrical stitching approach based on multicylinder projection optimization to reduce the visual stitching error of overlapping regions while improving the panoramic images’stitching accuracy.We hope that our study approach can be beneficial in computer vision and other related fields.First,we developed a panoramic camera system combined with eight low-cost network cameras.A set of indirect mapping equations between panoramic planar and perspective projections of subcameras were developed according to the rigorous imaging model for a multicamera panoramic system.Particularly,a cost function was developed that employs the minimizing projection error based on the projection constraint of homonymy points on the cylinder.Furthermore,the optimal solution of multiple cylinder radius was solved using the gradient descent approach.Thus,the approach of one cylinder projection is extended to numerous cylinder projections.Results and Discussions Table 1 shows the high-precision relative orientation elements between combined cameras.The panoramic indirect stitching imaging model was fabricated based on the precalibration findings.The experimental findings show that the multicylinder panoramic stitching approach offers a better visual effect than the traditional single-cylinder stitching approach(Fig.5).In this manner,the corresponding points’projection error in the panoramic images formed from the proposed approach was about 1/7th of the traditional single-cylinder stitching approach.The projection error of single-cylinder and multicylinder panoramic stitching approaches were calculated(Table 2).Furthermore,compared with the traditional direct approach,the difficulties of empty pixels and accumulation of pixels are solved(Fig.6).The stitching speed of a panoramic image generated from the proposed approach is 10 times that of the traditional SIFT feature-based approach and 55 times that of the mosaicing approach in the OpenCV package(Table 3).Conclusions The proposed multicylinder projection-based mosaicking approach is effective in solving the difficulty of panoramic mosaicking errors due to traditional single-cylinder panoramic imaging equation in a 360°panoramic environment with inconsistent object distances.Additionally,the indirect projection approach is employed to solve the difficulties of empty and multiple pixel projections in one pixel due to the direct panoramic stitching approach.The approach proposed in this study exhibits higher stitching accuracy and visual consistency and is suitable for the panoramic systems combined with multiple cameras,which are designed with eight cameras arranged cylindrically.Indeed,it can also be extended to the sphere panoramic camera system.