Efficient Unsupervised Image Stitching Using Attention Mechanism with Deep Homography Estimation

Traditional feature-based image stitching techniques often encounter obstacles when dealing with images lackingunique attributes or suffering from quality degradation. The scarcity of annotated datasets in real-life scenesseverely undermines the reliability of supervised learning methods in image stitching. Furthermore, existing deeplearning architectures designed for image stitching are often too bulky to be deployed on mobile and peripheralcomputing devices. To address these challenges, this study proposes a novel unsupervised image stitching methodbased on the YOLOv8 (You Only Look Once version 8) framework that introduces deep homography networksand attentionmechanisms. Themethodology is partitioned into three distinct stages. The initial stage combines theattention mechanism with a pooling pyramid model to enhance the detection and recognition of compact objectsin images, the task of the deep homography networks module is to estimate the global homography of the inputimages consideringmultiple viewpoints. The second stage involves preliminary stitching of the masks generated inthe initial stage and further enhancement through weighted computation to eliminate common stitching artifacts.The final stage is characterized by adaptive reconstruction and careful refinement of the initial stitching results.Comprehensive experiments acrossmultiple datasets are executed tometiculously assess the proposed model. Ourmethod’s Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity Index Measure (SSIM) improved by 10.6%and 6%. These experimental results confirm the efficacy and utility of the presented model in this paper.

Greedy calibration method for binocular camera system via characteristic homography matrix

A plane-based and linear camera calibration technique without considering lens distortion is proposed in a greedy and intuitive framework for the binocular camera system. Characteristic homography matrix and consistency constraints in close range are employed in this calibration. First, in order to calculate the internal geometries of the cameras, total least-square fitting as a robust tool for the geometrical cost function is exploited to recover the accurate principal point of each camera from all the characteristic lines of the homography matrices for all model planes. Secondly, generic prior knowledge of the aspect ratio of pixel cells is incorporated into the system to obtain the exact principal length in each camera. Thirdly, extrinsic geometries are accurately computed for all planar patterns with respect to each monocular camera. Finally, the rigid displacement between binocular cameras can be obtained by imposing the consistency constraints in 3-space geometry. Both simulation and real image experimental results indicate that reasonably reliable results can be obtained by this technique. And the proposed method is sufficient for applications where high precision is not required and can be easily performed by common computer users who are not experts in computer vision.

CAMSHIFT结合HOMOGRAPHY变换的算法

提出了一种基于CAMSHIFT算法结合HOMOGRAPHY变换的球员轨迹提取算法。在求取球员位置的过程中,CAMSHIFT算法通过球员的颜色直方图模型实现对球员的跟踪。在跟踪过程中引入KALMAN滤波进一步提高算法的跟踪效率。最后利用HOMOGRAPHY变换将场景视频帧转换为俯视投影图,真实反映球员在球场的位置信息。实验结果表明该算法能够较好的提取出球员的运动轨迹。

Local Homography Estimation on User-Specified Textureless Regions

This paper presents a novel deep neural network for designated point tracking(DPT)in a monocular RGB video,VideoInNet.More concretely,the aim is to track four designated points correlated by a local homography on a textureless planar region in the scene.DPT can be applied to augmented reality and video editing,especially in the field of video advertising.Existing methods predict the location of four designated points without appropriately considering the point correlation.To solve this problem,VideoInNet predicts the motion of the four designated points correlated by a local homography within the heatmap prediction framework.Our network refines the heatmaps of designated points through two stages.On the first stage,we introduce a context-aware and location-aware structure to learn a local homography for the designated plane in a supervised way.On the second stage,we introduce an iterative heatmap refinement module to improve the tracking accuracy.We propose a dataset focusing on textureless planar regions,named ScanDPT,for training and evaluation.We show that the error rate of VideoInNet is about 29%lower than that of the state-of-the-art approach when testing in the first 120 frames of testing videos on ScanDPT.

应用计算机视觉技术进行物体三维重构

基于计算机视觉技术，提出了一种三维物体重构方法；利用激光三角测量原理，建立了几何光学模型；采用新方法列摄像头进行了精确标定，获得了Homography矩阵，实现了三维小物体的重构。针对三维转动的不平行度和激光散斑的影响，提出了激光刀口斜率拟合校正法。对4～30mm的物体三维重构表明，测量精度达到0．05mm。

A method of automatic plane detection without random search

Plane detection is a prerequisite for many computer vision tasks. This paper proposes a new method which can automatically detect planes from two projective images. Firstly, we modify Scott’s feature point matching method by post-processing its result with the concept of similarity, and then get the lines matching according to feature points matching based on the approximate invariance of the features’ distribution between two images. Finally, we group all feature points into subsets in terms of their geometric relations with feature lines as initial sets to estimate homography rather than by a random search strategy (like RANSAC) as in most existing methods. The proposed method is especially suitable to detecting planes in man-made scenes. This method is validated on real images.

摄像机自由运动环境下的背景建模

提出了在摄像机运动情况下使用多层Homography匹配算法进行背景建模的方法。该方法中,场景可以被看作由多个平面所组成,使用RANSAC方法找到场景中不同的平面,即多层Homography。每个像素点肯定在某个平面上,通过所属平面相应的Homography变换,就能使相邻两帧重叠视野中的像素点进行匹配,这样就能对场景进行背景建模。实验结果表明,该方法能有效地在摄像机运动环境中进行像素点级别的背景建模。

无人机影像自动快速拼接研究及实现

针对目前无人机图像处理自动化程度不高,限制条件较多,大数据量处理耗时较长的缺陷,本文提出一种基于单应性矩阵(Homography Matrix)的整体优化无人机图像拼接算法。本文采用GPU(图形处理器)+CPU协同工作模式,在CUDA架构下来实现SIFT算法,并以此算法对无人机影像提取特征点,采用极线约束和RANSAC算法剔除错点及外点,并估算出各张影像的变换矩阵Homography各参数的初始值,再采用LM算法精确求解出每张影像对应于基准面的全局Homography,实现全局配准,进而实现无人机影像的快速拼接。

Feature point matching of curved surface and robust uncertainty

Stereo matching is an important research area in stereovision and stereo matching of curved surface is especially crucial A novel correspondence algorithm is presented and its matching uncertainty is computed robustly for feature points of curved surface. The comers are matched by using homography constraint besides epipolar constraint to solve the occlusion problem. The uncertainty sources are analyzed. A cost function is established and acts as an optimal rule to compute the matching uncertainty. An adaptive scheme Gauss weights are put forward to make the matching results robust to noises. It makes the practical application of comer matching possible. From the experimental results of an image pair of curved surface it is shown that computing uncertainty robustly can restrain the affection caused by noises to the matching precision.

Feature matching based on geometric constraints in weakly calibrated stereo views of curved scenes

The identification of the correspondences of points of views is an important task. A new feature matching algorithm for weakly calibrated stereo images of curved scenes is proposed, based on mere geometric constraints. After initial correspondences are built via the epipolar constraint, many point-to-point image mappings called homographies are set up to predict the matching position for feature points. To refine the predictions and reject false correspondences, four schemes are proposed. Extensive experiments on simulated data as well as on real images of scenes of variant depths show that the proposed method is effective and robust.

Architectural Building Detection and Tracking in Video Sequences Taken by Unmanned Aircraft System （UAS）

An Unmanned Aircraft System （UAS） is an aircraft or ground station that can be either remote controlled manually or is capable of flying autonomously under the guidance of pre-programmed Global Positioning System （GPS） waypoint flight plans or more complex onboard intelligent systems. The UAS aircrafts have recently found extensive applications in military reconnaissance and surveillance, homeland security, precision agriculture, fire monitoring and analysis, and other different kinds of aids needed in disasters. Through surveillance videos captured by a UAS digital imaging payload over the interest areas, the corresponding UAS missions can be conducted. In this paper, the authors present an effective method to detect and extract architectural buildings under rural environment from UAS video sequences. The SIFT points are chosen as image features. The planar homography is adopted as the motion model between different image frames. The proposed algorithm is tested on real UAS video data.

结合RefineNet的无监督图像拼接网络

为减少无监督图像拼接结果中存在的伪影和错位问题,提出了一种结合RefineNet的无监督图像拼接网络。该网络首先通过一个深度单应性(Deep Homography)算法计算两幅图像的投影矩阵,根据矩阵参数扭曲其中一副图像完成拼接过程的预对齐;其次在进行编解码处理时充分利用不同尺度的中间层特征图提取空间和语义信息,以加强网络对图像错位区域的感知能力;最后在恢复图像时引入链式残差池化结构捕获图像背景区域中的上下文特征,进一步提高了拼接的质量。实验结果表明:本文方法没有明显的拼接痕迹,亮度和颜色过渡自然,同时有效减少伪影和错位现象,结构相似性相较于其它3种拼接算法性能至少提高了1%,峰值信噪比较深度学习方法提高了0.26dB。

New multi-camera calibration algorithm based on 1D objects

A new calibration algorithm for multi-camera systems using 1D calibration objects is proposed. The algorithm inte- grates the rank-4 factorization with Zhang (2004)'s method. The intrinsic parameters as well as the extrinsic parameters are re- covered by capturing with cameras the 1D object's rotations around a fixed point. The algorithm is based on factorization of the scaled measurement matrix, the projective depth of which is estimated in an analytical equation instead of a recursive form. For more than three points on a 1D object, the approach of our algorithm is to extend the scaled measurement matrix. The obtained parameters are finally refined through the maximum likelihood inference. Simulations and experiments with real images verify that the proposed technique achieves a good trade-off between the intrinsic and extrinsic camera parameters.

An improved RANSAC algorithm for 3D wheel alignment

Aiming at the defects of traditional four-wheel aligner such as many sensors,complex operation and slow detection speed,a fast and accurate 3D four-wheel alignment detection method is studied.Firstly,a new and special circle center target board is designed to calibrate the camera,and then the registration of the homography matrix is optimized by using the improved RANSAC(Random sample consensus)algorithm combined with the designed special target board,and the parameters of the wheel alignment system are adjusted by using the space vector principle.Accurate measurements are made to obtain the parameters of the four-wheel alignment.Design a calibration comparison experiment between the traditional target board and the new type of target board,and conduct a comparative test with the existing four-wheel aligner of the depot.The experimental results show that the use of the new target board-binding optimization algorithm can improve the calibration efficiency by about 9%to 21%,while improving the calibration accuracy by about 10.6%to 17.8%.And through the real vehicle test,it is verified that the use of the new target combined with the optimization algorithm can ensure the accuracy and reliability of the four-wheel positioning.This method has a certain significance in the rapid detection of vehicle four-wheel alignment parameters.

Perspective Projection Algorithm Enabling Mobile Device’s Indoor Positioning

In order to improve the user’s satisfaction with the augmented reality (AR) technology and the accuracy of the service, it is important to obtain the exact position of the user. Frequently used techniques for finding outdoors locations is the global positioning system (GPS), which is less accurate indoors. Therefore, an indoor position is measured by comparing the reception level about access point (AP) signal of wireless fidelity (Wi-Fi) or using bluetooth low energy (BLE) tags. However, Wi-Fi and Bluetooth require additional hardware installation. In this paper, the proposed method of estimating the user’s position uses an indoor image and indoor coordinate map without additional hardware installation. The indoor image has several feature points extracted from fixed objects. By matching the feature points with the feature points of the user image, we can obtain the position of the user on the Indoor map by obtaining six or more pixel coordinates from the user image and solving the solution using the perspective projection formula. The experimental results show that the user position can be obtained more accurately in the indoor environment by using only the software without additional hardware installation.

Learning accurate template matching with differentiable coarseto-fine correspondence refinement

Template matching is a fundamental task in computer vision and has been studied for decades.It plays an essential role in manufacturing industry for estimating the poses of different parts,facilitating downstream tasks such as robotic grasping.Existing methods fail when the template and source images have different modalities,cluttered backgrounds,or weak textures.They also rarely consider geometric transformations via homographies,which commonly exist even for planar industrial parts.To tackle the challenges,we propose an accurate template matching method based on differentiable coarse-tofine correspondence refinement.We use an edge-aware module to overcome the domain gap between the mask template and the grayscale image,allowing robust matching.An initial warp is estimated using coarse correspondences based on novel structure-aware information provided by transformers.This initial alignment is passed to a refinement network using references and aligned images to obtain sub-pixel level correspondences which are used to give the final geometric transformation.Extensive evaluation shows that our method to be significantly better than state-of-the-art methods and baselines,providing good generalization ability and visually plausible results even on unseen real data.

Unsupervised Oral Endoscope Image Stitching Algorithm

Oral endoscope image stitching algorithm is studied to obtain wide-field oral images through regis-tration and stitching,which is of great significance for auxiliary diagnosis.Compared with natural images,oral images have lower textures and fewer features.However,traditional feature-based image stitching methods rely heavily on feature extraction quality,often showing an unsatisfactory performance when stitching images with few features.Moreover,due to the hand-held shooting,there are large depth and perspective disparities between the captured images,which also pose a challenge to image stitching.To overcome the above problems,we propose an unsupervised oral endoscope image stitching algorithm based on the extraction of overlapping regions and the loss of deep features.In the registration stage,we extract the overlapping region of the input images by sketching polygon intersection for feature points screening and estimate homography from coarse to fine on a three-layer feature pyramid structure.Moreover,we calculate loss using deep features instead of pixel values to emphasize the importance of depth disparities in homography estimation.Finally,we reconstruct the stitched images from feature to pixel,which can eliminate artifacts caused by large parallax.Our method is compared with both feature-based and previous deep-based methods on the UDIS-D dataset and our oral endoscopy image dataset.The experimental results show that our algorithm can achieve higher homography estimation accuracy,and better visual quality,and can be effectively applied to oral endoscope image stitching.

Single View Based Measurement on Space Planes

The plane metrology using a single uncalibrated image is studied in the paper, and three novel approaches are proposed. The first approach, namely key-line-based method, is an improvement over the widely used key-point-based method, which uses line correspondences directly to compute homography between the world plane and its image so as to increase the computational accuracy. The second and third approaches are both based on a pair of vanishing points from two orthogonal sets of parallel lines in the space plane together with two unparallel referential distances, but the two methods deal with the problem in different ways. One is from the algebraic viewpoint which first maps the image points to an affine space via a transformation constructed from the vanishing points, and then computes the metric distance according to the relationship between the affine space and the Euclidean space, while the other is from the geometrical viewpoint based on the invariance of cross ratios. The second and third methods avoid the selection of control points and are widely applicable. In addition, a brief description on how to retrieve other geometrical entities on the space plane, such as distance from a point to a line, angle formed by two lines, etc., is also presented in the paper. Extensive experiments on simulated data as well as on real images show that the first and the second approaches are of better precision and stronger robustness than the key-point-based one and the third one, since these two approaches are fundamentally based on line information.

A new navigation approach of terrain contour matching based on 3-D terrain reconstruction from onboard image sequence

This article presents a passive navigation method of terrain contour matching by reconstructing the 3-D terrain from the image sequence(acquired by the onboard camera).To achieve automation and simultaneity of the image sequence processing for navigation,a correspondence registration method based on control points tracking is proposed which tracks the sparse control points through the whole image sequence and uses them as correspondence in the relation geometry solution.Besides,a key frame selection method based on the images overlapping ratio and intersecting angles is explored,thereafter the requirement for the camera system configuration is provided.The proposed method also includes an optimal local homography estimating algorithm according to the control points,which helps correctly predict points to be matched and their speed corresponding.Consequently,the real-time 3-D terrain of the trajectory thus reconstructed is matched with the referenced terrain map,and the result of which provides navigating information.The digital simulation experiment and the real image based experiment have verified the proposed method.

A New Method of Manifold Mosaic for Large Displacement Images

In the traditional manifold mosaic, a single center strip is clipped out from each source image to create a large image. Therefore the displacement between neighboring views should be very small in order to fulfill effective strips cutting. In this paper, a method is proposed to create a manifold mosaic by images with relative large displacement by means of cutting out multiple strips in the overlap area according to the homography between images. These strips are then warped together to create a smooth mosaic. An improved RANSAC algorithm is also presented in order to improve the precision of homography calculation. Experimental results demonstrate the efficiency of the method.