Efficient Unsupervised Image Stitching Using Attention Mechanism with Deep Homography Estimation

Traditional feature-based image stitching techniques often encounter obstacles when dealing with images lackingunique attributes or suffering from quality degradation. The scarcity of annotated datasets in real-life scenesseverely undermines the reliability of supervised learning methods in image stitching. Furthermore, existing deeplearning architectures designed for image stitching are often too bulky to be deployed on mobile and peripheralcomputing devices. To address these challenges, this study proposes a novel unsupervised image stitching methodbased on the YOLOv8 (You Only Look Once version 8) framework that introduces deep homography networksand attentionmechanisms. Themethodology is partitioned into three distinct stages. The initial stage combines theattention mechanism with a pooling pyramid model to enhance the detection and recognition of compact objectsin images, the task of the deep homography networks module is to estimate the global homography of the inputimages consideringmultiple viewpoints. The second stage involves preliminary stitching of the masks generated inthe initial stage and further enhancement through weighted computation to eliminate common stitching artifacts.The final stage is characterized by adaptive reconstruction and careful refinement of the initial stitching results.Comprehensive experiments acrossmultiple datasets are executed tometiculously assess the proposed model. Ourmethod’s Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity Index Measure (SSIM) improved by 10.6%and 6%. These experimental results confirm the efficacy and utility of the presented model in this paper.

Research on Stitching Algorithm Based on Tree Branch Image

Branch identification technology is a key technology to achieve automated pruning of fruit tree branches, and one of its technical bottlenecks lies in the stitching of branch images. To this end, we propose a set of branch image stitching technology algorithms. The algorithm is based on the grey-scale prime centroid method to determine the detection feature points, and uses the coordinate transformation matrix H of the corresponding points of the image to carry out the image geometric transformation, and realises the feature matching through sample comparison and classification methods. The experimental results show that the matched point images are more correct and less time-consuming.

Underwater Terrain Image Stitching Based on Spatial Gradient Feature Block

At present,underwater terrain images are all strip-shaped small fragment images preprocessed by the side-scan sonar imaging system.However,the processed underwater terrain images have inconspicuous and few feature points.In order to better realize the stitching of underwater terrain images and solve the problems of slow traditional image stitching speed,we proposed an improved algorithm for underwater terrain image stitching based on spatial gradient feature block.First,the spatial gradient fuzzy C-Means algorithm is used to divide the underwater terrain image into feature blocks with the fusion of spatial gradient information.The accelerated-KAZE(AKAZE)algorithm is used to combine the feature block information to match the reference image and the target image.Then,the random sample consensus(RANSAC)is applied to optimize the matching results.Finally,image fusion is performed with the global homography and the optimal seam-line method to improve the accuracy of image overlay fusion.The experimental results show that the proposed method in this paper effectively divides images into feature blocks by combining spatial information and gradient information,which not only solves the problem of stitching failure of underwater terrain images due to unobvious features,and further reduces the sensitivity to noise,but also effectively reduces the iterative calculation in the feature point matching process of the traditional method,and improves the stitching speed.Ghosting and shape warping are significantly eliminated by re-optimizing the overlap of the image.

A survey on image and video stitching

Image/video stitching is a technology for solving the field of view(FOV)limitation of images/videos.It stitches multiple overlapping images/videos to generate a wide-FOV image/video,and has been used in various fields such as sports broadcasting,video surveillance,street view,and entertainment.This survey reviews image/video stitching algorithms,with a particular focus on those developed in recent years.Image stitching first calculates the corresponding relationships between multiple overlapping images,deforms and aligns the matched images,and then blends the aligned images to generate a wide-FOV image.A seamless method is always adopted to eliminate such potential flaws as ghosting and blurring caused by parallax or objects moving across the overlapping regions.Video stitching is the further extension of image stitching.It usually stitches selected frames of original videos to generate a stitching template by performing image stitching algorithms,and the subsequent frames can then be stitched according to the template.Video stitching is more complicated with moving objects or violent camera movement,because these factors introduce jitter,shakiness,ghosting,and blurring.Foreground detection technique is usually combined into stitching to eliminate ghosting and blurring,while video stabilization algorithms are adopted to solve the jitter and shakiness.This paper further discusses panoramic stitching as a special-extension of image/video stitching.Panoramic stitching is currently the most widely used application in stitching.This survey reviews the latest image/video stitching methods,and introduces the fundamental principles/advantages/weaknesses of image/video stitching algorithms.Image/video stitching faces long-term challenges such as wide baseline,large parallax,and low-texture problem in the overlapping region.New technologies may present new opportunities to address these issues,such as deep learning-based semantic correspondence,and 3D image stitching.Finally,this survey discusses the challenges of image/video stitching and proposes potential solutions.

Efficient Image Stitching in the Presence of Dynamic Objects and Structure Misalignment

This paper presents a new method for simultaneously eliminating visual artifacts caused by moving objects and structure misalignment in image stitching. Given that the input images are roughly aligned, our approach is implemented in two stages. In the first stage, we discover motions between input images, and then extract their corresponding regions through a multi-seed based region growing algorithm. In the second stage, with prior information provided by the extracted regions, we perform a graph cut optimization in gradient-domain to determine which pixels to use from each image to achieve seamless stitching. Our method is simple to implement and effective. The experimental results illustrate that the proposed approach can produce comparable or superior results in comparison with state-of-the-art methods.

Unsupervised Oral Endoscope Image Stitching Algorithm

Oral endoscope image stitching algorithm is studied to obtain wide-field oral images through regis-tration and stitching,which is of great significance for auxiliary diagnosis.Compared with natural images,oral images have lower textures and fewer features.However,traditional feature-based image stitching methods rely heavily on feature extraction quality,often showing an unsatisfactory performance when stitching images with few features.Moreover,due to the hand-held shooting,there are large depth and perspective disparities between the captured images,which also pose a challenge to image stitching.To overcome the above problems,we propose an unsupervised oral endoscope image stitching algorithm based on the extraction of overlapping regions and the loss of deep features.In the registration stage,we extract the overlapping region of the input images by sketching polygon intersection for feature points screening and estimate homography from coarse to fine on a three-layer feature pyramid structure.Moreover,we calculate loss using deep features instead of pixel values to emphasize the importance of depth disparities in homography estimation.Finally,we reconstruct the stitched images from feature to pixel,which can eliminate artifacts caused by large parallax.Our method is compared with both feature-based and previous deep-based methods on the UDIS-D dataset and our oral endoscopy image dataset.The experimental results show that our algorithm can achieve higher homography estimation accuracy,and better visual quality,and can be effectively applied to oral endoscope image stitching.

RecStitchNet:Learning to stitch images with rectangular boundaries

Irregular boundaries in image stitching naturally occur due to freely moving cameras.To deal with this problem,existing methods focus on optimizing mesh warping to make boundaries regular using the traditional explicit solution.However,previous methods always depend on hand-crafted features(e.g.,keypoints and line segments).Thus,failures often happen in overlapping regions without distinctive features.In this paper,we address this problem by proposing RecStitchNet,a reasonable and effective network for image stitching with rectangular boundaries.Considering that both stitching and imposing rectangularity are non-trivial tasks in the learning-based framework,we propose a three-step progressive learning based strategy,which not only simplifies this task,but gradually achieves a good balance between stitching and imposing rectangularity.In the first step,we perform initial stitching by a pre-trained state-of-the-art image stitching model,to produce initially warped stitching results without considering the boundary constraint.Then,we use a regression network with a comprehensive objective regarding mesh,perception,and shape to further encourage the stitched meshes to have rectangular boundaries with high content fidelity.Finally,we propose an unsupervised instance-wise optimization strategy to refine the stitched meshes iteratively,which can effectively improve the stitching results in terms of feature alignment,as well as boundary and structure preservation.Due to the lack of stitching datasets and the difficulty of label generation,we propose to generate a stitching dataset with rectangular stitched images as pseudo-ground-truth labels,and the performance upper bound induced from the it can be broken by our unsupervised refinement.Qualitative and quantitative results and evaluations demonstrate the advantages of our method over the state-of-the-art.

Underwater Image Bidirectional Matching for Localization Based on SIFT

For the purpose of identifying the stern of the SWATH （Small Waterplane Area Twin Hull） availably and perfecting the detection technique of the SWATH ship＇s performance, this paper presents a novel bidirectional image registration strategy and mosaicing technique based on the scale invariant feature transform （SIFT） algorithm. The proposed method can help us observe the stern with a great visual angle for analyzing the performance of the control fins of the SWATH. SIFT is one of the most effective local features of the scale, rotation and illumination invariant. However, there are a few false match rates in this algorithm. In terms of underwater machine vision, only by acquiring an accurate match rate can we find an underwater robot rapidly and identify the location of the object. Therefore, firstly, the selection of the match ratio principle is put forward in this paper; secondly, some advantages of the bidirectional registration algorithm are concluded by analyzing the characteristics of the unidirectional matching method. Finally, an automatic underwater image splicing method is proposed on the basis of fixed dimension, and then the edge of the image＇s overlapping section is merged by the principal components analysis algorithm. The experimental results achieve a better registration and smooth mosaicing effect, demonstrating that the proposed method is effective.

Measuring Structural Parameters of Knitted Fabrics by Digital Image Processing Techniques

In the knitting industry the measurements of the stitch density and the stitch length are usually done manually, which may lead to lower efficiency and less definition and also bring subjective ideas into the test results. In order to improve the effect we can measure with Digital Image Processing Techniques. A piece of sample is scanned into computer and changed into a digital image, which is processed with media filtering. To acquire the power spectrum, the image in the spatial domain is converted into the frequency domain. Picking up the characteristic points describing the stitch density and the stitch length separately in the power spectra and reconstructing them, the values of the stitch density and the stitch length could be calculated. When measuring the stitch length, we should establish a geometric model of the stitch based en the digital image processing, which provides a method to transform the stitch length in the two-dimensien space into the three-dimensien space and to measure the value of the stitch length more accurately. This method also provides a new way to measure the stitch length without damaging the fabric.

基于双反射镜的2D-DIC变形测量系统开发

二维数字图像相关(two-dimensional digital image correlation,2D-DIC)在测量过程中不可避免地会出现相机光轴与测量表面非垂直,由此产生的离面位移而将导致较大的测量误差,同时在视场受限的环境中难以通过单台相机完成大范围的变形测量。有鉴于此,该文开发了基于双反射镜的2D-DIC变形测量系统,使用双反射镜成像缓解离面运动对2D-DIC的影响,通过可移动相机实现小视场下的图像采集,提出基于频域移位的高精度图像拼接方法,并改进了融合函数,最终获得试样的高分辨率图像。单轴拉伸实验结果表明,轴向应变的平均相对误差相比传统2D-DIC方法降低12.82%,测量分辨率提高约34.92%,验证了测量系统的可行性和有效性。

面向青花瓷碎片图像的U-Net++拼接网络

针对现有图像拼接方法存在拼接处伪影以及非重叠区域内容失真,导致较低的准确性和鲁棒性的问题,提出一种基于U-Net++消除伪影的青花瓷碎片图像拼接方法.首先估计待拼接图像单应性矩阵;然后将单应性矩阵应用于结构拼接阶段,得到图像粗拼接结果;最后以图像粗拼接结果作为先验信息,在内容校正阶段改进现有的U-Net,利用U-Net++细化粗拼接结果,得到最终图像精确拼接.以青花瓷碎片图像数据集与相关经典方法进行实验的结果表明,在3个评价指标中,所提方法的峰值信噪比提高约13%,均方根误差降低约33%,均方误差降低57%左右;该方法具有较小的误差比,不仅能够提高图像拼接质量,而且表现出较好的鲁棒性.

基于无人机及YOLOX视觉算法的大跨度钢结构吊装过程位移监测

在大跨度钢结构吊装施工过程中,节点位移及结构变形关系到吊装施工的安全和质量。对于传统接触式监测方法存在的耗时、耗力且维护费用高等问题,提出了一种以无人机为载体的非接触式监测方式。首先,针对大跨度钢结构吊装过程中无人机近距采集视角受限的问题,采用Harris图像拼接算法进行全景拼接,并与图像加权融合相结合,消除图像拼接中产生的不利光标及拼接缝,实现整体、高精度的大跨度结构图像的无缝拼接;其次,采用加入卷积块注意力机制(convolutional block attention module, CBAM)的YOLOX视觉算法解决复杂背景下不同像素面积的小目标图像识别、坐标提取和位移监测;最后,对四种不同检测模型进行对比评估,并通过对比实验室不同工况试验和实际工程验证该方法在施工环境下对大跨度钢结构测点位移监测的可行性。试验结果表明,加入CBAM注意力机制的YOLOX检测模型的平均精度及置信度均优于其他三种网络模型,且视觉识别的位移信息与Leica全站仪的误差均在亚毫米级内,满足实际工程精度的要求,实现了复杂背景下的小目标位移监测,具备较高的经济效益和广泛的应用前景。

基于改进最佳缝合线的矿井图像拼接方法

煤矿井下掘进工作面高粉尘、低照度的恶劣环境导致图像信噪比较低,且有效特征点数量严重减少,处理后的图像存在较大色差和噪声,在使用最佳缝合线算法进行图像拼接时出现细节错位、缝合线处过渡不自然或拼接痕迹明显的现象。针对上述问题,提出了一种基于改进最佳缝合线的矿井图像拼接方法。首先,对原始图像进行HSV空间变换,采用改进的Retinex算法对亮度分量进行增强,利用双边滤波函数代替中心环绕函数,以解决亮度差异大处产生的光晕问题,通过增强算法有效提高特征点提取数量。然后,采用SIFT算法提取特征点,并以余弦距离作为匹配度指标;引入像素余弦相似度作为约束项,并采用形态学操作对颜色差异强度进行改进,利用动态规划法对最佳缝合线进行搜索,以避免图像拼接处的错位现象。最后,结合渐入渐出融合算法,使图像过渡平滑,实现煤矿井下掘进工作面的图像融合。模拟井下实际工况环境进行实验验证,结果表明:基于改进最佳缝合线的矿井图像拼接方法与传统最佳缝合线算法相比,避免了颜色差异和噪声引起的错位拼接现象,拼接缝处的图像过渡更加自然,避免了“鬼影”和明显拼接缝的产生,且图像平均梯度提高2.38%,拼接时间提高32.5%,使得融合区域更加平滑自然,提高了拼接质量。

大幅宽面阵相机多片探测器影像拼接方法

大幅宽面阵相机通常采用多片探测器拼幅成像的方式来增大幅宽。为了提高数据的使用效率,用户要求提供几何无损和视觉无缝的拼接产品。由于分片探测器间重叠区域小甚至无重叠区,拼接后影像畸变增大等原因,传统像方拼接方法难以适用。针对此问题,文章提出一种基于传感器校正的大幅宽面阵相机多片探测器影像高精度几何拼接方法。该方法首先根据卫星相机分片探测器的物理结构及其成像几何特性,虚拟一个完全理想的无畸变面阵影像,影像覆盖范围与原始分片探测器影像覆盖范围相同;在严格标定分片探测器像元指向的基础上,建立虚拟影像与原始分片影像的像点坐标换算关系,并将原始分片影像的像点灰度值赋给虚拟影像对应像点,进而实现对分片影像的无缝拼接和畸变校正。经对仿真数据和在轨数据的试验验证,结果表明:对于不同探测器拼接方式和搭接区重叠情况,文章中所提方法可获得视觉无缝和几何无损的拼接影像,影像的片间拼接精度和影像内部几何畸变均优于1个像素。

农田环境下无人机图像并行拼接识别算法

为改善在农田环境下无人机图像计算速度和效率,该研究提出了一种农田环境下无人机图像并行拼接识别算法。利用倒二叉树并行拼接识别算法,通过提取图像拼接中的变换矩阵,实现拼接识别同时进行。根据边缘设备的CPU核心数和图像数量自动将图像拼接识别任务划分为多个子进程,并分配到不同核心上执行,以提高在农田环境下的计算效率。试验结果表明:相同试验环境和数据集条件下,倒二叉树并行拼接算法的拼接耗时相较于其他算法平均减少了60%~90%左右;在农田环境下,倒二叉树并行拼接识别相较于串行拼接识别的耗时减少了70%,图像识别的平均像素交并比提升了10.17个百分点,说明在农田环境下采用多线程倒二叉树并行算法可以更好地利用农田环境下边缘设备的计算资源,大幅提升无人机图像的拼接和识别的速度,为无人机的快速实时监测提供技术支撑。

“图文缝合”:文学接受理论的当代探绎

“图文缝合”是指文学接受中基于语象互文与读者“期待视域”引导效应,文本表述与读者接受图文寓意缝合的文学现象。“图文缝合”研究的终极目标,意在揭示文学接受差异可控性的逻辑规律。文章整合叙事学与接受美学相关理论,以福克纳经典作品的图文缝合现象为研究介质,对语言与图像间互动缝合产生的文学效应展开深入探绎。本文旨在探赜文学图文缝合的形成规律与实现途径,拓展文学接受研究视域,促进当代文学批评理论的丰富与发展。

基于半投影翘曲的无人机图像拼接方法

为了解决无人机图像在拼接过程中出现的错位、重影等造成图像失真的问题,提出一种基于半投影翘曲的无人机图像拼接方法。首先估计图像的全局投影变换矩阵;然后根据待拼接图像的重叠情况,以及投影变换和相似性变换的连续性,求解出过渡变换矩阵和相似性变换矩阵,得到最终的半投影变换矩阵;最后构建图像重叠区域的差异矩阵,以此为基础获取重叠区域的差异性区域。使用分区融合策略,在差异性区域进行单采样,在其他区域进行距离加权融合。实验结果表明,该方法可以很好地拼接由于视角变化、地势起伏等造成视差的无人机图像,得到的拼接结果自然清晰,效果优于其他先进算法。该方法有效地解决了无人机图像拼接错位、重影的问题,在多项定量指标上表现良好。

大区域场景下基于无人机视角的目标计数方法

近年来,无人机因其灵活度高、机动性强在人群计数领域得到广泛应用。然而,现有的人群计数方法大多基于单视点,对于大范围、多摄像机场景下的多视点计数研究较少。为了解决这个问题,提出了一种基于无人机视角的目标计数方法以准确统计场景中的目标数量。选择临海区域进行数据采集,利用深度学习技术对采集的图像进行目标检测和图像拼接融合,在拼接后的图像中映射检测信息,并采用计数算法完成区域场景的计数任务。在公开数据集和该文制作的数据集上进行的实验验证了基于目标检测的计数算法的有效性。

改进网格单应性投影变换的文物多镜头光谱图像拼接方法

针对文物多镜头光谱图像序列拼接时的累计误差和多个景深视差导致的特征相对位置偏移问题,提出改进网格单应性投影变换的拼接方法。引入正射彩色全景图像作为参考图像,避免了以单一分镜头光谱图像为基准的累计误差问题。提出SIFT-LBP特征匹配算法,利用LBP纹理对SIFT特征匹配进行筛选和补充,获得准确的特征匹配点对;采用网格单应性变换,以特征匹配点之间的距离为权重,计算目标网格像素点的偏移位置,减少特征相对位置偏移问题;采用直方图匹配加权融合方法融合分镜头光谱图像,以平滑图像重叠区域。与主流图像拼接方法相比较,该方法拼接的图像视觉效果更优,且峰值信噪比较主流方法提高12.81%,结构相似性提高13.12%,为文物保护与研究提供可靠的全景光谱图像数据。

弱纹理飞机蒙皮曲面图像特征匹配及拼接

为了解决弱纹理飞机蒙皮特征点分布不均匀、正确匹配的特征点对较少的问题,提出了一种改进的LoFTR算法对飞机蒙皮图像进行拼接。根据相机位姿利用柱面反投影对蒙皮图像进行曲面校正;通过图像之间的重叠区域确定特征提取区域,从而减少错误匹配点对的生成;使用LoFTR算法进行特征提取,并且使用RANSAC算法对特征点进行筛选;根据图像分块的思想对重叠区域进行网格划分来对特征点进一步筛选,使得特征点分布更加均匀,得到更加准确的变换矩阵进行图像配准。实验在自研无人车采集的飞机蒙皮图像上进行了测试和验证,改进的方法与SIFT、SURF、ORB、BRISK以及AKAZE进行了特征匹配率比较实验,SIFT、SURF、ORB、BRISK和AKAZE匹配率分别为4.84%,0.47%、2.9%、0.86%、5.08%,提出的算法特征匹配率达到55.21%,SSIM平均值提高了44.38%~88.46%。该方法适用于对飞机蒙皮图像的拼接任务,且不存在因弱纹理而导致漏拼的问题。