Efficient Unsupervised Image Stitching Using Attention Mechanism with Deep Homography Estimation

Traditional feature-based image stitching techniques often encounter obstacles when dealing with images lackingunique attributes or suffering from quality degradation. The scarcity of annotated datasets in real-life scenesseverely undermines the reliability of supervised learning methods in image stitching. Furthermore, existing deeplearning architectures designed for image stitching are often too bulky to be deployed on mobile and peripheralcomputing devices. To address these challenges, this study proposes a novel unsupervised image stitching methodbased on the YOLOv8 (You Only Look Once version 8) framework that introduces deep homography networksand attentionmechanisms. Themethodology is partitioned into three distinct stages. The initial stage combines theattention mechanism with a pooling pyramid model to enhance the detection and recognition of compact objectsin images, the task of the deep homography networks module is to estimate the global homography of the inputimages consideringmultiple viewpoints. The second stage involves preliminary stitching of the masks generated inthe initial stage and further enhancement through weighted computation to eliminate common stitching artifacts.The final stage is characterized by adaptive reconstruction and careful refinement of the initial stitching results.Comprehensive experiments acrossmultiple datasets are executed tometiculously assess the proposed model. Ourmethod’s Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity Index Measure (SSIM) improved by 10.6%and 6%. These experimental results confirm the efficacy and utility of the presented model in this paper.

Research on Stitching Algorithm Based on Tree Branch Image

Branch identification technology is a key technology to achieve automated pruning of fruit tree branches, and one of its technical bottlenecks lies in the stitching of branch images. To this end, we propose a set of branch image stitching technology algorithms. The algorithm is based on the grey-scale prime centroid method to determine the detection feature points, and uses the coordinate transformation matrix H of the corresponding points of the image to carry out the image geometric transformation, and realises the feature matching through sample comparison and classification methods. The experimental results show that the matched point images are more correct and less time-consuming.

Underwater Terrain Image Stitching Based on Spatial Gradient Feature Block

At present,underwater terrain images are all strip-shaped small fragment images preprocessed by the side-scan sonar imaging system.However,the processed underwater terrain images have inconspicuous and few feature points.In order to better realize the stitching of underwater terrain images and solve the problems of slow traditional image stitching speed,we proposed an improved algorithm for underwater terrain image stitching based on spatial gradient feature block.First,the spatial gradient fuzzy C-Means algorithm is used to divide the underwater terrain image into feature blocks with the fusion of spatial gradient information.The accelerated-KAZE(AKAZE)algorithm is used to combine the feature block information to match the reference image and the target image.Then,the random sample consensus(RANSAC)is applied to optimize the matching results.Finally,image fusion is performed with the global homography and the optimal seam-line method to improve the accuracy of image overlay fusion.The experimental results show that the proposed method in this paper effectively divides images into feature blocks by combining spatial information and gradient information,which not only solves the problem of stitching failure of underwater terrain images due to unobvious features,and further reduces the sensitivity to noise,but also effectively reduces the iterative calculation in the feature point matching process of the traditional method,and improves the stitching speed.Ghosting and shape warping are significantly eliminated by re-optimizing the overlap of the image.

A survey on image and video stitching

Image/video stitching is a technology for solving the field of view(FOV)limitation of images/videos.It stitches multiple overlapping images/videos to generate a wide-FOV image/video,and has been used in various fields such as sports broadcasting,video surveillance,street view,and entertainment.This survey reviews image/video stitching algorithms,with a particular focus on those developed in recent years.Image stitching first calculates the corresponding relationships between multiple overlapping images,deforms and aligns the matched images,and then blends the aligned images to generate a wide-FOV image.A seamless method is always adopted to eliminate such potential flaws as ghosting and blurring caused by parallax or objects moving across the overlapping regions.Video stitching is the further extension of image stitching.It usually stitches selected frames of original videos to generate a stitching template by performing image stitching algorithms,and the subsequent frames can then be stitched according to the template.Video stitching is more complicated with moving objects or violent camera movement,because these factors introduce jitter,shakiness,ghosting,and blurring.Foreground detection technique is usually combined into stitching to eliminate ghosting and blurring,while video stabilization algorithms are adopted to solve the jitter and shakiness.This paper further discusses panoramic stitching as a special-extension of image/video stitching.Panoramic stitching is currently the most widely used application in stitching.This survey reviews the latest image/video stitching methods,and introduces the fundamental principles/advantages/weaknesses of image/video stitching algorithms.Image/video stitching faces long-term challenges such as wide baseline,large parallax,and low-texture problem in the overlapping region.New technologies may present new opportunities to address these issues,such as deep learning-based semantic correspondence,and 3D image stitching.Finally,this survey discusses the challenges of image/video stitching and proposes potential solutions.

Efficient Image Stitching in the Presence of Dynamic Objects and Structure Misalignment

This paper presents a new method for simultaneously eliminating visual artifacts caused by moving objects and structure misalignment in image stitching. Given that the input images are roughly aligned, our approach is implemented in two stages. In the first stage, we discover motions between input images, and then extract their corresponding regions through a multi-seed based region growing algorithm. In the second stage, with prior information provided by the extracted regions, we perform a graph cut optimization in gradient-domain to determine which pixels to use from each image to achieve seamless stitching. Our method is simple to implement and effective. The experimental results illustrate that the proposed approach can produce comparable or superior results in comparison with state-of-the-art methods.

Unsupervised Oral Endoscope Image Stitching Algorithm

Oral endoscope image stitching algorithm is studied to obtain wide-field oral images through regis-tration and stitching,which is of great significance for auxiliary diagnosis.Compared with natural images,oral images have lower textures and fewer features.However,traditional feature-based image stitching methods rely heavily on feature extraction quality,often showing an unsatisfactory performance when stitching images with few features.Moreover,due to the hand-held shooting,there are large depth and perspective disparities between the captured images,which also pose a challenge to image stitching.To overcome the above problems,we propose an unsupervised oral endoscope image stitching algorithm based on the extraction of overlapping regions and the loss of deep features.In the registration stage,we extract the overlapping region of the input images by sketching polygon intersection for feature points screening and estimate homography from coarse to fine on a three-layer feature pyramid structure.Moreover,we calculate loss using deep features instead of pixel values to emphasize the importance of depth disparities in homography estimation.Finally,we reconstruct the stitched images from feature to pixel,which can eliminate artifacts caused by large parallax.Our method is compared with both feature-based and previous deep-based methods on the UDIS-D dataset and our oral endoscopy image dataset.The experimental results show that our algorithm can achieve higher homography estimation accuracy,and better visual quality,and can be effectively applied to oral endoscope image stitching.

Underwater Image Bidirectional Matching for Localization Based on SIFT

为识别的目的 SWATH (小 Waterplane 区域双胞胎的赫尔) 严厉有效地并且完善 SWATH 轮船鈥檚 表演的察觉技术，这份报纸基于规模论述新奇双向图象登记策略和 mosaicing 技术不变的特征变换(筛) 算法。建议方法能帮助我们观察与为分析 SWATH 的控制鳍的性能的一个大视觉角度严厉。筛是规模，旋转和照明的最有效的本地特征之一不变。然而，在这个算法有一些假火柴率。以在水下机器视觉，仅仅由获得一个精确火柴率罐头，我们发现一在水下机器人很快并且识别目标的地点。第一，因此，火柴比率原则的选择在这份报纸被提出；第二，双向登记算法的一些优点被分析单向性的匹配方法的特征结束。最后，一自动在水下图象拼接方法根据图象的固定尺寸，然后边被建议鈥檚 重叠节被主要部件分析算法合并。试验性的结果完成更好的登记和光滑的 mosaicing 效果，证明建议方法是有效的。

Measuring Structural Parameters of Knitted Fabrics by Digital Image Processing Techniques

In the knitting industry the measurements of the stitch density and the stitch length are usually done manually, which may lead to lower efficiency and less definition and also bring subjective ideas into the test results. In order to improve the effect we can measure with Digital Image Processing Techniques. A piece of sample is scanned into computer and changed into a digital image, which is processed with media filtering. To acquire the power spectrum, the image in the spatial domain is converted into the frequency domain. Picking up the characteristic points describing the stitch density and the stitch length separately in the power spectra and reconstructing them, the values of the stitch density and the stitch length could be calculated. When measuring the stitch length, we should establish a geometric model of the stitch based on the digital image processing, which provides a method to transform the stitch length in the two-dimension space into the three-dimension space and to measure the value of the stitch length more accurately. This method also provides a new way to measure the stitch length without damaging the fabric.

基于双反射镜的2D-DIC变形测量系统开发

二维数字图像相关(two-dimensional digital image correlation,2D-DIC)在测量过程中不可避免地会出现相机光轴与测量表面非垂直,由此产生的离面位移而将导致较大的测量误差,同时在视场受限的环境中难以通过单台相机完成大范围的变形测量。有鉴于此,该文开发了基于双反射镜的2D-DIC变形测量系统,使用双反射镜成像缓解离面运动对2D-DIC的影响,通过可移动相机实现小视场下的图像采集,提出基于频域移位的高精度图像拼接方法,并改进了融合函数,最终获得试样的高分辨率图像。单轴拉伸实验结果表明,轴向应变的平均相对误差相比传统2D-DIC方法降低12.82%,测量分辨率提高约34.92%,验证了测量系统的可行性和有效性。

面向青花瓷碎片图像的U-Net++拼接网络

针对现有图像拼接方法存在拼接处伪影以及非重叠区域内容失真,导致较低的准确性和鲁棒性的问题,提出一种基于U-Net++消除伪影的青花瓷碎片图像拼接方法.首先估计待拼接图像单应性矩阵;然后将单应性矩阵应用于结构拼接阶段,得到图像粗拼接结果;最后以图像粗拼接结果作为先验信息,在内容校正阶段改进现有的U-Net,利用U-Net++细化粗拼接结果,得到最终图像精确拼接.以青花瓷碎片图像数据集与相关经典方法进行实验的结果表明,在3个评价指标中,所提方法的峰值信噪比提高约13%,均方根误差降低约33%,均方误差降低57%左右;该方法具有较小的误差比,不仅能够提高图像拼接质量,而且表现出较好的鲁棒性.

基于改进最佳缝合线的矿井图像拼接方法

煤矿井下掘进工作面高粉尘、低照度的恶劣环境导致图像信噪比较低,且有效特征点数量严重减少,处理后的图像存在较大色差和噪声,在使用最佳缝合线算法进行图像拼接时出现细节错位、缝合线处过渡不自然或拼接痕迹明显的现象。针对上述问题,提出了一种基于改进最佳缝合线的矿井图像拼接方法。首先,对原始图像进行HSV空间变换,采用改进的Retinex算法对亮度分量进行增强,利用双边滤波函数代替中心环绕函数,以解决亮度差异大处产生的光晕问题,通过增强算法有效提高特征点提取数量。然后,采用SIFT算法提取特征点,并以余弦距离作为匹配度指标;引入像素余弦相似度作为约束项,并采用形态学操作对颜色差异强度进行改进,利用动态规划法对最佳缝合线进行搜索,以避免图像拼接处的错位现象。最后,结合渐入渐出融合算法,使图像过渡平滑,实现煤矿井下掘进工作面的图像融合。模拟井下实际工况环境进行实验验证,结果表明:基于改进最佳缝合线的矿井图像拼接方法与传统最佳缝合线算法相比,避免了颜色差异和噪声引起的错位拼接现象,拼接缝处的图像过渡更加自然,避免了“鬼影”和明显拼接缝的产生,且图像平均梯度提高2.38%,拼接时间提高32.5%,使得融合区域更加平滑自然,提高了拼接质量。

农田环境下无人机图像并行拼接识别算法

为改善在农田环境下无人机图像计算速度和效率,该研究提出了一种农田环境下无人机图像并行拼接识别算法。利用倒二叉树并行拼接识别算法,通过提取图像拼接中的变换矩阵,实现拼接识别同时进行。根据边缘设备的CPU核心数和图像数量自动将图像拼接识别任务划分为多个子进程,并分配到不同核心上执行,以提高在农田环境下的计算效率。试验结果表明:相同试验环境和数据集条件下,倒二叉树并行拼接算法的拼接耗时相较于其他算法平均减少了60%~90%左右;在农田环境下,倒二叉树并行拼接识别相较于串行拼接识别的耗时减少了70%,图像识别的平均像素交并比提升了10.17个百分点,说明在农田环境下采用多线程倒二叉树并行算法可以更好地利用农田环境下边缘设备的计算资源,大幅提升无人机图像的拼接和识别的速度,为无人机的快速实时监测提供技术支撑。

“图文缝合”:文学接受理论的当代探绎

“图文缝合”是指文学接受中基于语象互文与读者“期待视域”引导效应,文本表述与读者接受图文寓意缝合的文学现象。“图文缝合”研究的终极目标,意在揭示文学接受差异可控性的逻辑规律。文章整合叙事学与接受美学相关理论,以福克纳经典作品的图文缝合现象为研究介质,对语言与图像间互动缝合产生的文学效应展开深入探绎。本文旨在探赜文学图文缝合的形成规律与实现途径,拓展文学接受研究视域,促进当代文学批评理论的丰富与发展。

基于半投影翘曲的无人机图像拼接方法

为了解决无人机图像在拼接过程中出现的错位、重影等造成图像失真的问题,提出一种基于半投影翘曲的无人机图像拼接方法。首先估计图像的全局投影变换矩阵;然后根据待拼接图像的重叠情况,以及投影变换和相似性变换的连续性,求解出过渡变换矩阵和相似性变换矩阵,得到最终的半投影变换矩阵;最后构建图像重叠区域的差异矩阵,以此为基础获取重叠区域的差异性区域。使用分区融合策略,在差异性区域进行单采样,在其他区域进行距离加权融合。实验结果表明,该方法可以很好地拼接由于视角变化、地势起伏等造成视差的无人机图像,得到的拼接结果自然清晰,效果优于其他先进算法。该方法有效地解决了无人机图像拼接错位、重影的问题,在多项定量指标上表现良好。

弱纹理飞机蒙皮曲面图像特征匹配及拼接

为了解决弱纹理飞机蒙皮特征点分布不均匀、正确匹配的特征点对较少的问题,提出了一种改进的LoFTR算法对飞机蒙皮图像进行拼接。根据相机位姿利用柱面反投影对蒙皮图像进行曲面校正;通过图像之间的重叠区域确定特征提取区域,从而减少错误匹配点对的生成;使用LoFTR算法进行特征提取,并且使用RANSAC算法对特征点进行筛选;根据图像分块的思想对重叠区域进行网格划分来对特征点进一步筛选,使得特征点分布更加均匀,得到更加准确的变换矩阵进行图像配准。实验在自研无人车采集的飞机蒙皮图像上进行了测试和验证,改进的方法与SIFT、SURF、ORB、BRISK以及AKAZE进行了特征匹配率比较实验,SIFT、SURF、ORB、BRISK和AKAZE匹配率分别为4.84%,0.47%、2.9%、0.86%、5.08%,提出的算法特征匹配率达到55.21%,SSIM平均值提高了44.38%~88.46%。该方法适用于对飞机蒙皮图像的拼接任务,且不存在因弱纹理而导致漏拼的问题。

基于全景视觉的无人船水面障碍物检测方法

无人船航行时水面障碍物检测因视角不足,导致漏检或误检,同时为满足无人船安全正常作业的需求,提出基于全景视觉的无人船水面障碍物目标检测方法。与传统的单目和双目视觉相比,全景视觉具有水平方向大视场监控的优点。基于多目全景视觉系统获得待拼接图像,在加速稳健特征(SURF)算法的基础上进行图像配准,引入k维树来构建数据索引,实现搜索空间级分类并进行快速匹配。通过M估计样本一致算法对匹配点进行优化,剔除误匹配点。对于图像融合中重叠区域出现的拼接缝隙或重影问题,设计一种基于圆弧函数的加权融合算法。提出改进的水面障碍物目标检测模型DS-YOLOv5s,将拼接好的全景图像作为训练好的模型作为输入,从而检测目标障碍物。实验结果表明,改进后的SURF算法与SURF算法相比特征点的匹配正确率提高11.47个百分点,在匹配时间上比SURF、RANSAC算法缩短5.83 s,DS-YOLOv5s模型的mAP@0.5达到95.7%,检测速度为51帧/s,符合实时目标检测标准。

基于深度学习的图像拼接算法研究综述

图像拼接是计算机视觉和计算机图形学中的一个重要分支,在三维成像等方面具有广泛的应用。相较于传统基于特征点检测的图像拼接框架,基于深度学习的图像拼接框架具有更强的场景泛化表现。目前虽然关于基于深度学习的图像拼接研究成果众多,但仍缺少相应研究的全面分析和总结。为了便于该领域后续工作的开展,梳理了该领域近10年的代表性成果。在对传统拼接方法与基于深度学习的图像拼接方法对比的基础上,从图像拼接研究领域中的单应性估计、图像拼接和图像矩形化三个子问题出发,进行了学习策略及模型架构设计、经典模型回顾、数据集等方面的整理与分析。总结了基于深度学习的图像拼接研究方法的一些特点和当前该领域的研究现状,并对未来研究前景进行了展望。

一种快速鲁棒性的视差图像拼接方法

当小型旋翼无人机超低空飞行时,不同位置拍摄的图像中极易出现结构视差。传统基于全局投影的拼接方法在处理视差图像时会出现伪影或畸变。最近的自适应拼接方法,如As-Projective-As-Possible(APAP)及其优化方法,使用“网格”来构建密集的局部投影场域,以改善视差图像拼接的对齐度和自然度,但计算复杂度较大。为此,提出了一种新颖的视差图像拼接方法,以深度稠密投影(Deep Dense Projection,Deep-DP)为特征匹配框架,设计了一种轻量化并行计算框架求解平滑形变场,以实现视差图像无缝拼接。在对比试验中,该算法平均运行时间达到0.2 s,显著优于其他先进算法,并且对齐质量和自然度达到了相近水平。提出的算法在兼顾图像拼接准确性的同时,计算效率显著提高。

大区域场景下基于无人机视角的目标计数方法

近年来,无人机因其灵活度高、机动性强在人群计数领域得到广泛应用。然而,现有的人群计数方法大多基于单视点,对于大范围、多摄像机场景下的多视点计数研究较少。为了解决这个问题,提出了一种基于无人机视角的目标计数方法以准确统计场景中的目标数量。选择临海区域进行数据采集,利用深度学习技术对采集的图像进行目标检测和图像拼接融合,在拼接后的图像中映射检测信息,并采用计数算法完成区域场景的计数任务。在公开数据集和该文制作的数据集上进行的实验验证了基于目标检测的计数算法的有效性。

基于图像矫正与小波变换融合的岩心图像拼接算法

为了提高岩心图像拼接的效率,针对在岩心采集出现的图像中的像素错位现象以及图像拼接处理出现的亮度差异、鬼影等问题,提出基于图像矫正与小波变换融合的岩心图像融合算法。首先改进AKAZE算法提高提取效率,对采集图像进行图像矫正得到矫正图像;再对矫正后的待拼接图像进行特征点提取与匹配,得到2幅图像之间配准关系;最后通过基于HSI域的小波融合与改进最佳缝合线融合得到拼接图像。实验结果表明,本文提出的矫正与拼接算法满足实际需求,图像无明显上下拼接缝与错位,且在DoEM和PSNR指标上与传统最佳缝合线和改进的渐入渐出算法相比均有所提升,融合区域避免了鬼影且亮度过渡均匀,无明显左右缝合线产生,拼接的图像符合人眼视觉。