Efficient Unsupervised Image Stitching Using Attention Mechanism with Deep Homography Estimation

Traditional feature-based image stitching techniques often encounter obstacles when dealing with images lackingunique attributes or suffering from quality degradation. The scarcity of annotated datasets in real-life scenesseverely undermines the reliability of supervised learning methods in image stitching. Furthermore, existing deeplearning architectures designed for image stitching are often too bulky to be deployed on mobile and peripheralcomputing devices. To address these challenges, this study proposes a novel unsupervised image stitching methodbased on the YOLOv8 (You Only Look Once version 8) framework that introduces deep homography networksand attentionmechanisms. Themethodology is partitioned into three distinct stages. The initial stage combines theattention mechanism with a pooling pyramid model to enhance the detection and recognition of compact objectsin images, the task of the deep homography networks module is to estimate the global homography of the inputimages consideringmultiple viewpoints. The second stage involves preliminary stitching of the masks generated inthe initial stage and further enhancement through weighted computation to eliminate common stitching artifacts.The final stage is characterized by adaptive reconstruction and careful refinement of the initial stitching results.Comprehensive experiments acrossmultiple datasets are executed tometiculously assess the proposed model. Ourmethod’s Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity Index Measure (SSIM) improved by 10.6%and 6%. These experimental results confirm the efficacy and utility of the presented model in this paper.

Underwater Terrain Image Stitching Based on Spatial Gradient Feature Block

At present,underwater terrain images are all strip-shaped small fragment images preprocessed by the side-scan sonar imaging system.However,the processed underwater terrain images have inconspicuous and few feature points.In order to better realize the stitching of underwater terrain images and solve the problems of slow traditional image stitching speed,we proposed an improved algorithm for underwater terrain image stitching based on spatial gradient feature block.First,the spatial gradient fuzzy C-Means algorithm is used to divide the underwater terrain image into feature blocks with the fusion of spatial gradient information.The accelerated-KAZE(AKAZE)algorithm is used to combine the feature block information to match the reference image and the target image.Then,the random sample consensus(RANSAC)is applied to optimize the matching results.Finally,image fusion is performed with the global homography and the optimal seam-line method to improve the accuracy of image overlay fusion.The experimental results show that the proposed method in this paper effectively divides images into feature blocks by combining spatial information and gradient information,which not only solves the problem of stitching failure of underwater terrain images due to unobvious features,and further reduces the sensitivity to noise,but also effectively reduces the iterative calculation in the feature point matching process of the traditional method,and improves the stitching speed.Ghosting and shape warping are significantly eliminated by re-optimizing the overlap of the image.

Unsupervised Oral Endoscope Image Stitching Algorithm

Oral endoscope image stitching algorithm is studied to obtain wide-field oral images through regis-tration and stitching,which is of great significance for auxiliary diagnosis.Compared with natural images,oral images have lower textures and fewer features.However,traditional feature-based image stitching methods rely heavily on feature extraction quality,often showing an unsatisfactory performance when stitching images with few features.Moreover,due to the hand-held shooting,there are large depth and perspective disparities between the captured images,which also pose a challenge to image stitching.To overcome the above problems,we propose an unsupervised oral endoscope image stitching algorithm based on the extraction of overlapping regions and the loss of deep features.In the registration stage,we extract the overlapping region of the input images by sketching polygon intersection for feature points screening and estimate homography from coarse to fine on a three-layer feature pyramid structure.Moreover,we calculate loss using deep features instead of pixel values to emphasize the importance of depth disparities in homography estimation.Finally,we reconstruct the stitched images from feature to pixel,which can eliminate artifacts caused by large parallax.Our method is compared with both feature-based and previous deep-based methods on the UDIS-D dataset and our oral endoscopy image dataset.The experimental results show that our algorithm can achieve higher homography estimation accuracy,and better visual quality,and can be effectively applied to oral endoscope image stitching.

Combined application of extended depth of field imaging, image stitching and polarized microscopy techniques in identification of Spatholobus suberectus

Objective:As traditional techniques for microscopic identification of Chinese medicines currently lack objective and high-quality reference images,here we developed a systemic procedure to be used in microscopic identification of Chinese medicines,which would lead to more objective,effective and accurate identification process.Methods:Spatholobi Caulis(Jixueteng in Chinese)was used as the specimen in the development of such procedure.Jixueteng samples were microscopically examined in bright-and dark-field microscopy.Microscopic images were obtained by regular,EDF,and image stitching techniques.Results:The microscopic images of the characteristics in pulverized Jixueteng were captured,thanks to EDF imaging and image stitching techniques which allowed the detailed and full sighting of each characteristic to be obtained simultaneously.Different layers in anatomical transverse section,including cork,phelloderm,cortex,phloem,cambium,xylem and pith,were distinctively observed.Moreover,by comparing images of bright-and dark-field microscopy,birefringent and non-birefringent components could readily be distinguished.Conclusion:With application of the developed procedure,high-definition,panoramic and microscopic images were acquired,which could be used as the reference images for microscopic identification of Chinese medicines.

A survey on image and video stitching

Image/video stitching is a technology for solving the field of view(FOV)limitation of images/videos.It stitches multiple overlapping images/videos to generate a wide-FOV image/video,and has been used in various fields such as sports broadcasting,video surveillance,street view,and entertainment.This survey reviews image/video stitching algorithms,with a particular focus on those developed in recent years.Image stitching first calculates the corresponding relationships between multiple overlapping images,deforms and aligns the matched images,and then blends the aligned images to generate a wide-FOV image.A seamless method is always adopted to eliminate such potential flaws as ghosting and blurring caused by parallax or objects moving across the overlapping regions.Video stitching is the further extension of image stitching.It usually stitches selected frames of original videos to generate a stitching template by performing image stitching algorithms,and the subsequent frames can then be stitched according to the template.Video stitching is more complicated with moving objects or violent camera movement,because these factors introduce jitter,shakiness,ghosting,and blurring.Foreground detection technique is usually combined into stitching to eliminate ghosting and blurring,while video stabilization algorithms are adopted to solve the jitter and shakiness.This paper further discusses panoramic stitching as a special-extension of image/video stitching.Panoramic stitching is currently the most widely used application in stitching.This survey reviews the latest image/video stitching methods,and introduces the fundamental principles/advantages/weaknesses of image/video stitching algorithms.Image/video stitching faces long-term challenges such as wide baseline,large parallax,and low-texture problem in the overlapping region.New technologies may present new opportunities to address these issues,such as deep learning-based semantic correspondence,and 3D image stitching.Finally,this survey discusses the challenges of image/video stitching and proposes potential solutions.

RecStitchNet:Learning to stitch images with rectangular boundaries

Irregular boundaries in image stitching naturally occur due to freely moving cameras.To deal with this problem,existing methods focus on optimizing mesh warping to make boundaries regular using the traditional explicit solution.However,previous methods always depend on hand-crafted features(e.g.,keypoints and line segments).Thus,failures often happen in overlapping regions without distinctive features.In this paper,we address this problem by proposing RecStitchNet,a reasonable and effective network for image stitching with rectangular boundaries.Considering that both stitching and imposing rectangularity are non-trivial tasks in the learning-based framework,we propose a three-step progressive learning based strategy,which not only simplifies this task,but gradually achieves a good balance between stitching and imposing rectangularity.In the first step,we perform initial stitching by a pre-trained state-of-the-art image stitching model,to produce initially warped stitching results without considering the boundary constraint.Then,we use a regression network with a comprehensive objective regarding mesh,perception,and shape to further encourage the stitched meshes to have rectangular boundaries with high content fidelity.Finally,we propose an unsupervised instance-wise optimization strategy to refine the stitched meshes iteratively,which can effectively improve the stitching results in terms of feature alignment,as well as boundary and structure preservation.Due to the lack of stitching datasets and the difficulty of label generation,we propose to generate a stitching dataset with rectangular stitched images as pseudo-ground-truth labels,and the performance upper bound induced from the it can be broken by our unsupervised refinement.Qualitative and quantitative results and evaluations demonstrate the advantages of our method over the state-of-the-art.

Automatic Seamless Stitching Method for CCD Images of Chang’E-1 Lunar Mission

A novel automatic seamless stitching method is presented. Compared to the traditional method, it can speed the processing and minimize the utilization of human resources to produce global lunar map. Meanwhile, a new global image map of the Moon with spatial resolution of -120 m has been completed by the proposed method from Chang＇E-1 CCD image data.

Automatic Stitching Method for Chang＇E-2 CCD Images of the Moon

The lunar map is a product of primary scientific objectives of lunar exploration. Aiming at the characteristics of the Chang＇E-2 CCD data, an automatic stitching method used for 2C level CCD data from Chang＇E-2 lunar mission is proposed. Combining with the image registration technique and the characteristics of Chang＇E CCD images, the fast method proposed not only can overcome the contradiction of the high spatial resolution of the CCD images and the low positioning accuracy of the location coordinates, but also can speed up the processing and minimize the utilization of human resources to produce lunar mosaic map. Meanwhile, a new lunar map from 70oN to 70oS with spatial resolution of less than 10 m has been completed by the proposed method. Its average relative location accuracy of the adjacent orbits CCD image data is less than 3 pixels.

Perceptual quality assessment of panoramic stitched contents for immersive applications:a prospective survey

The recent advancements in the field of Virtual Reality(VR)and Augmented Reality(AR)have a substantial impact on modern day technology by digitizing each and everything related to human life and open the doors to the next generation Software Technology(Soft Tech).VR and AR technology provide astonishing immersive contents with the help of high quality stitched panoramic contents and 360°imagery that widely used in the education,gaming,entertainment,and production sector.The immersive quality of VR and AR contents are greatly dependent on the perceptual quality of panoramic or 360°images,in fact a minor visual distortion can significantly degrade the overall quality.Thus,to ensure the quality of constructed panoramic contents for VR and AR applications,numerous Stitched Image Quality Assessment(SIQA)methods have been proposed to assess the quality of panoramic contents before using in VR and AR.In this survey,we provide a detailed overview of the SIQA literature and exclusively focus on objective SIQA methods presented till date.For better understanding,the objective SIQA methods are classified into two classes namely Full-Reference SIQA and No-Reference SIQA approaches.Each class is further categorized into traditional and deep learning-based methods and examined their performance for SIQA task.Further,we shortlist the publicly available benchmark SIQA datasets and evaluation metrices used for quality assessment of panoramic contents.In last,we highlight the current challenges in this area based on the existing SIQA methods and suggest future research directions that need to be target for further improvement in SIQA domain.

Short-working-distance optical imaging system and method for surface detection of underwater structures

In the surface imaging of underwater structures, long working distance will reduce image quality due to the turbidity of water. To acquire high definition and large field of view(FOV) images for surface detection, a short-working-distance underwater imaging system is proposed based on camera array. A multi-view calibration and rectification method is developed. A look-up table(LUT) method and a multi-resolution spline(MRS) method are applied to stitch array images real-time and seamlessly.Experiments both in the air and in the water are conducted. Strength and weakness of the LUT and MRS methods are discussed.Based on the results, the effectiveness in surface detection of underwater structures is verified.

Detecting large-scale underwater cracks based on remote operated vehicle and graph convolutional neural network

It is of great significance to quickly detect underwater cracks as they can seriously threaten the safety of underwater structures.Research to date has mainly focused on the detection of above-water-level cracks and hasn’t considered the large scale cracks.In this paper,a large-scale underwater crack examination method is proposed based on image stitching and segmentation.In addition,a purpose of this paper is to design a new convolution method to segment underwater images.An improved As-Projective-As-Possible(APAP)algorithm was designed to extract and stitch keyframes from videos.The graph convolutional neural network(GCN)was used to segment the stitched image.The GCN’s m-IOU is 24.02%higher than Fully convolutional networks(FCN),proving that GCN has great potential of application in image segmentation and underwater image processing.The result shows that the improved APAP algorithm and GCN can adapt to complex underwater environments and perform well in different study areas.

Corn ear test using SIFT-based panoramic photography and machine vision technology

Corn ear test is important to modern corn breeding.The test indexesmainly include lengths,radiuses,rows and numbers of corn ears and the kernels they bear,which can benefit the study on breeding new and fine corn varieties.These corn traits are often collected by traditional manual measurement,which is difficult to meet the needs of high throughput corn ear test.In this study,image sequences of corn ear samples were captured by building a panoramic photography collecting system.And then,to get the lengths and radiuses indexes,the corn area images were processed based on Lab color space and adaptive threshold segmentation.The sequence images were then matched and the panoramic image of a corn surface were extracted using Scale-invariant feature transform(SIFT).Finally,by using Exponential transformation(ETR)and Sobel-Hough algorithm,ears and rows indexes were acquired.Test results showed that the accuracy of the radiuses and lengths were 93.84%and 94.53%,respectively.Meanwhile,the accuracy of kernels and rows indexes were 98.12%and 96.14%,whichwere 4.03%and 7.25%higher than that of common mosaiced panoramic image.And the accuracy of kernel area and length-width ratio were 95.36%and 97.42%,respectively.All the results showed that the proposed method can be used for corn ear test effectively.