CMMCAN:Lightweight Feature Extraction and Matching Network for Endoscopic Images Based on Adaptive Attention

In minimally invasive surgery,endoscopes or laparoscopes equipped with miniature cameras and tools are used to enter the human body for therapeutic purposes through small incisions or natural cavities.However,in clinical operating environments,endoscopic images often suffer from challenges such as low texture,uneven illumination,and non-rigid structures,which affect feature observation and extraction.This can severely impact surgical navigation or clinical diagnosis due to missing feature points in endoscopic images,leading to treatment and postoperative recovery issues for patients.To address these challenges,this paper introduces,for the first time,a Cross-Channel Multi-Modal Adaptive Spatial Feature Fusion(ASFF)module based on the lightweight architecture of EfficientViT.Additionally,a novel lightweight feature extraction and matching network based on attention mechanism is proposed.This network dynamically adjusts attention weights for cross-modal information from grayscale images and optical flow images through a dual-branch Siamese network.It extracts static and dynamic information features ranging from low-level to high-level,and from local to global,ensuring robust feature extraction across different widths,noise levels,and blur scenarios.Global and local matching are performed through a multi-level cascaded attention mechanism,with cross-channel attention introduced to simultaneously extract low-level and high-level features.Extensive ablation experiments and comparative studies are conducted on the HyperKvasir,EAD,M2caiSeg,CVC-ClinicDB,and UCL synthetic datasets.Experimental results demonstrate that the proposed network improves upon the baseline EfficientViT-B3 model by 75.4%in accuracy(Acc),while also enhancing runtime performance and storage efficiency.When compared with the complex DenseDescriptor feature extraction network,the difference in Acc is less than 7.22%,and IoU calculation results on specific datasets outperform complex dense models.Furthermore,this method increases the F1 score by 33.2%and accelerates runtime by 70.2%.It is noteworthy that the speed of CMMCAN surpasses that of comparative lightweight models,with feature extraction and matching performance comparable to existing complex models but with faster speed and higher cost-effectiveness.

Road Traffic Monitoring from Aerial Images Using Template Matching and Invariant Features

Road traffic monitoring is an imperative topic widely discussed among researchers.Systems used to monitor traffic frequently rely on cameras mounted on bridges or roadsides.However,aerial images provide the flexibility to use mobile platforms to detect the location and motion of the vehicle over a larger area.To this end,different models have shown the ability to recognize and track vehicles.However,these methods are not mature enough to produce accurate results in complex road scenes.Therefore,this paper presents an algorithm that combines state-of-the-art techniques for identifying and tracking vehicles in conjunction with image bursts.The extracted frames were converted to grayscale,followed by the application of a georeferencing algorithm to embed coordinate information into the images.The masking technique eliminated irrelevant data and reduced the computational cost of the overall monitoring system.Next,Sobel edge detection combined with Canny edge detection and Hough line transform has been applied for noise reduction.After preprocessing,the blob detection algorithm helped detect the vehicles.Vehicles of varying sizes have been detected by implementing a dynamic thresholding scheme.Detection was done on the first image of every burst.Then,to track vehicles,the model of each vehicle was made to find its matches in the succeeding images using the template matching algorithm.To further improve the tracking accuracy by incorporating motion information,Scale Invariant Feature Transform(SIFT)features have been used to find the best possible match among multiple matches.An accuracy rate of 87%for detection and 80%accuracy for tracking in the A1 Motorway Netherland dataset has been achieved.For the Vehicle Aerial Imaging from Drone(VAID)dataset,an accuracy rate of 86%for detection and 78%accuracy for tracking has been achieved.

Image Feature Extraction and Matching of Augmented Solar Images in Space Weather

Augmented solar images were used to research the adaptability of four representative image extraction and matching algorithms in space weather domain.These include the scale-invariant feature transform algorithm,speeded-up robust features algorithm,binary robust invariant scalable keypoints algorithm,and oriented fast and rotated brief algorithm.The performance of these algorithms was estimated in terms of matching accuracy,feature point richness,and running time.The experiment result showed that no algorithm achieved high accuracy while keeping low running time,and all algorithms are not suitable for image feature extraction and matching of augmented solar images.To solve this problem,an improved method was proposed by using two-frame matching to utilize the accuracy advantage of the scale-invariant feature transform algorithm and the speed advantage of the oriented fast and rotated brief algorithm.Furthermore,our method and the four representative algorithms were applied to augmented solar images.Our application experiments proved that our method achieved a similar high recognition rate to the scale-invariant feature transform algorithm which is significantly higher than other algorithms.Our method also obtained a similar low running time to the oriented fast and rotated brief algorithm,which is significantly lower than other algorithms.

Fast matching pursuit for traffic images using differential evolution

To obtain the sparse decomposition and flexible representation of traffic images,this paper proposes a fast matching pursuit for traffic images using differential evolution. According to the structural features of traffic images,the introduced algorithm selects the image atoms in a fast and flexible way from an over-complete image dictionary to adaptively match the local structures of traffic images and therefore to implement the sparse decomposition. As compared with the traditional method and a genetic algorithm of matching pursuit by using extensive experiments,the differential evolution achieves much higher quality of traffic images with much less computational time,which indicates the effectiveness of the proposed algorithm.

A fast, accurate and dense feature matching algorithm for aerial images

Three-dimensional(3D)reconstruction based on aerial images has broad prospects,and feature matching is an important step of it.However,for high-resolution aerial images,there are usually problems such as long time,mismatching and sparse feature pairs using traditional algorithms.Therefore,an algorithm is proposed to realize fast,accurate and dense feature matching.The algorithm consists of four steps.Firstly,we achieve a balance between the feature matching time and the number of matching pairs by appropriately reducing the image resolution.Secondly,to realize further screening of the mismatches,a feature screening algorithm based on similarity judgment or local optimization is proposed.Thirdly,to make the algorithm more widely applicable,we combine the results of different algorithms to get dense results.Finally,all matching feature pairs in the low-resolution images are restored to the original images.Comparisons between the original algorithms and our algorithm show that the proposed algorithm can effectively reduce the matching time,screen out the mismatches,and improve the number of matches.

2D registration based on contour matching for partial matching images

The mean Hausdorff distance, though highly applicable in image registration, does not work well on partial matching images. An improvement upon traditional Hausdorff-distance-based image registration method is proposed, which consists of the following two aspects. One is to estimate transformation parameters between two images from the distributions of geometric property differences instead of establishing explicit feature correspondences. This procedure is treated as the pre-registration. The other aspect is that mean Hausdorff distance computation is replaced with the analysis of the second difference of generalized Hausdorff distance so as to eliminate the redundant points. Experimental results show that our registration method outperforms the method based on mean Hausdorff distance. The registration errors are noticeably reduced in the partial matching images.

A Fast Algorithm for Matching Remote Scene Images

An iterative algorithm to calculate mutual correlation using hierarchical key points and the search space mark principle is proposed. An effective algorithm is designed to improve the matching speed. By hi-erarchical key point algorithm and mutual correlation coefficients of the matching images, the important points can be iteratively calculated in the images hierarchically, and the correlation coefficient can be ob-tained with satisfactory precision. Massive spots in the parameter space which are impossible to match can be removed by the search space mark principle. Two approximate continuities in the correlation image matching process, the image gray level distribution continuity and the correlation coefficient value in the parameter space continuity, are considered in the method. The experiments show that the new algorithm can greatly enhance matching speed and achieve accurate matching results.

Improved HardNet and Stricter Outlier Filtering to Guide Reliable Matching

As the fundamental problem in the computer vision area,image matching has wide applications in pose estimation,3D reconstruction,image retrieval,etc.Suffering from the influence of external factors,the process of image matching using classical local detectors,e.g.,scale-invariant feature transform(SIFT),and the outlier filtering approaches,e.g.,Random sample consensus(RANSAC),show high computation speed and pool robustness under changing illumination and viewpoints conditions,while image matching approaches with deep learning strategy(such as HardNet,OANet)display reliable achievements in large-scale datasets with challenging scenes.However,the past learning-based approaches are limited to the distinction and quality of the dataset and the training strategy in the image-matching approaches.As an extension of the previous conference paper,this paper proposes an accurate and robust image matching approach using fewer training data in an end-to-end manner,which could be used to estimate the pose error This research first proposes a novel dataset cleaning and construction strategy to eliminate the noise and improve the training efficiency;Secondly,a novel loss named quadratic hinge triplet loss(QHT)is proposed to gather more effective and stable feature matching;Thirdly,in the outlier filtering process,the stricter OANet and bundle adjustment are applied for judging samples by adding the epipolar distance constraint and triangulation constraint to generate more outstanding matches;Finally,to recall the matching pairs,dynamic guided matching is used and then submit the inliers after the PyRANSAC process.Multiple evaluation metrics are used and reported in the 1st place in the Track1 of CVPR Image-Matching Challenge Workshop.The results show that the proposed method has advanced performance in large-scale and challenging Phototourism benchmark.

A Novel Fuzzy Inference System-Based Endmember Extraction in Hyperspectral Images

Spectral unmixing helps to identify different components present in the spectral mixtures which occur in the uppermost layer of the area owing to the low spatial resolution of hyperspectral images.Most spectral unmixing methods are globally based and do not consider the spectral variability among its endmembers that occur due to illumination,atmospheric,and environmental conditions.Here,endmember bundle extraction plays a major role in overcoming the above-mentioned limitations leading to more accurate abundance fractions.Accordingly,a two-stage approach is proposed to extract endmembers through endmember bundles in hyperspectral images.The divide and conquer method is applied as the first step in subset images with only the non-redundant bands to extract endmembers using the Vertex Component Analysis(VCA)and N-FINDR algorithms.A fuzzy rule-based inference system utilizing spectral matching parameters is proposed in the second step to categorize endmembers.The endmember with the minimum error is chosen as the final endmember in each specific category.The proposed method is simple and automatically considers endmember variability in hyperspectral images.The efficiency of the proposed method is evaluated using two real hyperspectral datasets.The average spectral angle and abundance angle are used to analyze the performance measures.

Outliers rejection in similar image matching

Background Image matching is crucial in numerous computer vision tasks such as 3D reconstruction and simultaneous visual localization and mapping.The accuracy of the matching significantly impacted subsequent studies.Because of their local similarity,when image pairs contain comparable patterns but feature pairs are positioned differently,incorrect recognition can occur as global motion consistency is disregarded.Methods This study proposes an image-matching filtering algorithm based on global motion consistency.It can be used as a subsequent matching filter for the initial matching results generated by other matching algorithms based on the principle of motion smoothness.A particular matching algorithm can first be used to perform the initial matching;then,the rotation and movement information of the global feature vectors are combined to effectively identify outlier matches.The principle is that if the matching result is accurate,the feature vectors formed by any matched point should have similar rotation angles and moving distances.Thus,global motion direction and global motion distance consistencies were used to reject outliers caused by similar patterns in different locations.Results Four datasets were used to test the effectiveness of the proposed method.Three datasets with similar patterns in different locations were used to test the results for similar images that could easily be incorrectly matched by other algorithms,and one commonly used dataset was used to test the results for the general image-matching problem.The experimental results suggest that the proposed method is more accurate than other state-of-the-art algorithms in identifying mismatches in the initial matching set.Conclusions The proposed outlier rejection matching method can significantly improve the matching accuracy for similar images with locally similar feature pairs in different locations and can provide more accurate matching results for subsequent computer vision tasks.

工业机械臂辅助机器视觉的工件识别与定位

传统的工业机器人系统由于缺乏感知判断能力,很难适应目标种类和位置的变化,而现代工业需要生产不同种类和规格的产品。为此,以多品种小批量生产的零件为背景提出了一种机械臂辅助机器视觉系统,该系统能够实现对工件的图像采集并进行预处理,通过改进的Canny轮廓的提取算法与基于Hausdorff距离的模版匹配完成对不同种类工件进行识别和定位。最后经实验的验证该系统的定位精度在0.35 mm内,基本满足了在实际生产中对小批量生产、不同形状且没有固定夹具的目标工件的抓取、搬运与检测等要求。

基于FPGA的小目标识别分类系统的设计与实现

为了提高小目标识别和分类的实时性,同时降低识别系统的资源消耗,本文提出了一种简易、高效的现场可编程门阵列(Field Programmable Gate Array,FPGA)小目标识别分类系统。该系统首先通过图像预处理消除图像噪点,并采用并行计算提升系统实时性。然后将处理后的图像与模板进行匹配计算得到识别结果,设计的模板匹配电路具有较小的硬件复杂度和较快的处理速度。实验结果表明,本文所提出的识别系统在680×480图像分辨下,可达137.5帧/s的处理速度,实时性强,同时仅消耗了9个块随机存储器(Block Random Access Memory,BRAM)和2个数字信号处理器(Digital Signal Processor,DSP),硬件资源消耗较少,在处理小目标识别和分类问题上有较好的实用价值。

基于几何约束的倾斜影像特征匹配方法

针对倾斜影像视角变换较大、重复纹理导致匹配数量少、匹配精度不高的问题,提出一种适用于倾斜影像的特征点、线分级匹配方法.首先,用直线提取(检测)算法(LSD)获取影像直线特征,并将直线特征以一定约束进行直线组对,构建直线对区域与改进的SIFT(scale-invariant feature transform)特征描述符进行匹配,使用RANSAC算法剔除误匹配,获得初始匹配结果后再进行核线约束;然后,利用已获得直线对区域进行影像局部纠正,在纠正后的局部影像上采用SIFT匹配并反算回原始影像,利用得到的同名点全局纠正倾斜影像,并进行特征点匹配与采用基于方格的运动统计算法(GMS)剔除误匹配,仍将匹配结果反算回原始影像上;最后,将仿射尺度不变特征变化结果与点拓展匹配结果进行合并,得到最终匹配结果.试验结果表明:本文方法匹配正确率与经典的仿射不变匹算法(ASIFT)的正确率相差不大,但匹配数量却是ASIFT算法的1倍~3倍.

面向无人机绝对定位的遥感影像快速检索方法

针对在复杂环境下无人机景象匹配导航中的视觉绝对定位问题,提出了一种聚合深度学习特征的实时影像快速检索方法。首先,引入可训练软分配深度学习框架—NetVLAD,结合VGG16网络提取并聚合生成影像稳定的全局特征表达向量;其次,在初始检索阶段,使用KD树结构对影像全局特征向量构建检索索引,在不损失检索精度的前提下提高检索速度;最后,使用皮尔逊积矩相关系数对初始检索结果进行快速预判断,自动过滤初始检索结果,对于需要重排序的影像则采用特征学习匹配算法——图神经网络SuperGlue进行匹配重排序。所提方法在公开的夏季和冬季遥感影像数据集分组进行实验,实验结果表明:未重排序条件下,初始检索结果第一张影像平均准确率达到了58.27%,部分特征较好地区准确率达到了85%,对不同时相遥感影像也有很好的适应性,平均检索一张影像耗时3.7 s,可为无人机景象匹配导航的初始定位提供参考。

车载激光点云道路标线分类提取方法

利用车载激光道路点云提取道路标线的难度较大。针对此问题,文章采用一种改进的基于点云特征图像的道路标线分类提取方法。首先将道路点云投影生成点云特征图像,通过结合图像梯度分析、图像二值化和连通域分析等操作,进一步进行道路标线像素提取;然后反投影到三维点云后,利用高斯混合模型对道路标线精细优化,从而提取出完整的道路标线点云;最后通过模板匹配分类,对道路标线点云进行分类提取。实验结果表明,该方法对不同道路环境下道路标线提取的准确度、完整度以及综合评价都超过90%。

基于直线段检测和LT描述符的矿井图像线特征匹配算法

图像匹配是同步定位与地图构建(SLAM)技术中极为重要的一环,用于根据图像之间的变换关系确定相机位姿。基于线特征的图像匹配方法具有较强的鲁棒性和抗噪能力,更加适用于井下图像匹配,基于深度学习的线描述符对线段遮挡等场景具有较高的鲁棒性,性能优于传统描述符,但卷积神经网络架构的描述符将可变长度线段抽象为固定维进行描述,不利于线段长度及视差变化较大图像的匹配。针对上述问题,提出一种基于直线段检测和线描述符的矿井图像线特征匹配算法。在频域利用单参数同态滤波降低图像的照射分量,并增强反射分量,提升亮度及对比度;在YUV空间利用对比度受限的自适应直方图均衡化(CLAHE)算法对亮度分量进行均衡,使亮度分布更加均匀;变换至RGB空间提取直线段检测(LSD)线,引入一种基于Transformer架构的LT描述符构建LSD线的特征向量,最后完成线特征匹配。实验结果表明:该算法结合了同态滤波和CLAHE算法的优点,增强后图像的亮度适中,对比度良好,灰度分布均匀,增强效果优于单参数同态滤波算法、EnlightenGAN算法;该算法提取的线特征数较原图平均提升了32.92%,在不同相似纹理占比、不同程度旋转与平移变化的井下图像匹配中鲁棒性好,平均正确匹配数为61.75对,平均精度为86.83%,优于线二进制描述符(LBD)算法、LBD_NNDR算法、LT算法,能够满足矿井图像稳健匹配的需求。

虚实坐标智能匹配与自适应三维轨迹规划

针对数字孪生增材修复任务中,物件仿真与真实坐标对应困难、三维轨迹误差较大的问题,提出虚实坐标智能匹配与自适应三维轨迹规划方法。采用图像分割方法获取实际场景下物件的分割图像,通过最小外接矩形预估物件真实坐标,仿真端据此执行旋转与平移操作,调整仿真物件的模型位置,实现坐标智能匹配;为最小化虚实坐标误差,设计基于Attention的多尺度残差UNet图像分割模型,利用多尺度残差模块获取物件多尺度特征,通过Attention策略增强网络对位置信息的描述。三维轨迹规划部分,设计融合特征高度的自适应分层算法,减少分层高度偏移,缩减阶梯误差;根据分层结果进行轮廓提取、填充,生成三维轨迹。结果表明,坐标匹配误差小于1mm,三维修复轨迹规划质量较高。

基于声全息法和单目视觉技术的柴油机噪声可视化实现

柴油机属于多噪声耦合动力装置,其噪声的识别和控制是内燃机研究领域的难点。为实现柴油机噪声的可视化,本研究结合了近场声全息法与单目视觉技术,开发了基于Labview的声像匹配模块,测试系统人机界面友好,并通过已知声源试验验证了声像匹配模块的正确性。将验证后的测试系统应用于高压共轨柴油机对主/次推力侧声源进行识别,结果表明:在大气压力为80kPa、转速为1800r/min的最大扭矩工况下,主推力侧出现较大辐射噪声的位置为中冷器进气管、曲轴定时齿形带轮、排气管、脚架和涡轮增压器等;次推力侧辐射噪声峰值出现在起动机位置。

联合多波束与侧扫声纳的海缆检测方法

为解决海底电缆路由检测中多波束扫测分辨率低、侧扫声纳(SSS)图像位置不准且存在误识别问题,提出了一种联合多波束与侧扫声纳的海底电缆路由自动检测方法,利用多波束图像高精度的位置信息纠正侧扫声纳图像,并顾及侧扫声纳图像海缆成像特点进行线状特征增强与边缘提取,实现海底电缆目标的自动、准确检测。实验结果表明,该方法能实现非掩埋海缆探测的准确探测,为海底电缆检测提供了多源测量信息融合的新思路,丰富了海底电缆检测的技术方法。

融合语义增强和位置编码的图文匹配方法

图文匹配是跨模态基础任务之一,其核心是如何准确评估图像语义与文本语义之间的相似度。现有方法是通过引入相关阈值,最大限度地区分相关和无关分布,以获得更好的语义对齐。然而,对于特征本身,其语义之间缺乏相互关联,且对于缺乏空间位置信息的图像区域与文本单词很难准确对齐,从而不可避免地限制了相关阈值的学习导致语义无法准确对齐。针对此问题,本文提出一种融合语义增强和位置编码的自适应相关性可学习注意力的图文匹配方法。首先,在初步提取特征的基础上构造图像(文本)无向全连通图,使用图注意力去聚合邻居的信息,获得语义增强的特征。然后,对图像区域的绝对位置信息编码,在具备了空间语义的图像区域与文本单词相似性的基础上获得最大程度区分的相关和无关分布,更好地学习两个分布之间的最优相关边界。最后,通过公开数据集Flickr 30 k和MSCOCO,利用Recall@K指标对比实验,验证本文方法的有效性。