融合边缘与灰度特征的形变工件精准定位方法

Precise positioning method of deformed workpiece by fusing edge and grayscale features

目的工业机器人视觉领域经常需要对一些由拼装、冲压或贴合等工艺造成的形变工件进行精准定位,工件的大部分特征表现出一定程度的非刚性,其他具备良好一致性的部分通常特征简单,导致一些常用的目标检测算法精度不足或鲁棒性不强,难以满足实际需求。针对这一问题,提出融合边缘与灰度特征的形变工件精准定位方法。方法第1阶段提出多归一化互相关的模板匹配MNCC(multi normalized cross correlation)方法检测形变目标,利用余弦距离下的灰度聚类获得均值模板,通过滑动窗口的方式,结合金字塔跟踪,自顶向下地优先搜索类均值模板,得到类匹配候选,然后进行类内细搜索获得最佳位置匹配。第2阶段提出一种改进的形状匹配方法T-SBM(truncated shape-based matching),通过改变原始SBM(shape-based matching)的梯度方向内积的计算方式,对负梯度极性方向截断,削弱目标背景不稳定导致局部梯度方向反转时对整体评分的负贡献,改善边缘稀疏或特征简单导致检测鲁棒性低的问题。最后提出二维高斯条件密度评价,将灰度特征、形状特征和形变量进行综合加权,获得理想目标评价,实现序贯检测。结果实验部分分别与SBM、归一化互相关匹配算法(normalized cross correlation,NCC)、LINE2D(linearizing the memory 2D)算法和YOLOv5s(you only look once version 5 small)算法在5种类型工件的472幅真实工业图像上进行了对比测试,在检出分值大于0.8(实际常用的阈值区间)时,提出算法的召回率优于其他几种测试算法;在IoU(intersection over union)阈值0.9时的平均检测准确率为81.7%,F1-Score为95%,两组指标相比其他测试算法分别至少提升了10.8%和8.3%。在平均定位精度方面,提出算法的定位偏差在IoU阈值0.9时达到了2.44像素,在5种测试算法中的表现也为最佳。结论提出了一种两阶段的定位方法,该方法适用于检测工业场景中由拼装、冲压和贴合等工艺制成的形变工件并能够进行精准定位,尤其适用于工业机器人视觉引导定位应用场景,并在实际项目中得到了应用。

Objective In industrial robot vision,accurately detecting deformed workpieces caused by assembly,stamping,or lamination is often necessary.These workpieces sometimes show non-rigid characteristics,such as dislocation or twist deformation.Most features do not maintain good shape consistency,while the remaining undeformed parts are generally simple,for example,sparse edges,which are not globally unique.In addition,obtaining massive training samples before workpieces are mass produced is not realistic.Hence,some commonly used object detection methods have insufficient accuracy or weak robustness,challenging meeting the actual needs.Method To address the problem,a two-stage precise positioning method of deformed workpieces by fusing edge and grayscale features is proposed.The first stage is the coarse position detection of deformed targets based on grayscale features,and the second stage is precise positioning based on shape features.The innovation of the first stage lies in that a multi normalized cross correlation(MNCC)matching method is proposed,which includes offline and online parts.In the offline part,the grayscale clustering algorithm at cosine distance is used to obtain the class-mean template,which characterizes a class center of similar features in the target deformation space.Therefore,fewer class-mean templates can be used to represent the grayscale features of the target’s deformation after discretization.In the online part,by sliding window combined with pyramid tracking,the class-mean template is searched preferentially from top to bottom to acquire the class-mean candidates.Then,a detailed search of the candidates within the class is carried out to obtain the best match,achieve the efficient matching of deformed workpieces,and complete the task of coarse position detection during the first stage.A truncated shape-based matching(T-SBM)method is proposed in the second stage to achieve precise positioning using the target edge.By changing the similarity measurement based on the gradient’s inner product,the gradient vector of opposite direction is truncated,so no negative evaluation of the local edge points exists.The improvement restricts the negative contribution on the overall similarity score when the local gradient direction is inverted due to the inconsistency of the target background.The simple representations of sparse edges leading to low robustness are prevented.Finally,a 2D Gaussian conditional density evaluation is proposed to combine grayscale features,shape features,and deformation quantity reasonably.The candidate with the top score wins the best match.The proposed evaluation provides a comprehensive estimation for the ideal target position detection under the 3-sigma principle,realizes the precise positioning of the deformed workpiece,and completes the sequential detection.Result In the experimental part,the proposed method is compared with classical shape-based matching(SBM),normalized cross-correlation(NCC),linearizing the memory 2D(LINE2D),and you only look once version 5 small(YOLOv5s)on 472 authentic industrial images consisting of five types of workpieces,namely,TV back,led panel,screw hole,metal tray,and aluminum plate.Industry vision software,HALCON,provides the implementation of SBM and NCC,and LINE2D is from OpenCV.The evaluation contains F1-score,recall,detection accuracy,and average pixel distance,where the first three and the last regards detection robustness and positioning accuracy,respectively.At intersection over union(IoU)of 0.9,a strict enough threshold for precise positioning,the average detection accuracy and the F1-score of the proposed method are 81.7%and 95%,respectively,and improve by 10.8%and 8.3%,compared with other test methods.When the minscore threshold is less than 0.8,the recall of the proposed method is slightly inferior to that of the NCC method.However,when the minscore is greater than 0.8,a commonly used threshold interval,the proposed method substantially outperforms the other methods.In terms of average positioning accuracy,the positioning error based on the Euclidean distance of the proposed method is as low as 2.44 pixels at the IoU threshold of 0.9,which is muchbetter than the that of the other test methods.Conclusion A two-stage precision positioning method for deformed workpieces made by assembly,stamping,or lamination is proposed.In the experiment,the proposed method outperforms the other test methods on detection robustness and positioning accuracy,which shows the proposed method is suitable for precisely positioning deformed workpieces in industrial scenes.