基于深度学习的螺纹钢焊牌机器人3D视觉计数和定位方法

Counting and Positioning Method of Threaded Steel Plate Welding Robot Based on Deep Learning and 3D Vision

为解决因螺纹钢截面形状不标准、发蓝、发黑,长短不一而导致无法用传统视觉方法计数和定位的问题,对基于深度学习的螺纹钢焊牌机器人视觉计数和定位方法进行了研究,并开发了螺纹钢自动化焊牌系统。提出基于Faster R-CNN的图像识别算法并对棒材进行目标检测。对数据集进行随机旋转、翻转、噪点、模糊、颜色变换等图像处理实现数据增强使数据集扩充32倍,再进行训练;建立机器人和三维相机的坐标模型,计算出图像像素坐标系转换到机器人工具坐标系的转换矩阵。通过多组图片拼接解决不平整导致的遮挡问题,使用加入采集图像时的位姿作为判定参数的改进SURF算法实现图像准确拼接;输入拼接的图片进行推理,统计棒材的支数并获取中心区域的一支棒材像素坐标,通过三维相机的映射关系获得其三维坐标,发送到机器人控制器作为焊牌位置。结果表明:所提方法可以完成整捆螺纹钢棒材的准确识别计数和机器人定位焊牌,焊牌位置准确,焊接质量稳固。

In order to solve the problems of non-standard section shape,blue,black and different length of rebar,which can not be counted and positioned by using traditional vision method,the visual counting and positioning method of rebar welding plate robot based on deep learning is studied,and the automatic welding plate system of rebar is developed.An image recognition algorithm based on Faster Regions with CNN(Faster R-CNN)is proposed for object detection of bars.Image processing such as random rotation,flipping,noise,blurring,and color transformation are performed on the dataset to enhance the data and expand the dataset by 32 times,and then the data are trained;coordinate models for robots and 3D cameras are established,the conversion matrix from image pixel coordinate system to robot tool coordinate system are calculated.Multiple sets of image stitching are used to solve the occlusion problem caused by unevenness,and an improved SURF algorithm that incorporates the pose of the collected image as a judgment parameter is used to achieve accurate image stitching;the concatenated images are input for reasoning and counting the number of bars to obtain the pixel coordinates of a bar in the central area,and its 3D coordinates are obtained through the mapping relationship of the 3D camera,which is sent to the robot controller as the welding plate position.The results show that by using the proposed method,the count of rebar rod with accurate positioning can be obtained,and the welding quality is stable.