Seam Tracking and Visual Control for Robotic Arc Welding Based on Structured Light Stereovision

A real-time arc welding robot visual control system based on a local network with a multi-level hierarchy is developed in this paper. It consists of an intelligence and human-machine interface level, a motion planning level, a motion control level and a servo control level. The last three levels form a local real-time open robot controller, which realizes motion planning and motion control of a robot. A camera calibration method based on the relative movement of the end-effector connected to a robot is proposed and a method for tracking weld seam based on the structured light stereovision is provided. Combining the parameters of the cameras and laser plane, three groups of position values in Cartesian space are obtained for each feature point in a stripe projected on the weld seam. The accurate three-dimensional position of the edge points in the weld seam can be calculated from the obtained parameters with an information fusion algorithm. By calculating the weld seam parameter from position and image data, the movement parameters of the robot used for tracking can be determined. A swing welding experiment of type V groove weld is successfully conducted, the results of which show that the system has high resolution seam tracking in real-time, and works stably and efficiently.

Vision Navigation Based PID Control for Line Tracking Robot

In a controlled indoor environment,line tracking has become the most practical and reliable navigation strategy for autonomous mobile robots.A line tracking robot is a self-mobile machine that can recognize and track a painted line on thefloor.In general,the path is set and can be visible,such as a black line on a white surface with high contrasting colors.The robot’s path is marked by a distinct line or track,which the robot follows to move.Several scientific contributions from the disciplines of vision and control have been made to mobile robot vision-based navigation.Localization,automated map generation,autonomous navigation and path tracking is all becoming more frequent in vision applications.A visual navigation line tracking robot should detect the line with a camera using an image processing technique.The paper focuses on combining computer vision techniques with a proportional-integral-derivative(PID)control-ler for automatic steering and speed control.A prototype line tracking robot is used to evaluate the proposed control strategy.

Learning-based force servoing control of a robot with vision in an unknown environment

A learning-based control approach is presented for force servoing of a robot with vision in an unknown environment. Firstly, mapping relationships between image features of the servoing object and the joint angles of the robot are derived and learned by a neural network. Secondly, a learning controller based on the neural network is designed for the robot to trace the object. Thirdly, a discrete time impedance control law is obtained for the force servoing of the robot, the on-line learning algorithms for three neural networks are developed to adjust the impedance parameters of the robot in the unknown environment. Lastly, wiping experiments are carried out by using a 6 DOF industrial robot with a CCD camera and a force/torque sensor in its end effector, and the experimental results confirm the effecti veness of the approach.

Vision-based Stabilization of Nonholonomic Mobile Robots by Integrating Sliding-mode Control and Adaptive Approach

Vision-based pose stabilization of nonholonomic mobile robots has received extensive attention. At present, most of the solutions of the problem do not take the robot dynamics into account in the controller design, so that these controllers are difficult to realize satisfactory control in practical application. Besides, many of the approaches suffer from the initial speed and torque jump which are not practical in the real world. Considering the kinematics and dynamics, a two-stage visual controller for solving the stabilization problem of a mobile robot is presented, applying the integration of adaptive control, sliding-mode control, and neural dynamics. In the first stage, an adaptive kinematic stabilization controller utilized to generate the command of velocity is developed based on Lyapunov theory. In the second stage, adopting the sliding-mode control approach, a dynamic controller with a variable speed function used to reduce the chattering is designed, which is utilized to generate the command of torque to make the actual velocity of the mobile robot asymptotically reach the desired velocity. Furthermore, to handle the speed and torque jump problems, the neural dynamics model is integrated into the above mentioned controllers. The stability of the proposed control system is analyzed by using Lyapunov theory. Finally, the simulation of the control law is implemented in perturbed case, and the results show that the control scheme can solve the stabilization problem effectively. The proposed control law can solve the speed and torque jump problems, overcome external disturbances, and provide a new solution for the vision-based stabilization of the mobile robot.

Intelligent Control of Mobile Robot with Redundant Manipulator & Stereovision: Quantum / Soft Computing Toolkit

The task of an intelligent control system design applying soft and quantum computational intelligence technologies discussed.An example of a control object as a mobile robot with redundant robotic manipulator and stereovision introduced.Design of robust knowledge bases is performed using a developed computational intelligence-quantum/soft computing toolkit(QC/SCOptKBTM).The knowledge base self-organization process of fuzzy homogeneous regulators through the application of end-to-end IT of quantum computing described.The coordination control between the mobile robot and redundant manipulator with stereovision based on soft computing described.The general design methodology of a generalizing control unit based on the physical laws of quantum computing(quantum information-thermodynamic trade-off of control quality distribution and knowledge base self-organization goal)is considered.The modernization of the pattern recognition system based on stereo vision technology presented.The effectiveness of the proposed methodology is demonstrated in comparison with the structures of control systems based on soft computing for unforeseen control situations with sensor system.The main objective of this article is to demonstrate the advantages of the approach based on quantum/soft computing.

Dynamic Coordination of Uncalibrated Hand/Eye Robotic System Based on Neural Network

A nonlinear visual mapping model is presented to replace the image Jacobian relation for uncalibrated hand/eye coordination. A new visual tracking controller based on artificial neural network is designed. Simulation results show that this method can drive the static tracking error to zero quickly and keep good robustness and adaptability at the same time. In addition, the algorithm is very easy to be implemented with low computational complexity.

Design of vision-based soccer robot using DSP

A new design of vision based soccer robot using the type TMS320F240 of DSPs for MiroSot series is presented. The DSP used enables cost effective control of DC motor, and features fewer external components, lower system cost and better performances than traditional microcontroller. The hardware architecture of robot is firstly presented in detail, and then the software design is briefly discussed. The control structure of decision making subsystem is illuminated also in this paper. The conclusion and prospect are given at last.

Visual Feedback Balance Control of a Robot Manipulator and Ball-Beam System

In this paper, we present a vision guided robotic ball-beam balancing control system, consisting of a robot manipulator (actuator), a ball-beam system (plant) and a machine vision system (feedback). The machine vision system feedbacks real-time beam angle and ball position data at a speed of 50 frames per second. Based on feedback data, the end-effector of a robot manipulator is driven to control the ball position by maneuvering of the inclination angle of the ball-beam system. The overall control system is implemented with two FPGA chips, one for machine vision processing, and one for robot joints servo PID controllers as well as ball position PD controller. Experiments are performed on a 5-axes robot manipulator to validate the proposed ball beam balancing control system.

Task-oriented Hierarchical Control of Modular Soft Robots with External Vision Guidance

General,high-precision theoretical modeling method is not well developed in the field of soft robotics,which holds back motion control and practical application of soft robots.The concept of modularization brings novel structure,novel locomotion patterns as well as novel control method for soft robots.This paper presents the concept of hierarchical control method for modular soft robot system and a H-configuration pneumatic modular soft robot is designed as the control object.The H-configuration modular soft robot is composed of two basic motion units that take worm-like locomotion principle.The locomotion principle of the basic motion unit is analyzed and the actuation sequence is optimized by evolution strategy in VOXCAD simulation software.The differential drive method is applied to the H-configuration modular soft robot with multi motion modes and vision sensor is used to control the motion mode of the robot.The H-configuration modular soft robot and the basic motion unit are assembled by a cubic soft module made of silicone rubber.Also,connection mechanism is designed to ensure that the soft modules can be assembled in any direction and posture.Experiments are conducted to verify the effect of the hierarchical control method of the modular soft robots.

Human-inspired lighting control in robot systems with the intention of glare avoidance

This paper presents some human-inspired strategies for lighting control in a robot system for best scene interpretation,where the main intention is to avoid possible glares or highlights occurring in images. It firstly compares the characteristics of human eyes and robot eyes. Then some evaluation criteria are addressed to assess the lighting conditions. A bio-inspired method is adopted to avoid the visual glare which is caused by either direct illumination from large light sources or indirect illumination reflected by smooth surfaces. Appropriate methods are proposed to optimize the pose and optical parameters of the light source and the vision camera.

基于Vision Card的机器人舵机高精度数字控制技术

针对目前机器人舵机控制技术调控性能差,工作过程不稳定的问题,引入Vision Card研究了一种新的高精度数字控制技术;同时分析Vision Card工作原理和机器人舵机控制原理,将二者融合,从视觉角度完成舵机控制;分别针对直流电机、驱动器和反馈电位器进行控制,通过计算直流电机的传递函数、反馈电位器上的电压和总行程,得到脉冲宽度调制(PWM)的电脉冲,利用得到的电脉冲来控制舵机;为验证控制技术有效性,与传统的开环控制技术、PID控制技术和模糊PID控制技术进行实验对比,结果表明,研究的控制技术稳定性高,调控性能好,可以有效提高机器人舵机的工作效率,保障机器人舵机的工作质量。

Scene images and text information‐based object location of robot grasping

With the rapid development of artificial intelligence(AI),the application of this technology in the medical field is becoming increasingly extensive,along with a gradual increase in the amount of intelligent equipment in hospitals.Service robots can save human resources and replace nursing staff to achieve some work.In view of the phenomenon of mobile service robots'grabbing and distribution of patients'drugs in hospitals,a real‐time object detection and positioning system based on image and text information is proposed,which realizes the precise positioning and tracking of the grabbing objects and completes the grasping of a specific object(medicine bottle).The lightweight object detection model NanoDet is used to learn the features of the grasping objects and the object category,and bounding boxes are regressed.Then,the images in the bounding boxes are enhanced to overcome unfavourable factors,such as a small object region.The text detection and recognition model PP‐OCR is used to detect and recognise the enhanced images and extract the text information.The object information provided by the two models is fused,and the text recognition result is matched with the object detection box to achieve the precise posi-tioning of the grasping object.The kernel correlation filter(KCF)tracking algorithm is introduced to achieve real‐time tracking of specific objects to precisely control the robot's grasping.Both deep learning models adopt lightweight networks to facilitate direct deployment.The experiments show that the proposed robot grasping detection system has high reliability,accuracy and real‐time performance.

基于计算机视觉的采摘机器人作业优化研究

为进一步提高我国采摘机器人智能化作业水平及采摘效率,基于计算机视觉应用技术展开优化研究。以采摘机器人结构组成为设计基点,运用视觉控制核心理念建立采摘机器人视觉识别处理与控制数学模型,实施相适应的采摘路径规划和系统采摘状态输出设计。同时,进行视觉采摘验证试验,结果表明:基于计算机视觉的采摘机器人系统优化正确可行,整机综合采摘效率可达94.61%,系统识别准确率与控制精度分别相对提升了6.12%和5.25%,机器人采摘成功率可提升至95.98%。因此,将计算机视觉处理技术有效应用至采摘机器人设计改进,对于类似农业采摘与收获装备开发研究可提供创新及借鉴思路,推广价值良好。

基于深度学习的巡检机器人避障轨迹自动控制系统设计

为了提高巡检机器人在复杂环境下的避障能力,使机器人能够安全地完成巡检任务,设计基于深度学习的巡检机器人避障轨迹自动控制系统;设计由CCD传感器、信号处理芯片等设备组成的工业智能视觉CCD相机,基于FPGA和USB2.0的视频采集卡传输采集数据,完成硬件部分的设计;在软件设计中,对采集的图像实施目标分割、双目目标匹配等预处理,通过对摄像头实施双目视觉标定获取障碍物空间位置三维信息,基于深度学习中的CRNN设计机器人自主避障规划网络模型,并设计模糊轨迹控制器,实现避障中的轨迹自动控制;系统测试结果表明,设计系统最终成功避开了3个动态障碍物,最大轨迹控制误差的最大值为1.45°,最小轨迹控制误差的最大值为0.62°,动态避障巡检速度始终在3.5 m/s左右,表现出了精准而稳定的轨迹控制效果。

基于TwinCAT3的一种开源机器人控制系统设计

目前主流工业机器人为封闭式控制结构,存在不开源、二次开发难的问题,因此设计一种基于TwinCAT3(the windows control and automation technology)的跨平台、可移植性好的机器人控制系统架构。该架构包含视觉、运动控制和算法集成与仿真控制模块,采用倍福自动化设备规范(automation device specification,ADS)通信技术和实时工业以太网总线技术(ethernet for control automation technology,EtherCAT),建立以计算机(personal computer,PC)和倍福控制器为EtherCAT主站,控制多组从站执行器的一主多从工作模式。该模式结合离线与在线控制、集成数字孪生技术,完成虚拟样机与物理样机的联动;采用开源可扩展架构,便于视觉算法、智能算法等算法集成。经实验验证,此架构具有拓展性好、实时性强的特点。

基于FPGA和机器视觉的多元特征模式水果分拣系统

水果自动分拣系统可以解放重复性人力劳动,提高工作效率,消除主观因素,提高工作科学化客观化。然而,目前水果分拣系统成本较高,尚无法大规模应用。因此,在FPGA平台上设计了基于多元特征机器视觉的低成本边缘式水果分拣系统。根据多元特征识别模式ZWM,定义了多元特征向量集和识别模式向量集等概念,进而建立了水果多元特征识别体系,可实现水果分类和水果分级识别。之后,通过将识别参数转化为舵机控制信息,机械臂可将识别出的水果搬运到指定位置,达到自动智能分拣效果。进而该系统可结合云端平台,通过手机实现分拣的远程控制。所提出的基于多元特征的低成本水果分拣系统,能够实现完成物品分类和分级融合分拣,可推动水果智能分拣系统的发展。

基于机器视觉与力控的发动机弯管焊缝自动化打磨系统研究

某系列尺寸不一的弯管由2部分在中间对焊而成,目前多采用人工方式进行弯管内壁焊缝的打磨,存在打磨质量与标准不一、效率低和劳动强度大等问题。为此,设计并实现了一种基于机器视觉技术和力控的发动机弯管焊缝自动化打磨系统。利用工业相机模组正对弯管两侧端面并拍摄图像,经图像处理算法处理计算得到弯管两侧端面焊缝起始位置坐标。机械臂根据该坐标调整打磨轨迹并搭载浮动主轴,保持打磨力的恒定并对焊缝不规则区域进行补偿,实现基于位置控制和力控的打磨自适应调整。试验结果表明,系统整体打磨合格率为96.03%,相比人工打磨效率提升95.90%。系统可以在较少人工的参与和干预下完成弯管焊缝的自动化打磨,具有良好的稳定性和使用柔性,并且允许弯管存在一定的变形。

基于Jetson Nano的智能投送机器人系统实现

针对室内狭小空间的作业环境,提出了一种基于视觉的智能投送机器人。设计了智能投送机器人的机械结构,搭建了智能投送机器人的软硬件控制系统。利用Jetson Nano处理图像识别并搜索物体信息;以STM32F4作为机器人底盘主控芯片,采用模糊分数阶PID控制算法控制电机速度,实现了机器人精准定位、自主避障循迹、物料的分选以及投送功能。