Drogue detection for autonomous aerial refueling via hybrid pigeon-inspired optimized color opponent and saliency aggregation

Drogue detection is one of the challenging tasks in autonomous aerial refueling due to the requirement for accuracy and rapidity.Saliency detection based on image intrinsic cues can achieve fast detection,but with poor accuracy.Recent studies reveal that optimization-based methods provide accurate and quick solutions for saliency detection.This paper presents a hybrid pigeon-inspired optimization method,the optimized color opponent,that aims to adjust the weight of color opponent channels to detect the drogue region.It can optimize the weights in the selected aerial refueling scene offline,and the results are applied for drogue detection in the scene.A novel algorithm aggregated by the optimized color opponent and robust background detection is presented to provide better precision and robustness.Experimental results on benchmark datasets and aerial refueling images show that the proposed method successfully extracts the saliency region or drogue and exhibits superior performance against the other saliency detection methods with intrinsic cues.The algorithm designed in this paper is competent for the drogue detection task of autonomous aerial refueling.

Verification of monocular and binocular pose estimation algorithms in vision-based UAVs autonomous aerial refueling system

With the high focus on autonomous aerial refueling(AAR), it becomes increasingly urgent to design efficient methods or algorithms for solving the AAR problems in complicated aerial environments. A vision-based technology for AAR is developed in this paper, and five monocular and binocular visual algorithms for pose estimation of the unmanned aerial vehicles(UAVs) are adopted and verified in this AAR system. The real-time on-board vision system is also designed for precise navigation in the UAVs docking phase. A series of out-door comparative experiments for different pose estimation algorithms are conducted to verify the feasibility and accuracy of the vision algorithms in AAR.

N-fold Bernoulli probability based adaptive fast-tracking algorithm and its application to autonomous aerial refuelling

Recently,deep learning has been widely utilized for object tracking tasks.However,deep learning encounters limits in tasks such as Autonomous Aerial Refueling(AAR),where the target object can vary substantially in size,requiring high-precision real-time performance in embedded systems.This paper presents a novel embedded adaptiveness single-object tracking framework based on an improved YOLOv4 detection approach and an n-fold Bernoulli probability theorem.First,an Asymmetric Convolutional Network(ACNet)and dense blocks are combined with the YOLOv4 architecture to detect small objects with high precision when similar objects are in the background.The prior object information,such as its location in the previous frame and its speed,is utilized to adaptively track objects of various sizes.Moreover,based on the n-fold Bernoulli probability theorem,we develop a filter that uses statistical laws to reduce the false positive rate of object tracking.To evaluate the efficiency of our algorithm,a new AAR dataset is collected,and extensive AAR detection and tracking experiments are performed.The results demonstrate that our improved detection algorithm is better than the original YOLOv4 algorithm on small and similar object detection tasks;the object tracking algorithm is better than state-of-the-art object tracking algorithms on refueling drogue tracking tasks.

Real-time drogue recognition and 3D locating for UAV autonomous aerial refueling based on monocular machine vision

Drogue recognition and 3D locating is a key problem during the docking phase of the autonomous aerial refueling （AAR）. To solve this problem, a novel and effective method based on monocular vision is presented in this paper. Firstly, by employing computer vision with red-ring-shape feature, a drogue detection and recognition algorithm is proposed to guarantee safety and ensure the robustness to the drogue diversity and the changes in environmental condi- tions, without using a set of infrared light emitting diodes （LEDs） on the parachute part of the dro- gue. Secondly, considering camera lens distortion, a monocular vision measurement algorithm for drogue 3D locating is designed to ensure the accuracy and real-time performance of the system, with the drogue attitude provided. Finally, experiments are conducted to demonstrate the effective- ness of the proposed method. Experimental results show the performances of the entire system in contrast with other methods, which validates that the proposed method can recognize and locate the drogue three dimensionally, rapidly and precisely.

Drogue detection for autonomous aerial refueling based on convolutional neural networks

Drogue detection is a fundamental issue during the close docking phase of autonomous aerial refueling（AAR）. To cope with this issue, a novel and effective method based on deep learning with convolutional neural networks（CNNs） is proposed. In order to ensure its robustness and wide application, a deep learning dataset of images was prepared by utilizing real data of ‘‘Probe and Drogue＂ aerial refueling, which contains diverse drogues in various environmental conditions without artificial features placed on the drogues. By employing deep learning ideas and graphics processing units（GPUs）, a model for drogue detection using a Caffe deep learning framework with CNNs was designed to ensure the method＇s accuracy and real-time performance. Experiments were conducted to demonstrate the effectiveness of the proposed method, and results based on real AAR data compare its performance to other methods, validating the accuracy, speed, and robustness of its drogue detection ability.

A guidance law for UAV autonomous aerial refueling based on the iterative computation method

The rendezvous and formation problem is a significant part for the unmanned aerial vehicle（UAV） autonomous aerial refueling（AAR） technique. It can be divided into two major phases： the long-range guidance phase and the formation phase. In this paper, an iterative computation guidance law（ICGL） is proposed to compute a series of state variables to get the solution of a control variable for a UAV conducting rendezvous with a tanker in AAR. The proposed method can make the control variable converge to zero when the tanker and the UAV receiver come to a formation flight eventually. For the long-range guidance phase, the ICGL divides it into two sub-phases： the correction sub-phase and the guidance sub-phase. The two sub-phases share the same iterative process. As for the formation phase, a velocity coordinate system is created by which control accelerations are designed to make the speed of the UAV consistent with that of the tanker.The simulation results demonstrate that the proposed ICGL is effective and robust against wind disturbance.

A monocular visual measurement system for UAV probe-and-drogue autonomous aerial refueling

Purpose–The purpose of this paper is to develop a monocular visual measurement system for autonomous aerial refueling(AAR)for unmanned aerial vehicle,which can process images from an infrared camera to estimate the pose of the drogue in the tanker with high accuracy and real-time performance.Design/methodology/approach–Methods and techniques for marker detection,feature matching and pose estimation have been designed and implemented in the visual measurement system.Findings–The simple blob detection(SBD)method is adopted,which outperforms the Laplacian of Gaussian method.And a novel noise-elimination algorithm is proposed for excluding the noise points.Besides,a novel feature matching algorithm based on perspective transformation is proposed.Comparative experimental results indicated the rapidity and effectiveness of the proposed methods.Practical implications–The visual measurement system developed in this paper can be applied to estimate the pose of the drogue with a fast speed and high accuracy and it is a feasible measurement strategy which will considerably increase the autonomy and reliability for AAR.Originality/value–The SBD method is used to detect the features and a novel noise-elimination algorithm is proposed.Besides,a novel feature matching algorithm based on perspective transformation is proposed which is robust and accurate.

基于Transformer特征金字塔的空中加油锥套实时检测

空中加油锥套的实时检测是实现自主空中加油的重要前提。针对现有目标检测算法在空中加油锥套检测中易受环境干扰导致精度不足的问题,提出了基于Transformer特征金字塔的空中加油锥套实时检测算法。首先,提出一种新的基于池化注意力的Transformer特征金字塔结构TPN对主干特征进行融合,实现更加高效的特征图增强;其次,使用线性注意力进一步降低TPN中注意力机制复杂性,提出轻量化检测模型DNet-LinTPN,减小其80%的内存占用。在自制的空中加油锥套数据集上的实验结果表明,基于TPN的模型在同等条件下精度、速度与模型大小均优于YOLOv7;轻量化检测模型DNet-LinTPN精度达到93.8%较YOLOv7-tiny提升9.4个百分点,参数量减少67.2%,运算量减少45.2%,鲁棒性明显提升。

一种空中自主加油扰动建模与自适应控制方法

针对加/受油机近距对接阶段设计气动扰动建模与对接控制方法。首先建立了大气紊流、受油机绕流和加油机尾流等干扰的等效模型,用以描述空中加油阶段气流扰动对锥套的推离规律以及对受油机气流角与气动力矩的影响。然后面向近距对接控制任务,设计了基于超螺旋滑模观测器的自适应干扰抑制控制器,在基线动态逆控制律的基础上引入外环二阶滑模干扰观测器用以补偿姿态扰动,内环引入角速率观测器并构造自适应项以抑制长时干扰。最后,通过综合视景仿真系统验证了基于滑模观测器的自适应干扰抑制控制器能够实现气流扰动下受油探管对锥套中心的精准跟踪。

自主空中加油能力需求及关键评价指标分析

从飞行试验角度而言,如何评价自主空中加受油能力,以及采用哪些关键指标进行评价是自主空中加受油试飞亟待解决的关键问题。从国内外空中加油相关标准要求出发,同时借鉴有人机空中加油的成功经验,分析了自主空中加油总体能力需求,提出了会合、编队、对接、加油及脱离等全过程关键评价指标,在数值仿真和硬件在回路两种环境下对评估方法进行了试验验证,结合试验结果,对影响自主空中加油对接成功的关键因素进行了分析并提出了评价指标建议,试验结果可为自主空中加受油流程设计、试飞验证等方面提供参考和借鉴。

基于仿鹰眼视觉的无人机自主空中加油

自主空中加油是提高无人机作战能力的重要技术支撑,而鹰在所有的动物中以视觉最为敏锐而著称。针对无人机自主空中加油中的精确对接需求,设计了基于仿鹰眼视觉原理的无人受油机视觉测量方法及半物理试验平台。采用鹰眼视觉注意机制提取虚拟视景中的兴趣区域,剔除噪声和干扰。在对受油机上的颜色标志点进行识别的基础上采用P3P算法,对加油机和受油机之间的相对位姿进行精确估计,并设计了基于比例-积分-微分的硬式加油伸缩管控制律。基于上述技术,在实验室环境下设计开发了无人机自主空中加油半物理仿真平台。实验结果表明:所设计的仿鹰眼视觉测量方法和自主控制律能满足无人机自主空中加油的性能要求,具有较强的实时性、准确性和鲁棒性。

视觉导航辅助的自主空中加油建模与仿真

研究了基于视觉导航方式的自主空中加油(AAR)建模问题。分析了影响自主空中加油对接阶段性能的两个最重要因素——相对距离的测量和风扰动,引入了多个摄像机,建立了适合AAR的视觉导航系统模型,建立了大气扰动和尾流等风扰动下的飞行器模型,用三次多项式规划了对接阶段的航迹并设计了LQR航迹跟踪控制律。对接过程仿真表明,视觉导航的估计精度和飞行控制律性能很好地满足了空中加油的要求。

基于计算机视觉的UAV自主空中加油半物理仿真

设计并实现了一种基于计算机视觉的无人机(UAV,Unmanned Aerial Vehicle)自主空中加油仿真平台,采用点匹配与位姿估计算法对软管式加油锥发光二极管(LED,Light Emitting Diodes)标志进行识别.对大气扰动和尾涡流进行了建模,建立了加油锥模型,并设计了加油锥运动状态预估算法.在此基础上,设计了基于线性二次型调节器(LQR,Linear Quadratic Regulator)的飞行控制律.基于上述技术,在实验室环境下设计开发了基于单目机载摄像机的无人机自主空中加油半物理仿真平台.半物理试验结果表明:所设计的位姿匹配算法和飞行控制律能够满足复杂环境下无人机自主空中加油的性能要求,具有较强的实时性、准确性和鲁棒性.

软管-锥套式空中加油动态建模与性能分析

提出了一种基于凯恩方程的自主空中加油软管-锥套动态模型及性能分析方法。该方法将软管视为由有限段以铰链形式连接的刚性杆组成,锥套为软管末端的一个质点。本文定义了描述系统状态的广义坐标及广义速率,导出了软管段位置多级递推公式和系统动态方程,估算了软管在加油机尾流、定常流和大气扰动下的气动载荷。通过数值仿真分析了平稳大气中锥套在加油机不同飞行条件下的稳态阻力和软管拖拽轨迹,验证了模型的正确性及系统稳定性。最后研究了大气扰动对锥套运动的影响及不同软管段的受扰运动。

基于双目视觉的自动空中加油近距导航方法

为解决插头锥管式自动空中加油中的近距精确导航问题,提出了一种基于双目视觉的近距导航方法.该方法使用受油机上的两个摄像机拍摄加油锥管端面上光学标记的图像,采用轮廓跟踪法检测图像中的光斑区域,并将其灰度重心作为光学标记的像点,根据对极几何约束完成像点的匹配,利用空间圆形拟合法计算加油锥管的位置和姿态.建立了空中加油的三维虚拟场景对该导航方法进行仿真验证,仿真结果表明,该方法在近距离加油对接时具有很高的位置和姿态测量精度,可以满足自动空中加油的需要.

自主空中加油视觉导航系统中的锥套检测算法

在软管式自主空中加油视觉相对导航系统中实现锥套的准确检测至关重要,由于加油机尾流及气流的影响,做随机运动的锥套区域的准确检测是具有一项挑战性的任务。将加油机锥套检测考虑为运动目标检测问题,提出了一种多尺度低秩和稀疏分解锥套检测算法。首先,对锥套图像序列进行平稳小波分解,得到多尺度低频图像序列;再将较粗尺度低频图像序列获得的目标作为下一较细尺度低频图像序列的目标可信图,再进行该尺度的低秩和稀疏分解获得稀疏项,即目标锥套区域。通过在真实加油机锥套图像数据上的实验结果表明,所提算法在进行锥套检测时是有效的。

基于近红外的自主空中加油相对位置解算

无人机(UAV)高空加油在受油口和加油锥套对接过程中,需在全天时和全天候下自主完成高精度的相对位置解算。由于近红外图像兼顾分辨率高及抗环境干扰能力强的优点,因此提出UAV自主空中软管式加油近红外相对位置解算方法。将4个850 nm波段近红外LED灯均匀布置在锥套内环,与850 nm带通滤镜和罗技C310摄像头一起构成视觉相对位置解算系统。首先进行锥套近红外成像,然后通过阈值分割、特征点检测和圆拟合实现锥套检测,进一步完成受油口与加油锥套相对位置解算。通过不同天气环境、不同角度、不同距离下的实验进行了算法性能验证,并给出了算法的适用范围,为工程应用提供借鉴。

不确定环境下的软管-锥套建模及控制研究

提出了一种能有效适应不确定性环境的锥套稳定控制方法。在对软管-锥套非线性动力学模型线性化及降阶处理的基础上,将大气紊流对软管-锥套的扰动视为随机扰动,并把这种扰动作为参数与软管-锥套系统降阶模型中的状态合并成增广的系统状态量,采用卡尔曼滤波方法对系统状态量进行实时估计,并设计LQG控制器,实现对锥套的稳定控制。仿真结果表明,该方法能实时跟踪大气紊流的变化,且具有较好的抗大气紊流干扰的能力,能保证锥套的稳定。

自主空中加油时变质量无人作战飞机非线性控制

针对自主空中加油中无人作战飞机(UCAV)位置保持问题,进行了时变质量UCAV的动力学建模与非线性控制设计.综合考虑了燃油传输对UCAV的质量、惯性矩阵和质心位置的影响,基于相对于惯性系的状态变量,推导了UCAV时变质量动力学方程.通过引入谱半径,将局部化自适应边界指令滤波反推方法应用于UCAV的位置保持控制.使用逼近器对未知模型不确定性进行在线逼近.对于固有逼近误差和外部扰动,采用局部化自适应边界进行补偿.通过指令滤波反推,设计了相对位置、速度、姿态角和角速度四个反馈回路来保证UCAV的稳定性.最后,三种不同加油方案下的非线性仿真验证了非线性飞行控制律的有效性.

机器视觉辅助的插头锥套式无人机自主空中加油仿真

为准确获取无人机自主空中加油对接阶段受油插头与加油锥套的相对位姿信息,提出一种机器视觉辅助的插头锥套式无人机自主空中加油方案。研究了机器视觉识别跟踪加油锥套的算法,利用卡尔曼滤波算法估计无人机与加油锥套的相对位姿。实验结果表明:机器视觉图像处理算法可精确识别跟踪加油锥套,滤波器估计的相对位姿误差收敛速度较快,满足插头锥套式无人机自主空中加油的需要。