Drogue detection for autonomous aerial refueling via hybrid pigeon-inspired optimized color opponent and saliency aggregation

Drogue detection is one of the challenging tasks in autonomous aerial refueling due to the requirement for accuracy and rapidity.Saliency detection based on image intrinsic cues can achieve fast detection,but with poor accuracy.Recent studies reveal that optimization-based methods provide accurate and quick solutions for saliency detection.This paper presents a hybrid pigeon-inspired optimization method,the optimized color opponent,that aims to adjust the weight of color opponent channels to detect the drogue region.It can optimize the weights in the selected aerial refueling scene offline,and the results are applied for drogue detection in the scene.A novel algorithm aggregated by the optimized color opponent and robust background detection is presented to provide better precision and robustness.Experimental results on benchmark datasets and aerial refueling images show that the proposed method successfully extracts the saliency region or drogue and exhibits superior performance against the other saliency detection methods with intrinsic cues.The algorithm designed in this paper is competent for the drogue detection task of autonomous aerial refueling.

Real-time drogue recognition and 3D locating for UAV autonomous aerial refueling based on monocular machine vision

Drogue recognition and 3D locating is a key problem during the docking phase of the autonomous aerial refueling （AAR）. To solve this problem, a novel and effective method based on monocular vision is presented in this paper. Firstly, by employing computer vision with red-ring-shape feature, a drogue detection and recognition algorithm is proposed to guarantee safety and ensure the robustness to the drogue diversity and the changes in environmental condi- tions, without using a set of infrared light emitting diodes （LEDs） on the parachute part of the dro- gue. Secondly, considering camera lens distortion, a monocular vision measurement algorithm for drogue 3D locating is designed to ensure the accuracy and real-time performance of the system, with the drogue attitude provided. Finally, experiments are conducted to demonstrate the effective- ness of the proposed method. Experimental results show the performances of the entire system in contrast with other methods, which validates that the proposed method can recognize and locate the drogue three dimensionally, rapidly and precisely.

Drogue detection for autonomous aerial refueling based on convolutional neural networks

Drogue detection is a fundamental issue during the close docking phase of autonomous aerial refueling（AAR）. To cope with this issue, a novel and effective method based on deep learning with convolutional neural networks（CNNs） is proposed. In order to ensure its robustness and wide application, a deep learning dataset of images was prepared by utilizing real data of ‘‘Probe and Drogue＂ aerial refueling, which contains diverse drogues in various environmental conditions without artificial features placed on the drogues. By employing deep learning ideas and graphics processing units（GPUs）, a model for drogue detection using a Caffe deep learning framework with CNNs was designed to ensure the method＇s accuracy and real-time performance. Experiments were conducted to demonstrate the effectiveness of the proposed method, and results based on real AAR data compare its performance to other methods, validating the accuracy, speed, and robustness of its drogue detection ability.

Verification of monocular and binocular pose estimation algorithms in vision-based UAVs autonomous aerial refueling system

With the high focus on autonomous aerial refueling(AAR), it becomes increasingly urgent to design efficient methods or algorithms for solving the AAR problems in complicated aerial environments. A vision-based technology for AAR is developed in this paper, and five monocular and binocular visual algorithms for pose estimation of the unmanned aerial vehicles(UAVs) are adopted and verified in this AAR system. The real-time on-board vision system is also designed for precise navigation in the UAVs docking phase. A series of out-door comparative experiments for different pose estimation algorithms are conducted to verify the feasibility and accuracy of the vision algorithms in AAR.

A guidance law for UAV autonomous aerial refueling based on the iterative computation method

The rendezvous and formation problem is a significant part for the unmanned aerial vehicle（UAV） autonomous aerial refueling（AAR） technique. It can be divided into two major phases： the long-range guidance phase and the formation phase. In this paper, an iterative computation guidance law（ICGL） is proposed to compute a series of state variables to get the solution of a control variable for a UAV conducting rendezvous with a tanker in AAR. The proposed method can make the control variable converge to zero when the tanker and the UAV receiver come to a formation flight eventually. For the long-range guidance phase, the ICGL divides it into two sub-phases： the correction sub-phase and the guidance sub-phase. The two sub-phases share the same iterative process. As for the formation phase, a velocity coordinate system is created by which control accelerations are designed to make the speed of the UAV consistent with that of the tanker.The simulation results demonstrate that the proposed ICGL is effective and robust against wind disturbance.

A monocular visual measurement system for UAV probe-and-drogue autonomous aerial refueling

Purpose–The purpose of this paper is to develop a monocular visual measurement system for autonomous aerial refueling(AAR)for unmanned aerial vehicle,which can process images from an infrared camera to estimate the pose of the drogue in the tanker with high accuracy and real-time performance.Design/methodology/approach–Methods and techniques for marker detection,feature matching and pose estimation have been designed and implemented in the visual measurement system.Findings–The simple blob detection(SBD)method is adopted,which outperforms the Laplacian of Gaussian method.And a novel noise-elimination algorithm is proposed for excluding the noise points.Besides,a novel feature matching algorithm based on perspective transformation is proposed.Comparative experimental results indicated the rapidity and effectiveness of the proposed methods.Practical implications–The visual measurement system developed in this paper can be applied to estimate the pose of the drogue with a fast speed and high accuracy and it is a feasible measurement strategy which will considerably increase the autonomy and reliability for AAR.Originality/value–The SBD method is used to detect the features and a novel noise-elimination algorithm is proposed.Besides,a novel feature matching algorithm based on perspective transformation is proposed which is robust and accurate.

N-fold Bernoulli probability based adaptive fast-tracking algorithm and its application to autonomous aerial refuelling

Recently,deep learning has been widely utilized for object tracking tasks.However,deep learning encounters limits in tasks such as Autonomous Aerial Refueling(AAR),where the target object can vary substantially in size,requiring high-precision real-time performance in embedded systems.This paper presents a novel embedded adaptiveness single-object tracking framework based on an improved YOLOv4 detection approach and an n-fold Bernoulli probability theorem.First,an Asymmetric Convolutional Network(ACNet)and dense blocks are combined with the YOLOv4 architecture to detect small objects with high precision when similar objects are in the background.The prior object information,such as its location in the previous frame and its speed,is utilized to adaptively track objects of various sizes.Moreover,based on the n-fold Bernoulli probability theorem,we develop a filter that uses statistical laws to reduce the false positive rate of object tracking.To evaluate the efficiency of our algorithm,a new AAR dataset is collected,and extensive AAR detection and tracking experiments are performed.The results demonstrate that our improved detection algorithm is better than the original YOLOv4 algorithm on small and similar object detection tasks;the object tracking algorithm is better than state-of-the-art object tracking algorithms on refueling drogue tracking tasks.