Enhancing Dense Small Object Detection in UAV Images Based on Hybrid Transformer

Transformer-based models have facilitated significant advances in object detection.However,their extensive computational consumption and suboptimal detection of dense small objects curtail their applicability in unmanned aerial vehicle(UAV)imagery.Addressing these limitations,we propose a hybrid transformer-based detector,H-DETR,and enhance it for dense small objects,leading to an accurate and efficient model.Firstly,we introduce a hybrid transformer encoder,which integrates a convolutional neural network-based cross-scale fusion module with the original encoder to handle multi-scale feature sequences more efficiently.Furthermore,we propose two novel strategies to enhance detection performance without incurring additional inference computation.Query filter is designed to cope with the dense clustering inherent in drone-captured images by counteracting similar queries with a training-aware non-maximum suppression.Adversarial denoising learning is a novel enhancement method inspired by adversarial learning,which improves the detection of numerous small targets by counteracting the effects of artificial spatial and semantic noise.Extensive experiments on the VisDrone and UAVDT datasets substantiate the effectiveness of our approach,achieving a significant improvement in accuracy with a reduction in computational complexity.Our method achieves 31.9%and 21.1%AP on the VisDrone and UAVDT datasets,respectively,and has a faster inference speed,making it a competitive model in UAV image object detection.

Real-time Rescue Target Detection Based on UAV Imagery for Flood Emergency Response

Timely acquisition of rescue target information is critical for emergency response after a flood disaster.Unmanned Aerial Vehicles(UAVs)equipped with remote sensing capabilities offer distinct advantages,including high-resolution imagery and exceptional mobility,making them well suited for monitoring flood extent and identifying rescue targets during floods.However,there are some challenges in interpreting rescue information in real time from flood images captured by UAVs,such as the complexity of the scenarios of UAV images,the lack of flood rescue target detection datasets and the limited real-time processing capabilities of the airborne on-board platform.Thus,we propose a real-time rescue target detection method for UAVs that is capable of efficiently delineating flood extent and identifying rescue targets(i.e.,pedestrians and vehicles trapped by floods).The proposed method achieves real-time rescue information extraction for UAV platforms by lightweight processing and fusion of flood extent extraction model and target detection model.The flood inundation range is extracted by the proposed method in real time and detects targets such as people and vehicles to be rescued based on this layer.Our experimental results demonstrate that the Intersection over Union(IoU)for flood water extraction reaches an impressive 80%,and the IoU for real-time flood water extraction stands at a commendable 76.4%.The information on flood stricken targets extracted by this method in real time can be used for flood emergency rescue.

Improved YOLOv8-Based Target Detection Algorithm for UAV Aerial Image

In response to the challenge of low detection accuracy and susceptibility to missed and false detections of small targets in unmanned aerial vehicles(UAVs)aerial images,an improved UAV image target detection algorithm based on YOLOv8 was proposed in this study.To begin with,the CoordAtt attention mechanism was employed to enhance the feature extraction capability of the backbone network,thereby reducing interference from backgrounds.Additionally,the BiFPN feature fusion network with an added small object detection layer was used to enhance the model's ability to perceive for small objects.Furthermore,a multi-level fusion module was designed and proposed to effectively integrate shallow and deep information.The use of an enhanced MPDIoU loss function further improved detection performance.The experimental results based on the publicly available VisDrone2019 dataset showed that the improved model outperformed the YOLOv8 baseline model,mAP@0.5 improved by 20%,and the improved method improved the detection accuracy of the model for small targets.

Bidirectional parallel multi-branch convolution feature pyramid network for target detection in aerial images of swarm UAVs

In this paper,based on a bidirectional parallel multi-branch feature pyramid network(BPMFPN),a novel one-stage object detector called BPMFPN Det is proposed for real-time detection of ground multi-scale targets by swarm unmanned aerial vehicles(UAVs).First,the bidirectional parallel multi-branch convolution modules are used to construct the feature pyramid to enhance the feature expression abilities of different scale feature layers.Next,the feature pyramid is integrated into the single-stage object detection framework to ensure real-time performance.In order to validate the effectiveness of the proposed algorithm,experiments are conducted on four datasets.For the PASCAL VOC dataset,the proposed algorithm achieves the mean average precision(mAP)of 85.4 on the VOC 2007 test set.With regard to the detection in optical remote sensing(DIOR)dataset,the proposed algorithm achieves 73.9 mAP.For vehicle detection in aerial imagery(VEDAI)dataset,the detection accuracy of small land vehicle(slv)targets reaches 97.4 mAP.For unmanned aerial vehicle detection and tracking(UAVDT)dataset,the proposed BPMFPN Det achieves the mAP of 48.75.Compared with the previous state-of-the-art methods,the results obtained by the proposed algorithm are more competitive.The experimental results demonstrate that the proposed algorithm can effectively solve the problem of real-time detection of ground multi-scale targets in aerial images of swarm UAVs.

Optimal Deep Reinforcement Learning for Intrusion Detection in UAVs

In recent years,progressive developments have been observed in recent technologies and the production cost has been continuously decreasing.In such scenario,Internet of Things(IoT)network which is comprised of a set of Unmanned Aerial Vehicles(UAV),has received more attention from civilian tomilitary applications.But network security poses a serious challenge to UAV networks whereas the intrusion detection system(IDS)is found to be an effective process to secure the UAV networks.Classical IDSs are not adequate to handle the latest computer networks that possess maximumbandwidth and data traffic.In order to improve the detection performance and reduce the false alarms generated by IDS,several researchers have employed Machine Learning(ML)and Deep Learning(DL)algorithms to address the intrusion detection problem.In this view,the current research article presents a deep reinforcement learning technique,optimized by BlackWidow Optimization(DRL-BWO)algorithm,for UAV networks.In addition,DRL involves an improved reinforcement learning-based Deep Belief Network(DBN)for intrusion detection.For parameter optimization of DRL technique,BWO algorithm is applied.It helps in improving the intrusion detection performance of UAV networks.An extensive set of experimental analysis was performed to highlight the supremacy of the proposed model.From the simulation values,it is evident that the proposed method is appropriate as it attained high precision,recall,F-measure,and accuracy values such as 0.985,0.993,0.988,and 0.989 respectively.

Using UAVs for detection of trees from digital surface models

A difficult problem in forestry is tree inventory.In this study, a GoProHero attached to a small unmanned aerial vehicle was used to capture images of a small area covered by pinus pinea trees. Then, a digital surface model was generated with image matching. The elevation model representing the terrain surface, a ‘digital terrain model’,was extracted from the digital surface model using morphological filtering. Individual trees were extracted by analyzing elevation flow on the digital elevation model because the elevation reached the highest value on the tree peaks compared to the neighborhood elevation pixels. The quality of the results was assessed by comparison with reference data for correctness of the estimated number of trees. The tree heights were calculated and evaluated with ground truth dataset. The results showed 80% correctness and 90% completeness.

An Edge-Boxes-Based Intruder Detection Algorithm for UAV Sense and Avoid System

With the great development of unmanned aircraft system(UAS)over the last decade,sense and avoid(SAA)system has been a crucial technology for integrating unmanned aircraft vehicle(UAV)into national airspace with reliable and safe operations.This paper mainly focuses on intruder detection for SAA system.A robust algorithm based on the combination of edge-boxes and spatial pyramid matching using sparse coding(sc-SPM)is presented.The algorithm is composed of three stages.First,edge-boxes method is adopted to obtain a large number of proposals;Second,the optimization program is presented to obtain intruder area-of-interest(ROI)regions;Third,sc-SPM is employed for feature representation of ROI regions and support vector machines(SVM)is adopted to detect the intruder.The algorithm is evaluated under different weather conditions.The recall reaches to 0.95 in dawn and sunny weather and 0.9 in cloudy weather.The experimental results indicate that the intruder detection algorithm is effective and robust with various weather under complex background.

Aerodynamic Effects Compensation on Multi-Rotor UAVs Based on a Neural Network Control Allocation Approach

This paper shows that the aerodynamic effects can be compensated in a quadrotor system by means of a control allocation approach using neural networks.Thus,the system performance can be improved by replacing the classic allocation matrix,without using the aerodynamic inflow equations directly.The network training is performed offline,which requires low computational power.The target system is a Parrot MAMBO drone whose flight control is composed of PD-PID controllers followed by the proposed neural network control allocation algorithm.Such a quadrotor is particularly susceptible to the aerodynamics effects of interest to this work,because of its small size.We compared the mechanical torques commanded by the flight controller,i.e.,the control input,to those actually generated by the actuators and established at the aircraft.It was observed that the proposed neural network was able to closely match them,while the classic allocation matrix could not achieve that.The allocation error was also determined in both cases.Furthermore,the closed-loop performance also improved with the use of the proposed neural network control allocation,as well as the quality of the thrust and torque signals,in which we perceived a much less noisy behavior.

CNN Based Automated Weed Detection System Using UAV Imagery

The problem of weeds in crops is a natural problem for farmers.Machine Learning(ML),Deep Learning(DL),and Unmanned Aerial Vehicles(UAV)are among the advanced technologies that should be used in order to reduce the use of pesticides while also protecting the environment and ensuring the safety of crops.Deep Learning-based crop and weed identification systems have the potential to save money while also reducing environmental stress.The accuracy of ML/DL models has been proven to be restricted in the past due to a variety of factors,including the selection of an efficient wavelength,spatial resolution,and the selection and tuning of hyperparameters.The purpose of the current research is to develop a new automated weed detecting system that uses Convolution Neural Network(CNN)classification for a real dataset of 4400 UAV pictures with 15336 segments.Snapshots were used to choose the optimal parameters for the proposed CNN LVQ model.The soil class achieved the user accuracy of 100%with the proposed CNN LVQ model,followed by soybean(99.79%),grass(98.58%),and broadleaf(98.32%).The developed CNN LVQ model showed an overall accuracy of 99.44%after rigorous hyperparameter tuning for weed detection,significantly higher than previously reported studies.

A Deep Learning Approach for Landmines Detection Based on Airborne Magnetometry Imaging and Edge Computing

Landmines continue to pose an ongoing threat in various regions around the world,with countless buried landmines affecting numerous human lives.The detonation of these landmines results in thousands of casualties reported worldwide annually.Therefore,there is a pressing need to employ diverse landmine detection techniques for their removal.One effective approach for landmine detection is UAV(Unmanned Aerial Vehicle)based AirborneMagnetometry,which identifies magnetic anomalies in the local terrestrial magnetic field.It can generate a contour plot or heat map that visually represents the magnetic field strength.Despite the effectiveness of this approach,landmine removal remains a challenging and resource-intensive task,fraughtwith risks.Edge computing,on the other hand,can play a crucial role in critical drone monitoring applications like landmine detection.By processing data locally on a nearby edge server,edge computing can reduce communication latency and bandwidth requirements,allowing real-time analysis of magnetic field data.It enables faster decision-making and more efficient landmine detection,potentially saving lives and minimizing the risks involved in the process.Furthermore,edge computing can provide enhanced security and privacy by keeping sensitive data close to the source,reducing the chances of data exposure during transmission.This paper introduces the MAGnetometry Imaging based Classification System(MAGICS),a fully automated UAV-based system designed for landmine and buried object detection and localization.We have developed an efficient deep learning-based strategy for automatic image classification using magnetometry dataset traces.By simulating the proposal in various network scenarios,we have successfully detected landmine signatures present in themagnetometry images.The trained models exhibit significant performance improvements,achieving a maximum mean average precision value of 97.8%.

A spatio-temporal multi-scale fusion algorithm for pine wood nematode disease tree detection

Pine wood nematode infection is a devastating disease.Unmanned aerial vehicle(UAV)remote sensing enables timely and precise monitoring.However,UAV aerial images are challenged by small target size and complex sur-face backgrounds which hinder their effectiveness in moni-toring.To address these challenges,based on the analysis and optimization of UAV remote sensing images,this study developed a spatio-temporal multi-scale fusion algorithm for disease detection.The multi-head,self-attention mechanism is incorporated to address the issue of excessive features generated by complex surface backgrounds in UAV images.This enables adaptive feature control to suppress redundant information and boost the model’s feature extraction capa-bilities.The SPD-Conv module was introduced to address the problem of loss of small target feature information dur-ing feature extraction,enhancing the preservation of key features.Additionally,the gather-and-distribute mechanism was implemented to augment the model’s multi-scale feature fusion capacity,preventing the loss of local details during fusion and enriching small target feature information.This study established a dataset of pine wood nematode disease in the Huangshan area using DJI(DJ-Innovations)UAVs.The results show that the accuracy of the proposed model with spatio-temporal multi-scale fusion reached 78.5%,6.6%higher than that of the benchmark model.Building upon the timeliness and flexibility of UAV remote sensing,the pro-posed model effectively addressed the challenges of detect-ing small and medium-size targets in complex backgrounds,thereby enhancing the detection efficiency for pine wood nematode disease.This facilitates early preemptive preser-vation of diseased trees,augments the overall monitoring proficiency of pine wood nematode diseases,and supplies technical aid for proficient monitoring.

Research on fast detection method of infrared small targets under resourceconstrained conditions

Infrared small target detection is a common task in infrared image processing.Under limited computa⁃tional resources.Traditional methods for infrared small target detection face a trade-off between the detection rate and the accuracy.A fast infrared small target detection method tailored for resource-constrained conditions is pro⁃posed for the YOLOv5s model.This method introduces an additional small target detection head and replaces the original Intersection over Union(IoU)metric with Normalized Wasserstein Distance(NWD),while considering both the detection accuracy and the detection speed of infrared small targets.Experimental results demonstrate that the proposed algorithm achieves a maximum effective detection speed of 95 FPS on a 15 W TPU,while reach⁃ing a maximum effective detection accuracy of 91.9 AP@0.5,effectively improving the efficiency of infrared small target detection under resource-constrained conditions.

Exploring Image Generation for UAV Change Detection

Change detection(CD)is becoming indispensable for unmanned aerial vehicles(UAVs),especially in the domain of water landing,rescue and search.However,even the most advanced models require large amounts of data for model training and testing.Therefore,sufficient labeled images with different imaging conditions are needed.Inspired by computer graphics,we present a cloning method to simulate inland-water scene and collect an auto-labeled simulated dataset.The simulated dataset consists of six challenges to test the effects of dynamic background,weather,and noise on change detection models.Then,we propose an image translation framework that translates simulated images to synthetic images.This framework uses shared parameters(encoder and generator)and 22×22 receptive fields(discriminator)to generate realistic synthetic images as model training sets.The experimental results indicate that:1)different imaging challenges affect the performance of change detection models;2)compared with simulated images,synthetic images can effectively improve the accuracy of supervised models.

Improved Weighted Local Contrast Method for Infrared Small Target Detection

In order to address the problem of high false alarm rate and low probabilities of infrared small target detection in complex low-altitude background,an infrared small target detection method based on improved weighted local contrast is proposed in this paper.First,the ratio information between the target and local background is utilized as an enhancement factor.The local contrast is calculated by incorporating the heterogeneity between the target and local background.Then,a local product weighted method is designed based on the spatial dissimilarity between target and background to further enhance target while suppressing background.Finally,the location of target is obtained by adaptive threshold segmentation.As experimental results demonstrate,the method shows superior performance in several evaluation metrics compared with six existing algorithms on different datasets containing targets such as unmanned aerial vehicles(UAV).

Eagle-Vision-Based Object Detection Method for UAV Formation in Hazy Weather

Inspired by eagle’s visual system,an eagle-vision-based object detection method for unmanned aerial vehicle(UAV)formation in hazy weather is proposed in this paper.To restore the hazy image,the values of atmospheric light and transmission are estimated on the basis of the signal processing mechanism of ON and OFF channels in eagle’s retina.Local features of the dehazed image are calculated according to the color antagonism mechanism and contrast sensitivity function of eagle’s visual system.A center-surround operation is performed to simulate the response of reception field.The final saliency map is generated by the Random Forest algorithm.Experimental results verify that the proposed method is capable to detect UAVs in hazy image and has superior performance over traditional methods.

Small objects detection in UAV aerial images based on improved Faster R-CNN

In order to solve the problem of small objects detection in unmanned aerial vehicle(UAV)aerial images with complex background,a general detection method for multi-scale small objects based on Faster region-based convolutional neural network(Faster R-CNN)is proposed.The bird’s nest on the high-voltage tower is taken as the research object.Firstly,we use the improved convolutional neural network ResNet101 to extract object features,and then use multi-scale sliding windows to obtain the object region proposals on the convolution feature maps with different resolutions.Finally,a deconvolution operation is added to further enhance the selected feature map with higher resolution,and then it taken as a feature mapping layer of the region proposals passing to the object detection sub-network.The detection results of the bird’s nest in UAV aerial images show that the proposed method can precisely detect small objects in aerial images.

Machine learning algorithm partially reconfigured on FPGA for an image edge detection system

Unmanned aerial vehicles(UAVs)have been widely used in military,medical,wireless communications,aerial surveillance,etc.One key topic involving UAVs is pose estimation in autonomous navigation.A standard procedure for this process is to combine inertial navigation system sensor information with the global navigation satellite system(GNSS)signal.However,some factors can interfere with the GNSS signal,such as ionospheric scintillation,jamming,or spoofing.One alternative method to avoid using the GNSS signal is to apply an image processing approach by matching UAV images with georeferenced images.But a high effort is required for image edge extraction.Here a support vector regression(SVR)model is proposed to reduce this computational load and processing time.The dynamic partial reconfiguration(DPR)of part of the SVR datapath is implemented to accelerate the process,reduce the area,and analyze its granularity by increasing the grain size of the reconfigurable region.Results show that the implementation in hardware is 68 times faster than that in software.This architecture with DPR also facilitates the low power consumption of 4 mW,leading to a reduction of 57%than that without DPR.This is also the lowest power consumption in current machine learning hardware implementations.Besides,the circuitry area is 41 times smaller.SVR with Gaussian kernel shows a success rate of 99.18%and minimum square error of 0.0146 for testing with the planning trajectory.This system is useful for adaptive applications where the user/designer can modify/reconfigure the hardware layout during its application,thus contributing to lower power consumption,smaller hardware area,and shorter execution time.

A Novel Unsupervised Change Detection Method with Structure Consistency and GFLICM Based on UAV Images

With the rapid development of Unmanned Aerial Vehicle(UAV)technology,change detection methods based on UAV images have been extensively studied.However,the imaging of UAV sensors is susceptible to environmental interference,which leads to great differences of same object between UAV images.Overcoming the discrepancy difference between UAV images is crucial to improving the accuracy of change detection.To address this issue,a novel unsupervised change detection method based on structural consistency and the Generalized Fuzzy Local Information C-means Clustering Model(GFLICM)was proposed in this study.Within this method,the establishment of a graph-based structural consistency measure allowed for the detection of change information by comparing structure similarity between UAV images.The local variation coefficient was introduced and a new fuzzy factor was reconstructed,after which the GFLICM algorithm was used to analyze difference images.Finally,change detection results were analyzed qualitatively and quantitatively.To measure the feasibility and robustness of the proposed method,experiments were conducted using two data sets from the cities of Yangzhou and Nanjing.The experimental results show that the proposed method can improve the overall accuracy of change detection and reduce the false alarm rate when compared with other state-of-the-art change detection methods.

Research on Surface Defect Detection Technology of Wind Turbine Blade Based on UAV Image

In the background of“double carbon,”vigorously developing new energy is particularly important.Wind power is an important clean energy source.In the field of new energy,wind power scale is also expanding.With the wind turbine,the probability of large-scale blade damage is also increasing.Because the large wind turbine blade crack detection cost is high and because of the poor working environment,this paper proposes a wind turbine blade surface defect detection method based on UAV acquisition images and digital image pro-cessing.The application of weighted averages to achieve grayscale processing,followed by median filtering to achieve image noise reduction,and an improved histogram equalization algorithm is proposed and used for the characteristics of the UAV acquisition images,which enhances the image by limiting the contrast adaptive his-togram equalization algorithm to make the details at the target area and defects more clear and complete,and improves the detection efficiency.The detection of the blade surface is achieved by separating and extracting the feature information from the defects through image foreground segmentation,threshold processing,and framing by the connected domain.The validity and accuracy of the proposed method in leaf detection were verified by experiments.

Fuzzy Logic Based UAV Suspicious Behavior Detection

In recent years,unmanned air vehicles(UAVs)are widely used in many military and civilian applications.With the big amount of UAVs operation in air space,the potential security and privacy problems are arising.This can lead to consequent harm for critical infrastructure in the event of these UAVs being used for criminal or terrorist purposes.Therefore,it is crucial to promptly identify the suspicious behaviors from the surrounding UAVs for some important regions.In this paper,a novel fuzzy logic based UAV behavior detection system has been presented to detect the different levels of risky behaviors of the incoming UAVs.The heading velocity and region type are two input indicators proposed for the risk indicator output in the designed fuzzy logic based system.The simulation has shown the effective and feasible of the proposed algorithm in terms of recall and precision of the detection.Especially,the suspicious behavior detection algorithm can provide a recall of 0.89 and a precision of 0.95 for the high risk scenario in the simulation.