Enhancing Dense Small Object Detection in UAV Images Based on Hybrid Transformer

Transformer-based models have facilitated significant advances in object detection.However,their extensive computational consumption and suboptimal detection of dense small objects curtail their applicability in unmanned aerial vehicle(UAV)imagery.Addressing these limitations,we propose a hybrid transformer-based detector,H-DETR,and enhance it for dense small objects,leading to an accurate and efficient model.Firstly,we introduce a hybrid transformer encoder,which integrates a convolutional neural network-based cross-scale fusion module with the original encoder to handle multi-scale feature sequences more efficiently.Furthermore,we propose two novel strategies to enhance detection performance without incurring additional inference computation.Query filter is designed to cope with the dense clustering inherent in drone-captured images by counteracting similar queries with a training-aware non-maximum suppression.Adversarial denoising learning is a novel enhancement method inspired by adversarial learning,which improves the detection of numerous small targets by counteracting the effects of artificial spatial and semantic noise.Extensive experiments on the VisDrone and UAVDT datasets substantiate the effectiveness of our approach,achieving a significant improvement in accuracy with a reduction in computational complexity.Our method achieves 31.9%and 21.1%AP on the VisDrone and UAVDT datasets,respectively,and has a faster inference speed,making it a competitive model in UAV image object detection.

Rapid identification of landslide,collapse and crack based on low-altitude remote sensing image of UAV

Landslides,collapses and cracks are the main types of geological hazards,which threaten the safety of human life and property at all times.In emergency surveying and mapping,it is timeconsuming and laborious to use the method of field artificial investigation and recognition and using satellite image to identify ground hazards,there are some problems,such as time lag,low resolution,and difficult to select the map on demand.In this paper,a10 cm per pixel resolution photogrammetry of a geological hazard-prone area of Taohuagou,Shanxi Province,China is carried out by DJ 4 UAV.The digital orthophoto model(DOM),digital surface model(DSM) and three-dimensional point cloud model(3 DPCM) are generated in this region.The method of visual interpretation of cracks based on DOM(as main)-3 DPCM(as auxiliary) and landslide and collapse based on 3 DPCM(as main)-DOM and DSM(as auxiliary) are proposed.Based on the low altitude remote sensing image of UAV,the shape characteristics,geological characteristics and distribution of the identified hazards are analyzed.The results show that using UAV low altitude remote sensing image,the method of combination of main and auxiliary data can quickly and accurately identify landslide,collapse and crack,the accuracy of crack identification is 93%,and the accuracy of landslide and collapse identification is 100%.It mainly occurs in silty clay and mudstone geology and is greatly affected by slope foot excavation.This study can play a great role in the recognition of sudden hazards by low altitude remote sensing images of UAV.

Road Traffic Monitoring from Aerial Images Using Template Matching and Invariant Features

Road traffic monitoring is an imperative topic widely discussed among researchers.Systems used to monitor traffic frequently rely on cameras mounted on bridges or roadsides.However,aerial images provide the flexibility to use mobile platforms to detect the location and motion of the vehicle over a larger area.To this end,different models have shown the ability to recognize and track vehicles.However,these methods are not mature enough to produce accurate results in complex road scenes.Therefore,this paper presents an algorithm that combines state-of-the-art techniques for identifying and tracking vehicles in conjunction with image bursts.The extracted frames were converted to grayscale,followed by the application of a georeferencing algorithm to embed coordinate information into the images.The masking technique eliminated irrelevant data and reduced the computational cost of the overall monitoring system.Next,Sobel edge detection combined with Canny edge detection and Hough line transform has been applied for noise reduction.After preprocessing,the blob detection algorithm helped detect the vehicles.Vehicles of varying sizes have been detected by implementing a dynamic thresholding scheme.Detection was done on the first image of every burst.Then,to track vehicles,the model of each vehicle was made to find its matches in the succeeding images using the template matching algorithm.To further improve the tracking accuracy by incorporating motion information,Scale Invariant Feature Transform(SIFT)features have been used to find the best possible match among multiple matches.An accuracy rate of 87%for detection and 80%accuracy for tracking in the A1 Motorway Netherland dataset has been achieved.For the Vehicle Aerial Imaging from Drone(VAID)dataset,an accuracy rate of 86%for detection and 78%accuracy for tracking has been achieved.

Remote sensing image encryption algorithm based on novel hyperchaos and an elliptic curve cryptosystem

Remote sensing images carry crucial ground information,often involving the spatial distribution and spatiotemporal changes of surface elements.To safeguard this sensitive data,image encryption technology is essential.In this paper,a novel Fibonacci sine exponential map is designed,the hyperchaotic performance of which is particularly suitable for image encryption algorithms.An encryption algorithm tailored for handling the multi-band attributes of remote sensing images is proposed.The algorithm combines a three-dimensional synchronized scrambled diffusion operation with chaos to efficiently encrypt multiple images.Moreover,the keys are processed using an elliptic curve cryptosystem,eliminating the need for an additional channel to transmit the keys,thus enhancing security.Experimental results and algorithm analysis demonstrate that the algorithm offers strong security and high efficiency,making it suitable for remote sensing image encryption tasks.

Machine learning algorithm partially reconfigured on FPGA for an image edge detection system

Unmanned aerial vehicles(UAVs)have been widely used in military,medical,wireless communications,aerial surveillance,etc.One key topic involving UAVs is pose estimation in autonomous navigation.A standard procedure for this process is to combine inertial navigation system sensor information with the global navigation satellite system(GNSS)signal.However,some factors can interfere with the GNSS signal,such as ionospheric scintillation,jamming,or spoofing.One alternative method to avoid using the GNSS signal is to apply an image processing approach by matching UAV images with georeferenced images.But a high effort is required for image edge extraction.Here a support vector regression(SVR)model is proposed to reduce this computational load and processing time.The dynamic partial reconfiguration(DPR)of part of the SVR datapath is implemented to accelerate the process,reduce the area,and analyze its granularity by increasing the grain size of the reconfigurable region.Results show that the implementation in hardware is 68 times faster than that in software.This architecture with DPR also facilitates the low power consumption of 4 mW,leading to a reduction of 57%than that without DPR.This is also the lowest power consumption in current machine learning hardware implementations.Besides,the circuitry area is 41 times smaller.SVR with Gaussian kernel shows a success rate of 99.18%and minimum square error of 0.0146 for testing with the planning trajectory.This system is useful for adaptive applications where the user/designer can modify/reconfigure the hardware layout during its application,thus contributing to lower power consumption,smaller hardware area,and shorter execution time.

Exploring Image Generation for UAV Change Detection

Change detection(CD)is becoming indispensable for unmanned aerial vehicles(UAVs),especially in the domain of water landing,rescue and search.However,even the most advanced models require large amounts of data for model training and testing.Therefore,sufficient labeled images with different imaging conditions are needed.Inspired by computer graphics,we present a cloning method to simulate inland-water scene and collect an auto-labeled simulated dataset.The simulated dataset consists of six challenges to test the effects of dynamic background,weather,and noise on change detection models.Then,we propose an image translation framework that translates simulated images to synthetic images.This framework uses shared parameters(encoder and generator)and 22×22 receptive fields(discriminator)to generate realistic synthetic images as model training sets.The experimental results indicate that:1)different imaging challenges affect the performance of change detection models;2)compared with simulated images,synthetic images can effectively improve the accuracy of supervised models.

Federated Learning with Blockchain Assisted Image Classification for Clustered UAV Networks

The evolving“Industry 4.0”domain encompasses a collection of future industrial developments with cyber-physical systems(CPS),Internet of things(IoT),big data,cloud computing,etc.Besides,the industrial Internet of things(IIoT)directs data from systems for monitoring and controlling the physical world to the data processing system.A major novelty of the IIoT is the unmanned aerial vehicles(UAVs),which are treated as an efficient remote sensing technique to gather data from large regions.UAVs are commonly employed in the industrial sector to solve several issues and help decision making.But the strict regulations leading to data privacy possibly hinder data sharing across autonomous UAVs.Federated learning(FL)becomes a recent advancement of machine learning(ML)which aims to protect user data.In this aspect,this study designs federated learning with blockchain assisted image classification model for clustered UAV networks(FLBIC-CUAV)on IIoT environment.The proposed FLBIC-CUAV technique involves three major processes namely clustering,blockchain enabled secure communication and FL based image classification.For UAV cluster construction process,beetle swarm optimization(BSO)algorithm with three input parameters is designed to cluster the UAVs for effective communication.In addition,blockchain enabled secure data transmission process take place to transmit the data from UAVs to cloud servers.Finally,the cloud server uses an FL with Residual Network model to carry out the image classification process.A wide range of simulation analyses takes place for ensuring the betterment of the FLBIC-CUAV approach.The experimental outcomes portrayed the betterment of the FLBIC-CUAV approach over the recent state of art methods.

Bidirectional parallel multi-branch convolution feature pyramid network for target detection in aerial images of swarm UAVs

In this paper,based on a bidirectional parallel multi-branch feature pyramid network(BPMFPN),a novel one-stage object detector called BPMFPN Det is proposed for real-time detection of ground multi-scale targets by swarm unmanned aerial vehicles(UAVs).First,the bidirectional parallel multi-branch convolution modules are used to construct the feature pyramid to enhance the feature expression abilities of different scale feature layers.Next,the feature pyramid is integrated into the single-stage object detection framework to ensure real-time performance.In order to validate the effectiveness of the proposed algorithm,experiments are conducted on four datasets.For the PASCAL VOC dataset,the proposed algorithm achieves the mean average precision(mAP)of 85.4 on the VOC 2007 test set.With regard to the detection in optical remote sensing(DIOR)dataset,the proposed algorithm achieves 73.9 mAP.For vehicle detection in aerial imagery(VEDAI)dataset,the detection accuracy of small land vehicle(slv)targets reaches 97.4 mAP.For unmanned aerial vehicle detection and tracking(UAVDT)dataset,the proposed BPMFPN Det achieves the mAP of 48.75.Compared with the previous state-of-the-art methods,the results obtained by the proposed algorithm are more competitive.The experimental results demonstrate that the proposed algorithm can effectively solve the problem of real-time detection of ground multi-scale targets in aerial images of swarm UAVs.

Pine wilt disease detection in high-resolution UAV images using object-oriented classification

Pine wilt disease(PWD)is currently one of the main causes of large-scale forest destruction.To control the spread of PWD,it is essential to detect affected pine trees quickly.This study investigated the feasibility of using the object-oriented multi-scale segmentation algorithm to identify trees discolored by PWD.We used an unmanned aerial vehicle(UAV)platform equipped with an RGB digital camera to obtain high spatial resolution images,and multiscale segmentation was applied to delineate the tree crown,coupling the use of object-oriented classification to classify trees discolored by PWD.Then,the optimal segmentation scale was implemented using the estimation of scale parameter(ESP2)plug-in.The feature space of the segmentation results was optimized,and appropriate features were selected for classification.The results showed that the optimal scale,shape,and compactness values of the tree crown segmentation algorithm were 56,0.5,and 0.8,respectively.The producer’s accuracy(PA),user’s accuracy(UA),and F1 score were 0.722,0.605,and 0.658,respectively.There were no significant classification errors in the final classification results,and the low accuracy was attributed to the low number of objects count caused by incorrect segmentation.The multi-scale segmentation and object-oriented classification method could accurately identify trees discolored by PWD with a straightforward and rapid processing.This study provides a technical method for monitoring the occurrence of PWD and identifying the discolored trees of disease using UAV-based high-resolution images.

Land cover classification in a mixed forest-grassland ecosystem using LResU-net and UAV imagery

Using an unmanned aerial vehicle (UAV) paired with image semantic segmentation to classify land cover within natural vegetation can promote the development of forest and grassland field. Semantic segmentation normally excels in medical and building classification, but its usefulness in mixed forest-grassland ecosystems in semi-arid to semi-humid climates is unknown. This study proposes a new semantic segmentation network of LResU-net in which residual convolution unit (RCU) and loop convolution unit (LCU) are added to the U-net framework to classify images of different land covers generated by UAV high resolution. The selected model enhanced classification accuracy by increasing gradient mapping via RCU and modifying the size of convolution layers via LCU as well as reducing convolution kernels. To achieve this objective, a group of orthophotos were taken at an altitude of 260 m for testing in a natural forest-grassland ecosystem of Keyouqianqi, Inner Mongolia, China, and compared the results with those of three other network models (U-net, ResU-net and LU-net). The results show that both the highest kappa coefficient (0.86) and the highest overall accuracy (93.7%) resulted from LResU-net, and the value of most land covers provided by the producer’s and user’s accuracy generated in LResU-net exceeded 0.85. The pixel-area ratio approach was used to calculate the real areas of 10 different land covers where grasslands were 67.3%. The analysis of the effect of RCU and LCU on the model training performance indicates that the time of each epoch was shortened from U-net (358 s) to LResU-net (282 s). In addition, in order to classify areas that are not distinguishable, unclassified areas were defined and their impact on classification. LResU-net generated significantly more accurate results than the other three models and was regarded as the most appropriate approach to classify land cover in mixed forest-grassland ecosystems.

A Novel Unsupervised Change Detection Method with Structure Consistency and GFLICM Based on UAV Images

With the rapid development of Unmanned Aerial Vehicle(UAV)technology,change detection methods based on UAV images have been extensively studied.However,the imaging of UAV sensors is susceptible to environmental interference,which leads to great differences of same object between UAV images.Overcoming the discrepancy difference between UAV images is crucial to improving the accuracy of change detection.To address this issue,a novel unsupervised change detection method based on structural consistency and the Generalized Fuzzy Local Information C-means Clustering Model(GFLICM)was proposed in this study.Within this method,the establishment of a graph-based structural consistency measure allowed for the detection of change information by comparing structure similarity between UAV images.The local variation coefficient was introduced and a new fuzzy factor was reconstructed,after which the GFLICM algorithm was used to analyze difference images.Finally,change detection results were analyzed qualitatively and quantitatively.To measure the feasibility and robustness of the proposed method,experiments were conducted using two data sets from the cities of Yangzhou and Nanjing.The experimental results show that the proposed method can improve the overall accuracy of change detection and reduce the false alarm rate when compared with other state-of-the-art change detection methods.

Research on Surface Defect Detection Technology of Wind Turbine Blade Based on UAV Image

In the background of“double carbon,”vigorously developing new energy is particularly important.Wind power is an important clean energy source.In the field of new energy,wind power scale is also expanding.With the wind turbine,the probability of large-scale blade damage is also increasing.Because the large wind turbine blade crack detection cost is high and because of the poor working environment,this paper proposes a wind turbine blade surface defect detection method based on UAV acquisition images and digital image pro-cessing.The application of weighted averages to achieve grayscale processing,followed by median filtering to achieve image noise reduction,and an improved histogram equalization algorithm is proposed and used for the characteristics of the UAV acquisition images,which enhances the image by limiting the contrast adaptive his-togram equalization algorithm to make the details at the target area and defects more clear and complete,and improves the detection efficiency.The detection of the blade surface is achieved by separating and extracting the feature information from the defects through image foreground segmentation,threshold processing,and framing by the connected domain.The validity and accuracy of the proposed method in leaf detection were verified by experiments.

基于UAV成像技术的‘高原红’海棠花期物候监测研究

以‘高原红’海棠(Malus‘Prairifire’)为试材,基于UAV成像技术采集物候期内图像样本并提取色彩参数值(CIE L^(*)a^(*)b^(*)),计算不同采样点及不同物候期的色差值(△E),通过对比△E的变化情况实现海棠物候的动态监测,为实时掌握海棠物候动态变化情况,适时作出调整策略提供技术参考,同时也为开展花期物候调控相关研究提供参考依据。结果表明:1)利用Photoshop提取航拍图像色彩参数,并基于物候色彩参数进行聚类分析,将其物候划分为萌芽期、展叶期、蕾期、初花期、盛花期、末花期、落叶期、结果期、休眠期,建立了‘高原红’海棠物候色彩参数库;2)计算‘高原红’海棠各物候期内色差△E_(1)及不同物候期间色差△E_(2)范围,并基于各物候期内最大色差△E_(1max)′及不同物候期间最小色差△E_(2min)′,建立了基于色差参数的物候判别方法;3)随机提取了2个日期(3月22日、4月10日)的‘高原红’海棠物候图像色彩参数,通过聚类分析判定2个日期物候图分别处于蕾期和盛花期,结果与实际相符;2个日期物候图间色差(△E_(2)′=21.8)大于蕾期至初花期最小色差(△E_(2min)=9.6),表明此时物候期跨度较大,结果与实际相符。综上,该方法可便捷高效判断海棠花期等物候期情况。

Heat transfer and temperature evolution in underground mininginduced overburden fracture and ground fissures: Optimal time window of UAV infrared monitoring

Heat transfer and temperature evolution in overburden fracture and ground fissures are one of the essential topics for the identification of ground fissures via unmanned aerial vehicle(UAV) infrared imager. In this study, discrete element software UDEC was employed to investigate the overburden fracture field under different mining conditions. Multiphysics software COMSOL were employed to investigate heat transfer and temperature evolution of overburden fracture and ground fissures under the influence of mining condition, fissure depth, fissure width, and month alternation. The UAV infrared field measurements also provided a calibration for numerical simulation. The results showed that for ground fissures connected to underground goaf(Fissure Ⅰ), the temperature difference increased with larger mining height and shallow buried depth. In addition, Fissure Ⅰ located in the boundary of the goaf have a greater temperature difference and is easier to be identified than fissures located above the mining goaf. For ground fissures having no connection to underground goaf(Fissure Ⅱ), the heat transfer is affected by the internal resistance of the overlying strata fracture when the depth of Fissure Ⅱ is greater than10 m, the temperature of Fissure Ⅱ gradually equals to the ground temperature as the fissures’ depth increases, and the fissures are difficult to be identified. The identification effect is most obvious for fissures larger than 16 cm under the same depth. In spring and summer, UAV infrared identification of mining fissures should be carried out during nighttime. This study provides the basis for the optimal time and season for the UAV infrared identification of different types of mining ground fissures.

Information Detection of Seismic Debris Flow by UAV High-resolution Image Based on Transfer Learning

A large number of debris flow disasters(called Seismic debris flows) would occur after an earthquake, which can cause a great amount of damage. UAV low-altitude remote sensing technology has become a means of quickly obtaining disaster information as it has the advantage of convenience and timeliness, but the spectral information of the image is so scarce, making it difficult to accurately detect the information of earthquake debris flow disasters. Based on the above problems, a seismic debris flow detection method based on transfer learning(TL) mechanism is proposed. On the basis of the constructed seismic debris flow disaster database, the features acquired from the training of the convolutional neural network(CNN) are transferred to the disaster information detection of the seismic debris flow. The automatic detection of earthquake debris flow disaster information is then completed, and the results of object-oriented seismic debris flow disaster information detection are compared and analyzed with the detection results supported by transfer learning.

Image Deraining for UAV Using Split Attention Based Recursive Network

Images captured in rainy days suffer from noticeable degradation of scene visibility.Unmanned aerial vehicles(UAVs),as important outdoor image acquisition systems,demand a proper rain removal algorithm to improve visual perception quality of captured images as well as the performance of many subsequent computer vision applications.To deal with rain streaks of different sizes and directions,this paper proposes to employ convolutional kernels of different sizes in a multi-path structure.Split attention is leveraged to enable communication across multiscale paths at feature level,which allows adaptive receptive field to tackle complex situations.We incorporate the multi-path convolution and the split attention operation into the basic residual block without increasing the channels of feature maps.Moreover,every block in our network is unfolded four times to compress the network volume without sacrificing the deraining performance.The performance on various benchmark datasets demonstrates that our method outperforms state-of-the-art deraining algorithms in both numerical and qualitative comparisons.

UAVAI-YOLO:无人机航拍图像的小目标检测模型

针对无人机航拍图像目标检测效果差的问题,提出改进的UAVAI-YOLO模型。首先,为使模型获得更加丰富的语义信息,使用改进可变形卷积网络(deformable convolutional networks,DCN)替换原骨干(back‐bone)网络部分通道到像素(channel-to-pixel,C2f)模块原始卷积。其次,为增加P2特征层而不增加模型参数量,提出Conv_C模块将骨干网络输出通道降维,同时避免通道降维导致的语义信息丢失,使用改进ODConv卷积替换颈部(neck)部分C2f模块原始卷积。然后,为充分利用上下文语义信息,引入双向特征金字塔网络(bidirectional feature pyramid network,BIFPN)。最后使用Wise-IoU替换原始损失函数,提高模型目标检测框的准确性。在公开的VisDrone2019数据集和UAVDT数据集的实验结果表明,UAVAI-YOLO模型相比于原YOLOv8n模型mAP@0.5分别提升了4.4%和1.1%。与其他主流目标检测模型相比具有较高的检测性精度。

Optimal combination of the correction model and parameters for the precision geometric correction of UAV hyperspectral images

Nowadays,with the rapid development of quantitative remote sensing represented by high-resolution UAV hyperspectral remote sensing observation technology,people have put forward higher requirements for the rapid preprocessing and geometric correction accuracy of hyperspectral images.The optimal geometric correction model and parameter combination of UAV hyperspectral images need to be determined to reduce unnecessary waste of time in the preprocessing and provide high-precision data support for the application of UAV hyperspectral images.In this study,the geometric correction accuracy under various geometric correction models(including affine transformation model,local triangulation model,polynomial model,direct linear transformation model,and rational function model)and resampling methods(including nearest neighbor resampling method,bilinear interpolation resampling method,and cubic convolution resampling method)were analyzed.Furthermore,the distribution,number,and accuracy of control points were analyzed based on the control variable method,and precise ground control points(GCPs)were analyzed.The results showed that the average geometric positioning error of UAV hyperspectral images(at 80 m altitude AGL)without geometric correction was as high as 3.4041 m(about 65 pixels).The optimal geometric correction model and parameter combination of the UAV hyperspectral image(at 80 m altitude AGL)used a local triangulation model,adopted a bilinear interpolation resampling method,and selected 12 edgemiddle distributed GCPs.The correction accuracy could reach 0.0493 m(less than one pixel).This study provides a reference for the geometric correction of UAV hyperspectral images.

Three-dimensional multi-constraint route planning of unmanned aerial vehicle low-altitude penetration based on coevolutionary multi-agent genetic algorithm

To address the issue of premature convergence and slow convergence rate in three-dimensional (3D) route planning of unmanned aerial vehicle (UAV) low-altitude penetration,a novel route planning method was proposed.First and foremost,a coevolutionary multi-agent genetic algorithm (CE-MAGA) was formed by introducing coevolutionary mechanism to multi-agent genetic algorithm (MAGA),an efficient global optimization algorithm.A dynamic route representation form was also adopted to improve the flight route accuracy.Moreover,an efficient constraint handling method was used to simplify the treatment of multi-constraint and reduce the time-cost of planning computation.Simulation and corresponding analysis show that the planning results of CE-MAGA have better performance on terrain following,terrain avoidance,threat avoidance (TF/TA2) and lower route costs than other existing algorithms.In addition,feasible flight routes can be acquired within 2 s,and the convergence rate of the whole evolutionary process is very fast.

Segmentation of High Spatial Resolution Remote Sensing Images of Mountainous Areas Based on the Improved Mean Shift Algorithm

Using conventional Mean Shift Algorithm to segment high spatial resolution Remote sensing images of mountainous areas usually leads to an unsatisfactory result, due to its rich texture information. In this paper, we propose an improved Mean Shift Algorithm in consideration of the characteristics of these images. First, images were classified into several homogeneous color regions and texture regions by conducting variance detection on the color space. Next, each homogeneous color region was directly segmented to generate the preliminary results by applying the Mean Shift Algorithm. For each texture region, we conduct a high-dimensional feature space by extracting information such as color, texture and shape comprehensively, and work out a proper bandwidth according to the normalized distribution density. Then the bandwidth variable Mean Shift Algorithm was applied to obtain segmentation results by conducting the pattern classification in feature space. Last, the final results were obtained by merging these regions by means of the constructed cost functions and removing the oversegmented regions from the merged regions. It has been experimentally segmented on the high spatial resolution remote sensing images collected by Quickbird and Unmanned Aerial Vehicle(UAV). We put forward an approach to evaluate the segmentation results by using the segmentation matching index(SMI). This takes into consideration both the area and the spectrum. The experimental results suggest that the improved Mean Shift Algorithm outperforms the conventional one in terms of accuracy of segmentation.