Tree species classification using deep learning and RGB optical images obtained by an unmanned aerial vehicle

The diversity of tree species and the complexity of land use in cities create challenging issues for tree species classification.The combination of deep learning methods and RGB optical images obtained by unmanned aerial vehicles(UAVs) provides a new research direction for urban tree species classification.We proposed an RGB optical image dataset with 10 urban tree species,termed TCC10,which is a benchmark for tree canopy classification(TCC).TCC10 dataset contains two types of data:tree canopy images with simple backgrounds and those with complex backgrounds.The objective was to examine the possibility of using deep learning methods(AlexNet,VGG-16,and ResNet-50) for individual tree species classification.The results of convolutional neural networks(CNNs) were compared with those of K-nearest neighbor(KNN) and BP neural network.Our results demonstrated:(1) ResNet-50 achieved an overall accuracy(OA) of 92.6% and a kappa coefficient of 0.91 for tree species classification on TCC10 and outperformed AlexNet and VGG-16.(2) The classification accuracy of KNN and BP neural network was less than70%,while the accuracy of CNNs was relatively higher.(3)The classification accuracy of tree canopy images with complex backgrounds was lower than that for images with simple backgrounds.For the deciduous tree species in TCC10,the classification accuracy of ResNet-50 was higher in summer than that in autumn.Therefore,the deep learning is effective for urban tree species classification using RGB optical images.

Road Traffic Monitoring from Aerial Images Using Template Matching and Invariant Features

Road traffic monitoring is an imperative topic widely discussed among researchers.Systems used to monitor traffic frequently rely on cameras mounted on bridges or roadsides.However,aerial images provide the flexibility to use mobile platforms to detect the location and motion of the vehicle over a larger area.To this end,different models have shown the ability to recognize and track vehicles.However,these methods are not mature enough to produce accurate results in complex road scenes.Therefore,this paper presents an algorithm that combines state-of-the-art techniques for identifying and tracking vehicles in conjunction with image bursts.The extracted frames were converted to grayscale,followed by the application of a georeferencing algorithm to embed coordinate information into the images.The masking technique eliminated irrelevant data and reduced the computational cost of the overall monitoring system.Next,Sobel edge detection combined with Canny edge detection and Hough line transform has been applied for noise reduction.After preprocessing,the blob detection algorithm helped detect the vehicles.Vehicles of varying sizes have been detected by implementing a dynamic thresholding scheme.Detection was done on the first image of every burst.Then,to track vehicles,the model of each vehicle was made to find its matches in the succeeding images using the template matching algorithm.To further improve the tracking accuracy by incorporating motion information,Scale Invariant Feature Transform(SIFT)features have been used to find the best possible match among multiple matches.An accuracy rate of 87%for detection and 80%accuracy for tracking in the A1 Motorway Netherland dataset has been achieved.For the Vehicle Aerial Imaging from Drone(VAID)dataset,an accuracy rate of 86%for detection and 78%accuracy for tracking has been achieved.

3D modeling of Unmanned Aerial Vehicles Tilt Photogrammetry

Unmanned Aerial Vehicles(UAV)tilt photogrammetry technology can quickly acquire image data in a short time.This technology has been widely used in all walks of life with the rapid development in recent years especially in the rapid acquisition of high-resolution remote sensing images,because of its advantages of high efficiency,reliability,low cost and high precision.Fully using the UAV tilt photogrammetry technology,the construction image progress can be observed by stages,and the construction site can be reasonably and optimally arranged through three-dimensional modeling to create a civilized,safe and tidy construction environment.

Heat transfer and temperature evolution in underground mininginduced overburden fracture and ground fissures: Optimal time window of UAV infrared monitoring

Heat transfer and temperature evolution in overburden fracture and ground fissures are one of the essential topics for the identification of ground fissures via unmanned aerial vehicle(UAV) infrared imager. In this study, discrete element software UDEC was employed to investigate the overburden fracture field under different mining conditions. Multiphysics software COMSOL were employed to investigate heat transfer and temperature evolution of overburden fracture and ground fissures under the influence of mining condition, fissure depth, fissure width, and month alternation. The UAV infrared field measurements also provided a calibration for numerical simulation. The results showed that for ground fissures connected to underground goaf(Fissure Ⅰ), the temperature difference increased with larger mining height and shallow buried depth. In addition, Fissure Ⅰ located in the boundary of the goaf have a greater temperature difference and is easier to be identified than fissures located above the mining goaf. For ground fissures having no connection to underground goaf(Fissure Ⅱ), the heat transfer is affected by the internal resistance of the overlying strata fracture when the depth of Fissure Ⅱ is greater than10 m, the temperature of Fissure Ⅱ gradually equals to the ground temperature as the fissures’ depth increases, and the fissures are difficult to be identified. The identification effect is most obvious for fissures larger than 16 cm under the same depth. In spring and summer, UAV infrared identification of mining fissures should be carried out during nighttime. This study provides the basis for the optimal time and season for the UAV infrared identification of different types of mining ground fissures.

结合Forstner与NCC约束的UAV图像配准研究

随着无人机(unmanned aerial vehicle,UAV)技术的飞快发展,UAV已成为航空遥感图像获取的重要手段。但与传统的大飞机航空摄影相比,UAV在平台的稳定性方面较差,采集图像时受自身配重、即时飞行环境等外界因素影响,使得最终获得的遥感图像存在复杂的几何变形,导致其图像配准过程存在很大的困难。针对此问题,首先基于UAV的POS数据进行图像重叠区域估算,利用Forstner算子提取图像中的特征点并结合信息熵对图像进行分块处理;然后通过基于旋转的归一化互相关(normalized cross-correlation,NCC)系数寻找相匹配的同名特征点,最终实现UAV图像的配准。实验结果证明该方法切实有效,并且保持了较好的鲁棒性。

基于SIFT的UAV载组合宽角相机影像匹配方法

针对无人驾驶飞机UAV(Unmanned Aerial Vehicle)航空组合相机获取的大像幅影像旋偏角较大、大尺度变化和颜色差异明显的问题,提出基于极几何和单应约束的SIFT(Scale Invariant Feature Transform)特征多尺度LSM(Least Squares Matching)算法。该算法顶层金字塔影像采用SIFT快速匹配,对匹配结果利用改进的RANSAC(Random Sample Consensus)算法计算影像间单应矩阵和基本矩阵;对影像进行Harris特征提取,根据极几何和单应约束采用双向一致性相关系数算法进行密集匹配;通过更新单应矩阵,设定阈值删除误匹配点;对匹配的同名点进行最小二乘匹配获取子像素级精度。通过对具有较大旋偏角、大尺度变化和颜色差异的3组实际航摄影像的试验对比表明,与传统方法相比,该算法具有较高的匹配成功率和较好的有效性。

UAV LiDAR在山区铁路勘察中的应用

随着无人机(UAV)激光雷达(LiDAR)技术的快速发展,其作业方式灵活,效率高、人工少等优点在铁路勘察中具有良好应用前景,横断面中高程精度,地形图中平面以及高程精度是线路设计应用中的关键影响因素,本文通过对横断面高程精度的研究,对地形图中设置的检核点以及实验区房屋角点坐标与人工测量数据进行对比统计分析,得出无人机LiDAR点云获取的横断面与地形图与人工测量成果精度相当,无人机LiDAR技术精度可以满足山区铁路勘察设计的需要,值得推广应用。

多旋翼无人机载4D成像雷达生命体征感知方法

无人机载雷达具有高机动灵活的特点,可解决传统非接触式生命体征感知中存在的探测范围小和探测场景受限等问题。该项研究工作将4D成像雷达搭载于多旋翼无人机上,提出一种基于点云配准的无人机载4D雷达生命体征感知方法。该方法通过对雷达点云进行配准和运动补偿,消除无人机在悬停状态时的运动误差干扰,进而对齐人体目标后实现生命体征信号的获取。仿真实验结果表明该方法能够对齐4D成像雷达点云序列,有效抑制无人机的运动干扰,从而准确提取人体目标的呼吸和心跳信号,为无人机载非接触式生命体征感知提供了一种新的技术途径。

基于改进YOLOv8的无人机可见光小目标检测方法研究

目的 :为解决目前无人机可见光系统检测小目标时准确率和实时性低的问题,提出一种基于改进YOLOv8的可见光小目标检测方法。方法:选取由主干网络(Backbone)、颈部模块(Neck)和头部模块(Head)组成的YOLOv8网络作为基础框架构建AGC-YOLO模型。首先,在Backbone部分融入卷积注意力模块(convolutional block attention module,CBAM),提高模型的特征表达能力;其次,将部分传统卷积模块替换为可改变核卷积模块AKconv,减少网络参数量;最后,在Neck部分采用Gold-YOLO模块,提高对不同尺寸目标的检测能力。选用VisDrone2019数据集分别进行消融实验和对比实验,通过平均精度均值(mean average precision,mAP)、每秒传输帧数(frames per second,FPS)、每秒10亿次的浮点运算数(giga floating-point operations per second,GFLOPs)和参数量(parameters)评估AGC-YOLO模型对小目标检测的效果。结果:AGC-YOLO模型的FPS为31.90,GFLOPs和Parameters分别为9.20和11.30 M,达到无人机实时性的检测速度要求(FPS不低于30)。虽然AGC-YOLO模型的GFLOPs和Parameters比轻量化模型YOLOv8n、Ghost-YOLO和YOLO-BiFPN有所增加,但是mAP50(mAP50表示在交并比为0.5时的mAP)分别提高了15%、6%和5%。结论:提出的方法在提高检测速度、减少参数量、保障检测精度方面表现良好,在无人机可见光小目标检测方面具有良好的应用前景。

Online Multi-Object Tracking Under Moving Unmanned Aerial Vehicle Platform Based on Object Detection and Feature Extraction Network

In order to solve the problem of small object size and low detection accuracy under the unmanned aerial vehicle(UAV)platform,the object detection algorithm based on deep aggregation network and high-resolution fusion module is studied.Furthermore,a joint network of object detection and feature extraction is studied to construct a real-time multi-object tracking algorithm.For the problem of object association failure caused by UAV movement,image registration is applied to multi-object tracking and a camera motion discrimination model is proposed to improve the speed of the multi-object tracking algorithm.The simulation results show that the algorithm proposed in this study can improve the accuracy of multi-object tracking under the UAV platform,and effectively solve the problem of association failure caused by UAV movement.

Adaptive cropping shallow attention network for defect detection of bridge girder steel using unmanned aerial vehicle images

Bridges are an important part of railway infrastructure and need regular inspection and maintenance.Using unmanned aerial vehicle(UAV)technology to inspect railway infrastructure is an active research issue.However,due to the large size of UAV images,flight distance,and height changes,the object scale changes dramatically.At the same time,the elements of interest in railway bridges,such as bolts and corrosion,are small and dense objects,and the sample data set is seriously unbalanced,posing great challenges to the accurate detection of defects.In this paper,an adaptive cropping shallow attention network(ACSANet)is proposed,which includes an adaptive cropping strategy for large UAV images and a shallow attention network for small object detection in limited samples.To enhance the accuracy and generalization of the model,the shallow attention network model integrates a coordinate attention(CA)mechanism module and an alpha intersection over union(α-IOU)loss function,and then carries out defect detection on the bolts,steel surfaces,and railings of railway bridges.The test results show that the ACSANet model outperforms the YOLOv5s model using adaptive cropping strategy in terms of the total mAP(an evaluation index)and missing bolt mAP by 5%and 30%,respectively.Also,compared with the YOLOv5s model that adopts the common cropping strategy,the total mAP and missing bolt mAP are improved by 10%and 60%,respectively.Compared with the YOLOv5s model without any cropping strategy,the total mAP and missing bolt mAP are improved by 40%and 67%,respectively.

Exploring Image Generation for UAV Change Detection

Change detection(CD)is becoming indispensable for unmanned aerial vehicles(UAVs),especially in the domain of water landing,rescue and search.However,even the most advanced models require large amounts of data for model training and testing.Therefore,sufficient labeled images with different imaging conditions are needed.Inspired by computer graphics,we present a cloning method to simulate inland-water scene and collect an auto-labeled simulated dataset.The simulated dataset consists of six challenges to test the effects of dynamic background,weather,and noise on change detection models.Then,we propose an image translation framework that translates simulated images to synthetic images.This framework uses shared parameters(encoder and generator)and 22×22 receptive fields(discriminator)to generate realistic synthetic images as model training sets.The experimental results indicate that:1)different imaging challenges affect the performance of change detection models;2)compared with simulated images,synthetic images can effectively improve the accuracy of supervised models.

无人机低空现代航测自动空三软件UAV-LAMapper系统的介绍及在石油行业的应用

无人机低空现代航测自动空三软件系统是一套全数字化的摄影测量处理系统,具有操作简单、高度自动化、运行速度快、能够处理各种数码相机拍摄的影像(包括处理最困难的飞艇影像)和传统胶片扫描影像,直接使用TIF、GeoTIFF、BMP和JPG等通用格式的影像,并且用四叉树技术在内存中产生金字塔影像,不占用硬盘空间。本系统适合在当前多种计算机(包括笔记本电脑)上进行内、外业一体化及立体检查作业。

Image Deraining for UAV Using Split Attention Based Recursive Network

Images captured in rainy days suffer from noticeable degradation of scene visibility.Unmanned aerial vehicles(UAVs),as important outdoor image acquisition systems,demand a proper rain removal algorithm to improve visual perception quality of captured images as well as the performance of many subsequent computer vision applications.To deal with rain streaks of different sizes and directions,this paper proposes to employ convolutional kernels of different sizes in a multi-path structure.Split attention is leveraged to enable communication across multiscale paths at feature level,which allows adaptive receptive field to tackle complex situations.We incorporate the multi-path convolution and the split attention operation into the basic residual block without increasing the channels of feature maps.Moreover,every block in our network is unfolded four times to compress the network volume without sacrificing the deraining performance.The performance on various benchmark datasets demonstrates that our method outperforms state-of-the-art deraining algorithms in both numerical and qualitative comparisons.

Railway Transport Infrastructure Monitoring by UAVs and Satellites

Improving the rail transport security requires development and implementation of neoteric monitoring and control facilities in conditions of increasing speed and intensity of the train movement and high level of terrorist threat. Use of Earth remote sensing (ERS), permitting to obtain information from large areas with a sufficiently high resolution, can provide significant assistance in solving the mentioned problems. This paper discusses the possibility of using various means of remote sensing such as satellites and unmanned aerial vehicles (UAV), also known as drones, for receiving information in different ranges of the electromagnetic spectrum. The paper states that joint using of both these means gives new possibilities in improving railroad security.

基于特征复用机制的航拍图像小目标检测算法

针对无人机(UAV)航拍图像检测存在的小目标检测精度低和模型参数量大的问题,提出轻量高效的航拍图像检测算法FS-YOLO.该算法以YOLOv8s为基准网络,通过降低通道维数和改进网络架构提出轻量的特征提取网络,实现对冗余特征信息的高效复用,在较少的参数量下产生更多特征图,提高模型对特征信息的提取和表达能力,同时显著减小模型大小.在特征融合阶段引入内容感知特征重组模块,加强对小目标显著语义信息的关注,提升网络对航拍图像的检测性能.使用无人机航拍数据集VisDrone进行实验验证,结果表明,所提算法以仅5.48 M的参数量实现了mAP0.5=47.0%的检测精度,比基准算法YOLOv8s的参数量降低了50.7%,精度提升了6.1%.在DIOR数据集上的实验表明,FS-YOLO的泛化能力较强,较其他先进算法更具竞争力.

高阶深度可分离无人机图像小目标检测算法

当前无人机图像中存在小目标数量众多、背景复杂的特点,目标检测中易造成漏检误检率较高的问题,针对这些问题,提出一种高阶深度可分离无人机图像小目标检测算法.首先,结合CSPNet结构与ConvMixer网络,深度可分离卷积核,获取梯度结合信息,并引入递归门控卷积C3模块,提升模型的高阶空间交互能力,增强网络对小目标的敏感度;其次,检测头采用两个头部进行解耦,分别输出特征图分类和位置信息,加快模型收敛速度;最后,使用边框损失函数EIoU,提高检测框精准度.在VisDrone2019数据集上的实验结果表明,该模型检测精度达到了35.1%,模型漏检率和误检率有明显下降,能够有效地应用于无人机图像小目标检测任务.在DOTA 1.0数据集和HRSID数据集上进行模型泛化能力测试,实验结果表明,该模型具有良好的鲁棒性.

城市森林结构多样性预测冠下地面温度的潜力研究

城市森林冠层具有调控城市森林微气候的能力,但现有研究尚未阐明冠层结构对冠下地面温度的影响及其预测潜力。文章基于无人机机载激光雷达(UAV-LiDAR)提取哈尔滨林业示范基地的城市森林冠层结构多样性特征指标,探究单一结构多样性特征对冠下地面温度的影响,以及结构多样性多因子组合对温度的预测潜力。结果表明:1)城市森林结构多样性的8个特征因子与冠下地面温度呈显著相关关系(P<0.05),其中深间隙(DG)、深间隙分数(DGF)、覆盖分数(CF)、间隙分数分布(GFP)表征了结构多样性的覆盖/开放度特征;冠层高度标准差(H_(std))、冠层高度最大值(H_(max))、95%分位点高度(ZQ_(95))表征了高度特征;垂直复杂指数(VCI)表征了异质性特征。2)城市森林冠层结构多样性的覆盖/开放度特征对冠下地面温度的响应更强(R^(2)为0.15~0.5),强于高度指标(R^(2)为0.14~0.19)以及异质性指标(R^(2)=0.14)。3)结合高度指标、覆盖/开放度指标以及异质性指标的多因子预测模型2(R^(2)=0.61,RMSE=0.51,MSE=0.26,AIC=62.74),对于冠下地面温度的预测性能更优。研究明晰了城市森林结构多样性的多因子变量及其特征组合预测冠下地面温度的潜力,为城市森林冠层结构调控内部小气候环境研究提供了科学参考。

基于无人机航拍的苎麻倒伏信息解译研究

茎杆倒伏是苎麻三麻培育中最常见的灾害,传统的监测方法具有耗时耗力、不及时等局限性。提出了一种基于无人机航拍获取苎麻倒伏信息的方法,首先利用Pix4D Mapper软件生成苎麻的冠层正射影像和数字表面模型(digital surface model,DSM),基于正射影像提取苎麻光谱、纹理及形状特征,基于DSM提取苎麻株高指标,最后结合3种机器学习算法构建正常/倒伏苎麻分类模型。结果表明,基于DSM提取的株高信息可以有效代替大田实测株高,模型R2为0.899。倒伏和正常苎麻在光谱、纹理、形状及株高特征上具有差异。在3种机器学习算法中,支持向量机和决策树模型的性能最好,准确率达到99%,能够高效地识别苎麻倒伏地块。以上研究结果为准确、快速评估作物倒伏情况提供了技术支撑。

基于无人机多光谱影像的云南松林蓄积量估测模型

【目的】无人机多光谱遥感影像较可见光影像具有更丰富的光谱信息,在森林蓄积量估测中具有较大潜力。以无人机载多光谱遥感影像为主要数据源,探索森林蓄积量的遥感估测模型,以克服传统地面调查工作量大、耗时长、成本高等弊端。【方法】以滇中地区典型天然云南松Pinusyunnanensis纯林为研究对象,利用无人机多光谱影像提取单波段反射率、各类植被指数、纹理特征等,计算各特征变量的标准地均值;筛选与云南松林蓄积量相关性显著的特征变量,采用多元线性、随机森林、支持向量机建立云南松林蓄积量估测模型,以决定系数(R^(2))、平均绝对误差(E_(MA))、均方根误差(E_(RMS))、平均相对误差(EMR)评价模型精度。【结果】①3种模型中,随机森林的精度最高(R^(2)=0.89,E_(MA)=4.69 m^(3)·hm^(-2),E_(RMS)=5.45 m^(3)·hm^(-2),EMR=14.5%),其次为支持向量机(R^(2)=0.74,E_(MA)=5.27 m^(3)·hm^(-2),E_(RMS)=8.31 m^(3)·hm^(-2),EMR=13.1%),最低为多元线性回归模型(R^(2)=0.35,E_(MA)=10.12 m^(3)·hm^(-2),E_(RMS)=12.85 m^(3)·hm^(-2),EMR=28.1%);3种模型在测试集上的估测精度均有所降低,随机森林的模型表现最好,支持向量机次之,多元线性最差。②3种模型在云南松林蓄积量估测中均存在一定的低值高估和高值低估现象。③基于无人机多光谱影像估测云南松林蓄积量,纹理特征仍是不可忽视的重要因子。【结论】基于无人机多光谱影像,在不进行单木分割的情景下,提取标准地的单波段反射率、植被指数、纹理特征均值,筛选适用于蓄积量估算的变量构建估测模型。通过对3种模型进行精度评价,随机森林为云南松林蓄积量估测的最佳模型。