Road Traffic Monitoring from Aerial Images Using Template Matching and Invariant Features

Road traffic monitoring is an imperative topic widely discussed among researchers.Systems used to monitor traffic frequently rely on cameras mounted on bridges or roadsides.However,aerial images provide the flexibility to use mobile platforms to detect the location and motion of the vehicle over a larger area.To this end,different models have shown the ability to recognize and track vehicles.However,these methods are not mature enough to produce accurate results in complex road scenes.Therefore,this paper presents an algorithm that combines state-of-the-art techniques for identifying and tracking vehicles in conjunction with image bursts.The extracted frames were converted to grayscale,followed by the application of a georeferencing algorithm to embed coordinate information into the images.The masking technique eliminated irrelevant data and reduced the computational cost of the overall monitoring system.Next,Sobel edge detection combined with Canny edge detection and Hough line transform has been applied for noise reduction.After preprocessing,the blob detection algorithm helped detect the vehicles.Vehicles of varying sizes have been detected by implementing a dynamic thresholding scheme.Detection was done on the first image of every burst.Then,to track vehicles,the model of each vehicle was made to find its matches in the succeeding images using the template matching algorithm.To further improve the tracking accuracy by incorporating motion information,Scale Invariant Feature Transform(SIFT)features have been used to find the best possible match among multiple matches.An accuracy rate of 87%for detection and 80%accuracy for tracking in the A1 Motorway Netherland dataset has been achieved.For the Vehicle Aerial Imaging from Drone(VAID)dataset,an accuracy rate of 86%for detection and 78%accuracy for tracking has been achieved.

Adaptive Boundary and Semantic Composite Segmentation Method for Individual Objects in Aerial Images

There are two types of methods for image segmentation.One is traditional image processing methods,which are sensitive to details and boundaries,yet fail to recognize semantic information.The other is deep learning methods,which can locate and identify different objects,but boundary identifications are not accurate enough.Both of them cannot generate entire segmentation information.In order to obtain accurate edge detection and semantic information,an Adaptive Boundary and Semantic Composite Segmentation method(ABSCS)is proposed.This method can precisely semantic segment individual objects in large-size aerial images with limited GPU performances.It includes adaptively dividing and modifying the aerial images with the proposed principles and methods,using the deep learning method to semantic segment and preprocess the small divided pieces,using three traditional methods to segment and preprocess original-size aerial images,adaptively selecting traditional results tomodify the boundaries of individual objects in deep learning results,and combining the results of different objects.Individual object semantic segmentation experiments are conducted by using the AeroScapes dataset,and their results are analyzed qualitatively and quantitatively.The experimental results demonstrate that the proposed method can achieve more promising object boundaries than the original deep learning method.This work also demonstrates the advantages of the proposed method in applications of point cloud semantic segmentation and image inpainting.

Bidirectional parallel multi-branch convolution feature pyramid network for target detection in aerial images of swarm UAVs

In this paper,based on a bidirectional parallel multi-branch feature pyramid network(BPMFPN),a novel one-stage object detector called BPMFPN Det is proposed for real-time detection of ground multi-scale targets by swarm unmanned aerial vehicles(UAVs).First,the bidirectional parallel multi-branch convolution modules are used to construct the feature pyramid to enhance the feature expression abilities of different scale feature layers.Next,the feature pyramid is integrated into the single-stage object detection framework to ensure real-time performance.In order to validate the effectiveness of the proposed algorithm,experiments are conducted on four datasets.For the PASCAL VOC dataset,the proposed algorithm achieves the mean average precision(mAP)of 85.4 on the VOC 2007 test set.With regard to the detection in optical remote sensing(DIOR)dataset,the proposed algorithm achieves 73.9 mAP.For vehicle detection in aerial imagery(VEDAI)dataset,the detection accuracy of small land vehicle(slv)targets reaches 97.4 mAP.For unmanned aerial vehicle detection and tracking(UAVDT)dataset,the proposed BPMFPN Det achieves the mAP of 48.75.Compared with the previous state-of-the-art methods,the results obtained by the proposed algorithm are more competitive.The experimental results demonstrate that the proposed algorithm can effectively solve the problem of real-time detection of ground multi-scale targets in aerial images of swarm UAVs.

Desertification Detection in Makkah Region based on Aerial Images Classification

Desertification has become a global threat and caused a crisis,especially in Middle Eastern countries,such as Saudi Arabia.Makkah is one of the most important cities in Saudi Arabia that needs to be protected from desertification.The vegetation area in Makkah has been damaged because of desertification through wind,floods,overgrazing,and global climate change.The damage caused by desertification can be recovered provided urgent action is taken to prevent further degradation of the vegetation area.In this paper,we propose an automatic desertification detection system based on Deep Learning techniques.Aerial images are classified using Convolutional Neural Networks(CNN)to detect land state variation in real-time.CNNs have been widely used for computer vision applications,such as image classification,image segmentation,and quality enhancement.The proposed CNN model was trained and evaluated on the Arial Image Dataset(AID).Compared to state-of-the-art methods,the proposed model has better performance while being suitable for embedded implementation.It has achieved high efficiency with 96.47% accuracy.In light of the current research,we assert the appropriateness of the proposed CNN model in detecting desertification from aerial images.

A fast, accurate and dense feature matching algorithm for aerial images

Three-dimensional(3D)reconstruction based on aerial images has broad prospects,and feature matching is an important step of it.However,for high-resolution aerial images,there are usually problems such as long time,mismatching and sparse feature pairs using traditional algorithms.Therefore,an algorithm is proposed to realize fast,accurate and dense feature matching.The algorithm consists of four steps.Firstly,we achieve a balance between the feature matching time and the number of matching pairs by appropriately reducing the image resolution.Secondly,to realize further screening of the mismatches,a feature screening algorithm based on similarity judgment or local optimization is proposed.Thirdly,to make the algorithm more widely applicable,we combine the results of different algorithms to get dense results.Finally,all matching feature pairs in the low-resolution images are restored to the original images.Comparisons between the original algorithms and our algorithm show that the proposed algorithm can effectively reduce the matching time,screen out the mismatches,and improve the number of matches.

Automatic detection and evaluation of sugarcane planting rows in aerial images

Sugarcane planting is an important and growing activity in Brazil.Thereupon,several techniques have been developed over the years to maximize crop productivity and profit,amongst them,processing of sugarcane field images.In this sense,this research aims to identify and analyze crop rows and measure their gaps from aerial images of sugarcane fields.For this,a small Remotely Piloted Aircraft captured the images,generating orthomosaics of the areas for analysis.Then,each orthomosaic is classified with the K-Nearest Neighbor algorithm to segment regions of interest.Planting row orientation is estimated using the RGB gradient filter.Morphological operations and computational geometry models are then used to detect and map rows and gaps along the planting row segment.To evaluate the results,crop rows are mapped and compared to manually taken measurements.Our technique obtained an error smaller than 2%when compared to gap length in crop rows from an orthomosaic with the area of 8.05 ha(ha).The proposed approach can map the positioning of the automatically generated row segments appropriately onto manually created segments.Moreover,our method also achieved similar results when confronted with a manual technique for differing growth stages(40 and 80 days after harvest)of the sugarcane crop.The proposed method presents a great potential to be adopted in sugarcane planting monitoring。

Fog removal and enhancement method for UAV aerial images based on dark channel prior

The existing UAV aerial image de-fog methods have low image contrast after de-fog,the difference between light and dark image is not obvious,leading to poor de-fog effect.Therefore,an aerial image de-fog enhancement method based on dark channel a priori is proposed.The image variance and absolute gradient mean are combined to get the weight coefficients,and the edge pixels are smoothed by using the multiple decomposition form.The image intensity is calculated and the noise is reduced.A convolution neural network is introduced to calculate the atmospheric transmittance in haze.Based on this,dark channel prior algorithm is used to enhance the light and shade difference of aerial photography image and realise the de-fog enhancement of aerial photography image.To verify the performance of the proposed method,simulation experiments are designed which were compared with the existing methods results in better fog-removing effect,higher contrast and shorter time.

An Accurate and Real-time Method of Self-blast Glass Insulator Location Based on Faster R-CNN and U-net with Aerial Images

This paper proposes a new deep learning framework for the location of broken insulators(in particular the self-blast glass insulator)in aerial images.We address the broken insulators location problem in a low signal-noise-ratio(SNR)setting.We deal with two modules:1)object detection based on Faster R-CNN,and 2)classification of pixels based on U-net.For the first time,our paper combines the above two modules.This combination is motivated as follows:Faster R-CNN is used to improve SNR,while the U-net is used for classification of pixels.A diverse aerial image set measured by a power grid in China is tested to validate the proposed approach.Furthermore,a comparison is made among different methods and the result shows that our approach is accurate in real time.

Improved seeded region growing for detection of water bodies in aerial images

In aerial images,near-specular and specular reflection often appear in water bodies.They often lead to irregular brightness or color changes in water bodies and even produce hot spots,harmful to radiometric normalization.Therefore,water bodies must be eliminated when calculating radiometric differences during radiometric normalization of aerial images.In this paper,a simple method to detect water bodies in aerial images based on texture features is presented,an improved seeded region growing(SRG)method.A texture feature is calculated using the relative standard deviation index(RSDI)and a coarse-to-fine procedure is employed.The proposed method includes a multiple partition strategy and a refinement in gradient image that improves the reliability and accuracy of water body detection.By fusing water bodies detected in multiple images,hot spots in these water bodies are also detected.Experiments validate the feasibility and effectiveness of the proposed method.

Gamma Correction for Brightness Preservation in Natural Images

Due to improper acquisition settings and other noise artifacts,the image degraded to yield poor mean preservation in brightness.The simplest way to improve the preservation is the implementation of histogram equalization.Because of over-enhancement,it failed to preserve the mean brightness and produce the poor quality of the image.This paper proposes a multi-scale decomposi-tion for brightness preservation using gamma correction.After transformation to hue,saturation and intensity(HSI)channel,the 2D-discrete wavelet transform decomposed the intensity component into low and high-pass coefficients.At the next phase,gamma correction is used by auto-tuning the scale value.The scale is the modified constant value used in the logarithmic function.Further,the scale value is optimized to obtain better visual quality in the image.The optimized value is the weighted distribution of standard deviation-mean of low pass coefficients.Finally,the experimental result is estimated in terms of quality assessment measures used as absolute mean brightness error,the measure of information detail,signal to noise ratio and patch-based contrast quality in the image.By comparison,the proposed method proved to be suitably remarkable in retaining the mean brightness and better visual quality of the image.

Archimedes Optimization with Deep Learning Based Aerial Image Classification for Cybersecurity Enabled UAV Networks

The recent adoption of satellite technologies,unmanned aerial vehicles(UAVs)and 5G has encouraged telecom networking to evolve into more stable service to remote areas and render higher quality.But,security concerns with drones were increasing as drone nodes have been striking targets for cyberattacks because of immensely weak inbuilt and growing poor security volumes.This study presents an Archimedes Optimization with Deep Learning based Aerial Image Classification and Intrusion Detection(AODL-AICID)technique in secure UAV networks.The presented AODLAICID technique concentrates on two major processes:image classification and intrusion detection.For aerial image classification,the AODL-AICID technique encompasses MobileNetv2 feature extraction,Archimedes Optimization Algorithm(AOA)based hyperparameter optimizer,and backpropagation neural network(BPNN)based classifier.In addition,the AODLAICID technique employs a stacked bi-directional long short-term memory(SBLSTM)model to accomplish intrusion detection for cybersecurity in UAV networks.At the final stage,the Nadam optimizer is utilized for parameter tuning of the SBLSTM approach.The experimental validation of the AODLAICID technique is tested and the obtained values reported the improved performance of the AODL-AICID technique over other models.

Adaptive cropping shallow attention network for defect detection of bridge girder steel using unmanned aerial vehicle images

Bridges are an important part of railway infrastructure and need regular inspection and maintenance.Using unmanned aerial vehicle(UAV)technology to inspect railway infrastructure is an active research issue.However,due to the large size of UAV images,flight distance,and height changes,the object scale changes dramatically.At the same time,the elements of interest in railway bridges,such as bolts and corrosion,are small and dense objects,and the sample data set is seriously unbalanced,posing great challenges to the accurate detection of defects.In this paper,an adaptive cropping shallow attention network(ACSANet)is proposed,which includes an adaptive cropping strategy for large UAV images and a shallow attention network for small object detection in limited samples.To enhance the accuracy and generalization of the model,the shallow attention network model integrates a coordinate attention(CA)mechanism module and an alpha intersection over union(α-IOU)loss function,and then carries out defect detection on the bolts,steel surfaces,and railings of railway bridges.The test results show that the ACSANet model outperforms the YOLOv5s model using adaptive cropping strategy in terms of the total mAP(an evaluation index)and missing bolt mAP by 5%and 30%,respectively.Also,compared with the YOLOv5s model that adopts the common cropping strategy,the total mAP and missing bolt mAP are improved by 10%and 60%,respectively.Compared with the YOLOv5s model without any cropping strategy,the total mAP and missing bolt mAP are improved by 40%and 67%,respectively.

Vision and Geolocation Data Combination for Precise Human Detection and Tracking in Search and Rescue Operations

In this paper, a study and evaluation of the combination of GPS/GNSS techniques and advanced image processing algorithms for distressed human detection, positioning and tracking, from a fully autonomous Unmanned Aerial Vehicle (UAV)-based rescue support system, are presented. In particular, the issue of human detection both on terrestrial and marine environment under several illumination and background conditions, as the human silhouette in water differs significantly from a terrestrial one, is addressed. A robust approach, including an adaptive distressed human detection algorithm running every N input image frames combined with a much faster tracking algorithm, is proposed. Real time or near-real-time distressed human detection rates achieved, using a single, low cost day/night NIR camera mounted onboard a fully autonomous UAV for Search and Rescue (SAR) operations. Moreover, the generation of our own dataset, for the image processing algorithms training is also presented. Details about both hardware and software configuration as well as the assessment of the proposed approach performance are fully discussed. Last, a comparison of the proposed approach to other human detection methods used in the literature is presented.

Unsupervised noise-robust feature extraction for aerial image classification

The rich data provided by satellites and unmanned aerial vehicles bring opportunities to directly model aerial image features by extracting their spatial and structural patterns.Although convolutional autoencoders(CAEs)have been attained a remarkable performance in ideal aerial image feature extraction,they are still challenging to extract information from noisy images which are generated from capture and transmission.In this paper,a novel CAE-based noise-robust unsupervised learning method is proposed for extracting high-level features accurately from aerial images and mitigating the effect of noise.Different from conventional CAEs,the proposed method introduces the noise-robust module between the encoder and the decoder.Besides,several pooling layers in CAEs are replaced by convolutional layers with stride=2.The performance of feature extraction is evaluated by the prediction accuracy and the accuracy loss in image classification experiments.A 5-classes aerial optical scene and a 9-classes hyperspectral image(HSI)data set are utilized for optical image and HSI feature extraction,respectively.Highlevel features extracted from aerial images are utilized for image classification by a linear support vector machine(SVM)classifier.Experimental results indicate that the proposed method improves the classification accuracy for noisy images(Gaussian noise 2Dσ=0.1,3Dσ=60)in both optical images(2D 87.5%)and HSIs(3D 85.6%)compared with the traditional CAE(2D 78.6%,3D 84.2%).The accuracy loss in classification experiments increases with the increment of noise.Compared with the traditional CAE(2D 15.7%,3D 11.8%),the proposed method shows the lower classification accuracy loss in experiments(2D 0.3%,3D 6.3%).The proposed unsupervised noise-robust feature extraction method attains desirable classification accuracy in ideal input and enhances the feature extraction capability from noisy input.

Analysis of large-scale UAV images using a multi-scale hierarchical representation

Unmanned aerial vehicle(UAV)-based imaging systems have many superiorities compared with other platforms,such as high flexibility and low cost in collecting images,providing wide application prospects.However,the acquisition of the UAV-based image commonly results in very high resolution and very large-scale images,which poses great challenges for subsequent applications.Therefore,an efficient representation of large-scale UAV images is necessary for the extraction of the required information in a reasonable time.In this work,we proposed a multi-scale hierarchical representation,i.e.binary partition tree,for analyzing large-scale UAV images.More precisely,we first obtained an initial partition of images by an oversegmentation algorithm,i.e.the simple linear iterative clustering.Next,we merged the similar superpixels to build an object-based hierarchical structure by fully considering the spectral and spatial information of the superpixels and their topological relationships.Moreover,objects of interest and optimal segmentation were obtained using object-based analysis methods with the hierarchical structure.Experimental results on processing the post-seismic UAV images of the 2013 Ya’an earthquake and the mosaic of images in the South-west of Munich demonstrate the effectiveness and efficiency of our proposed method.

Super-resolution enhancement of UAV images based on fractional calculus and POCS

A super-resolution enhancement algorithm was proposed based on the combination of fractional calculus and Projection onto Convex Sets(POCS)for unmanned aerial vehicles(UAVs)images.The representative problems of UAV images including motion blur,fisheye effect distortion,overexposed,and so on can be improved by the proposed algorithm.The fractional calculus operator is used to enhance the high-resolution and low-resolution reference frames for POCS.The affine transformation parameters between low-resolution images and reference frame are calculated by Scale Invariant Feature Transform(SIFT)for matching.The point spread function of POCS is simulated by a fractional integral filter instead of Gaussian filter for more clarity of texture and detail.The objective indices and subjective effect are compared between the proposed and other methods.The experimental results indicate that the proposed method outperforms other algorithms in most cases,especially in the structure and detail clarity of the reconstructed images.

Calibration of digital camera integration accuracy for low-cost oblique aerial photogrammetry

This work describes a calibration process for inexpensive consumer cameras integrated into a low cost and compact aerial multi-view imager for remote sensing and photogrammetry.The main advantage of this design is to make the filming component lightweight and rapidly deployable,as well as reducing cost when compared with mainstream commercial oblique imagery.An in situ flight test was carried out in Guiyang.In that experiment,a meridian convergence-based approach was adopted to adjust preprocessing,the residue error and the captured images’exterior orientation linear and angular parameters were calculated by means of the direct geo-referencing approach yielding a favorable outcome for exterior orientation linear parameters of the camera,around 0.2–0.3 m deviation from the actual measured results at 1000 m flight above ground level.The camera’s exterior orientation angular parametersφ,ωwhose difference compared with the standard aerial aero triangulation approach reached a high accuracy level within the intended endurance of 0.005°.These results indicate that the compact implementation of the oblique aerial imager comprised of consumer level off-the-shelf digital cameras achieved competitive accuracy at a low cost and high versatility.

The infuence of window size on remote sensing-based prediction of forest structural variables

Background:Determining the appropriate window size is a critical step in the estimation process of stand structural variables based on remote sensing data.Because the value of the reference laser and image metrics that afect the quality of the prediction model depends on window size.However,suitable window sizes are usually determined by trial and error.There are a limited number of published studies evaluating appropriate window sizes for diferent remote sensing data.This research investigated the efect of window size on predicting forest structural variables using airborne LiDAR data,digital aerial image and WorldView-3 satellite image.Results:In the WorldView-3 and digital aerial image,signifcant diferences were observed in the prediction accuracies of the structural variables according to diferent window sizes.For the estimation based on WorldView-3 in black pine stands,the optimal window sizes for stem number(N),volume(V),basal area(BA)and mean height(H)were determined as 1000 m^(2),100 m^(2),100 m^(2) and 600 m^(2),respectively.In oak stands,the R^(2) values of each moving window size were almost identical for N and BA.The optimal window size was 400 m^(2) for V and 600 m^(2) for H.For the estimation based on aerial image in black pine stands,the 800 m^(2) window size was optimal for N and H,the 600 m^(2) window size was optimal for V and the 1000 m^(2) window size was optimal for BA.In the oak stands,the optimal window sizes for N,V,BA and H were determined as 1000 m^(2),100 m^(2),100 m^(2) and 600 m^(2),respectively.The optimal window sizes may need to be scaled up or down to match the stand canopy components.In the LiDAR data,the R^(2) values of each window size were almost identical for all variables of the black pine and the oak stands.Conclusion:This study illustrated that the window size has an efect on the prediction accuracy in estimating forest structural variables based on remote sensing data.Moreover,the results showed that the optimal window size for forest structural variables varies according to remote sensing data and tree species composition.