Tree species classification using deep learning and RGB optical images obtained by an unmanned aerial vehicle

The diversity of tree species and the complexity of land use in cities create challenging issues for tree species classification.The combination of deep learning methods and RGB optical images obtained by unmanned aerial vehicles(UAVs) provides a new research direction for urban tree species classification.We proposed an RGB optical image dataset with 10 urban tree species,termed TCC10,which is a benchmark for tree canopy classification(TCC).TCC10 dataset contains two types of data:tree canopy images with simple backgrounds and those with complex backgrounds.The objective was to examine the possibility of using deep learning methods(AlexNet,VGG-16,and ResNet-50) for individual tree species classification.The results of convolutional neural networks(CNNs) were compared with those of K-nearest neighbor(KNN) and BP neural network.Our results demonstrated:(1) ResNet-50 achieved an overall accuracy(OA) of 92.6% and a kappa coefficient of 0.91 for tree species classification on TCC10 and outperformed AlexNet and VGG-16.(2) The classification accuracy of KNN and BP neural network was less than70%,while the accuracy of CNNs was relatively higher.(3)The classification accuracy of tree canopy images with complex backgrounds was lower than that for images with simple backgrounds.For the deciduous tree species in TCC10,the classification accuracy of ResNet-50 was higher in summer than that in autumn.Therefore,the deep learning is effective for urban tree species classification using RGB optical images.

Development of image-based wheat spike counter through a Faster R-CNN algorithm and application for genetic studies

Spike number(SN) per unit area is one of the major determinants of grain yield in wheat. Development of high-throughput techniques to count SN from large populations enables rapid and cost-effective selection and facilitates genetic studies. In the present study, we used a deep-learning algorithm, i.e., Faster Region-based Convolutional Neural Networks(Faster R-CNN) on Red-Green-Blue(RGB) images to explore the possibility of image-based detection of SN and its application to identify the loci underlying SN. A doubled haploid population of 101 lines derived from the Yangmai 16/Zhongmai 895 cross was grown at two sites for SN phenotyping and genotyped using the high-density wheat 660 K SNP array.Analysis of manual spike number(MSN) in the field, image-based spike number(ISN), and verification of spike number(VSN) by Faster R-CNN revealed significant variation(P < 0.001) among genotypes, with high heritability ranged from 0.71 to 0.96. The coefficients of determination(R^(2)) between ISN and VSN was 0.83, which was higher than that between ISN and MSN(R^(2)= 0.51), and between VSN and MSN(R^(2)= 0.50). Results showed that VSN data can effectively predict wheat spikes with an average accuracy of 86.7% when validated using MSN data. Three QTL Qsnyz.caas-4 DS, Qsnyz.caas-7 DS, and QSnyz.caas-7 DL were identified based on MSN, ISN and VSN data, while QSnyz.caas-7 DS was detected in all the three data sets. These results indicate that using Faster R-CNN model for image-based identification of SN per unit area is a precise and rapid phenotyping method, which can be used for genetic studies of SN in wheat.

An integrated rice panicle phenotyping method based on X-ray and RGB scanning and deep learning

Rice panicle phenotyping is required in rice breeding for high yield and grain quality.To fully evaluate spikelet and kernel traits without threshing and hulling,using X-ray and RGB scanning,we developed an integrated rice panicle phenotyping system and a corresponding image analysis pipeline.We compared five methods of counting spikelets and found that Faster R-CNN achieved high accuracy(R~2 of 0.99)and speed.Faster R-CNN was also applied to indica and japonica classification and achieved 91%accuracy.The proposed integrated panicle phenotyping method offers benefit for rice functional genetics and breeding.

Encryption and Decryption of Color Images through Random Disruption of Rows and Columns

In modern society,information is becoming increasingly interconnected through networks,and the rapid development of information technology has caused people to pay more attention to the encryption and the protection of information.Image encryption technology is a key technology for ensuring the security performance of images.We extracted single channel RGB component images from a color image using MATLAB programs,encrypted and decrypted the color images by randomly disrupting rows,columns and regions of the image.Combined with histograms and the visual judgments of encryption images,it is shown that the information of the original image cannot be obtained from the encryption image easily.The results show that the color-image encryptions with the algorithm we used have good effect and fast operation speed.Thus this algorithm has certain practical value.

Convolutional Neural Network-based Leakage Detection of Crude Oil Transmission Pipes

Due to the rapid development in the petroleum industry,the leakage detection of crude oil transmission pipes has become an increasingly crucial issue.At present,oil plants at home and abroad mostly use manual inspection method for detection.This traditional method is not only inefficient but also labor-intensive.The present paper proposes a novel convolutional neural network(CNN)architecture for automatic leakage level assessment of crude oil transmission pipes.An experimental setup is developed,where a visible camera and a thermal imaging camera are used to collect image data and analyze various leakage conditions.Specifically,images are collected from various pipes with no leaking and different leaking states.Apart from images from existing pipelines,images are collected from the experimental setup with different types of joints to simulate leakage conditions in the real world.The main contributions of the present paper are,developing a convolutional neural network to classify the information in red-green-blue(RGB)and thermal images,development of the experimental setup,conducting leakage experiments,and analyzing the data using the developed approach.By successfully combining the two types of images,the proposed method is able to achieve a higher classification accuracy,compared to other methods that use RGB images or thermal images alone.Especially,compared with the method that uses thermal images only,the accuracy increases from about 91%to over 96%.

Exploring 2D projection and 3D spatial information for aircraft 6D pose

The 6D pose estimation is important for the safe take-off and landing of the aircraft using a single RGB image. Due to the large scene and large depth, the exiting pose estimation methods have unstratified performance on the accuracy. To achieve precise 6D pose estimation of the aircraft, an end-to-end method using an RGB image is proposed. In the proposed method, the2D and 3D information of the keypoints of the aircraft is used as the intermediate supervision,and 6D pose information of the aircraft in this intermediate information will be explored. Specifically, an off-the-shelf object detector is utilized to detect the Region of the Interest(Ro I) of the aircraft to eliminate background distractions. The 2D projection and 3D spatial information of the pre-designed keypoints of the aircraft is predicted by the keypoint coordinate estimator(Kp Net).The proposed method is trained in an end-to-end fashion. In addition, to deal with the lack of the related datasets, this paper builds the Aircraft 6D Pose dataset to train and test, which captures the take-off and landing process of three types of aircraft from 11 views. Compared with the latest Wide-Depth-Range method on this dataset, our proposed method improves the average 3D distance of model points metric(ADD) and 5° and 5 m metric by 86.8% and 30.1%, respectively. Furthermore, the proposed method gets 9.30 ms, 61.0% faster than YOLO6D with 23.86 ms.

VNLSTM-PoseNet: A novel deep ConvNet for real-time 6-DOF camera relocalization in urban streets

Image-based relocalization is a renewed interest in outdoor environments,because it is an important problem with many applications.PoseNet introduces Convolutional Neural Network(CNN)for the first time to realize the real-time camera pose solution based on a single image.In order to solve the problem of precision and robustness of PoseNet and its improved algorithms in complex environment,this paper proposes and implements a new visual relocation method based on deep convolutional neural networks(VNLSTM-PoseNet).Firstly,this method directly resizes the input image without cropping to increase the receptive field of the training image.Then,the image and the corresponding pose labels are put into the improved Long Short-Term Memory based(LSTM-based)PoseNet network for training and the network is optimized by the Nadam optimizer.Finally,the trained network is used for image localization to obtain the camera pose.Experimental results on outdoor public datasets show our VNLSTM-PoseNet can lead to drastic improvements in relocalization performance compared to existing state-of-theart CNN-based methods.

3D Vehicle Detection Based on LiDAR and Camera Fusion

Nowadays,the deep learning for object detection has become more popular and is widely adopted in many fields.This paper focuses on the research of LiDAR and camera sensor fusion technology for vehicle detection to ensure extremely high detection accuracy.The proposed network architecture takes full advantage of the deep information of both the LiDAR point cloud and RGB image in object detection.First,the LiDAR point cloud and RGB image are fed into the system.Then a high-resolution feature map is used to generate a reliable 3D object proposal for both the LiDAR point cloud and RGB image.Finally,3D box regression is performed to predict the extent and orientation of vehicles in 3D space.Experiments on the challenging KITTI benchmark show that the proposed approach obtains ideal detection results and the detection time of each frame is about 0.12 s.This approach could establish a basis for further research in autonomous vehicles.