A Systematic Literature Review of Machine Learning and Deep Learning Approaches for Spectral Image Classification in Agricultural Applications Using Aerial Photography

Recently,there has been a notable surge of interest in scientific research regarding spectral images.The potential of these images to revolutionize the digital photography industry,like aerial photography through Unmanned Aerial Vehicles(UAVs),has captured considerable attention.One encouraging aspect is their combination with machine learning and deep learning algorithms,which have demonstrated remarkable outcomes in image classification.As a result of this powerful amalgamation,the adoption of spectral images has experienced exponential growth across various domains,with agriculture being one of the prominent beneficiaries.This paper presents an extensive survey encompassing multispectral and hyperspectral images,focusing on their applications for classification challenges in diverse agricultural areas,including plants,grains,fruits,and vegetables.By meticulously examining primary studies,we delve into the specific agricultural domains where multispectral and hyperspectral images have found practical use.Additionally,our attention is directed towards utilizing machine learning techniques for effectively classifying hyperspectral images within the agricultural context.The findings of our investigation reveal that deep learning and support vector machines have emerged as widely employed methods for hyperspectral image classification in agriculture.Nevertheless,we also shed light on the various issues and limitations of working with spectral images.This comprehensive analysis aims to provide valuable insights into the current state of spectral imaging in agriculture and its potential for future advancements.

MULTI-SPECTRAL AND HYPERSPECTRAL IMAGE FUSION USING 3-D WAVELET TRANSFORM

Image fusion is performed between one band of multi-spectral image and two bands of hyperspectral image to produce fused image with the same spatial resolution as source multi-spectral image and the same spectral resolution as source hyperspeetral image. According to the characteristics and 3-Dimensional （3-D） feature analysis of multi-spectral and hyperspectral image data volume, the new fusion approach using 3-D wavelet based method is proposed. This approach is composed of four major procedures： Spatial and spectral resampling, 3-D wavelet transform, wavelet coefficient integration and 3-D inverse wavelet transform. Especially, a novel method, Ratio Image Based Spectral Resampling （RIBSR）method, is proposed to accomplish data resampling in spectral domain by utilizing the property of ratio image. And a new fusion rule, Average and Substitution （A＆S） rule, is employed as the fusion rule to accomplish wavelet coefficient integration. Experimental results illustrate that the fusion approach using 3-D wavelet transform can utilize both spatial and spectral characteristics of source images more adequately and produce fused image with higher quality and fewer artifacts than fusion approach using 2-D wavelet transform. It is also revealed that RIBSR method is capable of interpolating the missing data more effectively and correctly, and A＆S rule can integrate coefficients of source images in 3-D wavelet domain to preserve both spatial and spectral features of source images more properly.

Robust key point descriptor for multi-spectral image matching

Histogram of collinear gradient-enhanced coding （HCGEC）, a robust key point descriptor for multi-spectral image matching, is proposed. The HCGEC mainly encodes rough structures within an image and suppresses detailed textural information, which is desirable in multi-spectral image matching. Experiments on two multi-spectral data sets demonstrate that the proposed descriptor can yield significantly better results than some state-of- the-art descriptors.

Extracting Feature Bands for Damaged Rice Leaves by Planthoppers Using Multi-spectral Imaging Technology

[Objective] The aim of this study was to extract effective feature bands of damaged rice leaves by planthoppers to make identification and classification rapidly from great amounts of imaging spectral data. [Method] The experiment, using multi-spectral imaging system, acquired the multi-spectral images of damaged rice leaves from band 400 to 720 nm by interval of 5 nm. [Result] According to the principle of band index, it was calculated that the bands at 515, 510, 710, 555, 630, 535, 505, 530 and 595 nm were having high band index value with rich information and little correlation. Furthermore, the experiment used two classification methods and calcu-lated the classification accuracy higher than 90.00% for feature bands and ful bands of damaged rice leaves by planthoppers respectively. [Conclusion] It can be con-cluded that these bands can be considered as effective feature bands to identify damaged rice leaves by planthoppers quickly from a large scale of crops.

Multi-Spectral and Fluorescence Imaging in Prevention of Overdose of Herbicides: The Case of Maize

Evaluation of the impact of herbicides on maize was done through multi- spectral and multi-modal imaging and multi-spectral fluorescence imaging combined with statistical methods. Spectra containing 13 wavelengths ranging from 375 nm to 940 nm were derived from multi-spectral images in transmission, reflection and scattering mode and fluorescence images obtained using high-pass filters (F450 nm, F500 nm, F550 nm, F600 nm, F650 nm) on control maize samples and maize samples treated with Herbextra herbicide were used. The appearance of the spectra allowed us to characterize the effect of the herbicide on the maize pigment concentration. The fluorescence images allowed us to track the fate of absorbed energy and through PLS-DA and SVM-DA to discriminate the two leaf categories with very low error rates for the test, i.e. 4.9% and 2% respectively. The results of this technique can be used in the context of precision agriculture.

Application of PCA Numalgorithm in Remote Sensing Image Processing

A numerical algorithm of principal component analysis (PCA) is proposed and its application in remote sensing image processing is introduced: (1) Multispectral image compression;(2) Multi-spectral image noise cancellation;(3) Information fusion of multi-spectral images and spot panchromatic images. The software experiments verify and evaluate the effectiveness and accuracy of the proposed algorithm.

Low-dimensional multi-spectral space for color reproduction based on nonnegative constrained principal component analysis

In order to overcome the shortcomings that the reconstructed spectral reflectance may be negative when using the classic principal component analysis （PCA）to reduce the dimensions of the multi-spectral data, a nonnegative constrained principal component analysis method is proposed to construct a low-dimensional multi-spectral space and accomplish the conversion between the new constructed space and the multispectral space. First, the reason behind the negative data is analyzed and a nonnegative constraint is imposed on the classic PCA. Then a set of nonnegative linear independence weight vectors of principal components is obtained, by which a lowdimensional space is constructed. Finally, a nonlinear optimization technique is used to determine the projection vectors of the high-dimensional multi-spectral data in the constructed space. Experimental results show that the proposed method can keep the reconstructed spectral data in [ 0, 1 ]. The precision of the space created by the proposed method is equivalent to or even higher than that by the PCA.

MULTI-SOURCE REMOTE SENSING IMAGE FUSION BASED ON SUPPORT VECTOR MACHINE

Remote Sensing image fusion is an effective way to use the large volume ofdata from multi-source images. This paper introduces a new method of remote sensing image fusionbased on support vector machine (SVM), using high spatial resolution data SPIN-2 and multi-spectralremote sensing data SPOT-4. Firstly, the new method is established by building a model of remotesensing image fusion based on SVM. Then by using SPIN-2 data and SPOT-4 data, image classificationfusion is tested. Finally, an evaluation of the fusion result is made in two ways. 1) Fromsubjectivity assessment, the spatial resolution of the fused image is improved compared to theSPOT-4. And it is clearly that the texture of the fused image is distinctive. 2) From quantitativeanalysis, the effect of classification fusion is better. As a whole, the re-suit shows that theaccuracy of image fusion based on SVM is high and the SVM algorithm can be recommended forapplication in remote sensing image fusion processes.

EDGE DETECTION METHOD OF REMOTE SENSING IMAGES BASED ON MATHEMATICAL MORPHOLOGY OF MULTI-STRUCTURE ELEMENTS

This paper puts forward an effective, specific algorithm for edge detection. Based on multi-structure elements of gray mathematics morphology, in the light of difference between noise and edge shape of RS images, the paper establishes multi-structure elements to detect edge by utilizing the grey form transformation principle. Compared with some classical edge detection operators, such as Sobel Edge Detection Operator, LOG Edge Detection Operator, and Canny Edge Detection Operator, the experiment indicates that this new algorithm possesses very good edge detection ability, which can detect edges more effectively, but its noise-resisting ability is relatively low. Because of the bigger noise & remote sensing image, the authors probe into putting forward other edge detection method based on combination of wavelet directivity checkout technology and small-scale Mathematical Morphology finally. So, position at the edge can be accurately located, the noise can be inhibited to a certain extent and the effect of edge detection is obvious.

USING COVARIANCE INTERSECTION FOR CHANGE DETECTION IN REMOTE SENSING IMAGES

In this paper,an unsupervised change detection technique for remote sensing images acquired on the same geographical area but at different time instances is proposed by conducting Covariance Intersection(CI) to perform unsupervised fusion of the final fuzzy partition matrices from the Fuzzy C-Means(FCM) clustering for the feature space by applying compressed sampling to the given remote sensing images.The proposed approach exploits a CI-based data fusion of the membership function matrices,which are obtained by taking the Fuzzy C-Means(FCM) clustering of the frequency-domain feature vectors and spatial-domain feature vectors,aimed at enhancing the unsupervised change detection performance.Compressed sampling is performed to realize the image local feature sampling,which is a signal acquisition framework based on the revelation that a small collection of linear projections of a sparse signal contains enough information for stable recovery.The experimental results demonstrate that the proposed algorithm has a good change detection results and also performs quite well on denoising purpose.

Analysis of Spectral Characteristics Based on Optical Remote Sensing and SAR Image Fusion

Because of cloudy and rainy weather in south China, optical remote sens-ing images often can＇t be obtained easily. With the regional trial results in Baoying, Jiangsu province, this paper explored the fusion model and effect of ENVISAT/SAR and HJ-1A satel ite multispectral remote sensing images. Based on the ARSIS strat-egy, using the wavelet transform and the Interaction between the Band Structure Model （IBSM）, the research progressed the ENVISAT satel ite SAR and the HJ-1A satel ite CCD images wavelet decomposition, and low/high frequency coefficient re-construction, and obtained the fusion images through the inverse wavelet transform. In the light of low and high-frequency images have different characteristics in differ-ent areas, different fusion rules which can enhance the integration process of self-adaptive were taken, with comparisons with the PCA transformation, IHS transfor-mation and other traditional methods by subjective and the corresponding quantita-tive evaluation. Furthermore, the research extracted the bands and NDVI values around the fusion with GPS samples, analyzed and explained the fusion effect. The results showed that the spectral distortion of wavelet fusion, IHS transform, PCA transform images was 0.101 6, 0.326 1 and 1.277 2, respectively and entropy was 14.701 5, 11.899 3 and 13.229 3, respectively, the wavelet fusion is the highest. The method of wavelet maintained good spectral capability, and visual effects while improved the spatial resolution, the information interpretation effect was much better than other two methods.

DiriNet:An Estimation Network for Spectral Response Function and Point Spread Function

Hyper-and multi-spectral image fusion is an important technology to produce hyper-spectral and hyper-resolution images,which always depends on the spectral response function andthe point spread function.However,few works have been payed on the estimation of the two degra-dation functions.To learn the two functions from image pairs to be fused,we propose a Dirichletnetwork,where both functions are properly constrained.Specifically,the spatial response function isconstrained with positivity,while the Dirichlet distribution along with a total variation is imposedon the point spread function.To the best of our knowledge,the neural network and the Dirichlet regularization are exclusively investigated,for the first time,to estimate the degradation functions.Both image degradation and fusion experiments demonstrate the effectiveness and superiority of theproposed Dirichlet network.

Multi-spectral image fusion method based on two channels non-separable wavelets

A construction method of two channels non-separable wavelets filter bank which dilation matrix is [1, 1; 1,-1] and its application in the fusion of multi-spectral image are presented. Many 4×4 filter banks are designed. The multi-spectral image fusion algorithm based on this kind of wavelet is proposed. Using this filter bank, multi-resolution wavelet decomposition of the intensity of multi-spectral image and panchromatic image is performed, and the two low-frequency components of the intensity and the panchromatic image are merged by using a tradeoff parameter. The experiment results show that this method is good in the preservation of spectral quality and high spatial resolution information. Its performance in preserving spectral quality and high spatial information is better than the fusion method based on DWFT and IHS. When the parameter t is closed to 1, the fused image can obtain rich spectral information from the original MS image. The amount of computation reduced to only half of the fusion method based on four channels wavelet transform.

Acquiring multi-spectral images by digital still cameras based on XYZLMS interim connection space

A method based on the XYZLMS interim connection space is proposed to accurately acquire the multi-spectral images by digital still cameras. The XYZLMS values are firstly predicted from RGB values by polynomial model with local training samples and then spectral reflectance is constructed from XYZLMS values by pseudo-inverse method. An experiment is implemented for multi-spectral image acquisition based on a commercial digital still camera. The results indicate that multi-spectral images can be accurately acquired except the very dark colors.

Multispectral image compression and encryption method based on tensor decomposition in wavelet domain

Multispectral image compression and encryption algorithms commonly suffer from issues such as low compression efficiency,lack of synchronization between the compression and encryption proces-ses,and degradation of intrinsic image structure.A novel approach is proposed to address these is-sues.Firstly,a chaotic sequence is generated using the Lorenz three-dimensional chaotic mapping to initiate the encryption process,which is XORed with each spectral band of the multispectral image to complete the initial encryption of the image.Then,a two-dimensional lifting 9/7 wavelet transform is applied to the processed image.Next,a key-sensitive Arnold scrambling technique is employed on the resulting low-frequency image.It effectively eliminates spatial redundancy in the multispectral image while enhancing the encryption process.To optimize the compression and encryption processes further,fast Tucker decomposition is applied to the wavelet sub-band tensor.It effectively removes both spectral redundancy and residual spatial redundancy in the multispectral image.Finally,the core tensor and pattern matrix obtained from the decomposition are subjected to entropy encoding,and real-time chaotic encryption is implemented during the encoding process,effectively integrating compression and encryption.The results show that the proposed algorithm is suitable for occasions with high requirements for compression and encryption,and it provides valuable insights for the de-velopment of compression and encryption in multispectral field.

A PIID-DTBT based semi-conducting polymer dots with broad and strong optical absorption in the visible-light region： Highly effective contrast agents for multiscale and multi-spectral photoacoustic imaging

As a hybrid imaging technique, photoacoustic imaging （PAI） can provide multiscale morphological information of tissues, and the use of multi-spectral PAI （MSPAI） can recover the spatial distribution of chromophores of interest, such as hemoglobin within tissues. Herein, we developed a contrast agent that can very effectively combine multiscale PAI with MSPAI for a more comprehensive characterization of complex biological tissues. Specifically, we developed novel PIID-DTBT based semi-conducting polymer dots （Pdots） that show broad and strong optical absorption in the visible-light region （500-700 nm）. The performances of gold nanoparticles （GNPs） and gold nanorods （GNRs）, which have been verified as excellent photoacoustic contrast agents, were compared with that of the Pdots based on the multiscale PAI system. Both ex vivo and in vivo experiments demonstrated that the Pdots have better photoacoustic conversion efficiency at 532 nm than GNPs and showed similar photoacoustic performance with GNRs at 700 nm at the same mass concentration. Photostability and toxicity tests demonstrated that the Pdots are photostable and biocompatible. More importantly, an in vivo MSPAI experiment indicated that the Pdots have better photoacoustic performance than the blood and therefore the signals can be accurately extracted from the background of vascular-rich tissues. Our work demonstrates the great potential of Pdots as highly effective contrast agents for the precise localization of lesions relative to the blood vessels based on multiscale PAI and MSPAI.

Integrative Multi-Spectral Sensor Device for Far-Infrared and Visible Light Fusion

Infrared and visible light image fusion technology is a hot spot in the research of multi-sensor fusion technology in recent years. Existing infrared and visible light fusion technologies need to register before fusion because of using two cameras. However, the application effect of the registration technology has yet to be improved. Hence, a novel integrative multi-spectral sensor device is proposed for infrared and visible light fusion, and by using the beam splitter prism, the coaxial light incident from the same lens is projected to the infrared charge coupled device （CCD） and visible light CCD, respectively. In this paper, the imaging mechanism of the proposed sensor device is studied with the process of the signals acquisition and fusion. The simulation experiment, which involves the entire process of the optic system, signal acquisition, and signal fusion, is constructed based on imaging effect model. Additionally, the quality evaluation index is adopted to analyze the simulation result. The experimental results demonstrate that the proposed sensor device is effective and feasible.

Development and application of crop monitoring system for detecting chlorophyll content of tomato seedlings

A crop monitoring system was developed to nondestructively monitor the crop growth status in the field.With a two channel multispectral camera with one lens,controlling platform,wireless remote control module and control software,the system was able to synchronously acquire visible image(red(R),green(G),blue(B):400-700 nm)and near-infrared(NIR)image(760-1000 nm).The tomato seedlings multi-spectral images collection experiment in the greenhouse was conducted by using the developed system from the seeding stage to fruiting stage.More than 240 couples of tomato seedlings pictures were acquired with the Soil and Plant Analyzer Development(SPAD)value measured at the same time.The obtained images were available to process,and some vegetation indexes,such as normalized difference vegetation index(NDVI),ratio vegetation index(RVI)and normalized difference green index(NDGI),were calculated.Considering the SPAD value and the correlation coefficient between SPAD and other parameters in different fertilization treatments,the multiple linear regressions(MLR)model for estimating tomato seedlings chlorophyll content was built based on the average gray value in red,green,blue and NIR,vegetable indexes,NDVI,RVI and NDGI in the 33.3%(N1),66.6%(N2),and 100%(N3)nutrient levels during seeding stage and blossom and fruit stage.The R2 of the model is 0.88.The results revealed that the developed crop monitoring system provided a feasible tool to detect the growth status of tomato.More filed experiments and multi-spectral image analysis will be investigated to evaluate the crop growth status in the near future.

Spiral volumetric optoacoustic tomography visualizes multi-scale dynamics in mice

Imaging dynamics at different temporal and spatial scales is essential for understanding the biological complexity of living organisms,disease state and progression.Optoacoustic imaging has been shown to offer exclusive applicability across multiple scales with excellent optical contrast and high resolution in deep-tissue observations.Yet,efficient visualization of multi-scale dynamics remained difficult with state-of-the-art systems due to inefficient trade-offs between image acquisition time and effective field of view.Herein,we introduce the spiral volumetric optoacoustic tomography technique that provides spectrally enriched highresolution contrast across multiple spatiotemporal scales.In vivo experiments in mice demonstrate a wide range of dynamic imaging capabilities,from three-dimensional high-frame-rate visualization of moving organs and contrast agent kinetics in selected areas to whole-body longitudinal studies with unprecedented image quality.The newly introduced paradigm shift in imaging of multi-scale dynamics adds to the multifarious advantages provided by the optoacoustic technology for structural,functional and molecular imaging.