Feature selection algorithm for text classification based on improved mutual information

In order to solve the poor performance in text classification when using traditional formula of mutual information (MI),a feature selection algorithm were proposed based on improved mutual information.The improved mutual information algorithm,which is on the basis of traditional improved mutual information methods that enhance the MI value of negative characteristics and feature's frequency,supports the concept of concentration degree and dispersion degree.In accordance with the concept of concentration degree and dispersion degree,formulas which embody concentration degree and dispersion degree were constructed and the improved mutual information was implemented based on these.In this paper,the feature selection algorithm was applied based on improved mutual information to a text classifier based on Biomimetic Pattern Recognition and it was compared with several other feature selection methods.The experimental results showed that the improved mutual information feature selection method greatly enhances the performance compared with traditional mutual information feature selection methods and the performance is better than that of information gain.Through the introduction of the concept of concentration degree and dispersion degree,the improved mutual information feature selection method greatly improves the performance of text classification system.

Identification of maize seed varieties based on near infrared reflectance spectroscopy and chemometrics

False seeds can often be seen in the maize seed market,leading to a serious decline in maize yield.Those existing variety identification methods are expensive,time consuming,and destructive to seeds.The aim of this study is to develop a cheap,fast and non-destructive method which can robustly identify large amounts of maize seed varieties based on near-infrared reflectance spectroscopy(NIRS)and chemometrics.Because it is difficult to establish models for every variety in the market,this study mainly investigated the performance of models based on a large number of samples(more than 40 major varieties in the market).The reflectance spectra of maize seeds were collected by two modes(bulk kernels mode and single kernel mode).Both collection modes can be applied to identification,but only the single kernel mode can be applied to purity sorting.The spectra were pretreated with smoothing,the first derivative and vector normalization;and then principal component analysis(PCA),linear discriminant analysis(LDA)and biomimetic pattern recognition(BPR)were applied to establish identification models.The environmental factors such as producing areas and years have a significant influence on the performance of the models.Therefore,the method to improve the robustness of the models was investigated in this study.New indexes(correct acceptance degree(CAD),correct rejection degree(CRD)and correct degree(CD))were defined to analyze the performance of the models more accurately.Finally,the models obtained a mean correct discrimination rate of over 90%,and exhibited robust properties for samples harvested from different areas and years.The results showed that NIR technology combined with chemometrics methods such as PCA,LDA,and BPR could be a suitable and alternative technique to identify the authenticity of maize seed varieties.