利用高性能混合深度学习网络提升光谱分类性能研究

Research on the Improvement of Spectra Classification Performance With the High-Performance Hybrid Deep Learning Network

随着观测设备的不断完善,人们获得的光谱数量持续上升,如何进一步提高光谱自动分类的性能引起广泛关注。为此,以恒星光谱为研究对象,在近年来新出现的BERT和CNN等深度学习模型的基础上,试图融合了BERT模型和CNN模型在特征提取和智能分类方面的优势,提出高性能混合深度学习网络BERT-CNN,用以探讨该模型在提升光谱分类性能方面的有效性。该模型首先将恒星光谱数据输入BERT模型;然后,利用BERT模型中的Transformer进行特征提取,得到特征向量;最后,将特征向量输入CNN模型,通过softmax分类器获得分类结果。该实验的编程语言为Python3.7,引入TensorFlow1.14作为深度学习模型框架,并以SDSS DR10中的K型、 F型、 G型的恒星光谱数据作为实验数据集。使用min-max方法对恒星光谱数据做归一化处理,通过与SVM、 CNN等分类模型的比较来验证BERT-CNN混合模型在恒星光谱分类中的有效性。引入网格搜索和10折交叉验证来获得模型的实验参数。实验包括两部分:一是利用精准率P、召回率R、调和平均值F1等指标对BERT-CNN模型的恒星光谱分类性能进行评价。当训练数据集占比实验数据集的30%~70%时,BERT-CNN模型处理K, F和G型恒星光谱数据集的精准率P、召回率R、调和平均值F1随训练样本数的增加而提升。在相同规模的训练样本条件下,BERT-CNN模型在K型恒星光谱数据集上的P,R和F1值均最高,其次是G型恒星光谱数据集,F型恒星光谱数据集上的分类效果较差。二是利用准确率对SVM, CNN和BERT-CNN等模型的对比实验结果进行评价。对K, F和G型恒星光谱数据集上,BERT-CNN模型分类效果最优,其次是CNN模型,SVM模型分类效果较差。表明,BERT-CNN模型有助于提升光谱分类性能。

With the development of observation apparatus, the spectra number rises constantly. How to further improve the classification performance deserves to be research. Because of this, the stellar spectra are taken as the research object, the high-performance hybrid deep learning network is proposed based on integrating the advantages of the BERT model in feature extraction and the CNN model in automatic classification, to verify the effectiveness of improving the spectra classification performance. Firstly, the stellar spectra are input into the the BERT model;And then the part in BERT model named Transformers are used to extract the features and based on which, the feature vectors are formed;Finally, the above feature vectors are input into the CNN model, and the stellar spectra classification results can be obtained with the help of softmax classifier. Python3.7 writes the models used in the experiment and the deep learning framework named TensorFlow is introduced. The K-, F-, G-type stellar spectra in SDSS DR10 are used for the experimental dataset, normalized by the min-max normalization method. The effectiveness of the BERT-CNN model is verified by comparing with the support vector machine models(SVM) and CNN. The performances of the above models are related to the parameters, and therefore, the ten cross-validation and the grid search method are used to obtain the optimal experimental parameters. There are two parts to the experiment. One is to evaluate the classification performances of BERT-CNN with precision P, recall R and F1 values. The proportion from 30% to 70% of the experimental dataset is respectively used for the training dataset, and the remainder is used for the test dataset. P, R and F1 values rise with the training size on the K-, F-, G-type stellar datasets. In the case of the same training size, the values of P, R and F1 arrive at the highest, followed by the performance on the G-type stellar dataset, the classification results on the F-type stellar dataset are much poorer. The other experiment is to evaluate the classification performances of SVM, CNN and BERT-CNN with accuracy. The classification performances of BERT-CNN on the K-, F-, G-type stellar datasets are all best, followed by CNN. The classification accuracies of SVM are much lower than the other two models. It indicates that the BERT-CNN model contributes to improving the spectra classification performance.