基于深度学习的词汇表示模型对比研究

A Comparative Study of Word Representation Models Based on Deep Learning

【目的】系统揭示传统深度表示模型与最新预训练模型的原理,探究其在文本挖掘任务中的效果差异。【方法】采用对比研究法,从模型侧和实验侧分别比较传统模型与最新模型在CR、MR、MPQA、Subj、SST-2和TREC六个数据集上的效果差异。【结果】在六个任务中,XLNet模型取得了最高的平均F1值(0.9186),优于ELMo(0.8090)、BERT(0.8983)、Word2Vec(0.7692)、GloVe(0.7576)和FastText(0.7506)。【局限】由于篇幅限制,实证研究以文本挖掘中的分类任务为主,尚未比较词汇表示学习方法在机器翻译、问答等其他任务中的效果。【结论】传统深度表示学习模型与最新预训练模型在文本挖掘任务中的表现存在较大差异。

[Objective]This study systematically explores the principles of traditional deep representation models and the latest pre-training ones,aiming to examine their performance in text mining tasks.[Methods]We compared these models’data mining results from the model side and the experimental side.All tests were conducted with six datasets of CR,MR,MPQA,Subj,SST-2 and TREC.[Results]The XLNet model achieved the best average F1 value(0.9186),which was higher than ELMo(0.8090),BERT(0.8983),Word2 Vec(0.7692),GloVe(0.7576)and FastText(0.7506).[Limitations]Our research focused on classification tasks of text mining,which did not compare the performance of vocabulary representation methods in machine translation,Q&A and other tasks.[Conclusions]The traditional deep representation learning models and the latest pre-training ones yield different results in text mining tasks.