预训练语言模型探究

On the Pre Training Language Models

随着人工智能应用的普及,自然语言处理作为认知智能的核心已经成为人工智能极大的挑战。以GPT、BERT为代表的预训练语言模型的兴起,使问答、摘要、阅读理解等多项自然语言处理任务取得一系列突破,效果显著提升。该文从预训练语言模型的概念、主流技术方法以及应用这3个方面,介绍主流预训练语言模型,以及在自然语言处理领域的影响。

With the popularity of artificial intelligence applications,natural language processing,as the core of cognitive intelligence,has become a great challenge for artificial intelligence.The rise of pre training language models represented by GPT and BERT has made a series of breakthroughs in many natural language processing tasks,such as question answering,abstracting and reading comprehension,and the effect has been significantly improved.This paper introduces the mainstream pre training language model and its influence in the field of natural language processing from three aspects:the concept of pre training language model,the mainstream technical methods and its application.