摘要
In recent years,there has been a surge of interest and rapid development in large-scale pre-training due to the explosive growth of both data and model parameters.Large-scale training has achieved impressive performance milestones across a wide range of practical problems,including natural language processing,computer vision,recommendation systems,robotics,and other basic research areas like bioinformatics.
