Boosting和Bagging综述

Boosting 和 Bagging 是两种用来提高学习算法准确度的方法,这两种方法通过构造一个预测函数系列,然后以一定的方式将它们组合成一个预测函数.文章将介绍这两种方法以及对他们进行的一些理论分析和实验,并对它们的应用以及将来可能的研究进行讨论.

Boosting算法综述

Boosting是近年来流行的一种用来提高学习算法精度的方法。以AdaBoost算法为例介绍了Boosting算法,并概括了它的各种理论分析,最后讨论了Boosting的应用及未来可能的发展方向。

Boosting算法研究

Boosting算法是近年来在机器学习领域中一种流行的用来提高学习精度的算法。文中以AdaBoost算法为例来介绍Boosting的基本原理。

北京地区体感温度误差订正方法研究

基于欧洲中期天气预报中心(European Centre for Medium-range weather Forecasts,ECMWF)模式的预报数据和北京地区气象站点的观测数据,使用两种机器学习算法(线性回归和梯度提升回归树)对站点的体感温度进行误差订正,并采用均方根误差(Root Mean Square Error,RMSE)对预报效果进行评估,进一步与传统订正方法模式输出统计(Model Output Statistics,MOS)得到的订正结果进行对比。结果表明:线性回归、梯度提升回归树、MOS和ECMWF预报得到的平均RMSE分别为3.12、3.06、3.45、4.06℃,即机器学习算法明显优于MOS和ECMWF模式原始预报。机器学习订正方法不仅在平原地区取得了较好的效果,在高海拔站点的订正效果更加突出,为北京冬奥会复杂山地条件下赛事正常运行提供了一定的技术保障。

两类集成学习算法在中长期径流预报中的应用

将前期130项遥相关气候指数作为预报因子,利用分类回归树算法(CART)作为基学习器,引入基于Bagging算法的随机森林模型(RF)与基于Boosting算法的梯度提升树模型(GBDT)的两类集成学习算法作为强学习预报模型,实现对丹江口水库未来1个月、1季度及1年3类径流序列的滚动预报,并通过相对误差绝对值的平均值(MAPE)、Nash效率系数(NSE)、相对均方根误差(RRMSE)、合格率(QR)等指标进行对比分析。研究结果表明,两类模型在验证期模拟精度相似,结果相仿,误差分布较均匀,可进一步用于集合径流预报;随着预报对象量级的增加,径流序列的不稳定性与极值序列分布的不均匀性得以降低,预报的准确度、可靠度以及稳定度得到提高。

Generalization Error Analysis of Neural Networks with Gradient Based Regularization

In this work,we study gradient-based regularization methods for neural networks.We mainly focus on two regularization methods:the total variation and the Tikhonov regularization.Adding the regularization term to the training loss is equivalent to using neural networks to solve some variational problems,mostly in high dimensions in practical applications.We introduce a general framework to analyze the error between neural network solutions and true solutions to variational problems.The error consists of three parts:the approximation errors of neural networks,the quadrature errors of numerical integration,and the optimization error.We also apply the proposed framework to two-layer networks to derive a priori error estimate when the true solution belongs to the so-called Barron space.Moreover,we conduct some numerical experiments to show that neural networks can solve corresponding variational problems sufficiently well.The networks with gradient-based regularization are much more robust in image applications.

Churn Prediction Task in MOOC

Churn prediction is a common task for machine learning applications in business.In this paper,this task is adapted for solving problem of low efficiency of massive open online courses(only 5%of all the students finish their course).The approach is presented on course“Methods and algorithms of the graph theory”held on national platform of online education in Russia.This paper includes all the steps to build an intelligent system to predict students who are active during the course,but not likely to finish it.The first part consists of constructing the right sample for prediction,EDA and choosing the most appropriate week of the course to make predictions on.The second part is about choosing the right metric and building models.Also,approach with using ensembles like stacking is proposed to increase the accuracy of predictions.As a result,a general approach to build a churn prediction model for online course is reviewed.This approach can be used for making the process of online education adaptive and intelligent for a separate student.

集成学习:Boosting算法综述

Boosting是近年来机器学习领域中一种流行的、用来提高学习精度的算法,本文首先以AdaBoost为例对Boosting算法进行简单的介绍,并对Boosting的各种不同理论分析进行概括,然后介绍了Boosting在回归问题中的理论研究,最后对Boosting的应用以及未来的研究方向进行了讨论。