基于特征结构不变性思想的自适应在线神经网络算法

Online adaptive neural network algorithm based on the idea of the feature structural invariance

针对传统的采用遗忘因子的在线学习方法难以实时精确地跟踪数据所发生的漂移问题,利用在线数据通常所具有的特征结构不变特性,提升在线学习模型的自适应能力.结合在线离散化和在线聚类技术,追踪和刻画数据的特征结构,并在聚类结构中,采用一种类似深度森林算法中的特征构造策略来提取辅助的在线结构特征.通过整合样本的原始特征和额外提取的结构特征共同动态地训练并更新在线神经网络模型,采用在线序列极限学习机算法作为在线神经网络的训练算法,通过8个基准的在线数据集验证算法的有效性、可行性和优越性.实验结果表明:文中算法可很好地追踪数据所发生的概念漂移,并具有较强的自适应性.

Online learning is an important branch in machine learning field,and it is generally confused by the concept drift(variance occurs in the data distribution)problem.It is difficult for the traditional online learning methods with forget factor strategy to accurately track the concept drifts in time.To address this problem,this paper presents the strategy from a new perspective,i.e.,taking advantage of the idea of the feature structural invariance to promote the adaptation of the online learner.First,the incremental discretization and incremental clustering techniques are integrated to track and describe the feature structure.Next,in the clustering structure,a deep forest-like feature construction strategy is adopted to capture the extra structural features.Finally,the original and the extra extracted structural features will be integrated to dynamically train and update the online neural network model.Without loss of generality,online-sequential extreme learning machine algorithm is used to train the online neural network learner.The effectiveness,feasibility and superiority of the algorithm presented in this paper is verified by 8 benchmark online streaming data sets.The results show that the proposed algorithm can track the concept drift occurring in the data stream well,and it has a strong adaptivity.