基于CMCP-LMCL的多分类深度神经网络及其应用

Deep Neural Network for Multi-Classification Based on CMCP-LMCL and Its Application

多分类问题涉及信用风险管理、股票走势预测等多个领域。深度神经网络(DNN)是常用于多分类预测的机器学习模型,然而输入特征维度较高且存在冗余信号时,将加重其可解释性不强和结构冗余等缺陷;同时,常用的Softmax损失也可能面临分类边界模糊导致预测效果不佳等问题。为此,本文针对多分类问题,提出一个新的深度神经网络CMCP-LMCL,利用CMCP变量选择方法压缩输入特征到第1隐藏层的权重。该方法融合权重的组结构,能够剔除无关特征以及不重要的连接;同时,对特征层之外的权重施加权重衰减L;2;惩罚,有利于改进过拟合问题。新方法的增强边缘余弦损失(LMCL)在Softmax基础上引入扩大参数和距离参数,增大分类决策边界的间隔以期提高分类预测性能。模拟分析表明,对比已有DNN和传统分类方法,无论特征以简单线性形式还是复杂非线性形式映射到因变量,本文所提出的方法均具有良好的特征选择性能和预测表现。基于信用贷款数据的实证分析表明,该方法能够有效选择风险指标并进行违约风险预警。

Multi-classification has appeared in many fields,such as credit risk management,stock trend prediction,and so on.Deep Neural Network(DNN)is a commonly used machine learning model capable of multi-classification prediction.However,when the feature dimension is high and there are redundant signals,it will aggravate its shortcomings such as poor interpretability and structural redundancy.At the same time,the commonly used Softmax loss may face some problems such as poor prediction results caused by fuzzy classification boundaries.Therefore,this paper proposes a new deep neural network(CMCP-LMCL)for multi-classification problems.It uses the CMCP variable selection method to compress the weight of the input feature to the first hidden layer,which incorporates the group structure of the weights and can eliminate both irrelevant features and unimportant connections.At the same time,the weight decay L2 penalty is imposed on the weights beyond the feature layer,which is helpful to improve the over-fitting problem.The LMCL loss function introduces extended parameters and distance parameters on the basis of Softmax to increase the interval of classification decision boundaries in order to improve the performance of classification prediction.Simulation analysis shows that compared with the existing deep neural networks(DNN)and traditional classification methods,the proposed method has good feature selection performance and prediction performance,whether the features are mapped to dependent variables in simple linear form or complex nonlinear form.The empirical analysis of credit data shows that this method can effectively select risk indicators and carry out early warning of default risk.