超高维删失数据的联合特征筛选方法研究

Joint feature screening method for ultrahigh dimensional censored data

针对超高维删失数据,通过降维技术可以进行特征选取,去除大数据中的噪声数据,以便挖掘高维大数据的重要信息,进行大数据的相关分析和应用.本文提出了一种稳健的偏相关系数来进行特征筛选,并引入逆概率加权方法来处理删失,发展出一种新的联合特征筛选方法.本文利用响应变量的条件分布函数来构造偏相关性度量,可以全面地刻画其与协变量间的相关性,且相较于传统的皮尔逊偏相关系数,该度量对于响应存在异常值,厚尾分布以及异方差结构时具有稳健性.其次,基于该度量所提出的联合特征筛选方法通过投影作用来消除由协变量之间的相关关系产生的干扰作用,故能够较好地改善假阴性错误、假阳性错误及协变量的共线性问题.我们推导了该方法的理论性质,给出了快速的迭代算法,并进一步通过模拟和实例分析来考察该算法在有限样本下的数值表现.

For ultra-high-dimensional censored data, feature screening can be performed to remove noise in big data, and classical statistic analysis can be applied after that. This paper proposes a robust partial correlation coefficient for feature screening, and introduces an inverse probability weighting method to deal with censoring. Based on that, a new joint feature screening method is developed. By incorporating the information of the entire conditional distribution of the failure time, our method can depict the relationship between the response and covariates comprehensively. Compared with the traditional Pearson partial correlation coefficient, this measurement is robust to outliers, heavy-tailed distribution and heteroscedasticity. Moreover,the joint feature screening method proposed based on this metric eliminates the interference caused by the correlation between the covariates through the projection effect, so as to reduce the false negative errors, false positive errors and tackle the problem of collinearity of covariates. We establish the sure screening property of our method and give the details of the iterative algorithm.The competence of our method is further confirmed through comprehensive simulation studies and a real data example.