A group adaptive elastic-net approach for variable selection in high-dimensional linear regression

In practice, predictors possess grouping structures spontaneously. Incorporation of such useful information can improve statistical modeling and inference. In addition, the high-dimensionality often leads to the collinearity problem. The elastic net is an ideal method which is inclined to reflect a grouping effect. In this paper, we consider the problem of group selection and estimation in the sparse linear regression model in which predictors can be grouped. We investigate a group adaptive elastic-net and derive oracle inequalities and model consistency for the cases where group number is larger than the sample size. Oracle property is addressed for the case of the fixed group number. We revise the locally approximated coordinate descent algorithm to make our computation. Simulation and real data studies indicate that the group adaptive elastic-net is an alternative and competitive method for model selection of high-dimensional problems for the cases of group number being larger than the sample size.

VARIABLE SELECTION FOR COVARIATE ADJUSTED REGRESSION MODEL

This paper employs the SCAD-penalized least squares method to simultaneously select variables and estimate the coefficients for high-dimensional covariate adjusted linear regression models.The distorted variables are assumed to be contaminated with a multiplicative factor that is determined by the value of an unknown function of an observable covariate.The authors show that under some appropriate conditions,the SCAD-penalized least squares estimator has the so called "oracle property".In addition,the authors also suggest a BIC criterion to select the tuning parameter,and show that BIC criterion is able to identify the true model consistently for the covariate adjusted linear regression models.Simulation studies and a real data are used to illustrate the efficiency of the proposed estimation algorithm.

Variable Selection for Varying Coefficient Models Via Kernel Based Regularized Rank Regression

A robust and efficient shrinkage-type variable selection procedure for varying coefficient models is proposed,selection consistency and oracle properties are established.Furthermore,a BIC-type criterion is suggested for shrinkage parameter selection and theoretical property is discussed.Numerical studies and real data analysis also are included to illustrate the finite sample performance of our method.

Variable selection for single-index varying-coefficient model

We consider the problem of variable selection for single-index varying-coefficient model, and present a regularized variable selection procedure by combining basis function approximations with SCAD penalty. The proposed procedure simultaneously selects significant covariates with functional coefficients and local significant variables with parametric coefficients. With appropriate selection of the tuning parameters, the consistency of the variable selection procedure and the oracle property of the estimators are established. The proposed method can naturally be applied to deal with pure single-index model and varying-coefficient model. Finite sample performances of the proposed method are illustrated by a simulation study and the real data analysis.

Model Detection and Variable Selection for Varying Coefficient Models with Longitudinal Data

In this puper, we consider the problem of variabie selection and model detection in varying coefficient models with longitudinM data. We propose a combined penalization procedure to select the significant variables, detect the true structure of the model and estimate the unknown regression coefficients simultaneously. With appropriate selection of the tuning parameters, we show that the proposed procedure is consistent in both variable selection and the separation of varying and constant coefficients, and the penalized estimators have the oracle property. Finite sample performances of the proposed method are illustrated by some simulation studies and the real data analysis.

Local Walsh-average-based estimation and variable selection for single-index models

We propose a robust estimation procedure based on local Walsh-average regression(LWR) for single-index models. Our novel method provides a root-n consistent estimate of the single-index parameter under some mild regularity conditions;the estimate of the unknown link function converges at the usual rate for the nonparametric estimation of a univariate covariate. We theoretically demonstrate that the new estimators show significant efficiency gain across a wide spectrum of non-normal error distributions and have almost no loss of efficiency for the normal error. Even in the worst case, the asymptotic relative efficiency(ARE) has a lower bound compared with the least squares(LS) estimates;the lower bounds of the AREs are 0.864 and 0.8896 for the single-index parameter and nonparametric function, respectively. Moreover, the ARE of the proposed LWR-based approach versus the ARE of the LS-based method has an expression that is closely related to the ARE of the signed-rank Wilcoxon test as compared with the t-test. In addition, to obtain a sparse estimate of the single-index parameter, we develop a variable selection procedure by combining the estimation method with smoothly clipped absolute deviation penalty;this procedure is shown to possess the oracle property. We also propose a Bayes information criterion(BIC)-type criterion for selecting the tuning parameter and further prove its ability to consistently identify the true model. We conduct some Monte Carlo simulations and a real data analysis to illustrate the finite sample performance of the proposed methods.

Distribution Approximation of Shrinkage Estimate in Censored Regression Model via Randomly Weighting Method

Censored regression("Tobit") model is a special case of limited dependent variable regression model, and plays an important role in econometrics. Based on this model, all kinds of methods for variable or group variable selection have been developed and the corresponding shrinkage parameter estimates are widely studied. However, asymptotic distributions of the shrinkage estimates involve unknown nuisance parameters,such as density function of error term. To avoid estimating nuisance parameters, this paper presents a randomly weighting method to approximate to the asymptotic distribution of the shrinkage estimate. A computation procedure of random approximation is provided and asymptotic properties of the randomly weighting estimates are also obtained. The proposed methods are evaluated with extensively numerical studies and a women labor supply example.

多元回归中选择自变量的一种简单方法

在线性回归模型建模中,回归自变量选择是一个受到广泛关注、文献众多,具有很强的理论和实际意义的问题.回归自变量选择子集的相合性是其中一个重要问题,如果某种自变量选择方法选择的子集在样本量趋于无穷时是相合的,而且预测均方误差较小,则这种方法是可取的.利用BIC准则可以挑选相合的自变量子集,但是在自变量个数很多时计算量过大;适应lasso方法具有较高计算效率,也能找到相合的自变量子集;本文提出一种更简单的自变量选择方法,只需要计算两次普通线性回归:第一次进行全集回归,得到全集的回归系数估计,然后利用这些回归系数估计挑选子集,然后只要在挑选的自变量子集上再进行一次普通线性回归就得到了回归结果.考虑如下的回归模型:Y_n=X_nβ~*+ε^((n)),其中回归系数β~*中非零分量下标的集合为J_O,设J_n是本文方法选择的自变量子集下标集合,β^((n))是本文方法估计的回归系数(未选中的自变量对应的系数为零),本文证明了,在适当条件下,(?)其中(β^((n))-β~*)J_O表示β^((n))-β~*的分量下标在J_O中的元素的组成的向量,σ~2是误差方差,∑,c是与矩阵(X_n^TX_n)/n极限有关的矩阵和常数.数值模拟结果表明本文方法具有很好的中小样本性质.

生长曲线模型的变量选择

生长曲线模型是一个典型的多元线性模型,在现代统计学上占有重要地位.文章首先基于Potthoff-.Roy变换后的生长曲线模型,采用自适应LASSO为惩罚函数给出了参数矩阵的惩罚最小二乘估计,实现了变量的选择.其次,基于局部渐近二次估计,对生长曲线模型的惩罚最小二乘估计给出了统一的近似估计表达式.接着,讨论了经过Potthoff-.Roy变换后模型的惩罚最小二乘估计,证明了自适应LASSO具有Oracle性质.最后对几种变量选择方法进行了数据模拟.结果表明自适应LASSO效果比较好.另外,综合考虑,Potthoff-Roy变换优于拉直变换.

定量结构-活性/性质相关性中变量的选择——正交变换法与最优子集回归法的比较

In this paper, the comparison of orthogonal descriptors and Leaps and Bounds regression analysis is performed. The results obtained by using orthogonal descriptors are better than that obtained by using Leaps and Bounds regression for the data set of nitrobenzenes used in this study. Leaps and Bounds regression can be used effectively for selection of variables in quantitative structure activity/property relationship(QSAR/QSPR) studies. Consequently, orthogonalisation of descriptors is also a good method for variable selection for studies on QSAR/QSPR.

高维广义线性模型的拟似然自适应Lasso估计

利用惩罚拟似然方法,讨论高维广义线性模型的拟似然自适应Lasso估计。该方法能同时进行变量选择和参数估计。在适当的条件下,证明了所得估计的相合性和Oracle性质,并利用数据模拟和实例分析说明了所提方法的优良性质。

异方差下正则化Expectile回归的变量选择

为了解决异方差存在时最小二乘回归不适用的问题,基于最小化非对称L2范数的Expectile回归,通过引入一种非凸惩罚(minimax concave penalty,MCP),提出带有MCP惩罚项的正则化Expectile回归模型,可以同时实现模型的变量选择和异方差检测,挖掘自变量与因变量之间更完整关系。传统方法假设随机误差项独立同分布且具有有限阶矩,本文方法将该假设弱化为误差项独立但不同分布,具有有限阶矩。证明了在一定条件下,带有MCP惩罚项的Expectile回归得到的估计量具有Oracle性质。数值模拟结果表明,该方法在变量选择上具有优良的表现,且通过不同Expectile权重值时的自变量集合变化,能有效检测出异方差。

部分线性模型的adaptive group lasso变量选择

对部分线性模型的aglasso(adaptive group lasso)参数估计及变量选择问题进行研究.通过构造aglasso的估计函数,将分组部分线性模型变量的选择问题转化为分组因子的选择问题.理论研究表明:该方法能相合地识别真实模型,并且估计具有oracle性质.最后通过模拟研究了所提方法的有限样本性质.

删失回归模型的分位数变量选择和压缩估计

删失回归模型是一种响应变量受限制的模型,广泛应用于计量经济学中.针对删失回归模型,借助于分位数估计方法和SCAD型惩罚函数,提出了一种变量选择和压缩估计方法.该方法可选出对模型有贡献的回归变量,即非0回归系数,同时给出非零参数的一个相合估计.另外,获得了变量选择方法的oracle性质.最后,利用数值模拟计算说明所提出方法的效果.

协变量随机缺失下部分线性模型的变量选择（英文）

考虑协变量随机缺失下部分线性模型,采用惩罚加权最小二乘提出了一种变量选择方法,研究了所提出方法的有限样本性质,证明了非零系数的估计具有Oracle性质.进一步,基于局部线性逼近方法给出了一步稀疏估计.通过模拟研究了所提出方法的有限样本性质.

自适应设计广义线性模型的自适应Lasso惩罚最小二乘的渐近性质

针对自适应设计广义线性模型,研究自适应Lasso惩罚最小二乘变量选择方法。在一定条件下,得到自适应Lasso惩罚最小二乘估计的相合性和Oracle性质,该结果将固定设计广义线性模型相关结果推广到自适应设计广义线性模型中。通过模拟可知,自适应Lasso惩罚方法优于Lasso惩罚方法。

自适应Lasso在Poisson对数线性回归模型下的性质

目的研究自适应Lasso在Poisson对数线性模型下的性质。方法利用数学分析及概率论中的性质。结果证明了在Poisson对数线性模型下自适应Lasso估计量具有稀疏性和渐进正态性。结论自适应Lasso可以有效选择Poisson对数线性模型中的变量,并同时估计变量系数。

删失回归模型中SCAD型变量选择与估计(英文)

删失回归模型是一种很重要的响应变量受限制的模型,它广泛应用于计量经济学中.基于SCAD罚函数,提出了SCAD型罚变量选择方法.该方法可选出重要的回归变量,即真参数中非零系数,同时给出非零参数相合估计.另外,证明了变量选择方法是相合的,具有oracle性质.最后,进行数值模拟计算说明所提出方法的效果。

一种自适应稀疏Group Lasso分位数回归估计

稀疏惩罚分位数回归是高维数据分析中进行变量选择和稳健估计的重要工具。对于具有分组解释变量的问题,期望达到组内和组间稀疏的理想效果,但是许多现有方法未能实现这一目标。文章将自适应Lasso和自适应Group Lasso相结合,构建了一种自适应稀疏Group Lasso惩罚分位数回归(Q-AdSGL)模型,给出了基于ADMM算法的模型求解方法,并讨论了估计量的Oracle性质。通过Monte Carlo模拟研究和实例分析证明了所提模型和算法的有效性。

Cox比例风险模型的弹性SCAD变量选择方法及应用

本文对基于Cox比例风险模型的弹性SCAD变量的选择方法进行了研究,证明了弹性SCAD方法在进行变量选择时具有Oracle性质、渐近正态性及组效应。通过数值模拟,将弹性SCAD与几种常见变量选择方法作了比较,得出几种不同参数设定下的变量选择结果。通过肺癌实际数据比较了弹性SCAD方法和其他常见变量选择方法。研究结果表明,弹性SCAD方法选出的变量与临床结论一致并发现了需要进一步深入研究的变量。