海量数据下线性混合效应模型的估计算法

Estimation algorithm for a linear mixed effect model with massive data

基于以往文献提出线性混合效应模型参数的三步估计方法,避免了繁杂的极大似然估计迭代步骤。同时为进一步解决海量数据下计算估计量时存在的存储瓶颈及计算时间过长问题,在海量纵向数据的两种不同数据格式下,分别基于三步估计方法利用分治算法计算模型参数的估计量。数值模拟和实证分析结果表明,本文所提出的三步估计方法和估计量的分治算法可以减轻计算负担,减少占用内存,解决内存不足的问题,并提高计算速度。

Based on the previous literature, a three-step method for the estimation of the parameters of a linear mixed effect model has been proposed, which avoids the complicated iterative steps of maximum likelihood estima-tion. At the same time, in order to further solve the storage bottleneck and calculation time when calculating the es-timator with massive data, the estimator of the model parameters has been calculated using the divide-and-conquer algorithm based on the three-step estimation method for two different data formats of massive vertical data. The re-sults of numerical simulation and empirical analysis show that the three-step estimation method and the estimator di-vide-and-conquer algorithm proposed in this paper can reduce the computational burden, reduce the memory con-sumption, solve the problem of insufficient memory, and improve the calculation speed.