分布式系统下基于分位数回归的统计诊断

Statistical Diagnosis Based on Quantile Regression for A Distributed System

随着大数据时代的到来,分布式系统已广泛应用于生活中。然而,由于分布式系统中服务器数量不限,各种服务器之间的异质性较高,可能会对统计推断的结果产生影响。因此,在分布式系统中进行统计诊断变得非常必要。为此,提出了一种适用于分布式系统下分位数回归模型的异常值检测方法。考虑到实际应用背景,采用了群组(分布式系统中的子集)删除的方法来捕捉边际相关性的影响,并在较为稳健的模型中进行统计诊断。在蒙特卡罗模拟研究中,该方法表现出色,并通过对空气质量监测站点实际数据的检测进一步验证了其有效性。

With the arrival of the era of big data,distributed systems have been widely used in our lives.However,due to the unlimited number of servers in the distributed system,the heterogeneity between the various servers is high,which may affect the results of statistical inference.Therefore,statistical diagnosis in distributed systems becomes very necessary and important.For this reason,an outlier detection method which is suitable for quantile regression model in a distributed system is proposed.Considering the practical application background,the method of group(subset in distributed system)deletion is used to capture the impact of marginal correlation,and make statistical diagnosis in a more robust model.In the Monte Carlo simulation study,the method performs well,and its effectiveness is further verified by the detection of the actual data of the air quality monitoring station.