This paper focuses on the parallel aggregation processing of data streams based on the shared-nothing architecture. A novel granularity-aware parallel aggregating model is proposed. It employs parallel sampling and li...This paper focuses on the parallel aggregation processing of data streams based on the shared-nothing architecture. A novel granularity-aware parallel aggregating model is proposed. It employs parallel sampling and linear regression to describe the characteristics of the data quantity in the query window in order to determine the partition granularity of tuples, and utilizes equal depth histogram to implement partitio ning. This method can avoid data skew and reduce communi cation cost. The experiment results on both synthetic data and actual data prove that the proposed method is efficient, practical and suitable for time-varying data streams processing.展开更多
内部电网地理信息系统(Geographic Information Systern,GIS)数据体量增加,对电网数据存储性能造成了极大的困难,为此,提出一种基于随机森林的电网GIS数据分布式存储方法。以跨域资源共享(Cross-Origin Resource Sharing,CORS)技术在电...内部电网地理信息系统(Geographic Information Systern,GIS)数据体量增加,对电网数据存储性能造成了极大的困难,为此,提出一种基于随机森林的电网GIS数据分布式存储方法。以跨域资源共享(Cross-Origin Resource Sharing,CORS)技术在电网GIS空间信息服务平台中获取的电网GIS数据为基础,根据类区分度数值选择电网GIS数据特征,引入随机森林算法分类处理电网GIS数据,将其合理分发给不同的服务器,采用并行处理手段存储分类数据,从而实现了电网GIS数据的分布式存储。实验数据显示:应用所提方法后,电网GIS数据分类精度达到了96.8%,电网GIS数据分布式存储时间最小值为5.2 s,充分证实了所提方法数据存储性能更佳。展开更多
On the basis of Space-Wise Least Square method, three numerical methods including Cholesky de- composition, pre-conditioned conjugate gradient and Open Multi-Processing parallel algorithm are applied into the determin...On the basis of Space-Wise Least Square method, three numerical methods including Cholesky de- composition, pre-conditioned conjugate gradient and Open Multi-Processing parallel algorithm are applied into the determination of gravity field with satellite gravity gradiometry data. The results show that, Cholesky de- composition method has been unable to meet the requirements of computation efficiency when the computer hardware is limited. Pre-conditioned conjugate gradient method can improve the computation efficiency of huge matrix inversion, but it also brings a certain loss of precision. The application of Open Multi-Processing parallel algorithm could achieve a good compromise between accuracy and computation efficiency.展开更多
This paper takes the Sobel operator as example to study parallel sequential algorithm onto a memory-sharing multiprocessor by using a virtual machine. Several different parallel algorithms using function decomposition...This paper takes the Sobel operator as example to study parallel sequential algorithm onto a memory-sharing multiprocessor by using a virtual machine. Several different parallel algorithms using function decomposition and/or data decomposition methods are compared and their performances are analyzed in terms of processor utilization, data traffic, shared memory access, and synchronization overhead. The analysis is validated through a simulation experiment on the virtual machine of 64 parallel processors. Conclusions are presented at the end of this paper.展开更多
基金Supported by Foundation of High Technology Pro-ject of Jiangsu (BG2004034) , Foundation of Graduate Creative Pro-gramof Jiangsu (xm04-36)
文摘This paper focuses on the parallel aggregation processing of data streams based on the shared-nothing architecture. A novel granularity-aware parallel aggregating model is proposed. It employs parallel sampling and linear regression to describe the characteristics of the data quantity in the query window in order to determine the partition granularity of tuples, and utilizes equal depth histogram to implement partitio ning. This method can avoid data skew and reduce communi cation cost. The experiment results on both synthetic data and actual data prove that the proposed method is efficient, practical and suitable for time-varying data streams processing.
基金supproted by the National Natural Science Foundation of China(40874012,40904003,40974016,41004007)
文摘On the basis of Space-Wise Least Square method, three numerical methods including Cholesky de- composition, pre-conditioned conjugate gradient and Open Multi-Processing parallel algorithm are applied into the determination of gravity field with satellite gravity gradiometry data. The results show that, Cholesky de- composition method has been unable to meet the requirements of computation efficiency when the computer hardware is limited. Pre-conditioned conjugate gradient method can improve the computation efficiency of huge matrix inversion, but it also brings a certain loss of precision. The application of Open Multi-Processing parallel algorithm could achieve a good compromise between accuracy and computation efficiency.
文摘This paper takes the Sobel operator as example to study parallel sequential algorithm onto a memory-sharing multiprocessor by using a virtual machine. Several different parallel algorithms using function decomposition and/or data decomposition methods are compared and their performances are analyzed in terms of processor utilization, data traffic, shared memory access, and synchronization overhead. The analysis is validated through a simulation experiment on the virtual machine of 64 parallel processors. Conclusions are presented at the end of this paper.
