This paper focuses on the parallel aggregation processing of data streams based on the shared-nothing architecture. A novel granularity-aware parallel aggregating model is proposed. It employs parallel sampling and li...This paper focuses on the parallel aggregation processing of data streams based on the shared-nothing architecture. A novel granularity-aware parallel aggregating model is proposed. It employs parallel sampling and linear regression to describe the characteristics of the data quantity in the query window in order to determine the partition granularity of tuples, and utilizes equal depth histogram to implement partitio ning. This method can avoid data skew and reduce communi cation cost. The experiment results on both synthetic data and actual data prove that the proposed method is efficient, practical and suitable for time-varying data streams processing.展开更多
【应用背景】快速射电暴(Fast Radio Burst,FRB)搜寻是500米口径球面射电望远镜(FAST)的重要科学目标之一,其计算复杂度高,数据量大,当前算法GPU利用率偏低,数据处理需较多的人工介入操作。【目的】在不修改算法实现的前提下,实现进程级...【应用背景】快速射电暴(Fast Radio Burst,FRB)搜寻是500米口径球面射电望远镜(FAST)的重要科学目标之一,其计算复杂度高,数据量大,当前算法GPU利用率偏低,数据处理需较多的人工介入操作。【目的】在不修改算法实现的前提下,实现进程级GPU并行优化,提高GPU整体资源利用率,简化算法运行调度,支持利用自动化脚本驱动计算过程。【方法】利用容器化封装FRB搜寻算法,结合GPU聚合技术实现多个FRB搜寻计算容器的多进程并行,支持GPU闲时复用。通过容器化封装屏蔽了GPU调用、依赖库管理等技术细节,减少人工介入操作。【结果】算法实验结果表明,在不修改原始算法、不增加GPU资源的前提下,将单GPU绑定6个计算进程,并行优化可实现FRB搜寻算法的加速比达到5.3,并行效率达到0.88,取得良好的并行效果。【结论】基于容器化封装及进程级GPU聚合的并行优化,可实现GPU利用率及计算效率的提升,有效支持自动化处理。该方法还具有良好的通用性,可适用于类似应用的并行优化。展开更多
In recent years, it has been difficult for manufactures and suppliers to forecast demand from a market for a given product precisely. Therefore, it has become important for them to cope with fluctuations in demand. Fr...In recent years, it has been difficult for manufactures and suppliers to forecast demand from a market for a given product precisely. Therefore, it has become important for them to cope with fluctuations in demand. From this viewpoint, the problem of planning or scheduling in production systems can be regarded as a mathematical problem with stochastic elements. However, in many previous studies, such problems are formulated without stochastic factors, treating stochastic elements as deterministic variables or parameters. Stochastic programming incorporates such factors into the mathematical formulation. In the present paper, we consider a multi-product, discrete, lotsizing and scheduling problem on parallel machines with stochastic demands. Under certain assumptions, this problem can be formulated as a stochastic integer programming problem. We attempt to solve this problem by a scenario aggregation method proposed by Rockafellar and Wets. The results from computational experiments suggest that our approach is able to solve large-scale problems, and that, under the condition of uncertainty, incorporating stochastic elements into the model gives better results than formulating the problem as a deterministic model.展开更多
在细观结构层次上将大坝混凝土作为骨料、固化水泥砂浆及其粘结界面组成的复合材料,建立了大坝混凝土三维细观力学数值模型。该模型既能够反映混凝土及其细观各相材料在荷载作用下的损伤演化过程,又考虑了动载作用的应变率强化效应。给...在细观结构层次上将大坝混凝土作为骨料、固化水泥砂浆及其粘结界面组成的复合材料,建立了大坝混凝土三维细观力学数值模型。该模型既能够反映混凝土及其细观各相材料在荷载作用下的损伤演化过程,又考虑了动载作用的应变率强化效应。给出了该数值模型求解方法,并编制出能够在普通PC机上运行的串行程序。加载过程既可按荷载控制又可按位移控制。同时,为了减少求解自由度应用了分尺度方法以使最小骨料和固化水泥砂浆混合后其力学性能与一种复合介质等效。通过混凝土湿筛和三级配试件的静、动(冲击)弯拉数值计算验证了本文计算方法和程序正确有效。另外,在串行程序的基础上,优化了刚度矩阵的存储方式,采用双门槛不完全Cholesky分解(ICT)预条件的共轭梯度法(CG),完成了能够在Sun Fire 6800服务器实现并行计算的并行程序改造,从而大大提高了计算效率。展开更多
基金Supported by Foundation of High Technology Pro-ject of Jiangsu (BG2004034) , Foundation of Graduate Creative Pro-gramof Jiangsu (xm04-36)
文摘This paper focuses on the parallel aggregation processing of data streams based on the shared-nothing architecture. A novel granularity-aware parallel aggregating model is proposed. It employs parallel sampling and linear regression to describe the characteristics of the data quantity in the query window in order to determine the partition granularity of tuples, and utilizes equal depth histogram to implement partitio ning. This method can avoid data skew and reduce communi cation cost. The experiment results on both synthetic data and actual data prove that the proposed method is efficient, practical and suitable for time-varying data streams processing.
文摘【应用背景】快速射电暴(Fast Radio Burst,FRB)搜寻是500米口径球面射电望远镜(FAST)的重要科学目标之一,其计算复杂度高,数据量大,当前算法GPU利用率偏低,数据处理需较多的人工介入操作。【目的】在不修改算法实现的前提下,实现进程级GPU并行优化,提高GPU整体资源利用率,简化算法运行调度,支持利用自动化脚本驱动计算过程。【方法】利用容器化封装FRB搜寻算法,结合GPU聚合技术实现多个FRB搜寻计算容器的多进程并行,支持GPU闲时复用。通过容器化封装屏蔽了GPU调用、依赖库管理等技术细节,减少人工介入操作。【结果】算法实验结果表明,在不修改原始算法、不增加GPU资源的前提下,将单GPU绑定6个计算进程,并行优化可实现FRB搜寻算法的加速比达到5.3,并行效率达到0.88,取得良好的并行效果。【结论】基于容器化封装及进程级GPU聚合的并行优化,可实现GPU利用率及计算效率的提升,有效支持自动化处理。该方法还具有良好的通用性,可适用于类似应用的并行优化。
文摘In recent years, it has been difficult for manufactures and suppliers to forecast demand from a market for a given product precisely. Therefore, it has become important for them to cope with fluctuations in demand. From this viewpoint, the problem of planning or scheduling in production systems can be regarded as a mathematical problem with stochastic elements. However, in many previous studies, such problems are formulated without stochastic factors, treating stochastic elements as deterministic variables or parameters. Stochastic programming incorporates such factors into the mathematical formulation. In the present paper, we consider a multi-product, discrete, lotsizing and scheduling problem on parallel machines with stochastic demands. Under certain assumptions, this problem can be formulated as a stochastic integer programming problem. We attempt to solve this problem by a scenario aggregation method proposed by Rockafellar and Wets. The results from computational experiments suggest that our approach is able to solve large-scale problems, and that, under the condition of uncertainty, incorporating stochastic elements into the model gives better results than formulating the problem as a deterministic model.
文摘在细观结构层次上将大坝混凝土作为骨料、固化水泥砂浆及其粘结界面组成的复合材料,建立了大坝混凝土三维细观力学数值模型。该模型既能够反映混凝土及其细观各相材料在荷载作用下的损伤演化过程,又考虑了动载作用的应变率强化效应。给出了该数值模型求解方法,并编制出能够在普通PC机上运行的串行程序。加载过程既可按荷载控制又可按位移控制。同时,为了减少求解自由度应用了分尺度方法以使最小骨料和固化水泥砂浆混合后其力学性能与一种复合介质等效。通过混凝土湿筛和三级配试件的静、动(冲击)弯拉数值计算验证了本文计算方法和程序正确有效。另外,在串行程序的基础上,优化了刚度矩阵的存储方式,采用双门槛不完全Cholesky分解(ICT)预条件的共轭梯度法(CG),完成了能够在Sun Fire 6800服务器实现并行计算的并行程序改造,从而大大提高了计算效率。
