期刊文献+
共找到274篇文章
< 1 2 14 >
每页显示 20 50 100
Symplectic partitioned Runge-Kutta method based onthe eighth-order nearly analytic discrete operator and its wavefield simulations 被引量:3
1
作者 张朝元 马啸 +1 位作者 杨磊 宋国杰 《Applied Geophysics》 SCIE CSCD 2014年第1期89-106,117,118,共20页
We propose a symplectic partitioned Runge-Kutta (SPRK) method with eighth-order spatial accuracy based on the extended Hamiltonian system of the acoustic waveequation. Known as the eighth-order NSPRK method, this te... We propose a symplectic partitioned Runge-Kutta (SPRK) method with eighth-order spatial accuracy based on the extended Hamiltonian system of the acoustic waveequation. Known as the eighth-order NSPRK method, this technique uses an eighth-orderaccurate nearly analytic discrete (NAD) operator to discretize high-order spatial differentialoperators and employs a second-order SPRK method to discretize temporal derivatives.The stability criteria and numerical dispersion relations of the eighth-order NSPRK methodare given by a semi-analytical method and are tested by numerical experiments. We alsoshow the differences of the numerical dispersions between the eighth-order NSPRK methodand conventional numerical methods such as the fourth-order NSPRK method, the eighth-order Lax-Wendroff correction (LWC) method and the eighth-order staggered-grid (SG)method. The result shows that the ability of the eighth-order NSPRK method to suppress thenumerical dispersion is obviously superior to that of the conventional numerical methods. Inthe same computational environment, to eliminate visible numerical dispersions, the eighth-order NSPRK is approximately 2.5 times faster than the fourth-order NSPRK and 3.4 timesfaster than the fourth-order SPRK, and the memory requirement is only approximately47.17% of the fourth-order NSPRK method and 49.41% of the fourth-order SPRK method,which indicates the highest computational efficiency. Modeling examples for the two-layermodels such as the heterogeneous and Marmousi models show that the wavefields generatedby the eighth-order NSPRK method are very clear with no visible numerical dispersion.These numerical experiments illustrate that the eighth-order NSPRK method can effectivelysuppress numerical dispersion when coarse grids are adopted. Therefore, this methodcan greatly decrease computer memory requirement and accelerate the forward modelingproductivity. In general, the eighth-order NSPRK method has tremendous potential value forseismic exploration and seismology research. 展开更多
关键词 SYMPLECTIC partitioned RUNGE-KUTTA method NEARLY ANALYTIC DISCRETE operATOR Numerical dispersion Wavefield simulation
下载PDF
Research on resonance parameters matching based on partitioned operation method of atmospheric pressure plasma reactor array
2
作者 Zhe YU Jialin ZHAO +3 位作者 Rui LIU Huijuan CAO Pu LIU Zhitao ZHANG 《Plasma Science and Technology》 SCIE EI CAS CSCD 2019年第5期34-42,共9页
Matching optimization of resonant parameters among the high power inverters,low power transformers and plasma reactors have significant effects on the performance and output of the reactor array when applying the part... Matching optimization of resonant parameters among the high power inverters,low power transformers and plasma reactors have significant effects on the performance and output of the reactor array when applying the partitioned operation method.In this paper,the Matlab/Simulink electrical model was established based on the method of partitioned operation.The matching relation between resonant parameters is analyzed on the basis of experimental result.As a consequence,transformer leakage inductance and working frequency are the important parameters influencing the operational efficiency of system,leakage inductance of transformer should be adjusted based on the equivalent capacitance of plasma reactor to realize the matching optimization of resonant parameters. 展开更多
关键词 ATMOSPHERIC pressure plasma reactor dielectric BARRIER DISCHARGE MATLAB/SIMULINK RESONANCE parameters partitioned operation method
下载PDF
用PartitionMagic实现多操作系统的同机安装 被引量:2
3
作者 任斌 《现代电子技术》 2004年第10期71-72,共2页
为使高校公共机房满足不同层次学生的需要 ,缓解上机压力 ,本文采用在单机上划分多主磁盘分区来安装多操作系统的方法予以解决。简要介绍了如何使用工具软件 Power Q uest Partition Magic来实现多操作系统的同机安装。
关键词 公共机房 分区 主磁盘分区 文件系统 多操作系统
下载PDF
Mathematical Models for a Social Partitioning Problem 被引量:1
4
作者 Vardges Melkonian 《American Journal of Computational Mathematics》 2021年第1期1-22,共22页
In this paper we develop modeling techniques for a social partitioning problem. Different social interaction regulations are imposed during pandemics to prevent the spread of diseases. We suggest partitioning a set of... In this paper we develop modeling techniques for a social partitioning problem. Different social interaction regulations are imposed during pandemics to prevent the spread of diseases. We suggest partitioning a set of company employees as an effective way to curb the spread, and use integer programming techniques to model it. The goal of the model is to maximize the number of direct interactions between employees who are essential for company’s work subject to the constraint that all employees should be partitioned into components of no more than a certain size implied by the regulations. Then we further develop the basic model to take into account different restrictions and provisions. We also give heuristics for solving the problem. Our computational results include sensitivity analysis on some of the models and analysis of the heuristic performance. 展开更多
关键词 Health Care operations Research Mathematical Models for Pandemics Graph partitioning Integer Linear Programming Heuristic Algorithms
下载PDF
Adaptive partition intuitionistic fuzzy time series forecasting model
5
作者 Xiaoshi Fan Yingjie Lei Yanan Wang 《Journal of Systems Engineering and Electronics》 SCIE EI CSCD 2017年第3期585-596,共12页
To enhance the accuracy of intuitionistic fuzzy time series forecasting model, this paper analyses the influence of universe of discourse partition and compares with relevant literature. Traditional models usually par... To enhance the accuracy of intuitionistic fuzzy time series forecasting model, this paper analyses the influence of universe of discourse partition and compares with relevant literature. Traditional models usually partition the global universe of discourse, which is not appropriate for all objectives. For example, the universe of the secular trend model is continuously variational. In addition, most forecasting methods rely on prior information, i.e., fuzzy relationship groups (FRG). Numerous relationship groups lead to the explosive growth of relationship library in a linear model and increase the computational complexity. To overcome problems above and ascertain an appropriate order, an intuitionistic fuzzy time series forecasting model based on order decision and adaptive partition algorithm is proposed. By forecasting the vector operator matrix, the proposed model can adjust partitions and intervals adaptively. The proposed model is tested on student enrollments of Alabama dataset, typical seasonal dataset Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX) and a secular trend dataset of total retail sales for social consumer goods in China. Experimental results illustrate the validity and applicability of the proposed method for different patterns of dataset. 展开更多
关键词 intuitionistic fuzzy set time series forecasting vector operator matrix order deciding adaptive partition
下载PDF
A Heuristic Algorithm to Solve the Task Partition Problem in MDOCEM
6
作者 He Yan-xiang Chen Xin-mong +1 位作者 Donald H. Cooley Zhang Jian-ping 《Wuhan University Journal of Natural Sciences》 CAS 1999年第2期27-30,共4页
This paper examines task partition problem in a Multiagent based Distributed Open Computing Environment Model(MDOCEM). We first present a formal method to describe the task partition problem, then give a heuristic al... This paper examines task partition problem in a Multiagent based Distributed Open Computing Environment Model(MDOCEM). We first present a formal method to describe the task partition problem, then give a heuristic algorithm to solve the task partition problem that gives an approximate optimum solution. 展开更多
关键词 distributed open computing environment task partition planning management operation set
下载PDF
Optimal Operation with Dynamic Partitioning Strategy for Centralized Shared Energy Storage Station with Integration of Large-scale Renewable Energy 被引量:2
7
作者 Jianlin Li Zhijin Fang +3 位作者 Qian Wang Mengyuan Zhang Yaxin Li Weijun Zhang 《Journal of Modern Power Systems and Clean Energy》 SCIE EI CSCD 2024年第2期359-370,共12页
As renewable energy continues to be integrated into the grid,energy storage has become a vital technique supporting power system development.To effectively promote the efficiency and economics of energy storage,centra... As renewable energy continues to be integrated into the grid,energy storage has become a vital technique supporting power system development.To effectively promote the efficiency and economics of energy storage,centralized shared energy storage(SES)station with multiple energy storage batteries is developed to enable energy trading among a group of entities.In this paper,we propose the optimal operation with dynamic partitioning strategy for the centralized SES station,considering the day-ahead demands of large-scale renewable energy power plants.We implement a multi-entity cooperative optimization operation model based on Nash bargaining theory.This model is decomposed into two subproblems:the operation profit maximization problem with energy trading and the leasing payment bargaining problem.The distributed alternating direction multiplier method(ADMM)is employed to address the subproblems separately.Simulations reveal that the optimal operation with a dynamic partitioning strategy improves the tracking of planned output of renewable energy entities,enhances the actual utilization rate of energy storage,and increases the profits of each participating entity.The results confirm the practicality and effectiveness of the strategy. 展开更多
关键词 Shared energy storage(SES) dynamic partitioning strategy optimal operation Nash bargaining theory actual utilization rate of energy storage
原文传递
ON FINITE RANK OPERATORS IN CSL ALGEBRAS Ⅲ
8
作者 Chen Peixin Lu Shijie Tao ChangliDept.of Math.,Zhejiang Univ.,Hangzhou 310027,China. 《Applied Mathematics(A Journal of Chinese Universities)》 SCIE CSCD 2002年第3期301-306,共6页
In terms of the exactly nonzero partition,the reducible projection-system and correlation matrices,two characterizations for a rank three operator in a CSL algebra can be completely decomposed are given.
关键词 CSL algebra finite rank operator exactly partition.
下载PDF
L^p-BOUNDEDNESS OF A KIND OF MULTI-PSEUDO-DIFFERENTIAL OPERATORS
9
作者 张成国 彭立中 《四川师范大学学报(自然科学版)》 CAS CSCD 1991年第1期93-94,共2页
1 Main ResultWe consider a kind of multi-pseudo-differential operators, which is introduced by R. Coifman and Y.Meyer [1].Let symbol function σ(x , a, ε)∈C<sup>∞</sup>(R<sup>n</sup>×... 1 Main ResultWe consider a kind of multi-pseudo-differential operators, which is introduced by R. Coifman and Y.Meyer [1].Let symbol function σ(x , a, ε)∈C<sup>∞</sup>(R<sup>n</sup>×R<sup>nm</sup>×R<sup>n</sup>) satisfy the following conditions:where x∈R<sup>n</sup>, a = (a<sub>1</sub>,…,a<sub>m</sub>) a-j∈R<sup>n</sup>,ζ∈R<sup>n</sup>, q = (q<sub>1</sub>,…,q<sub>m</sub>), q<sub>j</sub>∈(Z+)<sup>n</sup> and q =|q<sub>1</sub>|+…+|q<sub>m</sub>|,The multi-pseudo-differential operator with the symbol o(x,α,ζ) is defined as 展开更多
关键词 SYMBOL operators MULTI satisfy BOUNDEDNESS partition proof LEMMA 内任
下载PDF
Fast,Exact and Robust Set Operations on Polyhedrons Using Localized Constructive Solid Geometry Trees
10
作者 Ping Lu Xudong Jiang +2 位作者 Wei Lu Ran Wei Bin Sheng 《ZTE Communications》 2015年第3期57-66,共10页
Regularized Boolean operations have been widely used in 3D modeling systems. However, evaluating Boolean operations may be quite numerically unstable and time consuming, especially for iterated set operations. A novel... Regularized Boolean operations have been widely used in 3D modeling systems. However, evaluating Boolean operations may be quite numerically unstable and time consuming, especially for iterated set operations. A novel and unified technique is proposed in this paper for computing single and iterated set operations efficiently, robustly and exactly. An adaptive octree is combined with a nested constructive solid geometry (CSG) tree by this technique. The intersection handling is restricted to the cells in the octree where intersection actually occurs. Within those cells, a CSG tree template is instanced by the surfaces and the tree is converted to planebased binary space partitioning (BSP) for set evaluation; Moreover, the surface classification is restricted to the ceils in the octree where the surfaces only come from a model and are within the bounding-boxes of other polyhedrons. These two ways bring about the efficiency and scalability of the operations, in terms of runtime and memory. As all surfaces in such a cell have the same classification relation, they are classified as a whole. Robustness and exactness are achieved by integrating plane-based geometry representation with adaptive geometry predicate technique in intersection handling, and by applying divide-and-conquer arithmetic on surface classification. Experimental results demonstrate that the proposed approach can guarantee the robustness of Boolean computations and runs faster than other existing approaches. 展开更多
关键词 Boolean operations polyhedrons constructive solid geometry binary space partitioning tree
下载PDF
Analytic Expression of Arbitrary Matrix Elements for Boson Exponential Quadratic Polynomial Operators
11
作者 XU Xiu-Wei REN Ting-Qi LIU Shu-Yan MA Qiu-Ming LIU Sheng-Dian 《Communications in Theoretical Physics》 SCIE CAS CSCD 2007年第1期41-44,共4页
Making use of the transformation relation among usual, normal, and antinormal ordering for the multimode boson exponential quadratic polynomial operators (BEQPO's)I we present the analytic expression of arbitrary m... Making use of the transformation relation among usual, normal, and antinormal ordering for the multimode boson exponential quadratic polynomial operators (BEQPO's)I we present the analytic expression of arbitrary matrix elements for BEQPO's. As a preliminary application, we obtain the exact expressions of partition function about the boson quadratic polynomial system, matrix elements in particle-number, coordinate, and momentum representation, and P representation for the BEQPO's. 展开更多
关键词 Boson exponential quadratic polynomial operator matrix element P representation partition function of Boson quadratic polynomial system
下载PDF
Monotonic Vector Space Model (Ⅰ): Concepts and Operations
12
作者 Jianwen Hu 《American Journal of Operations Research》 2014年第1期30-41,共12页
Monotonic vector space (MVS), as a novel model in which there exist some monotonic mappings, is proposed. MVS model is an abstract of many practical problems (such as image processing, system capability engineering, e... Monotonic vector space (MVS), as a novel model in which there exist some monotonic mappings, is proposed. MVS model is an abstract of many practical problems (such as image processing, system capability engineering, etc.) and includes many useful important operations. This paper, as the first one of series papers, discusses the MVS framework, relative important concepts and important operations including partition, synthesis, screening, sampling, etc. And algorithms for these operations are the focus of this paper. The application of these operations in system capability engineering will be dealt with in the second part of this series of papers. 展开更多
关键词 MONOTONIC VECTOR SPACE operations partition SYNTHESIS SCREENING Sampling
下载PDF
套代数上的一类非线性中心化子
13
作者 纪玉德 吴冰 杨翠 《河北科技大学学报》 CAS 北大核心 2024年第2期176-180,共5页
为了推广算子代数中的基本理论,对一类非线性映射成为套代数上的可加中心化子的条件进行了研究。首先,基于Hilbert空间上的非平凡套定义与该套有关的套代数,并定义套代数上的一个非线性映射;其次,采用矩阵分块方法获得关于此映射的几个... 为了推广算子代数中的基本理论,对一类非线性映射成为套代数上的可加中心化子的条件进行了研究。首先,基于Hilbert空间上的非平凡套定义与该套有关的套代数,并定义套代数上的一个非线性映射;其次,采用矩阵分块方法获得关于此映射的几个性质;最后,证明套代数上满足某种条件的非线性映射为可加中心化子,给出刻画该映射的具体形式。结果表明,套代数上满足某种条件的非线性映射为可加中心化子,且可完全刻画。研究结果推广了非线性映射成为套代数上可加中心化子的结论,丰富了算子代数拓扑结构的分类问题,为套代数上其他类型非线性映射问题的刻画提供了借鉴与参考。 展开更多
关键词 算子代数 中心化子 非线性映射 HILBERT空间 矩阵分块
下载PDF
基于分区操作系统的文件系统设计
14
作者 杨琼 王冬 +1 位作者 张鹏 王济乾 《航空计算技术》 2024年第4期80-83,共4页
研究涉及一种基于分区操作系统的文件系统设计方法,解决分区操作系统下文件的安全访问问题。分区操作系统资源、空间彼此隔离,文件系统用于管理分区数据,不能违背数据资源的隔离需求。为此,通过将文件系统部署在分区操作系统的分区;分... 研究涉及一种基于分区操作系统的文件系统设计方法,解决分区操作系统下文件的安全访问问题。分区操作系统资源、空间彼此隔离,文件系统用于管理分区数据,不能违背数据资源的隔离需求。为此,通过将文件系统部署在分区操作系统的分区;分区内和分区外采用两层分区安全访问控制;采用快照管理实现文件系统自身对并行文件访问需求,实现文件系统设计。通过测试表明,该方法能够在满足分区操作系统设计约束下,实现文件的安全访问。 展开更多
关键词 分区操作系统 文件系统 安全访问控制 并行文件访问 快照管理
下载PDF
极端条件下多源配电网主动孤岛研究综述与展望
15
作者 陈春 万金金 +1 位作者 曹一家 孙充勃 《电力科学与技术学报》 CAS CSCD 北大核心 2024年第5期1-11,共11页
高比例分布式电源接入后的故障配电网,具备主动恢复与自治能力,有利于提升电力系统适应气候变化和灾害风险管理的水平,顺应配电网技术的发展方向。当前,配电网主动孤岛自恢复和稳定运行的研究受到广泛关注。首先,概述多源配电网主动孤... 高比例分布式电源接入后的故障配电网,具备主动恢复与自治能力,有利于提升电力系统适应气候变化和灾害风险管理的水平,顺应配电网技术的发展方向。当前,配电网主动孤岛自恢复和稳定运行的研究受到广泛关注。首先,概述多源配电网主动孤岛的内涵,以及多源配电网孤岛与微电网孤岛的区别,分析归纳配电网主动孤岛在分布式电源高比例接入下稳定运行存在的关键问题。其次,从主动孤岛划分、孤岛自组网波动平抑控制以及组网完成后主动孤岛小干扰稳定控制3个方面阐述国内外研究现状,并进行总结。最后,基于电力物联网、高电力电子化下的小干扰抑制以及能源互联网背景下的主动孤岛方案,对多源配电网主动孤岛的发展前景进行展望和总结。 展开更多
关键词 配电网主动孤岛 孤岛划分 孤岛自组网 孤岛稳定运行
下载PDF
基于Flexsim仿真的电梯群组上高峰分区分层运行管理模式比较研究
16
作者 朱丹 何雅菁 陈浩航 《机电工程技术》 2024年第5期16-20,共5页
按楼层序号奇偶属性分区分层运行管理模式与按楼层跨度分区分层运行管理模式是电梯调度管理的两种常见模式,但模式间的优胜性未见验证。为比较上述两种运行管理模式效能的优劣,选择以上高峰交通流为背景,应用Flexsim仿真方法对按楼层序... 按楼层序号奇偶属性分区分层运行管理模式与按楼层跨度分区分层运行管理模式是电梯调度管理的两种常见模式,但模式间的优胜性未见验证。为比较上述两种运行管理模式效能的优劣,选择以上高峰交通流为背景,应用Flexsim仿真方法对按楼层序号奇偶属性分区分层运行管理模式与按楼层跨度分区分层运行管理模式分别构建仿真模型,并从时效性、能耗性、舒适度、服务能力4个维度提出评价分区分层运行管理模式运行效能的指标体系。仿真结果表明,按楼层跨度分区分层运行管理模式的运行效能要优于按楼层序号奇偶属性分区分层运行管理模式的运行效能。在此基础上提出设置可移动式候梯设施、分区分层信息醒目提示、程序锁定电梯轿厢可停靠楼层等管理措施,为分区分层运行管理模式的施行创建良好条件。 展开更多
关键词 电梯群组 分区分层 运行管理模式 上高峰 Flexsim仿真 效能评价
下载PDF
基于临时支护新装置的工序优化与支护性能分析 被引量:1
17
作者 王云柱 成云海 +4 位作者 王贯东 苏畅 李峰辉 许文涛 马梦想 《煤炭工程》 北大核心 2024年第2期66-71,共6页
针对掘进工作面存在的掘支平行作业难、用时比重失调问题,为实现平行作业、快速支护,设计了新的临时支护装置,具有尺寸可调、快速响应、及时支护的特点。采用理论分析加数值模拟的方法进行研究;介绍了装置的结构设计;以新巨龙煤矿综掘... 针对掘进工作面存在的掘支平行作业难、用时比重失调问题,为实现平行作业、快速支护,设计了新的临时支护装置,具有尺寸可调、快速响应、及时支护的特点。采用理论分析加数值模拟的方法进行研究;介绍了装置的结构设计;以新巨龙煤矿综掘施工技术为例进行工艺优化,从理论上提出一种分区平行作业的施工技术;通过顶板两端简支岩梁力学模型,展开装置临时支护作用机理分析;并对装置的支护性能进行结构静力学分析。结果表明:在巷道顶板松动范围0~3.30 m时,装置支护所需承受的载荷为158.40 kN;仿真得到最大等效应力为57.37 MPa,在装置顶梁下端与耳座位置交接处,最大位移量出现在中间梁上端中部,为0.30 mm,装置的结构设计能满足支护要求。此研究可为实现快速掘进的平行作业,高效支护和安全生产,提供有益参考。 展开更多
关键词 掘进工作面 临时支护装置 分区平行作业 工序优化 支护性能
下载PDF
基于蜣螂优化的改进粒子群算法
18
作者 易云飞 王志勇 施运应 《重庆邮电大学学报(自然科学版)》 CSCD 北大核心 2024年第3期533-542,共10页
针对标准粒子群算法存在的局部最优、早熟和慢收敛等问题,提出了一种新的粒子群更新方法。改进了算法惯性权重,引入一种新的更新方式;借鉴蜣螂优化算法中蜣螂滚球、繁殖、觅食和偷窃行为,将基本粒子群的操作划分为寻优、变异、波动和跳... 针对标准粒子群算法存在的局部最优、早熟和慢收敛等问题,提出了一种新的粒子群更新方法。改进了算法惯性权重,引入一种新的更新方式;借鉴蜣螂优化算法中蜣螂滚球、繁殖、觅食和偷窃行为,将基本粒子群的操作划分为寻优、变异、波动和跳跃,从而提高了算法的全局寻优能力和收敛速度,并避免了早熟问题。通过与其他9种智能算法进行实验对比表明,在10个基准测试函数中,基于蜣螂优化的改进粒子群算法在寻优能力和收敛速度方面表现出色,证实了该算法的优越性。 展开更多
关键词 蜣螂优化 改进粒子群算法 操作划分 优越性
下载PDF
面向大规模农田的农机跨区作业紧急调配研究
19
作者 于春辉 常淑惠 +3 位作者 张璠 姚竟发 孟宇 郭亚倩 《中国农机化学报》 北大核心 2024年第8期196-203,共8页
针对大规模农田的粮食抢收工作,由于抢收约束时间的限制,导致可能无法在规定时间内完成所有粮食的收割任务,此时则会出现粮食损失值。构建以粮食损失值最小化为目标的农机跨区紧急调配模型,提出基于两阶段的农机紧急调配算法(Agricultur... 针对大规模农田的粮食抢收工作,由于抢收约束时间的限制,导致可能无法在规定时间内完成所有粮食的收割任务,此时则会出现粮食损失值。构建以粮食损失值最小化为目标的农机跨区紧急调配模型,提出基于两阶段的农机紧急调配算法(Agricultural Machinery Emergency Algorithm based on Two Stages,TSEA)。首先按照基于距离的分区策略对大规模农田进行分区,接着采用改进的遗传算法分别对各农田分区进行农机紧急作业调配。为验证算法的有效性,主要从紧急任务的粮食损失值、算法运行时间等方面将TSEA算法与GA、SA算法进行比较,试验结果表明,TSEA算法得到的紧急调度方案优于其他两种算法。为验证本文分区策略的有效性,分别采用TSEA算法和不分区的紧急调配算法从粮食损失值、算法运行时间等方面进行比较,验证分区策略的有效性。多组试验结果表明该文提出的策略和算法对于解决大规模农田的农机跨区紧急调配问题更有效,可为农机管理部门提供解决方案。 展开更多
关键词 农业机械 大规模农田 跨区作业 聚类分区 紧急调配
下载PDF
基于优先填补策略的Spark数据均衡分区方法
20
作者 何玉林 吴东彤 +1 位作者 Philippe Fournier-Viger 黄哲学 《电子学报》 EI CAS CSCD 北大核心 2024年第10期3322-3335,共14页
Spark作为基于内存计算的分布式大数据处理框架,运行速度快且通用性强.在任务计算过程中,Spark的默认分区器HashPartitioner在处理倾斜数据时,容易产生各个分区数据量不平衡的情况,导致资源利用率低且运行效率差.现存的Spark均衡分区改... Spark作为基于内存计算的分布式大数据处理框架,运行速度快且通用性强.在任务计算过程中,Spark的默认分区器HashPartitioner在处理倾斜数据时,容易产生各个分区数据量不平衡的情况,导致资源利用率低且运行效率差.现存的Spark均衡分区改进方法,例如多阶段分区、迁移分区和采样分区等,大多存在尺度把控难、通信开销成本高、对采样过度依赖等缺陷.为改善上述问题,本文提出了一种基于优先填补策略的分区方法,同时考虑了样本数据和非样本数据的分配,以便实现对全部数据的均衡分区.该方法在对数据采样并根据样本信息估算出每个键的权值后,将键按照权值大小降序排列,依次将键在满足分区容忍度的条件下分配到前面的分区中,为未被采样的键预留后面的分区空间,以获得针对样本数据的分区方案.Spark根据分区方案对样本中出现的键对应的数据进行分区,没有出现的键对应的数据则直接映射到可分配的最后一个分区中.实验结果表明,新分区方法能够有效实现Spark数据的均衡分区,在美国运输统计局发布的真实航空数据集上,基于该方法设计的优先填补分区器的总运行时间比HashPartitioner平均缩短了15.3%,比现有的均衡数据分区器和哈希键值重分配分区器分别平均缩短了38.7%和30.2%. 展开更多
关键词 均衡分区 优先填补策略 数据倾斜 Spark算子 大数据
下载PDF
上一页 1 2 14 下一页 到第
使用帮助 返回顶部