With the continuous expansion of the data center network scale, changing network requirements, and increasing pressure on network bandwidth, the traditional network architecture can no longer meet people’s needs. The...With the continuous expansion of the data center network scale, changing network requirements, and increasing pressure on network bandwidth, the traditional network architecture can no longer meet people’s needs. The development of software defined networks has brought new opportunities and challenges to future networks. The data and control separation characteristics of SDN improve the performance of the entire network. Researchers have integrated SDN architecture into data centers to improve network resource utilization and performance. This paper first introduces the basic concepts of SDN and data center networks. Then it discusses SDN-based load balancing mechanisms for data centers from different perspectives. Finally, it summarizes and looks forward to the study on SDN-based load balancing mechanisms and its development trend.展开更多
Providing highly efficient underwater transmission of mass multimedia data is challenging due to the particularities of the underwater environment. Although there are many schemes proposed to optimize the underwater a...Providing highly efficient underwater transmission of mass multimedia data is challenging due to the particularities of the underwater environment. Although there are many schemes proposed to optimize the underwater acoustic network communication protocols, from physical layer, data link layer, network layer to transport layer, the existing routing protocols for underwater wireless sensor network(UWSN) still cannot well deal with the problems in transmitting multimedia data because of the difficulties involved in high energy consumption, low transmission reliability or high transmission delay. It prevents us from applying underwater multimedia data to real-time monitoring of marine environment in practical application, especially in emergency search, rescue operation and military field. Therefore, the inefficient transmission of marine multimedia data has become a serious problem that needs to be solved urgently. In this paper, A Layered Load Balance Routing Protocol(L2-LBMT) is proposed for underwater multimedia data transmission. In L2-LBMT, we use layered and load-balance Ad Hoc Network to transmit data, and adopt segmented data reliable transfer(SDRT) protocol to improve the data transport reliability. And a 3-node variant of tornado(3-VT) code is also combined with the Ad Hoc Network to transmit little emergency data more quickly. The simulation results show that the proposed protocol can balance energy consumption of each node, effectively prolong the network lifetime and reduce transmission delay of marine multimedia data.展开更多
Energy-efficient data gathering in multi-hop wireless sensor networks was studied,considering that different node produces different amounts of data in realistic environments.A novel dominating set based clustering pr...Energy-efficient data gathering in multi-hop wireless sensor networks was studied,considering that different node produces different amounts of data in realistic environments.A novel dominating set based clustering protocol (DSCP) was proposed to solve the data gathering problem in this scenario.In DSCP,a node evaluates the potential lifetime of the network (from its local point of view) assuming that it acts as the cluster head,and claims to be a tentative cluster head if it maximizes the potential lifetime.When evaluating the potential lifetime of the network,a node considers not only its remaining energy,but also other factors including its traffic load,the number of its neighbors,and the traffic loads of its neighbors.A tentative cluster head becomes a final cluster head with a probability inversely proportional to the number of tentative cluster heads that cover its neighbors.The protocol can terminate in O(n/lg n) steps,and its total message complexity is O(n2/lg n).Simulation results show that DSCP can effectively prolong the lifetime of the network in multi-hop networks with unbalanced traffic load.Compared with EECT,the network lifetime is prolonged by 56.6% in average.展开更多
A new method of establishing rolling load distribution model was developed by online intelligent information-processing technology for plate rolling. The model combines knowledge model and mathematical model with usin...A new method of establishing rolling load distribution model was developed by online intelligent information-processing technology for plate rolling. The model combines knowledge model and mathematical model with using knowledge discovery in database (KDD) and data mining (DM) as the start. The online maintenance and optimization of the load model are realized. The effectiveness of this new method was testified by offline simulation and online application.展开更多
The backup requirement of data centres is tremendous as the size of data created by human is massive and is increasing exponentially.Single node deduplication cannot meet the increasing backup requirement of data cent...The backup requirement of data centres is tremendous as the size of data created by human is massive and is increasing exponentially.Single node deduplication cannot meet the increasing backup requirement of data centres.A feasible way is the deduplication cluster,which can meet it by adding storage nodes.The data routing strategy is the key of the deduplication cluster.DRSS(data routing strategy using semantics) improves the storage utilization of MCS(minimum chunk signature) data routing strategy a lot.However,for the large deduplication cluster,the load balance of DRSS is worse than MCS.To improve the load balance of DRSS,we propose a load balance strategy used for DRSS,namely DRSSLB.When a node is overloaded,DRSSLB iteratively migrates the current smallest container of the node to the smallest node in the deduplication cluster until this overloaded node becomes non-overloaded.A container is the minimum unit of data migration.Similar files sharing the same features or file names are stored in the same container.This ensures the similar data groups are still in the same node after rebalancing the nodes.We use the dataset from the real world to evaluate DRSSLB.Experimental results show that,for various numbers of nodes of the deduplication cluster,the data skews of DRSSLB are under predefined value while the storage utilizations of DRSSLB do not nearly increase compared with DRSS,with the low penalty(the data migration rate is only6.5% when the number of nodes is 64).展开更多
In data stream management systems (DSMSs), how to maintain the quality of queries is a difficult problem because both the processing cost and data arrival rates are highly unpredictable. When the system is overloaded,...In data stream management systems (DSMSs), how to maintain the quality of queries is a difficult problem because both the processing cost and data arrival rates are highly unpredictable. When the system is overloaded, quality degrades significantly and thus load shedding becomes necessary. Unlike processing overloading in the general way which is only by a feedback control (FB) loop to obtain a good and stable performance over data streams, a feedback plus feed-forward control (FFC) strategy is introduced in DSMSs, which have a good quality of service (QoS) in the aspects of miss ratio and processing delay. In this paper, a quality adaptation framework is proposed, in which the control-theory-based techniques are leveraged to adjust the application behavior with the considerations of the current system status. Compared to previous solutions, the FFC strategy achieves a good quality with a waste of fewer resources.展开更多
In a data center network (DCN), load balancing is required when servers transfer data on the same path. This is necessary to avoid congestion. Load balancing is challenged by the dynamic transferral of demands and c...In a data center network (DCN), load balancing is required when servers transfer data on the same path. This is necessary to avoid congestion. Load balancing is challenged by the dynamic transferral of demands and complex routing control. Because of the distributed nature of a traditional network, previous research on load balancing has mostly focused on improving the performance of the local network; thus, the load has not been optimally balanced across the entire network. In this paper, we propose a novel dynamic load-balancing algorithm for fat-tree. This algorithm avoids congestions to the great possible extent by searching for non-conflicting paths in a centralized way. We implement the algorithm in the popular software-defined networking architecture and evaluate the algorithm' s performance on the Mininet platform. The results show that our algorithm has higher bisection band- width than the traditional equal-cost multi-path load-balancing algorithm and thus more effectively avoids congestion.展开更多
With the development of drone technology and oblique photogrammetry technology, the acquisition of oblique photogrammetry models and basemap becomes more and more convenient and quickly. The increase in the number of ...With the development of drone technology and oblique photogrammetry technology, the acquisition of oblique photogrammetry models and basemap becomes more and more convenient and quickly. The increase in the number of basemap leads to excessively redundant basemap tiles requests in 3D GIS when loading oblique photogrammetry models, which slows down the system. Aiming at improving the speed of running system, this paper proposes a dynamic strategy for loading basemap tiles. Different from existing 3D GIS which loading oblique photogrammetry models and basemap tiles inde-pendently, this strategy dynamically loads basemap tiles depending on different height of view and the range of loaded oblique photogrammetry models. We achieve dynamic loading of basemap tiles by predetermining whether the basemap tiles will be covered by the oblique photogrammetry models. The experimental results show that this strategy can greatly reduce the num-ber of redundant requests from the client to the server while ensuring the user’s visual requirements for the oblique photogrammetric model.展开更多
The mining of the rules from the electrical load time series data which are collected from the EMS (Energy Management System) is discussed. The data from the EMS are too huge and sophisticated to be understood and use...The mining of the rules from the electrical load time series data which are collected from the EMS (Energy Management System) is discussed. The data from the EMS are too huge and sophisticated to be understood and used by the power system engineer, while useful information is hidden in the electrical load data. The authors discuss the use of fuzzy linguistic summary as data mining method to induce the rules from the electrical load time series. The data preprocessing techniques are also discussed in the paper.展开更多
目前数据中心网络(data center network,DCN)的负载均衡方法存在对大小流的调度缺乏全局实时检测等不足,部分大流会造成拥塞、负载不均衡和带宽碎片等问题.针对上述问题,提出了一种SDN网络流量负载均衡算法—DSA-D.首先,对流量进行分类...目前数据中心网络(data center network,DCN)的负载均衡方法存在对大小流的调度缺乏全局实时检测等不足,部分大流会造成拥塞、负载不均衡和带宽碎片等问题.针对上述问题,提出了一种SDN网络流量负载均衡算法—DSA-D.首先,对流量进行分类,为大流计算所有源至目的主机可达路径的最短跳数路径集;然后,根据LLDP和ECHO测量链路时延以求得时延最优路径集;最后,采用概率拟合算法分配路径,实现数据中心网络流量负载均衡.在相同场景下的实验结果表明,与ECMP、Hedera和DIFF算法相比,DSA-D算法具有更好的吞吐量、链路带宽利用率和平均往返时延.展开更多
According to the chaotic and non-linear characters of power load data,the time series matrix is established with the theory of phase-space reconstruction,and then Lyapunov exponents with chaotic time series are comput...According to the chaotic and non-linear characters of power load data,the time series matrix is established with the theory of phase-space reconstruction,and then Lyapunov exponents with chaotic time series are computed to determine the time delay and the embedding dimension.Due to different features of the data,data mining algorithm is conducted to classify the data into different groups.Redundant information is eliminated by the advantage of data mining technology,and the historical loads that have highly similar features with the forecasting day are searched by the system.As a result,the training data can be decreased and the computing speed can also be improved when constructing support vector machine(SVM) model.Then,SVM algorithm is used to predict power load with parameters that get in pretreatment.In order to prove the effectiveness of the new model,the calculation with data mining SVM algorithm is compared with that of single SVM and back propagation network.It can be seen that the new DSVM algorithm effectively improves the forecast accuracy by 0.75%,1.10% and 1.73% compared with SVM for two random dimensions of 11-dimension,14-dimension and BP network,respectively.This indicates that the DSVM gains perfect improvement effect in the short-term power load forecasting.展开更多
排序算法是计算机科学领域的一个基础算法,是大量应用的算法核心。在大数据时代,随着数据量的极速增长,并行排序算法受到广泛关注。现有的并行排序算法普遍存在通信开销过大、负载不均衡等问题,导致算法难以大规模扩展。针对以上问题,...排序算法是计算机科学领域的一个基础算法,是大量应用的算法核心。在大数据时代,随着数据量的极速增长,并行排序算法受到广泛关注。现有的并行排序算法普遍存在通信开销过大、负载不均衡等问题,导致算法难以大规模扩展。针对以上问题,提出一种大规模可扩展的正则采样并行排序(scalable parallel sorting by regular sampling,ScaPSRS)算法,摒弃传统正则采样并行排序(parallel sorting by regular sampling,PSRS)算法中由一个进程负责采样的做法,转而让所有进程参与正则采样,选出p-1个分隔元素,将整个数据集划分成p个不相交的子集,然后实施并行排序,避免了单一进程的采样瓶颈。此外,ScaPSRS采用一种新的迭代更新策略选择p-1个分隔元素,保证划分的p个子集尽可能大小相同,从而确保p个进程对各自的子集进行本地排序时的负载均衡。在天河二号超级计算机上进行的大量实验表明,ScaPSRS算法能够成功地扩展到32000个内核,性能比PSRS算法和Hofmann等人提出的分区算法分别提升了3.7倍和11.7倍。展开更多
文摘With the continuous expansion of the data center network scale, changing network requirements, and increasing pressure on network bandwidth, the traditional network architecture can no longer meet people’s needs. The development of software defined networks has brought new opportunities and challenges to future networks. The data and control separation characteristics of SDN improve the performance of the entire network. Researchers have integrated SDN architecture into data centers to improve network resource utilization and performance. This paper first introduces the basic concepts of SDN and data center networks. Then it discusses SDN-based load balancing mechanisms for data centers from different perspectives. Finally, it summarizes and looks forward to the study on SDN-based load balancing mechanisms and its development trend.
基金supported by the National Natural Science Foundation of China (No.61401413)the Digital Home Industry Cluster Oriented Technology Service Innovation Pilot Project in 2015
文摘Providing highly efficient underwater transmission of mass multimedia data is challenging due to the particularities of the underwater environment. Although there are many schemes proposed to optimize the underwater acoustic network communication protocols, from physical layer, data link layer, network layer to transport layer, the existing routing protocols for underwater wireless sensor network(UWSN) still cannot well deal with the problems in transmitting multimedia data because of the difficulties involved in high energy consumption, low transmission reliability or high transmission delay. It prevents us from applying underwater multimedia data to real-time monitoring of marine environment in practical application, especially in emergency search, rescue operation and military field. Therefore, the inefficient transmission of marine multimedia data has become a serious problem that needs to be solved urgently. In this paper, A Layered Load Balance Routing Protocol(L2-LBMT) is proposed for underwater multimedia data transmission. In L2-LBMT, we use layered and load-balance Ad Hoc Network to transmit data, and adopt segmented data reliable transfer(SDRT) protocol to improve the data transport reliability. And a 3-node variant of tornado(3-VT) code is also combined with the Ad Hoc Network to transmit little emergency data more quickly. The simulation results show that the proposed protocol can balance energy consumption of each node, effectively prolong the network lifetime and reduce transmission delay of marine multimedia data.
基金Projects(61173169,61103203)supported by the National Natural Science Foundation of ChinaProject(NCET-10-0798)supported by the Program for New Century Excellent Talents in University of ChinaProject supported by the Post-doctoral Program and the Freedom Explore Program of Central South University,China
文摘Energy-efficient data gathering in multi-hop wireless sensor networks was studied,considering that different node produces different amounts of data in realistic environments.A novel dominating set based clustering protocol (DSCP) was proposed to solve the data gathering problem in this scenario.In DSCP,a node evaluates the potential lifetime of the network (from its local point of view) assuming that it acts as the cluster head,and claims to be a tentative cluster head if it maximizes the potential lifetime.When evaluating the potential lifetime of the network,a node considers not only its remaining energy,but also other factors including its traffic load,the number of its neighbors,and the traffic loads of its neighbors.A tentative cluster head becomes a final cluster head with a probability inversely proportional to the number of tentative cluster heads that cover its neighbors.The protocol can terminate in O(n/lg n) steps,and its total message complexity is O(n2/lg n).Simulation results show that DSCP can effectively prolong the lifetime of the network in multi-hop networks with unbalanced traffic load.Compared with EECT,the network lifetime is prolonged by 56.6% in average.
文摘A new method of establishing rolling load distribution model was developed by online intelligent information-processing technology for plate rolling. The model combines knowledge model and mathematical model with using knowledge discovery in database (KDD) and data mining (DM) as the start. The online maintenance and optimization of the load model are realized. The effectiveness of this new method was testified by offline simulation and online application.
基金supported by the National Natural Science Foundation of China under Grant No.61373120the Aeronautical Science Foundation of China under Grant No.2014ZD53049
文摘The backup requirement of data centres is tremendous as the size of data created by human is massive and is increasing exponentially.Single node deduplication cannot meet the increasing backup requirement of data centres.A feasible way is the deduplication cluster,which can meet it by adding storage nodes.The data routing strategy is the key of the deduplication cluster.DRSS(data routing strategy using semantics) improves the storage utilization of MCS(minimum chunk signature) data routing strategy a lot.However,for the large deduplication cluster,the load balance of DRSS is worse than MCS.To improve the load balance of DRSS,we propose a load balance strategy used for DRSS,namely DRSSLB.When a node is overloaded,DRSSLB iteratively migrates the current smallest container of the node to the smallest node in the deduplication cluster until this overloaded node becomes non-overloaded.A container is the minimum unit of data migration.Similar files sharing the same features or file names are stored in the same container.This ensures the similar data groups are still in the same node after rebalancing the nodes.We use the dataset from the real world to evaluate DRSSLB.Experimental results show that,for various numbers of nodes of the deduplication cluster,the data skews of DRSSLB are under predefined value while the storage utilizations of DRSSLB do not nearly increase compared with DRSS,with the low penalty(the data migration rate is only6.5% when the number of nodes is 64).
基金Supported by the National Key R&D Program of China(2016YFC1401900)the National Science Foundation of China(61173029,61672144)
文摘In data stream management systems (DSMSs), how to maintain the quality of queries is a difficult problem because both the processing cost and data arrival rates are highly unpredictable. When the system is overloaded, quality degrades significantly and thus load shedding becomes necessary. Unlike processing overloading in the general way which is only by a feedback control (FB) loop to obtain a good and stable performance over data streams, a feedback plus feed-forward control (FFC) strategy is introduced in DSMSs, which have a good quality of service (QoS) in the aspects of miss ratio and processing delay. In this paper, a quality adaptation framework is proposed, in which the control-theory-based techniques are leveraged to adjust the application behavior with the considerations of the current system status. Compared to previous solutions, the FFC strategy achieves a good quality with a waste of fewer resources.
基金supported by the National Basic Research Program of China(973 Program)(2012CB315903)the Key Science and Technology Innovation Team Project of Zhejiang Province(2011R50010-05)+3 种基金the National Science and Technology Support Program(2014BAH24F01)863 Program of China(2012AA01A507)the National Natural Science Foundation of China(61379118 and 61103200)sponsored by the Research Fund of ZTE Corporation
文摘In a data center network (DCN), load balancing is required when servers transfer data on the same path. This is necessary to avoid congestion. Load balancing is challenged by the dynamic transferral of demands and complex routing control. Because of the distributed nature of a traditional network, previous research on load balancing has mostly focused on improving the performance of the local network; thus, the load has not been optimally balanced across the entire network. In this paper, we propose a novel dynamic load-balancing algorithm for fat-tree. This algorithm avoids congestions to the great possible extent by searching for non-conflicting paths in a centralized way. We implement the algorithm in the popular software-defined networking architecture and evaluate the algorithm' s performance on the Mininet platform. The results show that our algorithm has higher bisection band- width than the traditional equal-cost multi-path load-balancing algorithm and thus more effectively avoids congestion.
文摘With the development of drone technology and oblique photogrammetry technology, the acquisition of oblique photogrammetry models and basemap becomes more and more convenient and quickly. The increase in the number of basemap leads to excessively redundant basemap tiles requests in 3D GIS when loading oblique photogrammetry models, which slows down the system. Aiming at improving the speed of running system, this paper proposes a dynamic strategy for loading basemap tiles. Different from existing 3D GIS which loading oblique photogrammetry models and basemap tiles inde-pendently, this strategy dynamically loads basemap tiles depending on different height of view and the range of loaded oblique photogrammetry models. We achieve dynamic loading of basemap tiles by predetermining whether the basemap tiles will be covered by the oblique photogrammetry models. The experimental results show that this strategy can greatly reduce the num-ber of redundant requests from the client to the server while ensuring the user’s visual requirements for the oblique photogrammetric model.
文摘The mining of the rules from the electrical load time series data which are collected from the EMS (Energy Management System) is discussed. The data from the EMS are too huge and sophisticated to be understood and used by the power system engineer, while useful information is hidden in the electrical load data. The authors discuss the use of fuzzy linguistic summary as data mining method to induce the rules from the electrical load time series. The data preprocessing techniques are also discussed in the paper.
文摘目前数据中心网络(data center network,DCN)的负载均衡方法存在对大小流的调度缺乏全局实时检测等不足,部分大流会造成拥塞、负载不均衡和带宽碎片等问题.针对上述问题,提出了一种SDN网络流量负载均衡算法—DSA-D.首先,对流量进行分类,为大流计算所有源至目的主机可达路径的最短跳数路径集;然后,根据LLDP和ECHO测量链路时延以求得时延最优路径集;最后,采用概率拟合算法分配路径,实现数据中心网络流量负载均衡.在相同场景下的实验结果表明,与ECMP、Hedera和DIFF算法相比,DSA-D算法具有更好的吞吐量、链路带宽利用率和平均往返时延.
基金Project(70671039) supported by the National Natural Science Foundation of China
文摘According to the chaotic and non-linear characters of power load data,the time series matrix is established with the theory of phase-space reconstruction,and then Lyapunov exponents with chaotic time series are computed to determine the time delay and the embedding dimension.Due to different features of the data,data mining algorithm is conducted to classify the data into different groups.Redundant information is eliminated by the advantage of data mining technology,and the historical loads that have highly similar features with the forecasting day are searched by the system.As a result,the training data can be decreased and the computing speed can also be improved when constructing support vector machine(SVM) model.Then,SVM algorithm is used to predict power load with parameters that get in pretreatment.In order to prove the effectiveness of the new model,the calculation with data mining SVM algorithm is compared with that of single SVM and back propagation network.It can be seen that the new DSVM algorithm effectively improves the forecast accuracy by 0.75%,1.10% and 1.73% compared with SVM for two random dimensions of 11-dimension,14-dimension and BP network,respectively.This indicates that the DSVM gains perfect improvement effect in the short-term power load forecasting.
文摘排序算法是计算机科学领域的一个基础算法,是大量应用的算法核心。在大数据时代,随着数据量的极速增长,并行排序算法受到广泛关注。现有的并行排序算法普遍存在通信开销过大、负载不均衡等问题,导致算法难以大规模扩展。针对以上问题,提出一种大规模可扩展的正则采样并行排序(scalable parallel sorting by regular sampling,ScaPSRS)算法,摒弃传统正则采样并行排序(parallel sorting by regular sampling,PSRS)算法中由一个进程负责采样的做法,转而让所有进程参与正则采样,选出p-1个分隔元素,将整个数据集划分成p个不相交的子集,然后实施并行排序,避免了单一进程的采样瓶颈。此外,ScaPSRS采用一种新的迭代更新策略选择p-1个分隔元素,保证划分的p个子集尽可能大小相同,从而确保p个进程对各自的子集进行本地排序时的负载均衡。在天河二号超级计算机上进行的大量实验表明,ScaPSRS算法能够成功地扩展到32000个内核,性能比PSRS算法和Hofmann等人提出的分区算法分别提升了3.7倍和11.7倍。