Improved Harris Hawks Optimization Algorithm Based Data Placement Strategy for Integrated Cloud and Edge Computing

Cloud computing is considered to facilitate a more cost-effective way to deploy scientific workflows.The individual tasks of a scientific work-flow necessitate a diversified number of large states that are spatially located in different datacenters,thereby resulting in huge delays during data transmis-sion.Edge computing minimizes the delays in data transmission and supports the fixed storage strategy for scientific workflow private datasets.Therefore,this fixed storage strategy creates huge amount of bottleneck in its storage capacity.At this juncture,integrating the merits of cloud computing and edge computing during the process of rationalizing the data placement of scientific workflows and optimizing the energy and time incurred in data transmission across different datacentres remains a challenge.In this paper,Adaptive Cooperative Foraging and Dispersed Foraging Strategies-Improved Harris Hawks Optimization Algorithm(ACF-DFS-HHOA)is proposed for optimizing the energy and data transmission time in the event of placing data for a specific scientific workflow.This ACF-DFS-HHOA considered the factors influencing transmission delay and energy consumption of data centers into account during the process of rationalizing the data placement of scientific workflows.The adaptive cooperative and dispersed foraging strategy is included in HHOA to guide the position updates that improve population diversity and effectively prevent the algorithm from being trapped into local optimality points.The experimental results of ACF-DFS-HHOA confirmed its predominance in minimizing energy and data transmission time incurred during workflow execution.

Optimal Data Placement and Replication Approach for SIoT with Edge

Social networks(SNs)are sources with extreme number of users around the world who are all sharing data like images,audio,and video to their friends using IoT devices.This concept is the so-called Social Internet of Things(SIot).The evolving nature of edge-cloud computing has enabled storage of a large volume of data from various sources,and this task demands an efficient storage procedure.For this kind of large volume of data storage,the usage of data replication using edge with geo-distributed cloud service area is suited to fulfill the user’s expectations with low latency.The major issue is the way to store the data and replicate these large data items optimally and allocate the request from the data center efficiently.For efficient storage of these data,we use edge server,which is part of the cloud server,in this study.Thus,the data are distributed and stored with quick access,which will reduce the latency with response.The proposed data placement approach learns with machine learning(ML)algorithm called radial basis kernel function assisted with support vector machine(RBF-SVM)to classify the data center for storing the user and friend’s data from the SIoT devices.These learning algorithms will be used to predict the workload of the data stored in the data center as either edge or cloud depending on the existing time slots.The data placement with dynamic nature is also optimized using the proposed dynamic graph partitioning(GP)method to meet the individual user’s demand of low latency with minimum costs.This way will keep the SIoT data placement efficient and effective over time.Accordingly,this proposed data placement and replication approach introduces three kinds of innovations compared with the existing data placement approach.(i)Rather than storing the user data in a single cloud,this study uses the edge server closest to the SIoT devices for faster access with reduced response time.(ii)The classification algorithm called RBF-SVM is used to find storage for user for reducing data replication.(iii)Dynamic GP is introduced for data placement with reduced latency and minimum cost to fulfil the dynamic nature of the SN.The simulation result of this approach obtains reduced latency of 130 ms and minimum cost compared with those of the existing data placement approaches.Therefore,our proposed data placement with ML-based learning on edge provides promising results in terms of efficiency,effectiveness,and performance with reduced latency and minimum cost.

Graphical-based data placement algorithm for cloud workflow

When workflow task needs several datasets from different locations in cloud,data transfer becomes a challenge.To avoid the unnecessary data transfer,a graphical-based data placement algorithm for cloud workflow is proposed.The algorithm uses affinity graph to group datasets while keeping a polynomial time complexity.By integrating the algorithm,the workflow engine can intelligently select locations in which the data will reside to avoid the unnecessary data transfer during the initial stage and runtime stage.Simulations show that the proposed algorithm can effectively reduce data transfer during the workflow's execution.

Efficient Location-Aware Data Placement for Data-Intensive Applications in Geo-distributed Scientific Data Centers

Recent developments in cloud computing and big data have spurred the emergence of data-intensive applications for which massive scientific datasets are stored in globally distributed scientific data centers that have a high frequency of data access by scientists worldwide. Multiple associated data items distributed in different scientific data centers may be requested for one data processing task, and data placement decisions must respect the storage capacity limits of the scientific data centers. Therefore, the optimization of data access cost in the placement of data items in globally distributed scientific data centers has become an increasingly important goal.Existing data placement approaches for geo-distributed data items are insufficient because they either cannot cope with the cost incurred by the associated data access, or they overlook storage capacity limitations, which are a very practical constraint of scientific data centers. In this paper, inspired by applications in the field of high energy physics, we propose an integer-programming-based data placement model that addresses the above challenges as a Non-deterministic Polynomial-time（NP）-hard problem. In addition we use a Lagrangian relaxation based heuristics algorithm to obtain ideal data placement solutions. Our simulation results demonstrate that our algorithm is effective and significantly reduces overall data access cost.

Capability-Aware Data Placement for Heterogeneous Active Storage Systems

By moving computations from computing nodes to storage nodes, active storage technology provides an efficient for data-intensive high-performance computing applications. The existing studies have neglected the heterogeneity of storage nodes on the performance of active storage systems. We introduce CADP, a capability-aware data placement scheme for heterogeneous active storage systems to obtain high-performance data processing. The basic idea of CADP is to place data on storage nodes based on their computing capability and storage capability, so that the load-imbalance among heterogeneous servers can be avoided. We have implemented CADP under a parallel I/O system. The experimental results show that the proposed capability-aware data placement scheme can improve the active storage system performance significantly.

Novel Data Placement Algorithm for Distributed Storage System Based on Fault-Tolerant Domain

The 3-replica redundancy strategy is widely used to solve the problem of data reliability in large-scale distributed storage systems. However, its storage capacity utilization is only 33%. In this paper, a data placement algorithm based on fault-tolerant domain (FTD) is proposed. Owing to the fine-grained design of the FTD, the data reliability of systems using two replicas is comparable to that of current mainstream systems using three replicas, and the capacity utilization is increased to 50%. Moreover, the proposed FTD provides a new concept for the design of distributed storage systems. Distributed storage systems can take FTDs as the units for data placement, data migration, data repair and so on. In addition, fault detection can be performed independently and concurrently within the FTDs.

Sensor Placement for Sensing Coverage and Data Precision in Wireless Sensor Networks

We present a novel paradigm of sensor placement concerning data precision and estimation.Multiple abstract sensors are used to measure a quantity of a moving target in the scenario of a wireless sensor network.These sensors can cooperate with each other to obtain a precise estimate of the quantity in a real-time manner.We consider a problem on planning a minimum-cost scheme of sensor placement with desired data precision and resource consumption.Measured data is modeled as a Gaussian random variable with a changeable variance.A gird model is used to approximate the problem.We solve the problem with a heuristic algorithm using branch-and-bound method and tabu search.Our experiments demonstrate that the algorithm is correct in a certain tolerance,and it is also efficient and scalable.

A Data-Placement Strategy Based on Genetic Algorithm in Cloud Computing

With the development of Computerized Business Application, the amount of data is increasing exponentially. Cloud computing provides high performance computing resources and mass storage resources for massive data processing. In distributed cloud computing systems, data intensive computing can lead to data scheduling between data centers. Reasonable data placement can reduce data scheduling between the data centers effectively, and improve the data acquisition efficiency of users. In this paper, the mathematical model of data scheduling between data centers is built. By means of the global optimization ability of the genetic algorithm, generational evolution produces better approximate solution, and gets the best approximation of the data placement at last. The experimental results show that genetic algorithm can effectively work out the approximate optimal data placement, and minimize data scheduling between data centers.

An Improvement on Data-Driven Pole Placement for State Feedback Control and Model Identification

The recently proposed data-driven pole placement method is able to make use of measurement data to simultaneously identify a state space model and derive pole placement state feedback gain. It can achieve this precisely for systems that are linear time-invariant and for which noiseless measurement datasets are available. However, for nonlinear systems, and/or when the only noisy measurement datasets available contain noise, this approach is unable to yield satisfactory results. In this study, we investigated the effect on data-driven pole placement performance of introducing a prefilter to reduce the noise present in datasets. Using numerical simulations of a self-balancing robot, we demonstrated the important role that prefiltering can play in reducing the interference caused by noise.

云数据中心时延优化的数据放置方法

针对云数据中心数据获取效率低和服务器资源浪费问题,为优化云平台的数据访问和资源利用,文中提出了一种时延优化的云数据中心数据放置(LOP)方法。文中首先分析了云平台的性能,建立了云平台的资源利用和数据获取时间模型。然后基于非支配排序算法NSGA-Ⅲ实现了全局最优的数据放置策略,对数据资源进行合理部署,有效利用服务器的资源,提高了数据获取的效率。最后通过CloudSim仿真平台,对提出的数据放置方法进行了仿真和对比实验。实验结果表明,LOP方法能明显提高云服务器的资源利用率,缩短任务的数据获取时间。

基于数据驱动的高校学生毕业去向画像构建与分析

新时代背景下高校学生对择业提出更高的要求,推进高校学生个性化发展具有十分重要的意义,因此,帮助学生更好地规划适合自己的职业发展方向是必要的。文章采用智能算法对某高校的学情数据进行分析,以学生的毕业去向为导向构建画像并进行可视化,同时对各类画像进行比较,为高校学生发展提出合理建议。

Adaptive Controller Placement in Software Defined Wireless Networks

Controller placement problem(CPP)is a critical issue in software defined wireless networks(SDWN).Due to the limited power of wireless devices,CPP is facing the challenge of energy efficiency in SDWN.Nevertheless,the related research on CPP in SDWN hasn’t modeled the energy consumption of controllers so far.To prolong the lifetime of SDWN and improve the practicability of research,we rebuilt a CPP model considering the minimal transmitted power of controllers.An adaptive controller placement algorithm(ACPA)is proposed with the following two stages.First,data field method is adopted to determine sub-networks for different network topologies.Second,for each sub-network we adopt an exhaustive method to find the optimal location which meets the minimal average transmitted power to place controller.Compared with the other algorithms,the effectiveness and efficiency of the proposed scheme are validated through simulation.

Optimal Territorial Resources Placement for Multipurpose Wireless Services Using Genetic Algorithms

This paper presents a study for finding a solution to the placement of territorial resources for multipurpose wireless services considering also the restrictions imposed by the orography of the territory itself. To solve this problem genetic algorithms are used to identify sites where to place the resources for the optimal coverage of a given area. The used algorithm has demonstrated to be able to find optimal solutions in a variety of considered situations.

基于“推荐-学习”的两阶段数据布局策略

针对数据布局不合理导致云边协同集群服务质量下降和运营开销增加等问题,提出一种基于“推荐-学习”的两阶段数据副本管理机制.鉴于数据密集型移动应用的特点,综合考虑了云边环境下的数据访问延迟和放置代价之间的最优权衡.从理论上构建了副本放置的多目标数学模型,将决策问题描述为具有延迟和成本约束的双目标优化问题;在“推荐”阶段,设计了一个基于移动预测和反馈优化的副本推荐引擎,减少了副本创建的盲目性;在“学习”阶段,构建了一个基于异步优势行动者-评论家算法(A3C)的强化学习副本放置规则学习模型,改进了副本服务的全局性能指标.实验结果表明,基于“推荐-学习”的两阶段数据布局策略能够有效地减少等待延迟和节约成本开销,为现代云边协同系统的数据管理服务提供行之有效的解决方案,具有重要的理论意义和应用价值.

一种优化的Hadoop数据放置策略

Hadoop分布式文件系统(HDFS)的默认数据块放置策略均衡了数据存储的可靠性和读写速度,却没有考虑发挥集群的最佳性能。针对该问题提出了一种优化后的数据块放置算法。该算法为数据块设计2个指标,即被查询率与平均读取时间,用于评估集群执行任务对数据块的需求量。在符合HDFS默认数据放置算法基本规则的前提下,通过对数据块的需求量进行分析,然后重新计算数据块的放置位置,将需求量最多的数据转移到能够最快处理它们的节点上。通过实验数据证明:该算法可以使集群整体性能提高20%以上。优化后的数据块放置算法是有效的,并且不会增加对集群带宽的占用。

基于容器放置优化的云数据中心节能技术

为解决容器服务过程中带来的容器重复放置和“虚拟机-物理机”操作系统类型限制问题,提出一种离散白鼬优化容器放置方法(discrete stoat optimization container placement,DSO-CP)。以最小化能耗和最大化资源利用率为优化目标,建立多目标约束的容器放置优化模型,利用离散索引减少解向量结构维数,降低适应度函数复杂度,将容器放置和虚拟机放置两个阶段定义为一个问题,参考白鼬环状收缩捕猎模式构建DSO-CP算法进行联合优化,迭代获得最优解。仿真对虚拟机资源利用率和物理机激活数量进行分析,讨论3种不同环境下的能耗情况。仿真结果表明,该算法在降低数据中心能耗及提升其计算能力方面具有一定优势。

一种基于区域划分和内容流行度的NDN缓存策略

针对命名网络中路由器缓存能力有限的问题,提出了一种基于区域划分和内容流行度的NDN缓存策略(CCNCP).该策略根据拓扑结构划分区域,并综合衡量节点重要性选出局部中心节点和全局中心节点;引入分类的内容流行度,合理地将不同流行度的内容缓存在不同重要度的节点.首先采用经典的GN算法进行社区划分.定义了度中心性、紧密中心性、介数中心性等中心性度量指标.然后建立了系统模型,给出了缓存策略.最后对Zachary空手道俱乐部网络的社团结构进行改编,并引入真实拓扑GARR,使用Icarus模拟器进行仿真试验.结果表明:CCNCP在2个不同网络结构中表现得最好;当参数S等于0.25时,CCNCP和第2好缓存策略ProbCache的缓存命中率、延迟、路径拉伸比分别为30.9%和26.8%、52.9和56.2 ms、0.584和0.617,前者比后者分别提高4.1%、降低5.9%、降低5.3%,可见CCNCP在3个指标上与现有的典型策略相比都具有显著优势.

云环境下优化科学工作流执行性能的两阶段数据放置与任务调度策略

云环境中跨数据中心科学工作流的高效执行通常面临数据交互量大的问题.文中给出基于相关度的两阶段高效数据放置策略和任务调度策略:即在工作流建立阶段根据数据依赖关系图把关系紧密型数据集尽可能放置到同一数据中心;而后任务调度策略在运行阶段将任务调度到数据依赖最大的数据中心执行,并将新产生数据集放置到相关度最高的数据中心.实验表明,该策略能有效减少跨数据中心科学工作流执行时的数据传输量,从而能有效提升科学工作流的执行效率,并能减少资源的租赁费用.

云计算环境下面向数据密集型应用的数据布局策略与方法

云计算环境下面向流程的数据密集型应用已被广泛应用于多个领域.面对多数据中心的云计算环境,这类应用在数据布局方面遇到了新的挑战,主要表现在如何减少跨数据中心的数据传输、如何保持数据间的依赖性以及如何在提高效率的同时兼顾全局的负载均衡等.针对这些挑战,文中提出一种三阶段数据布局策略,分别针对跨数据中心数据传输、数据依赖关系和全局负载均衡三个目标对数据布局方案进行求解和优化.实验显示,文中提出的数据布局策略具有良好的综合性能,特别是在降低流程执行过程中由跨数据中心数据传输所导致的时间开销方面,效果尤为明显.