Network and data location aware approach for simultaneous job scheduling and data replication in large-scale data grid environments

Data Grid integrates graphically distributed resources for solving data intensive scientific applications. Effective scheduling in Grid can reduce the amount of data transferred among nodes by submitting a job to a node, where most of the requested data files are available. Scheduling is a traditional problem in parallel and distributed system. However, due to special issues and goals of Grid, traditional approach is not effective in this environment any more. Therefore, it is necessary to propose methods specialized for this kind of parallel and distributed system. Another solution is to use a data replication strategy to create multiple copies of files and store them in convenient locations to shorten file access times. To utilize the above two concepts, in this paper we develop a job scheduling policy, called hierarchical job scheduling strategy （HJSS）, and a dynamic data replication strategy, called advanced dynamic hierarchical replication strategy （ADHRS）, to improve the data access efficiencies in a hierarchical Data Grid. HJSS uses hierarchical scheduling to reduce the search time for an appropriate computing node. It considers network characteristics, number of jobs waiting in queue, file locations, and disk read speed of storage drive at data sources. Moreover, due to the limited storage capacity, a good replica replacement algorithm is needed. We present a novel replacement strategy which deletes files in two steps when free space is not enough for the new replica： first, it deletes those files with minimum time for transferring. Second, if space is still insufficient then it considers the last time the replica was requested, number of access, size of replica and file transfer time. The simulation results show that our proposed algorithm has better performance in comparison with other algorithms in terms of job execution time, number of intercommunications, number of replications, hit ratio, computing resource usage and storage usage.

Adaptive data replication strategy in cloud computing for performance improvement

Cloud computing is becoming a very popular word in industry and is receiving a large amount of atten- tion from the research community. Replica management is one of the most important issues in the cloud, which can offer fast data access time, high data availability and reliability. By keeping all replicas active, the replicas may enhance system task successful execution rate if the replicas and requests are reasonably distributed. However, appropriate replica place- ment in a large-scale, dynamically scalable and totally vir- tualized data centers is much more complicated. To provide cost-effective availability, minimize the response time of ap- plications and make load balancing for cloud storage, a new replica placement is proposed. The replica placement is based on five important parameters： mean service time, failure probability, load variance, latency and storage usage. How- ever, replication should be used wisely because the storage size of each site is limited. Thus, the site must keep only the important replicas. We also present a new replica replacement strategy based on the availability of the file, the last time the replica was requested, number of access, and size of replica. We evaluate our algorithm using the CloudSim simulator and find that it offers better performance in comparison with other algorithms in terms of mean response time, effective network usage, load balancing, replication frequency, and storage usage

Hierarchical data replication strategy to improve performance in cloud computing

Cloud computing environment is getting more interesting as a new trend of data management.Data replication has been widely applied to improve data access in distributed systems such as Grid and Cloud.However,due to the finite storage capacity of each site,copies that are useful for future jobs can be wastefully deleted and replaced with less valuable ones.Therefore,it is considerable to have appropriate replication strategy that can dynamically store the replicas while satisfying quality of service(QoS)requirements and storage capacity constraints.In this paper,we present a dynamic replication algorithm,named hierarchical data replication strategy(HDRS).HDRS consists of the replica creation that can adaptively increase replicas based on exponential growth or decay rate,the replica placement according to the access load and labeling technique,and finally the replica replacement based on the value of file in the future.We evaluate different dynamic data replication methods using CloudSim simulation.Experiments demonstrate that HDRS can reduce response time and bandwidth usage compared with other algorithms.It means that the HDRS can determine a popular file and replicates it to the best site.This method avoids useless replications and decreases access latency by balancing the load of sites.