This paper considers a time-constrained data collection problem from a network of ground sensors located on uneven terrain by an Unmanned Aerial Vehicle(UAV),a typical Unmanned Aerial System(UAS).The ground sensors ha...This paper considers a time-constrained data collection problem from a network of ground sensors located on uneven terrain by an Unmanned Aerial Vehicle(UAV),a typical Unmanned Aerial System(UAS).The ground sensors harvest renewable energy and are equipped with batteries and data buffers.The ground sensor model takes into account sensor data buffer and battery limitations.An asymptotically globally optimal method of joint UAV 3D trajectory optimization and data transmission schedule is developed.The developed method maximizes the amount of data transmitted to the UAV without losses and too long delays and minimizes the propulsion energy of the UAV.The developed algorithm of optimal trajectory optimization and transmission scheduling is based on dynamic programming.Computer simulations demonstrate the effectiveness of the proposed algorithm.展开更多
We have witnessed the fast-growing deployment of Hadoop,an open-source implementation of the MapReduce programming model,for purpose of data-intensive computing in the cloud.However,Hadoop was not originally designed ...We have witnessed the fast-growing deployment of Hadoop,an open-source implementation of the MapReduce programming model,for purpose of data-intensive computing in the cloud.However,Hadoop was not originally designed to run transient jobs in which us ers need to move data back and forth between storage and computing facilities.As a result,Hadoop is inefficient and wastes resources when operating in the cloud.This paper discusses the inefficiency of MapReduce in the cloud.We study the causes of this inefficiency and propose a solution.Inefficiency mainly occurs during data movement.Transferring large data to computing nodes is very time-con suming and also violates the rationale of Hadoop,which is to move computation to the data.To address this issue,we developed a dis tributed cache system and virtual machine scheduler.We show that our prototype can improve performance significantly when run ning different applications.展开更多
The implementation of artificial intelligence(AI)in a smart society,in which the analysis of human habits is mandatory,requires automated data scheduling and analysis using smart applications,a smart infrastructure,sm...The implementation of artificial intelligence(AI)in a smart society,in which the analysis of human habits is mandatory,requires automated data scheduling and analysis using smart applications,a smart infrastructure,smart systems,and a smart network.In this context,which is characterized by a large gap between training and operative processes,a dedicated method is required to manage and extract the massive amount of data and the related information mining.The method presented in this work aims to reduce this gap with near-zero-failure advanced diagnostics(AD)for smart management,which is exploitable in any context of Society 5.0,thus reducing the risk factors at all management levels and ensuring quality and sustainability.We have also developed innovative applications for a humancentered management system to support scheduling in the maintenance of operative processes,for reducing training costs,for improving production yield,and for creating a human–machine cyberspace for smart infrastructure design.The results obtained in 12 international companies demonstrate a possible global standardization of operative processes,leading to the design of a near-zero-failure intelligent system that is able to learn and upgrade itself.Our new method provides guidance for selecting the new generation of intelligent manufacturing and smart systems in order to optimize human–machine interactions,with the related smart maintenance and education.展开更多
In order to solve the problem that the existing data scheduling algorithm cannot make full use of neighbors' bandwidth resources when allocating data request among several senders in the multisender based P2P stre...In order to solve the problem that the existing data scheduling algorithm cannot make full use of neighbors' bandwidth resources when allocating data request among several senders in the multisender based P2P streaming system,a peer priority based scheduling algorithm is proposed.The algorithm calculates neighbors' priority based on peers' historical service evaluation as well as how many wanted data that the neighbor has.The data request allocated to each neighbor is adjusted dynamically according to the priority when scheduling.Peers with high priority are preferred to allocate more data request.Experiment shows the algorithm can make full use of neighbors' bandwidth resources to transmit data to reduce server pressure effectively and improve system scalability.展开更多
At present, big data is very popular, because it has proved to be much successful in many fields such as social media, E-commerce transactions, etc. Big data describes the tools and technologies needed to capture, man...At present, big data is very popular, because it has proved to be much successful in many fields such as social media, E-commerce transactions, etc. Big data describes the tools and technologies needed to capture, manage, store, distribute, and analyze petabyte or larger-sized datasets having different structures with high speed. Big data can be structured, unstructured, or semi structured. Hadoop is an open source framework that is used to process large amounts of data in an inexpensive and efficient way, and job scheduling is a key factor for achieving high performance in big data processing. This paper gives an overview of big data and highlights the problems and challenges in big data. It then highlights Hadoop Distributed File System (HDFS), Hadoop MapReduce, and various parameters that affect the performance of job scheduling algorithms in big data such as Job Tracker, Task Tracker, Name Node, Data Node, etc. The primary purpose of this paper is to present a comparative study of job scheduling algorithms along with their experimental results in Hadoop environment. In addition, this paper describes the advantages, disadvantages, features, and drawbacks of various Hadoop job schedulers such as FIFO, Fair, capacity, Deadline Constraints, Delay, LATE, Resource Aware, etc, and provides a comparative study among these schedulers.展开更多
With cloud computing technology becoming more mature, it is essential to combine the big data processing tool Hadoop with the Infrastructure as a Service(Iaa S) cloud platform. In this study, we first propose a new ...With cloud computing technology becoming more mature, it is essential to combine the big data processing tool Hadoop with the Infrastructure as a Service(Iaa S) cloud platform. In this study, we first propose a new Dynamic Hadoop Cluster on Iaa S(DHCI) architecture, which includes four key modules: monitoring,scheduling, Virtual Machine(VM) management, and VM migration modules. The load of both physical hosts and VMs is collected by the monitoring module and can be used to design resource scheduling and data locality solutions. Second, we present a simple load feedback-based resource scheduling scheme. The resource allocation can be avoided on overburdened physical hosts or the strong scalability of virtual cluster can be achieved by fluctuating the number of VMs. To improve the flexibility, we adopt the separated deployment of the computation and storage VMs in the DHCI architecture, which negatively impacts the data locality. Third, we reuse the method of VM migration and propose a dynamic migration-based data locality scheme using parallel computing entropy. We migrate the computation nodes to different host(s) or rack(s) where the corresponding storage nodes are deployed to satisfy the requirement of data locality. We evaluate our solutions in a realistic scenario based on Open Stack.Substantial experimental results demonstrate the effectiveness of our solutions that contribute to balance the workload and performance improvement, even under heavy-loaded cloud system conditions.展开更多
The scheduling of earth observation satellites(EOSs)data transmission is a complex combinatorial optimization problem. Current researches mainly deal with this problem on the assumption that the data transmission mode...The scheduling of earth observation satellites(EOSs)data transmission is a complex combinatorial optimization problem. Current researches mainly deal with this problem on the assumption that the data transmission mode is fixed, either playback or real-time transmission. Considering the characteristic of the problem, a multi-satellite real-time and playback data transmission scheduling model is established and a novel algorithm based on quantum discrete particle swarm optimization(QDPSO)is proposed. Furthermore, we design the longest compatible transmission chain mutation operator to enhance the performance of the algorithm. Finally, some experiments are implemented to validate correctness and practicability of the proposed algorithm.展开更多
Power saving is one of the key factors of emerging 4G mobile network as well as in IEEE 802.16e wireless metropolitan area networks (Wireless MAN). An efficient power saving mechanism is the heart for the guarantee of...Power saving is one of the key factors of emerging 4G mobile network as well as in IEEE 802.16e wireless metropolitan area networks (Wireless MAN). An efficient power saving mechanism is the heart for the guarantee of a long operating lifetime for a mobile subscriber station (MS), because MSs are normally driven by rechargeable batteries. It is a vital factor for Base Station (BS) of the same network. One of the most important features of 5G mobile is the extension of battery energy 10 times greater than the present days. In this context, the evaluation of duration of sleep mode of BS or MS based on traffic load of a network is now a new era of research work. In this paper, such analysis has been done based on two statistical models: Poisson’s pdf and Engset pdf. The concept of complete sharing and partitioning of user group of teletraffic engineering is applied to measure the possibility of getting leisure time of BS or MS. Both the traffic models used in the paper are applicable in both limited and unlimited user network, i.e. in micro and fem to cellular network of 4G and 5G.展开更多
To relieve the negative effect brought by the intricate wireless network environment and unstable user behavior in layered mobile peer-to peer(P2P) streaming service, an evolved layered P2P (E-LP2P) data schedulin...To relieve the negative effect brought by the intricate wireless network environment and unstable user behavior in layered mobile peer-to peer(P2P) streaming service, an evolved layered P2P (E-LP2P) data scheduling scheme in the process of service delivery is introduced in this paper. The data in base layer is scheduled according to its importance in streaming play to guarantee the basic play of streaming. The data in enhancement layer is scheduled according to the characters of streaming data, including its position and amount in server peer set in a multiple tied way towards the data in enhancement layer. To cope with the layer jitter caused by the fluctuation of bandwidth, jitter prevent mechanism is used to adjust the highest layer dynamically during the process of data scheduling. Simulation results show that the E-LP2P can provide good quality of service(QoS) performance in terms of throughput, layer delivery ratio, server load and useless packet ratio.展开更多
WiMAX distributed scheduling can be modeled as two procedures:three-way handshaking procedure and data subframe scheduling procedure.Due to manipulating data transmission directly,data subframe scheduling has a close...WiMAX distributed scheduling can be modeled as two procedures:three-way handshaking procedure and data subframe scheduling procedure.Due to manipulating data transmission directly,data subframe scheduling has a closer relationship with user Quality of Service(QoS) satisfaction,and has more severe impact on network performance,compared with handshaking procedure.A QoS guaranteed Throughput-Efficiency Optimal distributed data subframe Scheduling scheme,named as QoS-TEOS,is proposed.QoS-TEOS achieves QoS guarantee through modeling services into different ranks and assigning them with corresponding priorities.A service with higher priority is scheduled ahead of that with lower priority and offered with high QoS quality.Same kinds of services that request similar QoS quality are classified into one service set.Different service sets are scheduled with different strategies.QoS-TEOS promotes network performance through improving network throughput and efficiency.Theoretical analysis shows that the scheduled data transmission should balance data generation rate from upper layer and transmission rate of physical layer,to avoid network throughput and efficiency declining.Simulation results show that QoS-TEOS works excellently to achieve throughput-efficiency optimization and guarantee a high QoS.展开更多
基金funding from the Australian Government,via Grant No.AUSMURIB000001 associated with ONR MURI Grant No.N00014-19-1-2571。
文摘This paper considers a time-constrained data collection problem from a network of ground sensors located on uneven terrain by an Unmanned Aerial Vehicle(UAV),a typical Unmanned Aerial System(UAS).The ground sensors harvest renewable energy and are equipped with batteries and data buffers.The ground sensor model takes into account sensor data buffer and battery limitations.An asymptotically globally optimal method of joint UAV 3D trajectory optimization and data transmission schedule is developed.The developed method maximizes the amount of data transmitted to the UAV without losses and too long delays and minimizes the propulsion energy of the UAV.The developed algorithm of optimal trajectory optimization and transmission scheduling is based on dynamic programming.Computer simulations demonstrate the effectiveness of the proposed algorithm.
文摘We have witnessed the fast-growing deployment of Hadoop,an open-source implementation of the MapReduce programming model,for purpose of data-intensive computing in the cloud.However,Hadoop was not originally designed to run transient jobs in which us ers need to move data back and forth between storage and computing facilities.As a result,Hadoop is inefficient and wastes resources when operating in the cloud.This paper discusses the inefficiency of MapReduce in the cloud.We study the causes of this inefficiency and propose a solution.Inefficiency mainly occurs during data movement.Transferring large data to computing nodes is very time-con suming and also violates the rationale of Hadoop,which is to move computation to the data.To address this issue,we developed a dis tributed cache system and virtual machine scheduler.We show that our prototype can improve performance significantly when run ning different applications.
文摘The implementation of artificial intelligence(AI)in a smart society,in which the analysis of human habits is mandatory,requires automated data scheduling and analysis using smart applications,a smart infrastructure,smart systems,and a smart network.In this context,which is characterized by a large gap between training and operative processes,a dedicated method is required to manage and extract the massive amount of data and the related information mining.The method presented in this work aims to reduce this gap with near-zero-failure advanced diagnostics(AD)for smart management,which is exploitable in any context of Society 5.0,thus reducing the risk factors at all management levels and ensuring quality and sustainability.We have also developed innovative applications for a humancentered management system to support scheduling in the maintenance of operative processes,for reducing training costs,for improving production yield,and for creating a human–machine cyberspace for smart infrastructure design.The results obtained in 12 international companies demonstrate a possible global standardization of operative processes,leading to the design of a near-zero-failure intelligent system that is able to learn and upgrade itself.Our new method provides guidance for selecting the new generation of intelligent manufacturing and smart systems in order to optimize human–machine interactions,with the related smart maintenance and education.
基金Supported by the National High Technology Research and Development Program of China(No.2009AA01A339,2008AA01A317)the National Natural Science Foundation of China for Distinguished Young Scholars(No.60903218F0208)the Science and Technology Support Plan of China(No.2008BAH28B04)
文摘In order to solve the problem that the existing data scheduling algorithm cannot make full use of neighbors' bandwidth resources when allocating data request among several senders in the multisender based P2P streaming system,a peer priority based scheduling algorithm is proposed.The algorithm calculates neighbors' priority based on peers' historical service evaluation as well as how many wanted data that the neighbor has.The data request allocated to each neighbor is adjusted dynamically according to the priority when scheduling.Peers with high priority are preferred to allocate more data request.Experiment shows the algorithm can make full use of neighbors' bandwidth resources to transmit data to reduce server pressure effectively and improve system scalability.
文摘At present, big data is very popular, because it has proved to be much successful in many fields such as social media, E-commerce transactions, etc. Big data describes the tools and technologies needed to capture, manage, store, distribute, and analyze petabyte or larger-sized datasets having different structures with high speed. Big data can be structured, unstructured, or semi structured. Hadoop is an open source framework that is used to process large amounts of data in an inexpensive and efficient way, and job scheduling is a key factor for achieving high performance in big data processing. This paper gives an overview of big data and highlights the problems and challenges in big data. It then highlights Hadoop Distributed File System (HDFS), Hadoop MapReduce, and various parameters that affect the performance of job scheduling algorithms in big data such as Job Tracker, Task Tracker, Name Node, Data Node, etc. The primary purpose of this paper is to present a comparative study of job scheduling algorithms along with their experimental results in Hadoop environment. In addition, this paper describes the advantages, disadvantages, features, and drawbacks of various Hadoop job schedulers such as FIFO, Fair, capacity, Deadline Constraints, Delay, LATE, Resource Aware, etc, and provides a comparative study among these schedulers.
基金supported by the Open Project Program of Jiangsu High Technology Research Key Laboratory for Wireless Sensor Networks(No.WSNLBKF201503)the Fundamental Research Funds for the Central Universities(No.2016JBM011)+2 种基金Fundamental Research Funds for the Central Universities(No.2014ZD03-03)the Priority Academic Program Development of Jiangsu Higher Education InstitutionsJiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology
文摘With cloud computing technology becoming more mature, it is essential to combine the big data processing tool Hadoop with the Infrastructure as a Service(Iaa S) cloud platform. In this study, we first propose a new Dynamic Hadoop Cluster on Iaa S(DHCI) architecture, which includes four key modules: monitoring,scheduling, Virtual Machine(VM) management, and VM migration modules. The load of both physical hosts and VMs is collected by the monitoring module and can be used to design resource scheduling and data locality solutions. Second, we present a simple load feedback-based resource scheduling scheme. The resource allocation can be avoided on overburdened physical hosts or the strong scalability of virtual cluster can be achieved by fluctuating the number of VMs. To improve the flexibility, we adopt the separated deployment of the computation and storage VMs in the DHCI architecture, which negatively impacts the data locality. Third, we reuse the method of VM migration and propose a dynamic migration-based data locality scheme using parallel computing entropy. We migrate the computation nodes to different host(s) or rack(s) where the corresponding storage nodes are deployed to satisfy the requirement of data locality. We evaluate our solutions in a realistic scenario based on Open Stack.Substantial experimental results demonstrate the effectiveness of our solutions that contribute to balance the workload and performance improvement, even under heavy-loaded cloud system conditions.
基金supported by the National Natural Science Foundation of China(6110118461174159)
文摘The scheduling of earth observation satellites(EOSs)data transmission is a complex combinatorial optimization problem. Current researches mainly deal with this problem on the assumption that the data transmission mode is fixed, either playback or real-time transmission. Considering the characteristic of the problem, a multi-satellite real-time and playback data transmission scheduling model is established and a novel algorithm based on quantum discrete particle swarm optimization(QDPSO)is proposed. Furthermore, we design the longest compatible transmission chain mutation operator to enhance the performance of the algorithm. Finally, some experiments are implemented to validate correctness and practicability of the proposed algorithm.
文摘Power saving is one of the key factors of emerging 4G mobile network as well as in IEEE 802.16e wireless metropolitan area networks (Wireless MAN). An efficient power saving mechanism is the heart for the guarantee of a long operating lifetime for a mobile subscriber station (MS), because MSs are normally driven by rechargeable batteries. It is a vital factor for Base Station (BS) of the same network. One of the most important features of 5G mobile is the extension of battery energy 10 times greater than the present days. In this context, the evaluation of duration of sleep mode of BS or MS based on traffic load of a network is now a new era of research work. In this paper, such analysis has been done based on two statistical models: Poisson’s pdf and Engset pdf. The concept of complete sharing and partitioning of user group of teletraffic engineering is applied to measure the possibility of getting leisure time of BS or MS. Both the traffic models used in the paper are applicable in both limited and unlimited user network, i.e. in micro and fem to cellular network of 4G and 5G.
基金supported by the National Natural Science Foundation of China (60902047)the Fundamental Research Funds for the Central Universities (BUPT 2009RC0120)
文摘To relieve the negative effect brought by the intricate wireless network environment and unstable user behavior in layered mobile peer-to peer(P2P) streaming service, an evolved layered P2P (E-LP2P) data scheduling scheme in the process of service delivery is introduced in this paper. The data in base layer is scheduled according to its importance in streaming play to guarantee the basic play of streaming. The data in enhancement layer is scheduled according to the characters of streaming data, including its position and amount in server peer set in a multiple tied way towards the data in enhancement layer. To cope with the layer jitter caused by the fluctuation of bandwidth, jitter prevent mechanism is used to adjust the highest layer dynamically during the process of data scheduling. Simulation results show that the E-LP2P can provide good quality of service(QoS) performance in terms of throughput, layer delivery ratio, server load and useless packet ratio.
基金Supported by Intel Project under Grant No.4507336215Huawei Project under Grant No.YBCB2007025the University of Science and Technology of China Innovation Foundation under Grant No.KD2008053.
文摘WiMAX distributed scheduling can be modeled as two procedures:three-way handshaking procedure and data subframe scheduling procedure.Due to manipulating data transmission directly,data subframe scheduling has a closer relationship with user Quality of Service(QoS) satisfaction,and has more severe impact on network performance,compared with handshaking procedure.A QoS guaranteed Throughput-Efficiency Optimal distributed data subframe Scheduling scheme,named as QoS-TEOS,is proposed.QoS-TEOS achieves QoS guarantee through modeling services into different ranks and assigning them with corresponding priorities.A service with higher priority is scheduled ahead of that with lower priority and offered with high QoS quality.Same kinds of services that request similar QoS quality are classified into one service set.Different service sets are scheduled with different strategies.QoS-TEOS promotes network performance through improving network throughput and efficiency.Theoretical analysis shows that the scheduled data transmission should balance data generation rate from upper layer and transmission rate of physical layer,to avoid network throughput and efficiency declining.Simulation results show that QoS-TEOS works excellently to achieve throughput-efficiency optimization and guarantee a high QoS.