Policy Network-Based Dual-Agent Deep Reinforcement Learning for Multi-Resource Task Offloading in Multi-Access Edge Cloud Networks

The Multi-access Edge Cloud(MEC) networks extend cloud computing services and capabilities to the edge of the networks. By bringing computation and storage capabilities closer to end-users and connected devices, MEC networks can support a wide range of applications. MEC networks can also leverage various types of resources, including computation resources, network resources, radio resources,and location-based resources, to provide multidimensional resources for intelligent applications in 5/6G.However, tasks generated by users often consist of multiple subtasks that require different types of resources. It is a challenging problem to offload multiresource task requests to the edge cloud aiming at maximizing benefits due to the heterogeneity of resources provided by devices. To address this issue,we mathematically model the task requests with multiple subtasks. Then, the problem of task offloading of multi-resource task requests is proved to be NP-hard. Furthermore, we propose a novel Dual-Agent Deep Reinforcement Learning algorithm with Node First and Link features(NF_L_DA_DRL) based on the policy network, to optimize the benefits generated by offloading multi-resource task requests in MEC networks. Finally, simulation results show that the proposed algorithm can effectively improve the benefit of task offloading with higher resource utilization compared with baseline algorithms.

Combining neural network-based method with heuristic policy for optimal task scheduling in hierarchical edge cloud

Deploying service nodes hierarchically at the edge of the network can effectively improve the service quality of offloaded task requests and increase the utilization of resources.In this paper,we study the task scheduling problem in the hierarchically deployed edge cloud.We first formulate the minimization of the service time of scheduled tasks in edge cloud as a combinatorial optimization problem,blue and then prove the NP-hardness of the problem.Different from the existing work that mostly designs heuristic approximation-based algorithms or policies to make scheduling decision,we propose a newly designed scheduling policy,named Joint Neural Network and Heuristic Scheduling(JNNHSP),which combines a neural network-based method with a heuristic based solution.JNNHSP takes the Sequence-to-Sequence(Seq2Seq)model trained by Reinforcement Learning(RL)as the primary policy and adopts the heuristic algorithm as the auxiliary policy to obtain the scheduling solution,thereby achieving a good balance between the quality and the efficiency of the scheduling solution.In-depth experiments show that compared with a variety of related policies and optimization solvers,JNNHSP can achieve better performance in terms of scheduling error ratio,the degree to which the policy is affected by re-sources limitations,average service latency,and execution efficiency in a typical hierarchical edge cloud.

Edge Cloud Selection in Mobile Edge Computing(MEC)-Aided Applications for Industrial Internet of Things(IIoT)Services

In many IIoT architectures,various devices connect to the edge cloud via gateway systems.For data processing,numerous data are delivered to the edge cloud.Delivering data to an appropriate edge cloud is critical to improve IIoT service efficiency.There are two types of costs for this kind of IoT network:a communication cost and a computing cost.For service efficiency,the communication cost of data transmission should be minimized,and the computing cost in the edge cloud should be also minimized.Therefore,in this paper,the communication cost for data transmission is defined as the delay factor,and the computing cost in the edge cloud is defined as the waiting time of the computing intensity.The proposed method selects an edge cloud that minimizes the total cost of the communication and computing costs.That is,a device chooses a routing path to the selected edge cloud based on the costs.The proposed method controls the data flows in a mesh-structured network and appropriately distributes the data processing load.The performance of the proposed method is validated through extensive computer simulation.When the transition probability from good to bad is 0.3 and the transition probability from bad to good is 0.7 in wireless and edge cloud states,the proposed method reduced both the average delay and the service pause counts to about 25%of the existing method.

Super Resolution Sensing Technique for Distributed Resource Monitoring on Edge Clouds

With the vigorous development of mobile networks,the number of devices at the network edge is growing rapidly and the massive amount of data generated by the devices brings a huge challenge of response latency and communication burden.Existing resource monitoring systems are widely deployed in cloud data centers,but it is difficult for traditional resource monitoring solutions to handle the massive data generated by thousands of edge devices.To address these challenges,we propose a super resolution sensing(SRS)method for distributed resource monitoring,which can be used to recover reliable and accurate high‑frequency data from low‑frequency sampled resource monitoring data.Experiments based on the proposed SRS model are also conducted and the experimental results show that it can effectively reduce the errors generated when recovering low‑frequency monitoring data to high‑frequency data,and verify the effectiveness and practical value of applying SRS method for resource monitoring on edge clouds.

Correlation-Aware Replica Prefetching Strategy to Decrease Access Latency in Edge Cloud

With the number of connected devices increasing rapidly,the access latency issue increases drastically in the edge cloud environment.Massive low time-constrained and data-intensive mobile applications require efficient replication strategies to decrease retrieval time.However,the determination of replicas is not reasonable in many previous works,which incurs high response delay.To this end,a correlation-aware replica prefetching(CRP)strategy based on the file correlation principle is proposed,which can prefetch the files with high access probability.The key is to determine and obtain the implicit high-value files effectively,which has a significant impact on the performance of CRP.To achieve the goal of accelerating the acquisition of implicit highvalue files,an access rule management method based on consistent hashing is proposed,and then the storage and query mechanisms for access rules based on adjacency list storage structure are further presented.The theoretical analysis and simulation results corroborate that CRP shortens average response time over 4.8%,improves average hit ratio over 4.2%,reduces transmitting data amount over 8.3%,and maintains replication frequency at a reasonable level when compared to other schemes.

Cloud control for IIoT in a cloud-edge environment

The industrial Internet of Things(IIoT)is a new indus-trial idea that combines the latest information and communica-tion technologies with the industrial economy.In this paper,a cloud control structure is designed for IIoT in cloud-edge envi-ronment with three modes of 5G.For 5G based IIoT,the time sensitive network(TSN)service is introduced in transmission network.A 5G logical TSN bridge is designed to transport TSN streams over 5G framework to achieve end-to-end configuration.For a transmission control protocol(TCP)model with nonlinear disturbance,time delay and uncertainties,a robust adaptive fuzzy sliding mode controller(AFSMC)is given with control rule parameters.IIoT workflows are made up of a series of subtasks that are linked by the dependencies between sensor datasets and task flows.IIoT workflow scheduling is a non-deterministic polynomial(NP)-hard problem in cloud-edge environment.An adaptive and non-local-convergent particle swarm optimization(ANCPSO)is designed with nonlinear inertia weight to avoid falling into local optimum,which can reduce the makespan and cost dramatically.Simulation and experiments demonstrate that ANCPSO has better performances than other classical algo-rithms.

A PSO Improved with Imbalanced Mutation and Task Rescheduling for Task Offloading in End-Edge-Cloud Computing

To serve various tasks requested by various end devices with different requirements,end-edge-cloud(E2C)has attracted more and more attention from specialists in both academia and industry,by combining both benefits of edge and cloud computing.But nowadays,E2C still suffers from low service quality and resource efficiency,due to the geographical distribution of edge resources and the high dynamic of network topology and user mobility.To address these issues,this paper focuses on task offloading,which makes decisions that which resources are allocated to tasks for their processing.This paper first formulates the problem into binary non-linear programming and then proposes a particle swarm optimization(PSO)-based algorithm to solve the problem.The proposed algorithm exploits an imbalance mutation operator and a task rescheduling approach to improve the performance of PSO.The proposed algorithm concerns the resource heterogeneity by correlating the probability that a computing node is decided to process a task with its capacity,by the imbalance mutation.The task rescheduling approach improves the acceptance ratio for a task offloading solution,by reassigning rejected tasks to computing nodes with available resources.Extensive simulated experiments are conducted.And the results show that the proposed offloading algorithm has an 8.93%–37.0%higher acceptance ratio than ten of the classical and up-to-date algorithms,and verify the effectiveness of the imbalanced mutation and the task rescheduling.

Connected Vehicles Computation Task Offloading Based on Opportunism in Cooperative Edge Computing

The traditional multi-access edge computing (MEC) capacity isoverwhelmed by the increasing demand for vehicles, leading to acute degradationin task offloading performance. There is a tremendous number ofresource-rich and idle mobile connected vehicles (CVs) in the traffic network,and vehicles are created as opportunistic ad-hoc edge clouds to alleviatethe resource limitation of MEC by providing opportunistic computing services.On this basis, a novel scalable system framework is proposed in thispaper for computation task offloading in opportunistic CV-assisted MEC.In this framework, opportunistic ad-hoc edge cloud and fixed edge cloudcooperate to form a novel hybrid cloud. Meanwhile, offloading decision andresource allocation of the user CVs must be ascertained. Furthermore, thejoint offloading decision and resource allocation problem is described asa Mixed Integer Nonlinear Programming (MINLP) problem, which optimizesthe task response latency of user CVs under various constraints. Theoriginal problem is decomposed into two subproblems. First, the Lagrangedual method is used to acquire the best resource allocation with the fixedoffloading decision. Then, the satisfaction-driven method based on trial anderror (TE) learning is adopted to optimize the offloading decision. Finally, acomprehensive series of experiments are conducted to demonstrate that oursuggested scheme is more effective than other comparison schemes.

Cloud-Assisted Distributed Edge Brains for Multi-Cell Joint Beamforming Optimization for 6G

In 5G networks,optimization of antenna beam weights of base stations has become the key application of AI for network optimization.For 6G,higher frequency bands and much denser cells are expected,and the importance of automatic and accurate beamforming assisted by AI will become more prominent.In existing network,servers are“patched”to network equipment to act as a centralized brain for model training and inference leading to high transmission overhead,large inference latency and potential risks of data security.Decentralized architectures have been proposed to achieve flexible parameter configuration and fast local response,but it is inefficient in collecting and sharing global information among base stations.In this paper,we propose a novel solution based on a collaborative cloud edge architecture for multi-cell joint beamforming optimization.We analyze the performance and costs of the proposed solution with two other architectural solutions by simulation.Compared with the centralized solution,our solution improves prediction accuracy by 24.66%,and reduces storage cost by 83.82%.Compared with the decentralized solution,our solution improves prediction accuracy by 68.26%,and improves coverage performance by 0.4 dB.At last,the future research work is prospected.

Intelligent Task Offloading and Collaborative Computation in Multi-UAV-Enabled Mobile Edge Computing

This article establishes a three-tier mobile edge computing(MEC) network, which takes into account the cooperation between unmanned aerial vehicles(UAVs). In this MEC network, we aim to minimize the processing delay of tasks by jointly optimizing the deployment of UAVs and offloading decisions,while meeting the computing capacity constraint of UAVs. However, the resulting optimization problem is nonconvex, which cannot be solved by general optimization tools in an effective and efficient way. To this end, we propose a two-layer optimization algorithm to tackle the non-convexity of the problem by capitalizing on alternating optimization. In the upper level algorithm, we rely on differential evolution(DE) learning algorithm to solve the deployment of the UAVs. In the lower level algorithm, we exploit distributed deep neural network(DDNN) to generate offloading decisions. Numerical results demonstrate that the two-layer optimization algorithm can effectively obtain the near-optimal deployment of UAVs and offloading strategy with low complexity.

Enhancing resource allocation in edge and fog-cloud computing with genetic algorithm and particle swarm optimization

Evolutionary algorithms have gained significant attention from researchers as effective solutions for various optimization problems.Genetic Algorithm(GA)is widely popular due to its logical approach,broad applicability,and ability to tackle complex issues encountered in engineering systems.However,GA is known for its high implementation cost and typically requires a large number of iterations.On the other hand,Particle Swarm Optimization(PSO)is a relatively new heuristic technique inspired by the collective behaviors of real organisms.Both GA and PSO algorithms are prominent heuristic optimization methods that belong to the population-based approaches family.While they are often seen as competitors,their efficiency heavily relies on the parameter values chosen and the specific optimization problem at hand.In this study,we aim to compare the runtime performance of GA and PSO algorithms within a cutting-edge edge and fog cloud architecture.Through extensive experiments and performance evaluations,the authors demonstrate the effectiveness of GA and PSO algorithms in improving resource allocation in edge and fog cloud computing scenarios using FogWorkflowSim simulator.The comparative analysis sheds light on the strengths and limitations of each algorithm,providing valuable insights for researchers and practitioners in the field.

A Reverse Auction Mechanism for Time-Varying Multidimensional Resource Allocation in Vehicular Fog Computing with Cloud and Edge Collaboration

It is a hot issue to allocate resources using auction mechanisms in vehicular fog computing(VFC)with cloud and edge collaboration.However,most current research faces the limitation of only considering single type resource allocation,which cannot satisfy the resource requirements of users.In addition,the resource requirements of users are satisfied with a fixed amount of resources during the usage time,which may result in high cost of users and even cause a waste of resources.In fact,the actual resource requirements of users may change with time.Besides,existing allocation algorithms in the VFC of cloud and edge collaboration cannot be directly applied to time-varying multidimensional resource allocation.Therefore,in order to minimize the cost of users,we propose a reverse auction mechanism for the time-varying multidimensional resource allocation problem(TMRAP)in VFC with cloud and edge collaboration based on VFC parking assistance and transform the resource allocation problem into an integer programming(IP)model.And we also design a heuristic resource allocation algorithm to approximate the solution of the model.We apply a dominant-resource-based strategy for resource allocation to improve resource utilization and obtain the lowest cost of users for resource pricing.Furthermore,we prove that the algorithm satisfies individual rationality and truthfulness,and can minimize the cost of users and improve resource utilization through comparison with other similar methods.Above all,we combine VFC smart parking assistance with reverse auction mechanisms to encourage resource providers to offer resources,so that more vehicle users can obtain services at lower prices and relieve traffic pressure.

Multi-stage online task assignment driven by offline data under spatio-temporal crowdsourcing

In the era of the Internet of Things(IoT),the crowdsourcing process is driven by data collected by devices that interact with each other and with the physical world.As a part of the IoT ecosystem,task assignment has become an important goal of the research community.Existing task assignment algorithms can be categorized as offline(performs better with datasets but struggles to achieve good real-life results)or online(works well with real-life input but is difficult to optimize regarding in-depth assignments).This paper proposes a Cross-regional Online Task(CROT)assignment problem based on the online assignment model.Given the CROT problem,an Online Task Assignment across Regions based on Prediction(OTARP)algorithm is proposed.OTARP is a two-stage graphics-driven bilateral assignment strategy that uses edge cloud and graph embedding to complete task assignments.The first stage uses historical data to make offline predictions,with a graph-driven method for offline bipartite graph matching.The second stage uses a bipartite graph to complete the online task assignment process.This paper proposes accelerating the task assignment process through multiple assignment rounds and optimizing the process by combining offline guidance and online assignment strategies.To encourage crowd workers to complete crowd tasks across regions,an incentive strategy is designed to encourage crowd workers’movement.To avoid the idle problem in the process of crowd worker movement,a drop-by-rider problem is used to help crowd workers accept more crowd tasks,optimize the number of assignments,and increase utility.Finally,through comparison experiments on real datasets,the performance of the proposed algorithm on crowd worker utility value and the matching number is evaluated.

Mobility Management in Small Cell Cluster of Cellular Network

The installation of small cells in a 5G network extends the maximum coverage and provides high availability.However,this approach increases the handover overhead in the Core Network(CN)due to frequent handoffs.The variation of user density and movement inside a region of small cells also increases the handover overhead in CN.However,the present 5G system cannot reduce the handover overhead in CN under such circumstances because it relies on a traditionally rigid and complex hierarchical sequence for a handover procedure.Recently,Not Only Stack(NO Stack)architecture has been introduced for Radio Access Network(RAN)to reduce the signaling during handover.This paper proposes a system based on NO Stack architecture and solves the aforementioned problem by adding a dedicated local mobility controller to the edge cloud for each cluster.The dedicated cluster controller manages the user mobility locally inside a cluster and also maintains the forwarding data of a mobile user locally.To reduce the latency for X2-based handover requests,an edge cloud infrastructure has been also developed to provide high-computing for dedicated controllers at the edge of a cellular network.The proposed system is also compared with the traditional 3GPP architecture and other works in the context of overhead and delay caused by X2-based handover requests during user mobility.Simulated results show that the inclusion of a dedicated local controller for small clusters together with the implementation of NO Stack framework reduces the significant amount of overhead of X2-based handover requests at CN.

Efficient and Cost-Effective Vehicle Detection in Foggy Weather for Edge/Fog-Enabled Traffic Surveillance and Collision Avoidance Systems

Vision-based vehicle detection in adverse weather conditions such as fog,haze,and mist is a challenging research area in the fields of autonomous vehicles,collision avoidance,and Internet of Things(IoT)-enabled edge/fog computing traffic surveillance and monitoring systems.Efficient and cost-effective vehicle detection at high accuracy and speed in foggy weather is essential to avoiding road traffic collisions in real-time.To evaluate vision-based vehicle detection performance in foggy weather conditions,state-of-the-art Vehicle Detection in Adverse Weather Nature(DAWN)and Foggy Driving(FD)datasets are self-annotated using the YOLO LABEL tool and customized to four vehicle detection classes:cars,buses,motorcycles,and trucks.The state-of-the-art single-stage deep learning algorithms YOLO-V5,and YOLO-V8 are considered for the task of vehicle detection.Furthermore,YOLO-V5s is enhanced by introducing attention modules Convolutional Block Attention Module(CBAM),Normalized-based Attention Module(NAM),and Simple Attention Module(SimAM)after the SPPF module as well as YOLO-V5l with BiFPN.Their vehicle detection accuracy parameters and running speed is validated on cloud(Google Colab)and edge(local)systems.The mAP50 score of YOLO-V5n is 72.60%,YOLOV5s is 75.20%,YOLO-V5m is 73.40%,and YOLO-V5l is 77.30%;and YOLO-V8n is 60.20%,YOLO-V8s is 73.50%,YOLO-V8m is 73.80%,and YOLO-V8l is 72.60%on DAWN dataset.The mAP50 score of YOLO-V5n is 43.90%,YOLO-V5s is 40.10%,YOLO-V5m is 49.70%,and YOLO-V5l is 57.30%;and YOLO-V8n is 41.60%,YOLO-V8s is 46.90%,YOLO-V8m is 42.90%,and YOLO-V8l is 44.80%on FD dataset.The vehicle detection speed of YOLOV5n is 59 Frame Per Seconds(FPS),YOLO-V5s is 47 FPS,YOLO-V5m is 38 FPS,and YOLO-V5l is 30 FPS;and YOLO-V8n is 185 FPS,YOLO-V8s is 109 FPS,YOLO-V8m is 72 FPS,and YOLO-V8l is 63 FPS on DAWN dataset.The vehicle detection speed of YOLO-V5n is 26 FPS,YOLO-V5s is 24 FPS,YOLO-V5m is 22 FPS,and YOLO-V5l is 17 FPS;and YOLO-V8n is 313 FPS,YOLO-V8s is 182 FPS,YOLO-V8m is 99 FPS,and YOLO-V8l is 60 FPS on FD dataset.YOLO-V5s,YOLO-V5s variants and YOLO-V5l_BiFPN,and YOLO-V8 algorithms are efficient and cost-effective solution for real-time vision-based vehicle detection in foggy weather.

Clustered Single-Board Devices with Docker Container Big Stream Processing Architecture

The expanding amounts of information created by Internet of Things(IoT)devices places a strain on cloud computing,which is often used for data analysis and storage.This paper investigates a different approach based on edge cloud applications,which involves data filtering and processing before being delivered to a backup cloud environment.This Paper suggest designing and implementing a low cost,low power cluster of Single Board Computers(SBC)for this purpose,reducing the amount of data that must be transmitted elsewhere,using Big Data ideas and technology.An Apache Hadoop and Spark Cluster that was used to run a test application was containerized and deployed using a Raspberry Pi cluster and Docker.To obtain system data and analyze the setup’s performance a Prometheusbased stack monitoring and alerting solution in the cloud based market is employed.This Paper assesses the system’s complexity and demonstrates how containerization can improve fault tolerance and maintenance ease,allowing the suggested solution to be used in industry.An evaluation of the overall performance is presented to highlight the capabilities and limitations of the suggested architecture,taking into consideration the suggested solution’s resource use in respect to device restrictions.

A Multi-Layer Collaboration Framework for Industrial Parks with 5G Vehicle-to-Everything Networks

The fifth-generation(5G)wireless communication networks are expected to play an essential role in the transformation of vertical industries.Among many exciting applications to be enabled by 5G,logistics tasks in industry parks can be performed more efficiently via vehicle-to-everything(V2X)communications.In this paper,a multi-layer collaboration framework enabled by V2X is proposed for logistics management in industrial parks.The proposed framework includes three layers:a perception and execution layer,a logistics layer,and a configuration layer.In addition to the collaboration among these three layers,this study addresses the collaboration among devices,edge servers,and cloud services.For effective logistics in industrial parks,task collaboration is achieved through four functions:environmental perception and map construction,task allocation,path planning,and vehicle movement.To dynamically coordinate these functions,device–edge–cloud collaboration,which is supported by 5G slices and V2X communication technology,is applied.Then,the analytical target cascading method is adopted to configure and evaluate the collaboration schemes of industrial parks.Finally,a logistics analytical case study in industrial parks is employed to demonstrate the feasibility of the proposed collaboration framework.

COLLABORATIVE RESOURCE ALLOCATION OVER A HYBRID CLOUD CENTER AND EDGE SERVER NETWORK

This paper considers the collaborative resource allocation problem over a hybrid cloud center and edge server network, an emerging infrastructure for efficient Internet services. The cloud center acts as a pool of inexhaustible computation and storage powers. The edge servers often have limited computation and storage powers but are able to provide quick responses to service requests from end users. Upon receiving service requests, edge servers assign them to themselves, their neighboring edge servers, as well as the cloud center, aiming at minimizing the overall network cost. This paper first establishes an optimization model for this problem. Second, in light of the separable structure of the optimization model, we utilize the alternating direction method of multipliers （ADMM） to develop a fully collaborative resource allocation algorithm. The edge servers and the cloud center autonomously collaborate to compute their local optimization variables and prices of network resources, and reach an optimal solution. Numerical experiments demonstrate the effectiveness of the hybrid network infrastructure as well as the proposed algorithm.

Industry-Oriented Cloud Edge Intelligent Assembly Guidance System

In view of the low efficiency of production and assembly in traditional industries,we use AR technology to replace traditional assembly instructions and design an industry-oriented cloud-edge intelligent assembly guidance system.Since the computing power ofARglasses cannot meet the high complexity requirements of scene understanding,we adopt the joint solution of Cloud-Edge.First,the sensor data collected by the AR glasses are streamed to the edge server using high-speed and low-latency wireless interconnection technology.Then,the product artifacts in the data scene are identified and understood through the instance segmentation network BlendMask based on deep learning.Then,the 3D pose of the object is calculated in real time by combining pose estimation and 3D reconstruction.Furthermore,an accurate 3D guidance animation is generated,and the virtual 3D model in the AR glasses is accurately superimposed on the real object to determine whether the assembly is correct in real time.Experiments show that the system effectively combines artificial intelligence and intelligent manufacturing,integrates various elements in the scene in real time to provide operators with multimodal and multidimensional immersive guidance,and corrects in time when assembly errors occur.It can not only quickly guide the operator to complete the learning of the assembly process but also assist the staff in the assembly in real time.Ultimately,it improves assembly speed and accuracy,which in turn improves enterprise productivity.

Data management,communication systems and the edge:Challenges for the future of transportation

Edge Clouds.Edge cloud systems have emerged as a new promising alternative to address the needs of many emerging latency-critical applications such as autonomous vehicles.These applications are ill-suited for traditional clouds due to the end-to-end latency and the limited bandwidth between the cloud's(few)data centers and these applications.In the edge cloud model,a myriad of small-scale computing clusters are brought next to the applications at the edge of the network.Many Applications in transportation engineering such as autonomous vehicle collision avoidance,and fleet management,requiring more global decision making for safety and correctness of operation,can thus make use of edge cloud systems,offloading their computations to these edge clusters.The edge can then analyze data from these vehicles to optimize the traffic flow,reduce accidents,and provide transportation systems with more autonomy.The idea of edge computing today forms a cornerstone in the design of many future systems,including,5G networks and autonomous vehicles,among many others(see Fig.1).