Task Offloading in Edge Computing Using GNNs and DQN

In a network environment composed of different types of computing centers that can be divided into different layers(clod,edge layer,and others),the interconnection between them offers the possibility of peer-to-peer task offloading.For many resource-constrained devices,the computation of many types of tasks is not feasible because they cannot support such computations as they do not have enough available memory and processing capacity.In this scenario,it is worth considering transferring these tasks to resource-rich platforms,such as Edge Data Centers or remote cloud servers.For different reasons,it is more exciting and appropriate to download various tasks to specific download destinations depending on the properties and state of the environment and the nature of the functions.At the same time,establishing an optimal offloading policy,which ensures that all tasks are executed within the required latency and avoids excessive workload on specific computing centers is not easy.This study presents two alternatives to solve the offloading decision paradigm by introducing two well-known algorithms,Graph Neural Networks(GNN)and Deep Q-Network(DQN).It applies the alternatives on a well-known Edge Computing simulator called PureEdgeSimand compares them with the two defaultmethods,Trade-Off and Round Robin.Experiments showed that variants offer a slight improvement in task success rate and workload distribution.In terms of energy efficiency,they provided similar results.Finally,the success rates of different computing centers are tested,and the lack of capacity of remote cloud servers to respond to applications in real-time is demonstrated.These novel ways of finding a download strategy in a local networking environment are unique as they emulate the state and structure of the environment innovatively,considering the quality of its connections and constant updates.The download score defined in this research is a crucial feature for determining the quality of a download path in the GNN training process and has not previously been proposed.Simultaneously,the suitability of Reinforcement Learning(RL)techniques is demonstrated due to the dynamism of the network environment,considering all the key factors that affect the decision to offload a given task,including the actual state of all devices.

Energy-Efficient Traffic Offloading for RSMA-Based Hybrid Satellite Terrestrial Networks with Deep Reinforcement Learning

As the demands of massive connections and vast coverage rapidly grow in the next wireless communication networks, rate splitting multiple access(RSMA) is considered to be the new promising access scheme since it can provide higher efficiency with limited spectrum resources. In this paper, combining spectrum splitting with rate splitting, we propose to allocate resources with traffic offloading in hybrid satellite terrestrial networks. A novel deep reinforcement learning method is adopted to solve this challenging non-convex problem. However, the neverending learning process could prohibit its practical implementation. Therefore, we introduce the switch mechanism to avoid unnecessary learning. Additionally, the QoS constraint in the scheme can rule out unsuccessful transmission. The simulation results validates the energy efficiency performance and the convergence speed of the proposed algorithm.

IoT Task Offloading in Edge Computing Using Non-Cooperative Game Theory for Healthcare Systems

In this paper,we present a comprehensive system model for Industrial Internet of Things(IIoT)networks empowered by Non-Orthogonal Multiple Access(NOMA)and Mobile Edge Computing(MEC)technologies.The network comprises essential components such as base stations,edge servers,and numerous IIoT devices characterized by limited energy and computing capacities.The central challenge addressed is the optimization of resource allocation and task distribution while adhering to stringent queueing delay constraints and minimizing overall energy consumption.The system operates in discrete time slots and employs a quasi-static approach,with a specific focus on the complexities of task partitioning and the management of constrained resources within the IIoT context.This study makes valuable contributions to the field by enhancing the understanding of resourceefficient management and task allocation,particularly relevant in real-time industrial applications.Experimental results indicate that our proposed algorithmsignificantly outperforms existing approaches,reducing queue backlog by 45.32% and 17.25% compared to SMRA and ACRA while achieving a 27.31% and 74.12% improvement in Qn O.Moreover,the algorithmeffectively balances complexity and network performance,as demonstratedwhen reducing the number of devices in each group(Ng)from 200 to 50,resulting in a 97.21% reduction in complexity with only a 7.35% increase in energy consumption.This research offers a practical solution for optimizing IIoT networks in real-time industrial settings.

Policy Network-Based Dual-Agent Deep Reinforcement Learning for Multi-Resource Task Offloading in Multi-Access Edge Cloud Networks

The Multi-access Edge Cloud(MEC) networks extend cloud computing services and capabilities to the edge of the networks. By bringing computation and storage capabilities closer to end-users and connected devices, MEC networks can support a wide range of applications. MEC networks can also leverage various types of resources, including computation resources, network resources, radio resources,and location-based resources, to provide multidimensional resources for intelligent applications in 5/6G.However, tasks generated by users often consist of multiple subtasks that require different types of resources. It is a challenging problem to offload multiresource task requests to the edge cloud aiming at maximizing benefits due to the heterogeneity of resources provided by devices. To address this issue,we mathematically model the task requests with multiple subtasks. Then, the problem of task offloading of multi-resource task requests is proved to be NP-hard. Furthermore, we propose a novel Dual-Agent Deep Reinforcement Learning algorithm with Node First and Link features(NF_L_DA_DRL) based on the policy network, to optimize the benefits generated by offloading multi-resource task requests in MEC networks. Finally, simulation results show that the proposed algorithm can effectively improve the benefit of task offloading with higher resource utilization compared with baseline algorithms.

Online Learning-Based Offloading Decision and Resource Allocation in Mobile Edge Computing-Enabled Satellite-Terrestrial Networks

Mobile edge computing(MEC)-enabled satellite-terrestrial networks(STNs)can provide Internet of Things(IoT)devices with global computing services.Sometimes,the network state information is uncertain or unknown.To deal with this situation,we investigate online learning-based offloading decision and resource allocation in MEC-enabled STNs in this paper.The problem of minimizing the average sum task completion delay of all IoT devices over all time periods is formulated.We decompose this optimization problem into a task offloading decision problem and a computing resource allocation problem.A joint optimization scheme of offloading decision and resource allocation is then proposed,which consists of a task offloading decision algorithm based on the devices cooperation aided upper confidence bound(UCB)algorithm and a computing resource allocation algorithm based on the Lagrange multiplier method.Simulation results validate that the proposed scheme performs better than other baseline schemes.

Outage Analysis of Optimal UAV Cooperation with IRS via Energy Harvesting Enhancement Assisted Computational Offloading

The utilization of mobile edge computing(MEC)for unmanned aerial vehicle(UAV)communication presents a viable solution for achieving high reliability and low latency communication.This study explores the potential of employing intelligent reflective surfaces(IRS)andUAVs as relay nodes to efficiently offload user computing tasks to theMEC server system model.Specifically,the user node accesses the primary user spectrum,while adhering to the constraint of satisfying the primary user peak interference power.Furthermore,the UAV acquires energy without interrupting the primary user’s regular communication by employing two energy harvesting schemes,namely time switching(TS)and power splitting(PS).The selection of the optimal UAV is based on the maximization of the instantaneous signal-to-noise ratio.Subsequently,the analytical expression for the outage probability of the system in Rayleigh channels is derived and analyzed.The study investigates the impact of various system parameters,including the number of UAVs,peak interference power,TS,and PS factors,on the system’s outage performance through simulation.The proposed system is also compared to two conventional benchmark schemes:the optimal UAV link transmission and the IRS link transmission.The simulation results validate the theoretical derivation and demonstrate the superiority of the proposed scheme over the benchmark schemes.

Joint offloading strategy based on quantum particle swarm optimization for MEC-enabled vehicular networks

With the development of the mobile communication technology,a wide variety of envisioned intelligent transportation systems have emerged and put forward more stringent requirements for vehicular communications.Most of computation-intensive and power-hungry applications result in a large amount of energy consumption and computation costs,which bring great challenges to the on-board system.It is necessary to exploit traffic offloading and scheduling in vehicular networks to ensure the Quality of Experience(QoE).In this paper,a joint offloading strategy based on quantum particle swarm optimization for the Mobile Edge Computing(MEC)enabled vehicular networks is presented.To minimize the delay cost and energy consumption,a task execution optimization model is formulated to assign the task to the available service nodes,which includes the service vehicles and the nearby Road Side Units(RSUs).For the task offloading process via Vehicle to Vehicle(V2V)communication,a vehicle selection algorithm is introduced to obtain an optimal offloading decision sequence.Next,an improved quantum particle swarm optimization algorithm for joint offloading is proposed to optimize the task delay and energy consumption.To maintain the diversity of the population,the crossover operator is introduced to exchange information among individuals.Besides,the crossover probability is defined to improve the search ability and convergence speed of the algorithm.Meanwhile,an adaptive shrinkage expansion factor is designed to improve the local search accuracy in the later iterations.Simulation results show that the proposed joint offloading strategy can effectively reduce the system overhead and the task completion delay under different system parameters.

A review of optimization methods for computation offloading in edge computing networks

Handling the massive amount of data generated by Smart Mobile Devices(SMDs)is a challenging computational problem.Edge Computing is an emerging computation paradigm that is employed to conquer this problem.It can bring computation power closer to the end devices to reduce their computation latency and energy consumption.Therefore,this paradigm increases the computational ability of SMDs by collaboration with edge servers.This is achieved by computation offloading from the mobile devices to the edge nodes or servers.However,not all applications benefit from computation offloading,which is only suitable for certain types of tasks.Task properties,SMD capability,wireless channel state,and other factors must be counted when making computation offloading decisions.Hence,optimization methods are important tools in scheduling computation offloading tasks in Edge Computing networks.In this paper,we review six types of optimization methods-they are Lyapunov optimization,convex optimization,heuristic techniques,game theory,machine learning,and others.For each type,we focus on the objective functions,application areas,types of offloading methods,evaluation methods,as well as the time complexity of the proposed algorithms.We discuss a few research problems that are still open.Our purpose for this review is to provide a concise summary that can help new researchers get started with their computation offloading researches for Edge Computing networks.

Task offloading mechanism based on federated reinforcement learning in mobile edge computing

With the arrival of 5G,latency-sensitive applications are becoming increasingly diverse.Mobile Edge Computing(MEC)technology has the characteristics of high bandwidth,low latency and low energy consumption,and has attracted much attention among researchers.To improve the Quality of Service(QoS),this study focuses on computation offloading in MEC.We consider the QoS from the perspective of computational cost,dimensional disaster,user privacy and catastrophic forgetting of new users.The QoS model is established based on the delay and energy consumption and is based on DDQN and a Federated Learning(FL)adaptive task offloading algorithm in MEC.The proposed algorithm combines the QoS model and deep reinforcement learning algorithm to obtain an optimal offloading policy according to the local link and node state information in the channel coherence time to address the problem of time-varying transmission channels and reduce the computing energy consumption and task processing delay.To solve the problems of privacy and catastrophic forgetting,we use FL to make distributed use of multiple users’data to obtain the decision model,protect data privacy and improve the model universality.In the process of FL iteration,the communication delay of individual devices is too large,which affects the overall delay cost.Therefore,we adopt a communication delay optimization algorithm based on the unary outlier detection mechanism to reduce the communication delay of FL.The simulation results indicate that compared with existing schemes,the proposed method significantly reduces the computation cost on a device and improves the QoS when handling complex tasks.

A Review of the Current Task Offloading Algorithms,Strategies and Approach in Edge Computing Systems

Task offloading is an important concept for edge computing and the Internet of Things(IoT)because computationintensive tasksmust beoffloaded tomore resource-powerful remote devices.Taskoffloading has several advantages,including increased battery life,lower latency,and better application performance.A task offloading method determines whether sections of the full application should be run locally or offloaded for execution remotely.The offloading choice problem is influenced by several factors,including application properties,network conditions,hardware features,and mobility,influencing the offloading system’s operational environment.This study provides a thorough examination of current task offloading and resource allocation in edge computing,covering offloading strategies,algorithms,and factors that influence offloading.Full offloading and partial offloading strategies are the two types of offloading strategies.The algorithms for task offloading and resource allocation are then categorized into two parts:machine learning algorithms and non-machine learning algorithms.We examine and elaborate on algorithms like Supervised Learning,Unsupervised Learning,and Reinforcement Learning(RL)under machine learning.Under the non-machine learning algorithm,we elaborate on algorithms like non(convex)optimization,Lyapunov optimization,Game theory,Heuristic Algorithm,Dynamic Voltage Scaling,Gibbs Sampling,and Generalized Benders Decomposition(GBD).Finally,we highlight and discuss some research challenges and issues in edge computing.

Task Offloading and Resource Allocation for Edge-Enabled Mobile Learning

Mobile learning has evolved into a new format of education based on communication and computer technology that is favored by an increasing number of learning users thanks to the development of wireless communication networks,mobile edge computing,artificial intelligence,and mobile devices.However,due to the constrained data processing capacity of mobile devices,efficient and effective interactive mobile learning is a challenge.Therefore,for mobile learning,we propose a"Cloud,Edge and End"fusion system architecture.Through task offloading and resource allocation for edge-enabled mobile learning to reduce the time and energy consumption of user equipment.Then,we present the proposed solutions that uses the minimum cost maximum flow(MCMF)algorithm to deal with the offloading problem and the deep Q network(DQN)algorithm to deal with the resource allocation problem respectively.Finally,the performance evaluation shows that the proposed offloading and resource allocation scheme can improve system performance,save energy,and satisfy the needs of learning users.

Latency-Aware Dynamic Second Offloading Service in SDN-Based Fog Architecture

Task offloading is a key strategy in Fog Computing (FC). Thedefinition of resource-constrained devices no longer applies to sensors andInternet of Things (IoT) embedded system devices alone. Smart and mobileunits can also be viewed as resource-constrained devices if the power, cloudapplications, and data cloud are included in the set of required resources. Ina cloud-fog-based architecture, a task instance running on an end device mayneed to be offloaded to a fog node to complete its execution. However, ina busy network, a second offloading decision is required when the fog nodebecomes overloaded. The possibility of offloading a task, for the second time,to a fog or a cloud node depends to a great extent on task importance, latencyconstraints, and required resources. This paper presents a dynamic service thatdetermines which tasks can endure a second offloading. The task type, latencyconstraints, and amount of required resources are used to select the offloadingdestination node. This study proposes three heuristic offloading algorithms.Each algorithm targets a specific task type. An overloaded fog node can onlyissue one offloading request to execute one of these algorithms accordingto the task offloading priority. Offloading requests are sent to a SoftwareDefined Networking (SDN) controller. The fog node and controller determinethe number of offloaded tasks. Simulation results show that the average timerequired to select offloading nodes was improved by 33% when compared tothe dynamic fog-to-fog offloading algorithm. The distribution of workloadconverges to a uniform distribution when offloading latency-sensitive nonurgenttasks. The lowest offloading priority is assigned to latency-sensitivetasks with hard deadlines. At least 70% of these tasks are offloaded to fognodes that are one to three hops away from the overloaded node.

Joint computation offloading and resource allocation in vehicular edge computing networks

Vehicular Edge Computing(VEC)is a promising technique to accommodate the computation-intensive and delaysensitive tasks through offloading the tasks to the RoadSide-Unit(RSU)equipped with edge computing servers or neighboring vehicles.Nevertheless,the limited computation resources of edge computing servers and the mobility of vehicles make the offloading policy design very challenging.In this context,through considering the potential transmission gains brought by the mobility of vehicles,we propose an efficient computation offloading and resource allocation scheme in VEC networks with two kinds of offloading modes,i.e.,Vehicle to Vehicle(V2V)and Vehicle to RSU(V2R).We define a new cost function for vehicular users by incorporating the vehicles’offloading delay,energy consumption,and expenses with a differentiated pricing strategy,as well as the transmission gain.An optimization problem is formulated to minimize the average cost of all the task vehicles under the latency and computation capacity constraints.A distributed iterative algorithm is proposed by decoupling the problem into two subproblems for the offloading mode selection and the resource allocation.Matching theorybased and Lagrangian-based algorithms are proposed to solve the two subproblems,respectively.Simulation results show the proposed algorithm achieves low complexity and significantly improves the system performance compared with three benchmark schemes.

Task Offloading Optimization for AGVs with Fixed Routes in Industrial IoT Environment

In order to solve the delay requirements of computing intensive tasks in industrial Internet of things,edge computing is moving from theoretical research to practical applications.Edge servers(ESs)have been deployed in factories,and on-site auto guided vehicles(AGVs),besides doing their regular transportation tasks,can partly act as mobile collectors and distributors of computing data and tasks.Since AGVs may offload tasks to the same ES if they have overlapping path segments,resource allocation conflicts are inevitable.In this paper,we study the problem of efficient task offloading from AGVs to ESs,along their fixed trajectories.We propose a multi-AGV task offloading optimization algorithm(MATO),which first uses the weighted polling algorithm to preliminarily allocate tasks for individual AGVs based on load balancing,and then uses the Deep Q-Network(DQN)model to obtain the updated offloading strategy for the AGV group.The simulation results show that,compared with the existing methods,the proposed MATO algorithm can significantly reduce the maximum completion time of tasks and be stable under various parameter settings.

Stochastic programming based multi-arm bandit offloading strategy for internet of things

In order to solve the high latency of traditional cloud computing and the processing capacity limitation of Internet of Things(IoT)users,Multi-access Edge Computing(MEC)migrates computing and storage capabilities from the remote data center to the edge of network,providing users with computation services quickly and directly.In this paper,we investigate the impact of the randomness caused by the movement of the IoT user on decision-making for offloading,where the connection between the IoT user and the MEC servers is uncertain.This uncertainty would be the main obstacle to assign the task accurately.Consequently,if the assigned task cannot match well with the real connection time,a migration(connection time is not enough to process)would be caused.In order to address the impact of this uncertainty,we formulate the offloading decision as an optimization problem considering the transmission,computation and migration.With the help of Stochastic Programming(SP),we use the posteriori recourse to compensate for inaccurate predictions.Meanwhile,in heterogeneous networks,considering multiple candidate MEC servers could be selected simultaneously due to overlapping,we also introduce the Multi-Arm Bandit(MAB)theory for MEC selection.The extensive simulations validate the improvement and effectiveness of the proposed SP-based Multi-arm bandit Method(SMM)for offloading in terms of reward,cost,energy consumption and delay.The results showthat SMMcan achieve about 20%improvement compared with the traditional offloading method that does not consider the randomness,and it also outperforms the existing SP/MAB based method for offloading.

Adaptive delay-energy balanced partial offloading strategy in Mobile Edge Computing networks

Mobile Edge Computing(MEC)-based computation offloading is a promising application paradigm for serving large numbers of users with various delay and energy requirements.In this paper,we propose a flexible MECbased requirement-adaptive partial offloading model to accommodate each user's specific preference regarding delay and energy consumption.To address the dimensional differences between time and energy,we introduce two normalized parameters and then derive the computational overhead of processing tasks.Different from existing works,this paper considers practical variations in the user request patterns,and exploits a flexible partial offloading mode to minimize computation overheads subject to tolerable delay,task workload and power constraints.Since the resulting problem is non-convex,we decouple it into two convex subproblems and present an iterative algorithm to obtain a feasible offloading solution.Numerical experiments show that our proposed scheme achieves a significant improvement in computation overheads compared with existing schemes.

Multi-Agent Deep Reinforcement Learning for Efficient Computation Offloading in Mobile Edge Computing

Mobile-edge computing(MEC)is a promising technology for the fifth-generation(5G)and sixth-generation(6G)architectures,which provides resourceful computing capabilities for Internet of Things(IoT)devices,such as virtual reality,mobile devices,and smart cities.In general,these IoT applications always bring higher energy consumption than traditional applications,which are usually energy-constrained.To provide persistent energy,many references have studied the offloading problem to save energy consumption.However,the dynamic environment dramatically increases the optimization difficulty of the offloading decision.In this paper,we aim to minimize the energy consumption of the entireMECsystemunder the latency constraint by fully considering the dynamic environment.UnderMarkov games,we propose amulti-agent deep reinforcement learning approach based on the bi-level actorcritic learning structure to jointly optimize the offloading decision and resource allocation,which can solve the combinatorial optimization problem using an asymmetric method and compute the Stackelberg equilibrium as a better convergence point than Nash equilibrium in terms of Pareto superiority.Our method can better adapt to a dynamic environment during the data transmission than the single-agent strategy and can effectively tackle the coordination problem in the multi-agent environment.The simulation results show that the proposed method could decrease the total computational overhead by 17.8%compared to the actor-critic-based method and reduce the total computational overhead by 31.3%,36.5%,and 44.7%compared with randomoffloading,all local execution,and all offloading execution,respectively.

Joint multiple resource allocation for offloading cost minimization in IRS-assisted MEC networks with NOMA

Effective resource allocation can exploit the advantage of intelligent reflective surface(IRS)assisted mobile edge computing(MEC)fully.However,it is challenging to balance the limited energy of MTs and the strict delay requirement of their tasks.In this paper,in order to tackle the challenge,we jointly optimize the offloading delay and energy consumption of mobile terminals(MTs)to realize the delay-energy tradeoff in an IRS-assisted MEC network,in which non-orthogonal multiple access(NOMA)and multiantenna are applied to improve spectral efficiency.To achieve the optimal delay-energy tradeoff,an offloading cost minimization model is proposed,in which the edge computing resource allocation,signal detecting vector,uplink transmission power,and IRS phase shift coefficient are needed to be jointly optimized.The optimization of the model is a multi-level fractional problem in complex fields with some coupled high dimension variables.To solve the intractable problem,we decouple the original problem into a computing subproblem and a wireless transmission subproblem based on the uncoupled relationship between different variable types.The computing subproblem is proved convex and the closed-form solution is obtained for the edge computing resource allocation.Further,the wireless transmission subproblem is solved iteratively through decoupling the residual variables.In each iteration,the closed-form solution of residual variables is obtained through different successive convex approximation(SCA)methods.We verify the proposed algorithm can converge to an optimum with polynomial complexity.Simulation results indicate the proposed method achieves average saved costs of 65.64%,11.24%,and 9.49%over three benchmark methods respectively.

Multi-Objective Optimization for NOMA-Based Mobile Edge Computing Offloading by Maximizing System Utility

Mobile Edge Computing(MEC)is a technology for the fifth-generation(5G)wireless communications to enable User Equipment(UE)to offload tasks to servers deployed at the edge of network.However,taking both delay and energy consumption into consideration in the 5G MEC system is usually complex and contradictory.Non-orthogonal multiple access(NOMA)enable more UEs to offload their computing tasks to MEC servers using the same spectrum resources to enhance the spectrum efficiency for 5G,which makes the problem even more complex in the NOMA-MEC system.In this work,a system utility maximization model is present to NOMA-MEC system,and two optimization algorithms based on Newton method and greedy algorithm respectively are proposed to jointly optimize the computing resource allocation,SIC order,transmission time slot allocation,which can easily achieve a better trade-off between the delay and energy consumption.The simulation results prove that the proposed method is effective for NOMA-MEC systems.

Twin delayed deep deterministic policy gradient-based intelligent computation offloading for IoT

In view of the randomness distribution of multiple users in the dynamic large-scale Internet of Things(IoT)scenario,comprehensively formulating available resources for fog nodes in the area and achieving computation services at low cost have become great challenges.As a result,this paper studies an efficient and intelligent computation offloading mechanism with resource allocation.Specifically,an optimization problem is formulated to minimize the total energy consumption of all tasks under the joint optimization of computation offloading decisions,bandwidth resources and transmission power.Meanwhile,a Twin Delayed Deep Deterministic Policy Gradient-based Intelligent Computation Offloading(TD3PG-ICO)algorithm is proposed to solve this optimization problem.By combining the concept of the actor critic algorithm,the proposed algorithm designs two independent critic networks that can avoid the subjective prediction of a single critic network and better guide the policy network to generate the global optimal computation offloading policy.Additionally,this algorithm introduces a continuous variable discretization operation to select the target offloading node with random probability.The available resources of the target node are dynamically allocated to improve the model decision-making effect.Finally,the simulation results show that this proposed algorithm has faster convergence speed and good robustness.It can always approach the greedy algorithm with respect to the lowest total energy consumption.Furthermore,compared with full local and Deep Q-learning Network(DQN)-based computation offloading schemes,the total energy consumption can be reduced by an average of 15.53%and 6.41%,respectively.