Task Offloading and Resource Allocation in NOMA-VEC:A Multi-Agent Deep Graph Reinforcement Learning Algorithm

Vehicular edge computing(VEC)is emerging as a promising solution paradigm to meet the requirements of compute-intensive applications in internet of vehicle(IoV).Non-orthogonal multiple access(NOMA)has advantages in improving spectrum efficiency and dealing with bandwidth scarcity and cost.It is an encouraging progress combining VEC and NOMA.In this paper,we jointly optimize task offloading decision and resource allocation to maximize the service utility of the NOMA-VEC system.To solve the optimization problem,we propose a multiagent deep graph reinforcement learning algorithm.The algorithm extracts the topological features and relationship information between agents from the system state as observations,outputs task offloading decision and resource allocation simultaneously with local policy network,which is updated by a local learner.Simulation results demonstrate that the proposed method achieves a 1.52%∼5.80%improvement compared with the benchmark algorithms in system service utility.

Two-Stage IoT Computational Task Offloading Decision-Making in MEC with Request Holding and Dynamic Eviction

The rapid development of Internet of Things(IoT)technology has led to a significant increase in the computational task load of Terminal Devices(TDs).TDs reduce response latency and energy consumption with the support of task-offloading in Multi-access Edge Computing(MEC).However,existing task-offloading optimization methods typically assume that MEC’s computing resources are unlimited,and there is a lack of research on the optimization of task-offloading when MEC resources are exhausted.In addition,existing solutions only decide whether to accept the offloaded task request based on the single decision result of the current time slot,but lack support for multiple retry in subsequent time slots.It is resulting in TD missing potential offloading opportunities in the future.To fill this gap,we propose a Two-Stage Offloading Decision-making Framework(TSODF)with request holding and dynamic eviction.Long Short-Term Memory(LSTM)-based task-offloading request prediction and MEC resource release estimation are integrated to infer the probability of a request being accepted in the subsequent time slot.The framework learns optimized decision-making experiences continuously to increase the success rate of task offloading based on deep learning technology.Simulation results show that TSODF reduces total TD’s energy consumption and delay for task execution and improves task offloading rate and system resource utilization compared to the benchmark method.

Task Offloading in Edge Computing Using GNNs and DQN

In a network environment composed of different types of computing centers that can be divided into different layers(clod,edge layer,and others),the interconnection between them offers the possibility of peer-to-peer task offloading.For many resource-constrained devices,the computation of many types of tasks is not feasible because they cannot support such computations as they do not have enough available memory and processing capacity.In this scenario,it is worth considering transferring these tasks to resource-rich platforms,such as Edge Data Centers or remote cloud servers.For different reasons,it is more exciting and appropriate to download various tasks to specific download destinations depending on the properties and state of the environment and the nature of the functions.At the same time,establishing an optimal offloading policy,which ensures that all tasks are executed within the required latency and avoids excessive workload on specific computing centers is not easy.This study presents two alternatives to solve the offloading decision paradigm by introducing two well-known algorithms,Graph Neural Networks(GNN)and Deep Q-Network(DQN).It applies the alternatives on a well-known Edge Computing simulator called PureEdgeSimand compares them with the two defaultmethods,Trade-Off and Round Robin.Experiments showed that variants offer a slight improvement in task success rate and workload distribution.In terms of energy efficiency,they provided similar results.Finally,the success rates of different computing centers are tested,and the lack of capacity of remote cloud servers to respond to applications in real-time is demonstrated.These novel ways of finding a download strategy in a local networking environment are unique as they emulate the state and structure of the environment innovatively,considering the quality of its connections and constant updates.The download score defined in this research is a crucial feature for determining the quality of a download path in the GNN training process and has not previously been proposed.Simultaneously,the suitability of Reinforcement Learning(RL)techniques is demonstrated due to the dynamism of the network environment,considering all the key factors that affect the decision to offload a given task,including the actual state of all devices.

Policy Network-Based Dual-Agent Deep Reinforcement Learning for Multi-Resource Task Offloading in Multi-Access Edge Cloud Networks

The Multi-access Edge Cloud(MEC) networks extend cloud computing services and capabilities to the edge of the networks. By bringing computation and storage capabilities closer to end-users and connected devices, MEC networks can support a wide range of applications. MEC networks can also leverage various types of resources, including computation resources, network resources, radio resources,and location-based resources, to provide multidimensional resources for intelligent applications in 5/6G.However, tasks generated by users often consist of multiple subtasks that require different types of resources. It is a challenging problem to offload multiresource task requests to the edge cloud aiming at maximizing benefits due to the heterogeneity of resources provided by devices. To address this issue,we mathematically model the task requests with multiple subtasks. Then, the problem of task offloading of multi-resource task requests is proved to be NP-hard. Furthermore, we propose a novel Dual-Agent Deep Reinforcement Learning algorithm with Node First and Link features(NF_L_DA_DRL) based on the policy network, to optimize the benefits generated by offloading multi-resource task requests in MEC networks. Finally, simulation results show that the proposed algorithm can effectively improve the benefit of task offloading with higher resource utilization compared with baseline algorithms.

Multi-Agent Deep Deterministic Policy Gradien-Based Task Offloading Resource Allocation Joint Offloading

With the advancement of technology and the continuous innovation of applications, low-latency applications such as drones, online games and virtual reality are gradually becoming popular demands in modern society. However, these applications pose a great challenge to the traditional centralized mobile cloud computing paradigm, and it is obvious that the traditional cloud computing model is already struggling to meet such demands. To address the shortcomings of cloud computing, mobile edge computing has emerged. Mobile edge computing provides users with computing and storage resources by offloading computing tasks to servers at the edge of the network. However, most existing work only considers single-objective performance optimization in terms of latency or energy consumption, but not balanced optimization in terms of latency and energy consumption. To reduce task latency and device energy consumption, the problem of joint optimization of computation offloading and resource allocation in multi-cell, multi-user, multi-server MEC environments is investigated. In this paper, a dynamic computation offloading algorithm based on Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is proposed to obtain the optimal policy. The experimental results show that the algorithm proposed in this paper reduces the delay by 5 ms compared to PPO, 1.5 ms compared to DDPG and 10.7 ms compared to DQN, and reduces the energy consumption by 300 compared to PPO, 760 compared to DDPG and 380 compared to DQN. This fully proves that the algorithm proposed in this paper has excellent performance.

Task offloading mechanism based on federated reinforcement learning in mobile edge computing

With the arrival of 5G,latency-sensitive applications are becoming increasingly diverse.Mobile Edge Computing(MEC)technology has the characteristics of high bandwidth,low latency and low energy consumption,and has attracted much attention among researchers.To improve the Quality of Service(QoS),this study focuses on computation offloading in MEC.We consider the QoS from the perspective of computational cost,dimensional disaster,user privacy and catastrophic forgetting of new users.The QoS model is established based on the delay and energy consumption and is based on DDQN and a Federated Learning(FL)adaptive task offloading algorithm in MEC.The proposed algorithm combines the QoS model and deep reinforcement learning algorithm to obtain an optimal offloading policy according to the local link and node state information in the channel coherence time to address the problem of time-varying transmission channels and reduce the computing energy consumption and task processing delay.To solve the problems of privacy and catastrophic forgetting,we use FL to make distributed use of multiple users’data to obtain the decision model,protect data privacy and improve the model universality.In the process of FL iteration,the communication delay of individual devices is too large,which affects the overall delay cost.Therefore,we adopt a communication delay optimization algorithm based on the unary outlier detection mechanism to reduce the communication delay of FL.The simulation results indicate that compared with existing schemes,the proposed method significantly reduces the computation cost on a device and improves the QoS when handling complex tasks.

Optimization Scheme of Trusted Task Offloading in IIoT Scenario Based on DQN

With the development of the Industrial Internet of Things(IIoT),end devices(EDs)are equipped with more functions to capture information.Therefore,a large amount of data is generated at the edge of the network and needs to be processed.However,no matter whether these computing tasks are offloaded to traditional central clusters or mobile edge computing(MEC)devices,the data is short of security and may be changed during transmission.In view of this challenge,this paper proposes a trusted task offloading optimization scheme that can offer low latency and high bandwidth services for IIoT with data security.Blockchain technology is adopted to ensure data consistency.Meanwhile,to reduce the impact of low throughput of blockchain on task offloading performance,we design the processes of consensus and offloading as a Markov decision process(MDP)by defining states,actions,and rewards.Deep reinforcement learning(DRL)algorithm is introduced to dynamically select offloading actions.To accelerate the optimization,we design a novel reward function for the DRL algorithm according to the scale and computational complexity of the task.Experiments demonstrate that compared with methods without optimization,our mechanism performs better when it comes to the number of task offloading and throughput of blockchain.

A Review of the Current Task Offloading Algorithms,Strategies and Approach in Edge Computing Systems

Task offloading is an important concept for edge computing and the Internet of Things(IoT)because computationintensive tasksmust beoffloaded tomore resource-powerful remote devices.Taskoffloading has several advantages,including increased battery life,lower latency,and better application performance.A task offloading method determines whether sections of the full application should be run locally or offloaded for execution remotely.The offloading choice problem is influenced by several factors,including application properties,network conditions,hardware features,and mobility,influencing the offloading system’s operational environment.This study provides a thorough examination of current task offloading and resource allocation in edge computing,covering offloading strategies,algorithms,and factors that influence offloading.Full offloading and partial offloading strategies are the two types of offloading strategies.The algorithms for task offloading and resource allocation are then categorized into two parts:machine learning algorithms and non-machine learning algorithms.We examine and elaborate on algorithms like Supervised Learning,Unsupervised Learning,and Reinforcement Learning(RL)under machine learning.Under the non-machine learning algorithm,we elaborate on algorithms like non(convex)optimization,Lyapunov optimization,Game theory,Heuristic Algorithm,Dynamic Voltage Scaling,Gibbs Sampling,and Generalized Benders Decomposition(GBD).Finally,we highlight and discuss some research challenges and issues in edge computing.

Task Offloading Optimization for AGVs with Fixed Routes in Industrial IoT Environment

In order to solve the delay requirements of computing intensive tasks in industrial Internet of things,edge computing is moving from theoretical research to practical applications.Edge servers(ESs)have been deployed in factories,and on-site auto guided vehicles(AGVs),besides doing their regular transportation tasks,can partly act as mobile collectors and distributors of computing data and tasks.Since AGVs may offload tasks to the same ES if they have overlapping path segments,resource allocation conflicts are inevitable.In this paper,we study the problem of efficient task offloading from AGVs to ESs,along their fixed trajectories.We propose a multi-AGV task offloading optimization algorithm(MATO),which first uses the weighted polling algorithm to preliminarily allocate tasks for individual AGVs based on load balancing,and then uses the Deep Q-Network(DQN)model to obtain the updated offloading strategy for the AGV group.The simulation results show that,compared with the existing methods,the proposed MATO algorithm can significantly reduce the maximum completion time of tasks and be stable under various parameter settings.

A Hybrid Heuristic Service Caching and Task Offloading Method for Mobile Edge Computing

Computing-intensive and latency-sensitive user requests pose significant challenges to traditional cloud computing.In response to these challenges,mobile edge computing(MEC)has emerged as a new paradigm that extends the computational,caching,and communication capabilities of cloud computing.By caching certain services on edge nodes,computational support can be provided for requests that are offloaded to the edges.However,previous studies on task offloading have generally not considered the impact of caching mechanisms and the cache space occupied by services.This oversight can lead to problems,such as high delays in task executions and invalidation of offloading decisions.To optimize task response time and ensure the availability of task offloading decisions,we investigate a task offloading method that considers caching mechanism.First,we incorporate the cache information of MEC into the model of task offloading and reduce the task offloading problem as a mixed integer nonlinear programming(MINLP)problem.Then,we propose an integer particle swarm optimization and improved genetic algorithm(IPSO_IGA)to solve the MINLP.IPSO_IGA exploits the evolutionary framework of particle swarm optimization.And it uses a crossover operator to update the positions of particles and an improved mutation operator to maintain the diversity of particles.Finally,extensive simulation experiments are conducted to evaluate the performance of the proposed algorithm.The experimental results demonstrate that IPSO_IGA can save 20%to 82%of the task completion time,compared with state-of-theart and classical algorithms.Moreover,IPSO_IGA is suitable for scenarios with complex network structures and computing-intensive tasks.

A New Partial Task Offloading Method in a Cooperation Mode under Multi-Constraints for Multi-UE

In Multi-access Edge Computing(MEC),to deal with multiple user equipment(UE)’s task offloading problem of parallel relationships under the multi-constraints,this paper proposes a cooperation partial task offloading method(named CPMM),aiming to reduce UE’s energy and computation consumption,while meeting the task completion delay as much as possible.CPMM first studies the task offloading of single-UE and then considers the task offloading ofmulti-UE based on single-UE task offloading.CPMMuses the critical path algorithmto divide the modules into key and non-key modules.According to some constraints of UE-self when offloading tasks,it gives priority to non-key modules for offloading and uses the evaluation decision method to select some appropriate key modules for offloading.Based on fully considering the competition between multiple UEs for communication resources and MEC service resources,CPMM uses the weighted queuing method to alleviate the competition for communication resources and uses the branch decision algorithm to determine the location of module offloading by BS according to the MEC servers’resources.It achieves its goal by selecting reasonable modules to offload and using the cooperation ofUE,MEC,andCloudCenter to determine the execution location of themodules.Extensive experiments demonstrate that CPMM obtains superior performances in task computation consumption reducing around 6%on average,task completion delay reducing around 5%on average,and better task execution success rate than other similar methods.

A PSO Improved with Imbalanced Mutation and Task Rescheduling for Task Offloading in End-Edge-Cloud Computing

To serve various tasks requested by various end devices with different requirements,end-edge-cloud(E2C)has attracted more and more attention from specialists in both academia and industry,by combining both benefits of edge and cloud computing.But nowadays,E2C still suffers from low service quality and resource efficiency,due to the geographical distribution of edge resources and the high dynamic of network topology and user mobility.To address these issues,this paper focuses on task offloading,which makes decisions that which resources are allocated to tasks for their processing.This paper first formulates the problem into binary non-linear programming and then proposes a particle swarm optimization(PSO)-based algorithm to solve the problem.The proposed algorithm exploits an imbalance mutation operator and a task rescheduling approach to improve the performance of PSO.The proposed algorithm concerns the resource heterogeneity by correlating the probability that a computing node is decided to process a task with its capacity,by the imbalance mutation.The task rescheduling approach improves the acceptance ratio for a task offloading solution,by reassigning rejected tasks to computing nodes with available resources.Extensive simulated experiments are conducted.And the results show that the proposed offloading algorithm has an 8.93%–37.0%higher acceptance ratio than ten of the classical and up-to-date algorithms,and verify the effectiveness of the imbalanced mutation and the task rescheduling.

Adaptive Partial Task Offloading and Virtual Resource Placement in SDN/NFV-Based Network Softwarization

Edge intelligence brings the deployment of applied deep learning(DL)models in edge computing systems to alleviate the core backbone network congestions.The setup of programmable software-defined networking(SDN)control and elastic virtual computing resources within network functions virtualization(NFV)are cooperative for enhancing the applicability of intelligent edge softwarization.To offer advancement for multi-dimensional model task offloading in edge networks with SDN/NFV-based control softwarization,this study proposes a DL mechanism to recommend the optimal edge node selection with primary features of congestion windows,link delays,and allocatable bandwidth capacities.Adaptive partial task offloading policy considered the DL-based recommendation to modify efficient virtual resource placement for minimizing the completion time and termination drop ratio.The optimization problem of resource placement is tackled by a deep reinforcement learning(DRL)-based policy following the Markov decision process(MDP).The agent observes the state spaces and applies value-maximized action of available computation resources and adjustable resource allocation steps.The reward formulation primarily considers taskrequired computing resources and action-applied allocation properties.With defined policies of resource determination,the orchestration procedure is configured within each virtual network function(VNF)descriptor using topology and orchestration specification for cloud applications(TOSCA)by specifying the allocated properties.The simulation for the control rule installation is conducted using Mininet and Ryu SDN controller.Average delay and task delivery/drop ratios are used as the key performance metrics.

Task Offloading Based on Vehicular Edge Computing for Autonomous Platooning

Autonomous platooning technology is regarded as one of the promising technologies for the future and the research is conducted actively.The autonomous platooning task generally requires highly complex computations so it is difficult to process only with the vehicle’s processing units.To solve this problem,there are many studies on task offloading technique which transfers complex tasks to their neighboring vehicles or computation nodes.However,the existing task offloading techniques which mainly use learning-based algorithms are difficult to respond to the real-time changing road environment due to their complexity.They are also challenging to process computation tasks within 100 ms which is the time limit for driving safety.In this paper,we propose a novel offloading scheme that can support autonomous platooning tasks being processed within the limit and ensure driving safety.The proposed scheme can handle computation tasks by considering the communication bandwidth,delay,and amount of computation.We also conduct simulations in the highway environment to evaluate the existing scheme and the proposed scheme.The result shows that our proposed scheme improves the utilization of nearby computing nodes,and the offloading tasks can be processed within the time for driving safety.

Deep reinforcement learning based task offloading in blockchain enabled smart city

With the expansion of cities and emerging complicated application,smart city has become an in-telligent management mechanism.In order to guarantee the information security and quality of service(QoS)of the Internet of Thing(IoT)devices in the smart city,a mobile edge computing(MEC)en-abled blockchain system is considered as the smart city scenario where the offloading process of com-puting tasks is a key aspect infecting the system performance in terms of service profit and latency.The task offloading process is formulated as a Markov decision process(MDP)and the optimal goal is the cumulative profit for the offloading nodes considering task profit and service latency cost,under the restriction of system timeout as well as processing resource.Then,a policy gradient based task of-floading(PG-TO)algorithm is proposed to solve the optimization problem.Finally,the numerical re-sult shows that the proposed PG-TO has better performance than the comparison algorithm,and the system performance as well as QoS is analyzed respectively.The testing result indicates that the pro-posed method has good generalization.

An intelligent task offloading algorithm(iTOA)for UAV edge computing network

Unmanned Aerial Vehicle(UAV)has emerged as a promising technology for the support of human activities,such as target tracking,disaster rescue,and surveillance.However,these tasks require a large computation load of image or video processing,which imposes enormous pressure on the UAV computation platform.To solve this issue,in this work,we propose an intelligent Task Offloading Algorithm(iTOA)for UAV edge computing network.Compared with existing methods,iTOA is able to perceive the network’s environment intelligently to decide the offloading action based on deep Monte Calor Tree Search(MCTS),the core algorithm of Alpha Go.MCTS will simulate the offloading decision trajectories to acquire the best decision by maximizing the reward,such as lowest latency or power consumption.To accelerate the search convergence of MCTS,we also proposed a splitting Deep Neural Network(sDNN)to supply the prior probability for MCTS.The sDNN is trained by a self-supervised learning manager.Here,the training data set is obtained from iTOA itself as its own teacher.Compared with game theory and greedy search-based methods,the proposed iTOA improves service latency performance by 33%and 60%,respectively.

Intelligent Task Offloading and Collaborative Computation over D2D Communication

In this paper,the problem of computation offloading in the edge server is studied in a mobile edge computation(MEC)-enabled cell networks that consists of a base station(BS)integrating edge servers,several terminal devices and collaborators.In the considered networks,we develop an intelligent task offloading and collaborative computation scheme to achieve the optimal computation offloading.First,a distance-based collaborator screening method is proposed to get collaborators within the distance threshold and with high power.Second,based on the Lyapunov stochastic optimization theory,the system stability problem is transformed into a queue stability issue,and the optimal computation offloading is obtained by solving these three sub-problems:task allocation control,task execution control and queue update,respectively.Moreover,rigorous experimental simulation shows that our proposed computation offloading algorithm can achieve the joint optimization among the system efficiency,energy consumption and time delay compared to the mobility-aware and migration-enabled approach,Full BS and Full local.

Investigating and Modelling of Task Offloading Latency in Edge-Cloud Environment

Recently,the number of Internet of Things(IoT)devices connected to the Internet has increased dramatically as well as the data produced by these devices.This would require offloading IoT tasks to release heavy computation and storage to the resource-rich nodes such as Edge Computing and Cloud Computing.However,different service architecture and offloading strategies have a different impact on the service time performance of IoT applications.Therefore,this paper presents an Edge-Cloud system architecture that supports scheduling offloading tasks of IoT applications in order to minimize the enormous amount of transmitting data in the network.Also,it introduces the offloading latency models to investigate the delay of different offloading scenarios/schemes and explores the effect of computational and communication demand on each one.A series of experiments conducted on an EdgeCloudSim show that different offloading decisions within the Edge-Cloud system can lead to various service times due to the computational resources and communications types.Finally,this paper presents a comprehensive review of the current state-of-the-art research on task offloading issues in the Edge-Cloud environment.

Multi-objective optimization for task offloading based on network calculus in fog environments

With the widespread application of wireless communication technology and continuous improvements to Internet of Things(IoT)technology,fog computing architecture composed of edge,fog,and cloud layers have become a research hotspot.This architecture uses Fog Nodes(FNs)close to users to implement certain cloud functions while compensating for cloud disadvantages.However,because of the limited computing and storage capabilities of a single FN,it is necessary to offload tasks to multiple cooperating FNs for task completion.To effectively and quickly realize task offloading,we use network calculus theory to establish an overall performance model for task offloading in a fog computing environment and propose a Globally Optimal Multi-objective Optimization algorithm for Task Offloading(GOMOTO)based on the performance model.The results show that the proposed model and algorithm can effectively reduce the total delay and total energy consumption of the system and improve the network Quality of Service(QoS).

Deep reinforcement learning-based optimization of lightweight task offloading for multi-user mobile edge computing

To improve the quality of computation experience for mobile devices,mobile edge computing(MEC)is a promising paradigm by providing computing capabilities in close proximity within a sliced radio access network,which supports both traditional communication and MEC services.However,this kind of intensive computing problem is a high dimensional NP hard problem,and some machine learning methods do not have a good effect on solving this problem.In this paper,the Markov decision process model is established to find the excellent task offloading scheme,which maximizes the long-term utility performance,so as to make the best offloading decision according to the queue state,energy queue state and channel quality between mobile users and BS.In order to explore the curse of high dimension in state space,a candidate network is proposed based on edge computing optimize offloading(ECOO)algorithm with the application of deep deterministic policy gradient algorithm.Through simulation experiments,it is proved that the ECOO algorithm is superior to some deep reinforcement learning algorithms in terms of energy consumption and time delay.So the ECOO is good at dealing with high dimensional problems.