A review of optimization methods for computation offloading in edge computing networks

Handling the massive amount of data generated by Smart Mobile Devices(SMDs)is a challenging computational problem.Edge Computing is an emerging computation paradigm that is employed to conquer this problem.It can bring computation power closer to the end devices to reduce their computation latency and energy consumption.Therefore,this paradigm increases the computational ability of SMDs by collaboration with edge servers.This is achieved by computation offloading from the mobile devices to the edge nodes or servers.However,not all applications benefit from computation offloading,which is only suitable for certain types of tasks.Task properties,SMD capability,wireless channel state,and other factors must be counted when making computation offloading decisions.Hence,optimization methods are important tools in scheduling computation offloading tasks in Edge Computing networks.In this paper,we review six types of optimization methods-they are Lyapunov optimization,convex optimization,heuristic techniques,game theory,machine learning,and others.For each type,we focus on the objective functions,application areas,types of offloading methods,evaluation methods,as well as the time complexity of the proposed algorithms.We discuss a few research problems that are still open.Our purpose for this review is to provide a concise summary that can help new researchers get started with their computation offloading researches for Edge Computing networks.

Multi-Agent Deep Reinforcement Learning for Efficient Computation Offloading in Mobile Edge Computing

Mobile-edge computing(MEC)is a promising technology for the fifth-generation(5G)and sixth-generation(6G)architectures,which provides resourceful computing capabilities for Internet of Things(IoT)devices,such as virtual reality,mobile devices,and smart cities.In general,these IoT applications always bring higher energy consumption than traditional applications,which are usually energy-constrained.To provide persistent energy,many references have studied the offloading problem to save energy consumption.However,the dynamic environment dramatically increases the optimization difficulty of the offloading decision.In this paper,we aim to minimize the energy consumption of the entireMECsystemunder the latency constraint by fully considering the dynamic environment.UnderMarkov games,we propose amulti-agent deep reinforcement learning approach based on the bi-level actorcritic learning structure to jointly optimize the offloading decision and resource allocation,which can solve the combinatorial optimization problem using an asymmetric method and compute the Stackelberg equilibrium as a better convergence point than Nash equilibrium in terms of Pareto superiority.Our method can better adapt to a dynamic environment during the data transmission than the single-agent strategy and can effectively tackle the coordination problem in the multi-agent environment.The simulation results show that the proposed method could decrease the total computational overhead by 17.8%compared to the actor-critic-based method and reduce the total computational overhead by 31.3%,36.5%,and 44.7%compared with randomoffloading,all local execution,and all offloading execution,respectively.

Twin delayed deep deterministic policy gradient-based intelligent computation offloading for IoT

In view of the randomness distribution of multiple users in the dynamic large-scale Internet of Things(IoT)scenario,comprehensively formulating available resources for fog nodes in the area and achieving computation services at low cost have become great challenges.As a result,this paper studies an efficient and intelligent computation offloading mechanism with resource allocation.Specifically,an optimization problem is formulated to minimize the total energy consumption of all tasks under the joint optimization of computation offloading decisions,bandwidth resources and transmission power.Meanwhile,a Twin Delayed Deep Deterministic Policy Gradient-based Intelligent Computation Offloading(TD3PG-ICO)algorithm is proposed to solve this optimization problem.By combining the concept of the actor critic algorithm,the proposed algorithm designs two independent critic networks that can avoid the subjective prediction of a single critic network and better guide the policy network to generate the global optimal computation offloading policy.Additionally,this algorithm introduces a continuous variable discretization operation to select the target offloading node with random probability.The available resources of the target node are dynamically allocated to improve the model decision-making effect.Finally,the simulation results show that this proposed algorithm has faster convergence speed and good robustness.It can always approach the greedy algorithm with respect to the lowest total energy consumption.Furthermore,compared with full local and Deep Q-learning Network(DQN)-based computation offloading schemes,the total energy consumption can be reduced by an average of 15.53%and 6.41%,respectively.

Deep Reinforcement Learning-Based Computation Offloading for 5G Vehicle-Aware Multi-Access Edge Computing Network

Multi-access Edge Computing(MEC)is one of the key technologies of the future 5G network.By deploying edge computing centers at the edge of wireless access network,the computation tasks can be offloaded to edge servers rather than the remote cloud server to meet the requirements of 5G low-latency and high-reliability application scenarios.Meanwhile,with the development of IOV(Internet of Vehicles)technology,various delay-sensitive and compute-intensive in-vehicle applications continue to appear.Compared with traditional Internet business,these computation tasks have higher processing priority and lower delay requirements.In this paper,we design a 5G-based vehicle-aware Multi-access Edge Computing network(VAMECN)and propose a joint optimization problem of minimizing total system cost.In view of the problem,a deep reinforcement learningbased joint computation offloading and task migration optimization(JCOTM)algorithm is proposed,considering the influences of multiple factors such as concurrent multiple computation tasks,system computing resources distribution,and network communication bandwidth.And,the mixed integer nonlinear programming problem is described as a Markov Decision Process.Experiments show that our proposed algorithm can effectively reduce task processing delay and equipment energy consumption,optimize computing offloading and resource allocation schemes,and improve system resource utilization,compared with other computing offloading policies.

Deep Q-Learning Based Computation Offloading Strategy for Mobile Edge Computing

To reduce the transmission latency and mitigate the backhaul burden of the centralized cloud-based network services,the mobile edge computing(MEC)has been drawing increased attention from both industry and academia recently.This paper focuses on mobile users’computation offloading problem in wireless cellular networks with mobile edge computing for the purpose of optimizing the computation offloading decision making policy.Since wireless network states and computing requests have stochastic properties and the environment’s dynamics are unknown,we use the modelfree reinforcement learning(RL)framework to formulate and tackle the computation offloading problem.Each mobile user learns through interactions with the environment and the estimate of its performance in the form of value function,then it chooses the overhead-aware optimal computation offloading action(local computing or edge computing)based on its state.The state spaces are high-dimensional in our work and value function is unrealistic to estimate.Consequently,we use deep reinforcement learning algorithm,which combines RL method Q-learning with the deep neural network(DNN)to approximate the value functions for complicated control applications,and the optimal policy will be obtained when the value function reaches convergence.Simulation results showed that the effectiveness of the proposed method in comparison with baseline methods in terms of total overheads of all mobile users.

Joint Resource Allocation and Coordinated Computation Offloading for Fog Radio Access Networks

The cloud radio access network(C-RAN) and the fog computing have been recently proposed to tackle the dramatically increasing traffic demands and to provide better quality of service(QoS) to user equipment(UE).Considering the better computation capability of the cloud RAN(10 times larger than that of the fog RAN) and the lower transmission delay of the fog computing,we propose a joint resource allocation and coordinated computation offloading algorithm for the fog RAN(F-RAN),which takes the advantage of C-RAN and fog computing.Specifically,the F-RAN splits a computation task into the fog computing part and the cloud computing part.Based on the constraints of maximum transmission delay tolerance,fronthaul and backhaul capacity limits,we minimize the energy cost and obtain optimal computational resource allocation for multiple UE,transmission power allocation of each UE and the event splitting factor.Numerical results have been proposed with the comparison of existing methods.

Stochastic Learning for Opportunistic Peer-to-Peer Computation Offloading in IoT Edge Computing

Peer-to-peer computation offloading has been a promising approach that enables resourcelimited Internet of Things(IoT)devices to offload their computation-intensive tasks to idle peer devices in proximity.Different from dedicated servers,the spare computation resources offered by peer devices are random and intermittent,which affects the offloading performance.The mutual interference caused by multiple simultaneous offloading requestors that share the same wireless channel further complicates the offloading decisions.In this work,we investigate the opportunistic peer-to-peer task offloading problem by jointly considering the stochastic task arrivals,dynamic interuser interference,and opportunistic availability of peer devices.Each requestor makes decisions on both local computation frequency and offloading transmission power to minimize its own expected long-term cost on tasks completion,which takes into consideration its energy consumption,task delay,and task loss due to buffer overflow.The dynamic decision process among multiple requestors is formulated as a stochastic game.By constructing the post-decision states,a decentralized online offloading algorithm is proposed,where each requestor as an independent learning agent learns to approach the optimal strategies with its local observations.Simulation results under different system parameter configurations demonstrate the proposed online algorithm achieves a better performance compared with some existing algorithms,especially in the scenarios with large task arrival probability or small helper availability probability.

Efficient Computation Offloading in Mobile Cloud Computing for Video Streaming Over 5G

In this paper,we investigate video quality enhancement using computation offloading to the mobile cloud computing(MCC)environment.Our objective is to reduce the computational complexity required to covert a low-resolution video to high-resolution video while minimizing computation at the mobile client and additional communication costs.To do so,we propose an energy-efficient computation offloading framework for video streaming services in a MCC over the fifth generation(5G)cellular networks.In the proposed framework,the mobile client offloads the computational burden for the video enhancement to the cloud,which renders the side information needed to enhance video without requiring much computation by the client.The cloud detects edges from the upsampled ultra-high-resolution video(UHD)and then compresses and transmits them as side information with the original low-resolution video(e.g.,full HD).Finally,the mobile client decodes the received content and integrates the SI and original content,which produces a high-quality video.In our extensive simulation experiments,we observed that the amount of computation needed to construct a UHD video in the client is 50%-60% lower than that required to decode UHD video compressed by legacy video encoding algorithms.Moreover,the bandwidth required to transmit a full HD video and its side information is around 70% lower than that required for a normal UHD video.The subjective quality of the enhanced UHD is similar to that of the original UHD video even though the client pays lower communication costs with reduced computing power.

Deep Reinforcement Learning Based Joint Partial Computation Offloading and Resource Allocation in Mobility-Aware MEC System

Mobile edge computing(MEC)emerges as a paradigm to free mobile devices(MDs)from increasingly dense computing workloads in 6G networks.The quality of computing experience can be greatly improved by offloading computing tasks from MDs to MEC servers.Renewable energy harvested by energy harvesting equipments(EHQs)is considered as a promising power supply for users to process and offload tasks.In this paper,we apply the uniform mobility model of MDs to derive a more realistic wireless channel model in a multi-user MEC system with batteries as EHQs to harvest and storage energy.We investigate an optimization problem of the weighted sum of delay cost and energy cost of MDs in the MEC system.We propose an effective joint partial computation offloading and resource allocation(CORA)algorithm which is based on deep reinforcement learning(DRL)to obtain the optimal scheduling without prior knowledge of task arrival,renewable energy arrival as well as channel condition.The simulation results verify the efficiency of the proposed algorithm,which undoubtedly minimizes the cost of MDs compared with other benchmarks.

Joint Optimization of Task Caching,Computation Offloading and Resource Allocation for Mobile Edge Computing

Applications with sensitive delay and sizeable data volumes,such as interactive gaming and augmented reality,have become popular in recent years.These applications pose a huge challenge for mobile users with limited resources.Computation offloading is a mainstream technique to reduce execution delay and save energy for mobile users.However,computation offloading requires communication between mobile users and mobile edge computing(MEC) servers.Such a mechanism would difficultly meet users’ demand in some data-hungry and computation-intensive applications because the energy consumption and delay caused by transmissions are considerable expenses for users.Caching task data can effectively reduce the data transmissions when users offload their tasks to the MEC server.The limited caching space at the MEC server calls for judiciously decide which tasks should be cached.Motivated by this,we consider the joint optimization of computation offloading and task caching in a cellular network.In particular,it allows users to proactively cache or offload their tasks at the MEC server.The objective of this paper is to minimize the system cost,which is defined as the weighted sum of task execution delay and energy consumption for all users.Aiming at establishing optimal performance bound for the system design,we formulate an optimization problem by jointly optimizing the task caching,computation offloading,and resource allocation.The problem is a challenging mixed-integer non-linear programming problem and is NP-hard in general.To solve it efficiently,by using convex optimization,Karmarkar ’s algorithm and the proposed fast search algorithm,we obtain an optimal solution of the formulated problem with manageable computational complexity.Extensive simulation results show that in comparison to some representative benchmark methods,the proposed solution can effectively reduce the system cost.

Efficient Computation Offloading of IoT-Based Workflows Using Discrete Teaching Learning-Based Optimization

As the Internet of Things(IoT)and mobile devices have rapidly proliferated,their computationally intensive applications have developed into complex,concurrent IoT-based workflows involving multiple interdependent tasks.By exploiting its low latency and high bandwidth,mobile edge computing(MEC)has emerged to achieve the high-performance computation offloading of these applications to satisfy the quality-of-service requirements of workflows and devices.In this study,we propose an offloading strategy for IoT-based workflows in a high-performance MEC environment.The proposed task-based offloading strategy consists of an optimization problem that includes task dependency,communication costs,workflow constraints,device energy consumption,and the heterogeneous characteristics of the edge environment.In addition,the optimal placement of workflow tasks is optimized using a discrete teaching learning-based optimization(DTLBO)metaheuristic.Extensive experimental evaluations demonstrate that the proposed offloading strategy is effective at minimizing the energy consumption of mobile devices and reducing the execution times of workflows compared to offloading strategies using different metaheuristics,including particle swarm optimization and ant colony optimization.

Distributed Deep Learning for Cooperative Computation Offloading in Low Earth Orbit Satellite Networks

Low earth orbit(LEO) satellite network is an important development trend for future mobile communication systems, which can truly realize the“ubiquitous connection” of the whole world. In this paper, we present a cooperative computation offloading in the LEO satellite network with a three-tier computation architecture by leveraging the vertical cooperation among ground users, LEO satellites, and the cloud server, and the horizontal cooperation between LEO satellites. To improve the quality of service for ground users, we optimize the computation offloading decisions to minimize the total execution delay for ground users subject to the limited battery capacity of ground users and the computation capability of each LEO satellite. However, the formulated problem is a large-scale nonlinear integer programming problem as the number of ground users and LEO satellites increases, which is difficult to solve with general optimization algorithms. To address this challenging problem, we propose a distributed deep learning-based cooperative computation offloading(DDLCCO) algorithm, where multiple parallel deep neural networks(DNNs) are adopted to learn the computation offloading strategy dynamically. Simulation results show that the proposed algorithm can achieve near-optimal performance with low computational complexity compared with other computation offloading strategies.

Knowledge Distillation for Mobile Edge Computation Offloading

Edge computation offloading allows mobile end devices to execute compute-inten?sive tasks on edge servers. End devices can decide whether the tasks are offloaded to edge servers, cloud servers or executed locally according to current network condition and devic?es'profiles in an online manner. In this paper, we propose an edge computation offloading framework based on deep imitation learning (DIL) and knowledge distillation (KD), which assists end devices to quickly make fine-grained decisions to optimize the delay of computa?tion tasks online. We formalize a computation offloading problem into a multi-label classifi?cation problem. Training samples for our DIL model are generated in an offline manner. Af?ter the model is trained, we leverage KD to obtain a lightweight DIL model, by which we fur?ther reduce the model's inference delay. Numerical experiment shows that the offloading de?cisions made by our model not only outperform those made by other related policies in laten?cy metric, but also have the shortest inference delay among all policies.

Outage Analysis of Optimal UAV Cooperation with IRS via Energy Harvesting Enhancement Assisted Computational Offloading

The utilization of mobile edge computing(MEC)for unmanned aerial vehicle(UAV)communication presents a viable solution for achieving high reliability and low latency communication.This study explores the potential of employing intelligent reflective surfaces(IRS)andUAVs as relay nodes to efficiently offload user computing tasks to theMEC server system model.Specifically,the user node accesses the primary user spectrum,while adhering to the constraint of satisfying the primary user peak interference power.Furthermore,the UAV acquires energy without interrupting the primary user’s regular communication by employing two energy harvesting schemes,namely time switching(TS)and power splitting(PS).The selection of the optimal UAV is based on the maximization of the instantaneous signal-to-noise ratio.Subsequently,the analytical expression for the outage probability of the system in Rayleigh channels is derived and analyzed.The study investigates the impact of various system parameters,including the number of UAVs,peak interference power,TS,and PS factors,on the system’s outage performance through simulation.The proposed system is also compared to two conventional benchmark schemes:the optimal UAV link transmission and the IRS link transmission.The simulation results validate the theoretical derivation and demonstrate the superiority of the proposed scheme over the benchmark schemes.

Multiuser Computation Offloading for Long-Term Sequential Tasks in Mobile Edge Computing Environments

Mobile edge computing has shown its potential in serving emerging latency-sensitive mobile applications in ultra-dense 5G networks via offloading computation workloads from the remote cloud data center to the nearby network edge.However,current computation offloading studies in the heterogeneous edge environment face multifaceted challenges:Dependencies among computational tasks,resource competition among multiple users,and diverse long-term objectives.Mobile applications typically consist of several functionalities,and one huge category of the applications can be viewed as a series of sequential tasks.In this study,we first proposed a novel multiuser computation offloading framework for long-term sequential tasks.Then,we presented a comprehensive analysis of the task offloading process in the framework and formally defined the multiuser sequential task offloading problem.Moreover,we decoupled the long-term offloading problem into multiple single time slot offloading problems and proposed a novel adaptive method to solve them.We further showed the substantial performance advantage of our proposed method on the basis of extensive experiments.

User Satisfaction-Aware Edge Computation Offloading in 5G Multi-Scenario

Edge computation offloading has made some progress in the fifth generation mobile network(5G).However,load balancing in edge computation offloading is still a challenging problem.Meanwhile,with the continuous pursuit of low execution latency in 5G multi-scenario,the functional requirements of edge computation offloading are further exacerbated.Given the above challenges,we raise a unique edge computation offloading method in 5G multi-scenario,and consider user satisfaction.The method consists of three functional parts:offloading strategy generation,offloading strategy update,and offloading strategy optimization.First,the offloading strategy is generated by means of a deep neural network(DNN),then update the offloading strategy by updating the DNN parameters.Finally,we optimize the offloading strategy based on changes in user satisfaction.In summary,compared to existing optimization methods,our proposal can achieve performance close to the optimum.Massive simulation results indicate the latency of the execution of our method on the CPU is under 0.1 seconds while improving the average computation rate by about 10%.

Energy-efficient multiuser and multitask computation offloading optimization method

For dynamic application scenarios of Mobile Edge Computing(MEC),an Energy-efficient Multiuser and Multitask Computation Offloading(EMMCO)optimization method is proposed.Under the consideration of multiuser and multitask computation offloading,first,the EMMCO method takes into account the existence of dependencies among different tasks within an implementation,abstracts these dependencies as a Directed Acyclic Graph(DAG),and models the computation offloading problem as a Markov decision process.Subsequently,the task embedding sequence in the DAG is fed to the RNN encoder-decoder neural network with combination of the attention mechanism,the long-term dependencies among different tasks are successfully captured by this scheme.Finally,the Improved Policy Loss Clip-based PPO2(IPLC-PPO2)algorithm is developed,and the RNN encoder-decoder neural network is trained by the developed algorithm.The loss function in the IPLC-PPO2 algorithm is utilized as a preference for the training process,and the neural network parameters are continuously updated to select the optimal offloading scheduling decisions.Simulation results demonstrate that the proposed EMMCO method can achieve lower latency,reduce energy consumption,and obtain a significant improvement in the Quality of Service(QoS)than the compared algorithms under different situations of mobile edge network.

Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems

Mobile Edge Computing(MEC)is one of the most promising techniques for next-generation wireless communication systems.In this paper,we study the problem of dynamic caching,computation offloading,and resource allocation in cache-assisted multi-user MEC systems with stochastic task arrivals.There are multiple computationally intensive tasks in the system,and each Mobile User(MU)needs to execute a task either locally or remotely in one or more MEC servers by offloading the task data.Popular tasks can be cached in MEC servers to avoid duplicates in offloading.The cached contents can be either obtained through user offloading,fetched from a remote cloud,or fetched from another MEC server.The objective is to minimize the long-term average of a cost function,which is defined as a weighted sum of energy consumption,delay,and cache contents’fetching costs.The weighting coefficients associated with the different metrics in the objective function can be adjusted to balance the tradeoff among them.The optimum design is performed with respect to four decision parameters:whether to cache a given task,whether to offload a given uncached task,how much transmission power should be used during offloading,and how much MEC resources to be allocated for executing a task.We propose to solve the problems by developing a dynamic scheduling policy based on Deep Reinforcement Learning(DRL)with the Deep Deterministic Policy Gradient(DDPG)method.A new decentralized DDPG algorithm is developed to obtain the optimum designs for multi-cell MEC systems by leveraging on the cooperations among neighboring MEC servers.Simulation results demonstrate that the proposed algorithm outperforms other existing strategies,such as Deep Q-Network(DQN).

Deep reinforcement learning based computation offloading and resource allocation for low-latency fog radio access networks

Fog Radio Access Networks(F-RANs)have been considered a groundbreaking technique to support the services of Internet of Things by leveraging edge caching and edge computing.However,the current contributions in computation offloading and resource allocation are inefficient;moreover,they merely consider the static communication mode,and the increasing demand for low latency services and high throughput poses tremendous challenges in F-RANs.A joint problem of mode selection,resource allocation,and power allocation is formulated to minimize latency under various constraints.We propose a Deep Reinforcement Learning(DRL)based joint computation offloading and resource allocation scheme that achieves a suboptimal solution in F-RANs.The core idea of the proposal is that the DRL controller intelligently decides whether to process the generated computation task locally at the device level or offload the task to a fog access point or cloud server and allocates an optimal amount of computation and power resources on the basis of the serving tier.Simulation results show that the proposed approach significantly minimizes latency and increases throughput in the system.

Joint offloading strategy based on quantum particle swarm optimization for MEC-enabled vehicular networks

With the development of the mobile communication technology,a wide variety of envisioned intelligent transportation systems have emerged and put forward more stringent requirements for vehicular communications.Most of computation-intensive and power-hungry applications result in a large amount of energy consumption and computation costs,which bring great challenges to the on-board system.It is necessary to exploit traffic offloading and scheduling in vehicular networks to ensure the Quality of Experience(QoE).In this paper,a joint offloading strategy based on quantum particle swarm optimization for the Mobile Edge Computing(MEC)enabled vehicular networks is presented.To minimize the delay cost and energy consumption,a task execution optimization model is formulated to assign the task to the available service nodes,which includes the service vehicles and the nearby Road Side Units(RSUs).For the task offloading process via Vehicle to Vehicle(V2V)communication,a vehicle selection algorithm is introduced to obtain an optimal offloading decision sequence.Next,an improved quantum particle swarm optimization algorithm for joint offloading is proposed to optimize the task delay and energy consumption.To maintain the diversity of the population,the crossover operator is introduced to exchange information among individuals.Besides,the crossover probability is defined to improve the search ability and convergence speed of the algorithm.Meanwhile,an adaptive shrinkage expansion factor is designed to improve the local search accuracy in the later iterations.Simulation results show that the proposed joint offloading strategy can effectively reduce the system overhead and the task completion delay under different system parameters.