Policy Network-Based Dual-Agent Deep Reinforcement Learning for Multi-Resource Task Offloading in Multi-Access Edge Cloud Networks

The Multi-access Edge Cloud(MEC) networks extend cloud computing services and capabilities to the edge of the networks. By bringing computation and storage capabilities closer to end-users and connected devices, MEC networks can support a wide range of applications. MEC networks can also leverage various types of resources, including computation resources, network resources, radio resources,and location-based resources, to provide multidimensional resources for intelligent applications in 5/6G.However, tasks generated by users often consist of multiple subtasks that require different types of resources. It is a challenging problem to offload multiresource task requests to the edge cloud aiming at maximizing benefits due to the heterogeneity of resources provided by devices. To address this issue,we mathematically model the task requests with multiple subtasks. Then, the problem of task offloading of multi-resource task requests is proved to be NP-hard. Furthermore, we propose a novel Dual-Agent Deep Reinforcement Learning algorithm with Node First and Link features(NF_L_DA_DRL) based on the policy network, to optimize the benefits generated by offloading multi-resource task requests in MEC networks. Finally, simulation results show that the proposed algorithm can effectively improve the benefit of task offloading with higher resource utilization compared with baseline algorithms.

DQN-Based Proactive Trajectory Planning of UAVs in Multi-Access Edge Computing

The main aim of future mobile networks is to provide secure,reliable,intelligent,and seamless connectivity.It also enables mobile network operators to ensure their customer’s a better quality of service(QoS).Nowadays,Unmanned Aerial Vehicles(UAVs)are a significant part of the mobile network due to their continuously growing use in various applications.For better coverage,cost-effective,and seamless service connectivity and provisioning,UAVs have emerged as the best choice for telco operators.UAVs can be used as flying base stations,edge servers,and relay nodes in mobile networks.On the other side,Multi-access EdgeComputing(MEC)technology also emerged in the 5G network to provide a better quality of experience(QoE)to users with different QoS requirements.However,UAVs in a mobile network for coverage enhancement and better QoS face several challenges such as trajectory designing,path planning,optimization,QoS assurance,mobilitymanagement,etc.The efficient and proactive path planning and optimization in a highly dynamic environment containing buildings and obstacles are challenging.So,an automated Artificial Intelligence(AI)enabled QoSaware solution is needed for trajectory planning and optimization.Therefore,this work introduces a well-designed AI and MEC-enabled architecture for a UAVs-assisted future network.It has an efficient Deep Reinforcement Learning(DRL)algorithm for real-time and proactive trajectory planning and optimization.It also fulfills QoS-aware service provisioning.A greedypolicy approach is used to maximize the long-term reward for serving more users withQoS.Simulation results reveal the superiority of the proposed DRL mechanism for energy-efficient and QoS-aware trajectory planning over the existing models.

Deep Reinforcement Learning-Based Computation Offloading for 5G Vehicle-Aware Multi-Access Edge Computing Network

Multi-access Edge Computing(MEC)is one of the key technologies of the future 5G network.By deploying edge computing centers at the edge of wireless access network,the computation tasks can be offloaded to edge servers rather than the remote cloud server to meet the requirements of 5G low-latency and high-reliability application scenarios.Meanwhile,with the development of IOV(Internet of Vehicles)technology,various delay-sensitive and compute-intensive in-vehicle applications continue to appear.Compared with traditional Internet business,these computation tasks have higher processing priority and lower delay requirements.In this paper,we design a 5G-based vehicle-aware Multi-access Edge Computing network(VAMECN)and propose a joint optimization problem of minimizing total system cost.In view of the problem,a deep reinforcement learningbased joint computation offloading and task migration optimization(JCOTM)algorithm is proposed,considering the influences of multiple factors such as concurrent multiple computation tasks,system computing resources distribution,and network communication bandwidth.And,the mixed integer nonlinear programming problem is described as a Markov Decision Process.Experiments show that our proposed algorithm can effectively reduce task processing delay and equipment energy consumption,optimize computing offloading and resource allocation schemes,and improve system resource utilization,compared with other computing offloading policies.

Connected Vehicles Computation Task Offloading Based on Opportunism in Cooperative Edge Computing

The traditional multi-access edge computing (MEC) capacity isoverwhelmed by the increasing demand for vehicles, leading to acute degradationin task offloading performance. There is a tremendous number ofresource-rich and idle mobile connected vehicles (CVs) in the traffic network,and vehicles are created as opportunistic ad-hoc edge clouds to alleviatethe resource limitation of MEC by providing opportunistic computing services.On this basis, a novel scalable system framework is proposed in thispaper for computation task offloading in opportunistic CV-assisted MEC.In this framework, opportunistic ad-hoc edge cloud and fixed edge cloudcooperate to form a novel hybrid cloud. Meanwhile, offloading decision andresource allocation of the user CVs must be ascertained. Furthermore, thejoint offloading decision and resource allocation problem is described asa Mixed Integer Nonlinear Programming (MINLP) problem, which optimizesthe task response latency of user CVs under various constraints. Theoriginal problem is decomposed into two subproblems. First, the Lagrangedual method is used to acquire the best resource allocation with the fixedoffloading decision. Then, the satisfaction-driven method based on trial anderror (TE) learning is adopted to optimize the offloading decision. Finally, acomprehensive series of experiments are conducted to demonstrate that oursuggested scheme is more effective than other comparison schemes.

Intelligent Immunity Based Security Defense System for Multi-Access Edge Computing Network

In this paper,the security problem for the multi-access edge computing(MEC)network is researched,and an intelligent immunity-based security defense system is proposed to identify the unauthorized mobile users and to protect the security of whole system.In the proposed security defense system,the security is protected by the intelligent immunity through three functions,identification function,learning function,and regulation function,respectively.Meanwhile,a three process-based intelligent algorithm is proposed for the intelligent immunity system.Numerical simulations are given to prove the effeteness of the proposed approach.

Integration of Communication and Computing in Blockchain-Enabled Multi-Access Edge Computing Systems

Blockchain and multi-access edge com-puting(MEC)are two emerging promising tech-nologies that have received extensive attention from academia and industry.As a brand-new information storage,dissemination and management mechanism,blockchain technology achieves the reliable transmis-sion of data and value.While as a new computing paradigm,multi-access edge computing enables the high-frequency interaction and real-time transmission of data.The integration of communication and com-puting in blockchain-enabled multi-access edge com-puting networks has been studied without a systemat-ical view.In the survey,we focus on the integration of communication and computing,explores the mu-tual empowerment and mutual promotion effects be-tween the blockchain and MEC,and introduces the resource integration architecture of blockchain and multi-access edge computing.Then,the paper sum-marizes the applications of the resource integration ar-chitecture,resource management,data sharing,incen-tive mechanism,and consensus mechanism,and ana-lyzes corresponding applications in real-world scenar-ios.Finally,future challenges and potentially promis-ing research directions are discussed and present in de-tail.

A Review in the Core Technologies of 5G: Device-to-Device Communication, Multi-Access Edge Computing and Network Function Virtualization

5G is a new generation of mobile networking that aims to achieve unparalleled speed and performance. To accomplish this, three technologies, Device-to-Device communication (D2D), multi-access edge computing (MEC) and network function virtualization (NFV) with ClickOS, have been a significant part of 5G, and this paper mainly discusses them. D2D enables direct communication between devices without the relay of base station. In 5G, a two-tier cellular network composed of traditional cellular network system and D2D is an efficient method for realizing high-speed communication. MEC unloads work from end devices and clouds platforms to widespread nodes, and connects the nodes together with outside devices and third-party providers, in order to diminish the overloading effect on any device caused by enormous applications and improve users’ quality of experience (QoE). There is also a NFV method in order to fulfill the 5G requirements. In this part, an optimized virtual machine for middle-boxes named ClickOS is introduced, and it is evaluated in several aspects. Some middle boxes are being implemented in the ClickOS and proved to have outstanding performances.

基于高空平台的边缘计算卸载:网络、算法和展望

高空平台(high altitude platform,HAP)技术与多接入边缘计算(multi-access edge computing,MEC)技术的结合将MEC服务器部署区域由地面扩展到空中,打破传统地面MEC网络的局限性,为用户提供无处不在的计算卸载服务。针对基于HAP的MEC卸载研究进行综述,首先,从HAP计算节点的优势、网络组成部分、网络结构、主要挑战及其应对技术4个方面分析基于HAP的MEC网络;其次,分别从图论、博弈论、机器学习、联邦学习等理论的角度对基于HAP的MEC卸载算法进行横向分析和纵向对比;最后,指出基于HAP的MEC卸载技术目前存在的问题,并对该技术的未来研究方向进行展望。

Two-Stage IoT Computational Task Offloading Decision-Making in MEC with Request Holding and Dynamic Eviction

The rapid development of Internet of Things(IoT)technology has led to a significant increase in the computational task load of Terminal Devices(TDs).TDs reduce response latency and energy consumption with the support of task-offloading in Multi-access Edge Computing(MEC).However,existing task-offloading optimization methods typically assume that MEC’s computing resources are unlimited,and there is a lack of research on the optimization of task-offloading when MEC resources are exhausted.In addition,existing solutions only decide whether to accept the offloaded task request based on the single decision result of the current time slot,but lack support for multiple retry in subsequent time slots.It is resulting in TD missing potential offloading opportunities in the future.To fill this gap,we propose a Two-Stage Offloading Decision-making Framework(TSODF)with request holding and dynamic eviction.Long Short-Term Memory(LSTM)-based task-offloading request prediction and MEC resource release estimation are integrated to infer the probability of a request being accepted in the subsequent time slot.The framework learns optimized decision-making experiences continuously to increase the success rate of task offloading based on deep learning technology.Simulation results show that TSODF reduces total TD’s energy consumption and delay for task execution and improves task offloading rate and system resource utilization compared to the benchmark method.

Associative Tasks Computing Offloading Scheme in Internet of Medical Things with Deep Reinforcement Learning

The Internet of Medical Things(Io MT) is regarded as a critical technology for intelligent healthcare in the foreseeable 6G era. Nevertheless, due to the limited computing power capability of edge devices and task-related coupling relationships, Io MT faces unprecedented challenges. Considering the associative connections among tasks, this paper proposes a computing offloading policy for multiple-user devices(UDs) considering device-to-device(D2D) communication and a multi-access edge computing(MEC)technique under the scenario of Io MT. Specifically,to minimize the total delay and energy consumption concerning the requirement of Io MT, we first analyze and model the detailed local execution, MEC execution, D2D execution, and associated tasks offloading exchange model. Consequently, the associated tasks’ offloading scheme of multi-UDs is formulated as a mixed-integer nonconvex optimization problem. Considering the advantages of deep reinforcement learning(DRL) in processing tasks related to coupling relationships, a Double DQN based associative tasks computing offloading(DDATO) algorithm is then proposed to obtain the optimal solution, which can make the best offloading decision under the condition that tasks of UDs are associative. Furthermore, to reduce the complexity of the DDATO algorithm, the cacheaided procedure is intentionally introduced before the data training process. This avoids redundant offloading and computing procedures concerning tasks that previously have already been cached by other UDs. In addition, we use a dynamic ε-greedy strategy in the action selection section of the algorithm, thus preventing the algorithm from falling into a locally optimal solution. Simulation results demonstrate that compared with other existing methods for associative task models concerning different structures in the Io MT network, the proposed algorithm can lower the total cost more effectively and efficiently while also providing a tradeoff between delay and energy consumption tolerance.

Beyond 5G Networks: Integration of Communication, Computing, Caching, and Control

In recent years,the exponential proliferation of smart devices with their intelligent applications poses severe challenges on conventional cellular networks.Such challenges can be potentially overcome by integrating communication,computing,caching,and control(i4C)technologies.In this survey,we first give a snapshot of different aspects of the i4C,comprising background,motivation,leading technological enablers,potential applications,and use cases.Next,we describe different models of communication,computing,caching,and control(4C)to lay the foundation of the integration approach.We review current stateof-the-art research efforts related to the i4C,focusing on recent trends of both conventional and artificial intelligence(AI)-based integration approaches.We also highlight the need for intelligence in resources integration.Then,we discuss the integration of sensing and communication(ISAC)and classify the integration approaches into various classes.Finally,we propose open challenges and present future research directions for beyond 5G networks,such as 6G.

基于5G专网的智能电网业务实现方案

针对光纤网络、电力无线专网、Wi-Fi网络等电力通信网难以支撑输电线/变电站智能巡检、配网差动保护、广域相量测量、低压集抄等智能电网业务开展的情况,本文提出了基于5G专网的智能电网业务实现方案。即采用网络切片、边缘计算、高精度授时等5G技术,解决视频监测信号、“三遥”信号、电力终端感知信号等数据的灵活接入、实时传输和安全隔离等问题,实现电网运营数据的实时采集和智能控制。模拟仿真验证结果表明采用5G技术能满足智能电网业务对带宽、时延、抖动和安全的要求,可有效支撑智能电网业务实现。

Jointly Optimized Request Dispatching and Service Placement for MEC in LEO Network

Integrating Multi-access Edge Computing(MEC) in Low Earth Orbit(LEO) network is an important way to provide globally seamless low-delay service. In this paper, we consider the scenario that MEC platforms with computation and storage resource are deployed on LEO satellites, which is called "LEO-MEC". Service request dispatching decision is very important for resource utilization of the whole LEO-MEC system and Qo E of MEC users. Another important problem is service placement that is closely coupled with request dispatching. This paper models the joint service request dispatching and service placement problem as an optimization problem, which is a Mixed Integer Linear Programming(MILP). Our proposed mechanism solves this problem and uses the solved decision variables to dispatch requests and place services. Simulation results show that our proposed mechanism can achieve better performance in terms of ratio of served users and average hop count compared with baseline mechanism.

Joint resource allocation for hybrid NOMA-assisted MEC in 6G networks

Multi-access Edge Computing(MEC)is an essential technology for expanding computing power of mobile devices,which can combine the Non-Orthogonal Multiple Access(NOMA)in the power domain to multiplex signals to improve spectral efficiency.We study the integration of the MEC with the NOMA to improve the computation service for the Beyond Fifth-Generation(B5G)and the Sixth-Generation(6G)wireless networks.This paper aims to minimize the energy consumption of a hybrid NOMA-assisted MEC system.In a hybrid NOMA system,a user can offload its task during a time slot shared with another user by the NOMA,and then upload the remaining data during an exclusive time duration served by Orthogonal Multiple Access(OMA).The original energy minimization problem is non-convex.To efficiently solve it,we first assume that the user grouping is given,and focuses on the one group case.Then,a multilevel programming method is proposed to solve the non-convex problem by decomposing it into three subproblems,i.e.,power allocation,time slot scheduling,and offloading task assignment,which are solved optimally by carefully studying their convexity and monotonicity.The derived solution is optimal to the original problem by substituting the closed expressions obtained from those decomposed subproblems.Furthermore,we investigate the multi-user case,in which a close-to-optimal algorithm with lowcomplexity is proposed to form users into different groups with unique time slots.The simulation results verify the superior performance of the proposed scheme compared with some benchmarks,such as OMA and pure NOMA.

User association and resource allocation in green mobile edge networks using deep reinforcement learning

In order to meet the emerging requirements for high computational complexity, low delay and energy consumption of the 5 th generation wireless systems(5 G) network, ultra-dense networks(UDNs) combined with multi-access edge computing(MEC) can further improve network capacity and computing capability. In addition, the integration of green energy can effectively reduce the on-grid energy consumption of system and realize green computation. This paper studies the joint optimization of user association(UA) and resource allocation(RA) in MEC enabled UDNs under the green energy supply pattern, users need to perceive the green energy status of base stations(BSs) and choose the one with abundant resources to associate. To minimize the computation cost for all users, the optimization problem is formulated as a mixed integer nonlinear programming(MINLP) which is NP-hard. In order to solve the problem, a deep reinforcement learning(DRL)-based association and optimized allocation(DAOA) scheme is designed to solve it in two stages. The simulation results show that the proposed scheme has good performance in terms of computation cost and time out ratio, as well achieve load balancing potentially.

A multi-agent collaboration scheme for energy-efficient task scheduling in a 3D UAV-MEC space

Multi-access edge computing(MEC)presents computing services at the edge of networks to address the enormous processing requirements of intelligent applications.Due to the maneuverability of unmanned aerial vehicles(UAVs),they can be used as temporal aerial edge nodes for providing edge services to ground users in MEC.However,MEC environment is usually dynamic and complicated.It is a challenge for multiple UAVs to select appropriate service strategies.Besides,most of existing works study UAV-MEC with the assumption that the flight heights of UAVs are fixed;i.e.,the flying is considered to occur with reference to a two-dimensional plane,which neglects the importance of the height.In this paper,with consideration of the co-channel interference,an optimization problem of energy efficiency is investigated to maximize the number of fulfilled tasks,where multiple UAVs in a threedimensional space collaboratively fulfill the task computation of ground users.In the formulated problem,we try to obtain the optimal flight and sub-channel selection strategies for UAVs and schedule strategies for tasks.Based on the multi-agent deep deterministic policy gradient(MADDPG)algorithm,we propose a curiosity-driven and twin-networks-structured MADDPG(CTMADDPG)algorithm to solve the formulated problem.It uses the inner reward to facilitate the state exploration of agents,avoiding convergence at the sub-optimal strategy.Furthermore,we adopt the twin critic networks for update stabilization to reduce the probability of Q value overestimation.The simulation results show that CTMADDPG is outstanding in maximizing the energy efficiency of the whole system and outperforms the other benchmarks.